[argobots-discuss] Idle Execution Stream

[Phil Carns]

FWIW, we use the ABT_POOL_FIFO_WAIT in the Mochi project as our default 
scheduler.  It works well as a tradeoff between CPU usage and response time.


That pool must also be used in conjunction with the ABT_SCHED_BASIC_WAIT 
scheduler to get the full benefit.  There is a simple example of setting 
both the waiting scheduler and pool at 
https://github.com/mochi-hpc/mochi-abt-io/blob/main/src/abt-io.c#L1351. 
That example is for a dedicated (explicitly created) pool, but you can 
also do something similar for the implicit main execution stream as well 
if necessary using ABT_xstream_set_main_sched_basic().


-Phil


On 5/7/21 6:45 PM, Iwasaki, Shintaro via discuss wrote:
> Hello Houjun,
>
> Thanks for your question.
> It depends on the setting. By default, the execution stream will 
> busily check the pool, so it wastes its underlying core if the pool is 
> empty.
> If so, it competes with other non-Argobots threads.
>
> To let idle execution streams sleep, Argobots provides a few options:
>
> 1. If you enable the scheduler sleep at configuration time, the ES 
> that uses a builtin scheduler will sleep when it cannot consecutively 
> find a ready ULT in its associated pools.
> ./configure --enable-sched-sleep
> The sleep time is configurable via ABT_SCHED_SLEEP_NSEC=XXX
> ABT_SCHED_SLEEP_NSEC=100 ./a.out
> This implementation is very naive but it is easy for users to try.
>
> 2. If ABT_POOL_FIFO_WAIT is used to create a pool, the ES will 
> timed-wait on an empty pool until a ULT is pushed to that pool.
> Internally, it uses a condition variable, so it is smarter. This needs 
> some efforts to use (at least you need to change the code).
> ...
> Perhaps you might be interested in the following example.
> https://github.com/pmodels/argobots/blob/main/examples/profiling/async_engine.c 
> <https://github.com/pmodels/argobots/blob/main/examples/profiling/async_engine.c>
> The following PR is related. This (indirectly) shows how to check the 
> CPU resource consumption by ESs.
> https://github.com/pmodels/argobots/pull/208 
> <https://github.com/pmodels/argobots/pull/208>
>
> If you have any further questions, please feel free to ask!
>
> Thanks,
> Shintaro
>
>
>
> ------------------------------------------------------------------------
> *From:* Houjun Tang via discuss <discuss at lists.argobots.org>
> *Sent:* Friday, May 7, 2021 5:25 PM
> *To:* discuss at argobots.org <discuss at argobots.org>
> *Cc:* Houjun Tang <htang4 at lbl.gov>
> *Subject:* [argobots-discuss] Idle Execution Stream
> Hi,
>
> From what I understand, an Argobots execution stream is mapped to a 
> pthread, and the ULTs are executed by it. What happens when there is 
> no ULT in the ES, does the pthread sleep and wait on a conditional 
> variable until a new ULT is scheduled for execution? Does it compete 
> with other non-Argobots threads for the CPU resources while idle?
>
> -- 
> Thanks,
> Houjun Tang
>
> _______________________________________________
> discuss mailing list
> discuss at lists.argobots.org
> https://lists.argobots.org/mailman/listinfo/discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.argobots.org/pipermail/discuss/attachments/20210510/0a16477b/attachment.html>