<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body>
<p>FWIW, we use the ABT_POOL_FIFO_WAIT in the Mochi project as our
default scheduler. It works well as a tradeoff between CPU usage
and response time.<br>
</p>
<p><br>
</p>
<p>That pool must also be used in conjunction with the
ABT_SCHED_BASIC_WAIT scheduler to get the full benefit. There is
a simple example of setting both the waiting scheduler and pool at
<a class="moz-txt-link-freetext" href="https://github.com/mochi-hpc/mochi-abt-io/blob/main/src/abt-io.c#L1351">https://github.com/mochi-hpc/mochi-abt-io/blob/main/src/abt-io.c#L1351</a>.
That example is for a dedicated (explicitly created) pool, but you
can also do something similar for the implicit main execution
stream as well if necessary using
ABT_xstream_set_main_sched_basic().</p>
<p><br>
</p>
<p>-Phil<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 5/7/21 6:45 PM, Iwasaki, Shintaro
via discuss wrote:<br>
</div>
<blockquote type="cite" cite="mid:DM6PR09MB575080F43BFE8CF3440830CBD5579@DM6PR09MB5750.namprd09.prod.outlook.com">
<style type="text/css" style="display:none;">P {margin-top:0;margin-bottom:0;}</style>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
Hello Houjun,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
Thanks for your question.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
It depends on the setting. By default, the execution stream will
busily check the pool, so it wastes its underlying core if the
pool is empty.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
If so, it competes with other non-Argobots threads.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
To let idle execution streams sleep, Argobots provides a few
options:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
1. If you enable the scheduler sleep at configuration time, the
ES that uses a builtin scheduler will sleep when it cannot
<span style="background-color:rgb(255, 255, 255);display:inline
!important">consecutively </span>find a ready ULT in its
associated pools.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
./configure --enable-sched-sleep<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
The sleep time is configurable via ABT_SCHED_SLEEP_NSEC=XXX</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
ABT_SCHED_SLEEP_NSEC=100 ./a.out<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
This implementation is very naive but it is easy for users to
try.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
2. If ABT_POOL_FIFO_WAIT is used to create a pool, the ES will
timed-wait on an empty pool until a ULT is pushed to that pool.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<span style="margin:0px;font-size:12pt">Internally, it uses a
condition variable, so it is smarter. </span><span style="font-size: 12pt;">This needs some efforts to use (at
least you need to change the code).</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<span style="margin:0px;font-size:12pt"></span> </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
...</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
Perhaps you might be interested in the following example.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<a href="https://github.com/pmodels/argobots/blob/main/examples/profiling/async_engine.c" moz-do-not-send="true">https://github.com/pmodels/argobots/blob/main/examples/profiling/async_engine.c</a><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
The following PR is related. This (indirectly) shows how to
check the CPU resource consumption by ESs.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<a href="https://github.com/pmodels/argobots/pull/208" moz-do-not-send="true">https://github.com/pmodels/argobots/pull/208</a><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
If you have any further questions, please feel free to ask!</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
Thanks,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
Shintaro</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Houjun
Tang via discuss <a class="moz-txt-link-rfc2396E" href="mailto:discuss@lists.argobots.org"><discuss@lists.argobots.org></a><br>
<b>Sent:</b> Friday, May 7, 2021 5:25 PM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:discuss@argobots.org">discuss@argobots.org</a> <a class="moz-txt-link-rfc2396E" href="mailto:discuss@argobots.org"><discuss@argobots.org></a><br>
<b>Cc:</b> Houjun Tang <a class="moz-txt-link-rfc2396E" href="mailto:htang4@lbl.gov"><htang4@lbl.gov></a><br>
<b>Subject:</b> [argobots-discuss] Idle Execution Stream</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>Hi,</div>
<div><br>
</div>
<div>From what I understand, an Argobots execution stream is
mapped to a pthread, and the ULTs are executed by it. What
happens when there is no ULT in the ES, does the pthread
sleep and wait on a conditional variable until a new ULT is
scheduled for execution? Does it compete with other
non-Argobots threads for the CPU resources while idle?<br>
</div>
<div><br>
-- <br>
<div dir="ltr" class="x_gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>Thanks,</div>
<div>Houjun Tang</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:discuss@lists.argobots.org">discuss@lists.argobots.org</a>
<a class="moz-txt-link-freetext" href="https://lists.argobots.org/mailman/listinfo/discuss">https://lists.argobots.org/mailman/listinfo/discuss</a>
</pre>
</blockquote>
</body>
</html>