<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>Hi Shintaro,</p>
<p><br>
</p>
<p>I just confirmed your description with a Margo test program.
I've filled in some description of the test case (and why this is
an issue for our use case) in a new github issue so we can track
it and discuss possible solutions there.</p>
<p><br>
</p>
<p> <a class="moz-txt-link-freetext" href="https://github.com/pmodels/argobots/issues/361">https://github.com/pmodels/argobots/issues/361</a>.</p>
<p><br>
</p>
<p>thanks!</p>
<p>-Phil</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 7/23/21 1:36 PM, Iwasaki, Shintaro
wrote:<br>
</div>
<blockquote type="cite" cite="mid:DM6PR09MB5750544F659013F2C3E0D2FFD5E59@DM6PR09MB5750.namprd09.prod.outlook.com">
<style type="text/css" style="display:none;">P {margin-top:0;margin-bottom:0;}</style>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
Hello Phil,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
> We use ABT_SCHED_BASIC_WAIT and ABT_POOL_FIFO_WAIT by
default in Mochi (Margo). Even so, if the default (simple)
mutex is busy-yielding, that will effectively become a busy spin
if the ES has no other units to execute, right?
<div><br>
</div>
<div>If a ULT is waiting in a lock (e.g., ABT_mutex_lock() or
ABT_rwlock_rwlock()), yes, you're right. Sorry for the
confusing behavior. It should have been clarified somewhere.</div>
<div><br>
</div>
<span>This "default" behavior is only for a historical reason.
If `--disable-simple-mutex` solves some performance issues
(this configuration option works for Argobots 1.1), please let
us know. I just need to change the default option for this.</span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<span><br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<span>Thanks,</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<span>Shintaro</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Phil
Carns via discuss <a class="moz-txt-link-rfc2396E" href="mailto:discuss@lists.argobots.org"><discuss@lists.argobots.org></a><br>
<b>Sent:</b> Friday, July 23, 2021 12:25 PM<br>
<b>To:</b> Iwasaki, Shintaro via discuss
<a class="moz-txt-link-rfc2396E" href="mailto:discuss@lists.argobots.org"><discuss@lists.argobots.org></a><br>
<b>Cc:</b> Carns, Philip H. <a class="moz-txt-link-rfc2396E" href="mailto:carns@mcs.anl.gov"><carns@mcs.anl.gov></a><br>
<b>Subject:</b> Re: [argobots-discuss] condition variable and
rwlock</font>
<div> </div>
</div>
<div>
<p><br>
</p>
<div class="x_moz-cite-prefix">On 7/23/21 11:53 AM, Iwasaki,
Shintaro via discuss wrote:<br>
</div>
<blockquote type="cite">
<style type="text/css" style="display:none">p
{margin-top:0;
margin-bottom:0}</style>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
<div>Hi Matthieu,</div>
<div><br>
</div>
<div>By default, ABT_rwlock() and ABT_mutex() waits in a
busy-yield loop (keeps yielding until a flag is set). If
that configuration is set, waiters will suspend, not
yield.</div>
</div>
</blockquote>
<p><br>
</p>
<p>I'm confused :)<br>
</p>
<p><br>
</p>
<p>We use ABT_SCHED_BASIC_WAIT and ABT_POOL_FIFO_WAIT by default
in Mochi (Margo). Even so, if the default (simple) mutex is
busy-yielding, that will effectively become a busy spin if the
ES has no other units to execute, right?
<br>
</p>
<p><br>
</p>
<p>If so, that wasn't our intention. I thought that the ES
would idle while trying to acquire an abt mutex if no other
ULTs were runnable. We might need to do some testing to
better understand what's going on there.</p>
<p><br>
</p>
<p>thanks,</p>
<p>-Phil<br>
</p>
<p><br>
</p>
<p><br>
</p>
<blockquote type="cite">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
<div><br>
</div>
<div>This behavior becomes default because of a historical
reason (<a class="x_moz-txt-link-freetext" href="https://github.com/pmodels/argobots/pull/102" moz-do-not-send="true">https://github.com/pmodels/argobots/pull/102</a>).
There's a performance trade-off between a simple mutex
(current default) and a non-simple mutex, so we hesitate
to change it silently.</div>
<div><br>
</div>
<div>If this default behavior is not what the user expects,
I will create a PR to change it (if so I'd be happy if you
create a GitHub issue that briefly describes this issue). </div>
<div><br>
</div>
<div>Thanks,</div>
<span>Shintaro</span><br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Dorier, Matthieu
<a class="x_moz-txt-link-rfc2396E" href="mailto:mdorier@anl.gov" moz-do-not-send="true"><mdorier@anl.gov></a><br>
<b>Sent:</b> Friday, July 23, 2021 10:34 AM<br>
<b>To:</b> <a class="x_moz-txt-link-abbreviated" href="mailto:discuss@lists.argobots.org" moz-do-not-send="true">
discuss@lists.argobots.org</a> <a class="x_moz-txt-link-rfc2396E" href="mailto:discuss@lists.argobots.org" moz-do-not-send="true">
<discuss@lists.argobots.org></a>; Iwasaki,
Shintaro <a class="x_moz-txt-link-rfc2396E" href="mailto:siwasaki@anl.gov" moz-do-not-send="true">
<siwasaki@anl.gov></a><br>
<b>Subject:</b> Re: condition variable and rwlock</font>
<div> </div>
</div>
<div>
<div dir="auto" style="direction:ltr; margin:0; padding:0;
font-family:sans-serif; font-size:11pt; color:black">
Thanks, I'm curious about your comment about
--disable-simple-mutex. What you describe is what I would
expect rwlock to do by default... How do rwlock by
default, then?<br>
<br>
</div>
<div dir="auto" style="direction:ltr; margin:0; padding:0;
font-family:sans-serif; font-size:11pt; color:black">
Thanks<br>
</div>
<div dir="auto" style="direction:ltr; margin:0; padding:0;
font-family:sans-serif; font-size:11pt; color:black">
Matthieu<span id="x_x_ms-outlook-android-cursor"></span><br>
<br>
</div>
<div dir="auto" style="direction:ltr; margin:0; padding:0;
font-family:sans-serif; font-size:11pt; color:black">
<span id="x_x_OutlookSignature">
<div dir="auto" style="direction:ltr; margin:0;
padding:0; font-family:sans-serif; font-size:11pt;
color:black">
Get <a href="https://aka.ms/ghei36" moz-do-not-send="true">Outlook for Android</a></div>
</span><br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Iwasaki, Shintaro
<a class="x_moz-txt-link-rfc2396E" href="mailto:siwasaki@anl.gov" moz-do-not-send="true"><siwasaki@anl.gov></a><br>
<b>Sent:</b> Friday, July 23, 2021 3:51:21 PM<br>
<b>To:</b> <a class="x_moz-txt-link-abbreviated" href="mailto:discuss@lists.argobots.org" moz-do-not-send="true">
discuss@lists.argobots.org</a> <a class="x_moz-txt-link-rfc2396E" href="mailto:discuss@lists.argobots.org" moz-do-not-send="true">
<discuss@lists.argobots.org></a>; Dorier,
Matthieu <a class="x_moz-txt-link-rfc2396E" href="mailto:mdorier@anl.gov" moz-do-not-send="true">
<mdorier@anl.gov></a><br>
<b>Subject:</b> Re: condition variable and rwlock</font>
<div> </div>
</div>
<style type="text/css" style="display:none">p
{margin-top:0;
margin-bottom:0}</style>
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
Hello Matthieu,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
Thanks for your question!</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
> Is there a way of using an Argobots condition
variable with an rwlock instead of a mutex?<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
No. The user cannot combine ABT_rwlock with ABT_cond.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
> my use-case is a structure that receives many read
requests and a few writes, clearly justifying using a
rwlock instead of a mutex, but I may want some of the
reads to block until a write has happened, which means I
need a condition variable
<div><br>
</div>
<div>First, if readers encounter an rwlock locked by a
writer, they will suspend until the writer releases
the rwlock (if --disable-simple-mutex is set, which is
not set by default).</div>
<div><br>
</div>
<div>For this specific purpose(especially if "some", not
"all" of the readers should block), what I came up
with in my mind first is the following. It looks fine
except for a seemingly complex structure that uses
multiple synchronization objects.</div>
<div>
<div><br>
</div>
<div>void reader() {</div>
<div> while (1) {</div>
<div> if (work_queue.is_empty() and
I_AM_SOME_OF_READERS()) {</div>
<div> // 1. Internally ABT_self_suspend()-like
mechanism needs</div>
<div> // to take a lock (even if it is in the
readers' lock)</div>
<div> // since multiple readers might access
the same data</div>
<div> // structure (for example, a
user-maintained</div>
<div> // suspended ULT list).</div>
<div> // 2. Anyway this path is not performance
sensitive</div>
<div> ABT_mutex_lock(mutex);</div>
<div> ABT_cond_wait(cond, mutex);</div>
<div> ABT_mutex_unlock(mutex);</div>
<div> // Now someone woke me up after pushing
work.</div>
<div> }</div>
<div> ABT_rwlock_rdlock(rwlock);</div>
<div> if (!work_queue.is_empty())</div>
<div> ; // Do real work.</div>
<div> ABT_rwlock_unlock(rwlock);</div>
<div> }</div>
<div>}</div>
<div><br>
</div>
<div>void writer() {</div>
<div> ABT_rwlock_wrlock(rwlock);</div>
<div> work_queue.push_work(work);</div>
<div> // You do not need to take a mutex to call
ABT_cond_broadcast.</div>
<div> // It is fine even if there's no waiter.</div>
<div> ABT_cond_broadcast(cond);</div>
<div> ABT_rwlock_unlock(rwlock);</div>
<span>}</span><br>
</div>
<div><br>
</div>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
<span>I would welcome any suggestions! (For example,
does ABT_rwlock_trywrlock() help?)<br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
<span><br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
<span>Best,</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12pt; color:rgb(0,0,0)">
<span>Shintaro</span></div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_x_x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Dorier, Matthieu via
discuss
<a class="x_moz-txt-link-rfc2396E" href="mailto:discuss@lists.argobots.org" moz-do-not-send="true"><discuss@lists.argobots.org></a><br>
<b>Sent:</b> Friday, July 23, 2021 5:03 AM<br>
<b>To:</b> <a class="x_moz-txt-link-abbreviated" href="mailto:discuss@argobots.org" moz-do-not-send="true">
discuss@argobots.org</a> <a class="x_moz-txt-link-rfc2396E" href="mailto:discuss@argobots.org" moz-do-not-send="true">
<discuss@argobots.org></a><br>
<b>Cc:</b> Dorier, Matthieu <a class="x_moz-txt-link-rfc2396E" href="mailto:mdorier@anl.gov" moz-do-not-send="true">
<mdorier@anl.gov></a><br>
<b>Subject:</b> [argobots-discuss] condition variable
and rwlock</font>
<div> </div>
</div>
<style>@font-face
{font-family:"Cambria Math"}@font-face
{font-family:Calibri}p.x_x_x_x_MsoNormal, li.x_x_x_x_MsoNormal, div.x_x_x_x_MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Calibri",sans-serif}span.x_x_x_x_EmailStyle17
{font-family:"Calibri",sans-serif;
color:windowtext}.x_x_x_x_MsoChpDefault
{font-size:12.0pt;
font-family:"Calibri",sans-serif}</style>
<div style="word-wrap:break-word" lang="EN-GB">
<div class="x_x_x_x_WordSection1">
<p class="x_x_x_x_MsoNormal"><span style="font-size:11.0pt" lang="EN-US">Hi,</span></p>
<p class="x_x_x_x_MsoNormal"><span style="font-size:11.0pt" lang="EN-US"> </span></p>
<p class="x_x_x_x_MsoNormal"><span style="font-size:11.0pt" lang="EN-US">I suspect
the answer is no, but is there a way of using an
Argobots condition variable with an rwlock instead
of a mutex?</span></p>
<p class="x_x_x_x_MsoNormal"><span style="font-size:11.0pt" lang="EN-US">(my use-case
is a structure that receives many read requests
and a few writes, clearly justifying using a
rwlock instead of a mutex, but I may want some of
the reads to block until a write has happened,
which means I need a condition variable).</span></p>
<p class="x_x_x_x_MsoNormal"><span style="font-size:11.0pt" lang="EN-US">Thanks,</span></p>
<p class="x_x_x_x_MsoNormal"><span style="font-size:11.0pt" lang="EN-US"> </span></p>
<p class="x_x_x_x_MsoNormal"><span style="font-size:11.0pt" lang="EN-US">Matthieu
</span></p>
</div>
</div>
</div>
</div>
<br>
<fieldset class="x_mimeAttachmentHeader"></fieldset>
<pre class="x_moz-quote-pre">_______________________________________________
discuss mailing list
<a class="x_moz-txt-link-abbreviated" href="mailto:discuss@lists.argobots.org" moz-do-not-send="true">discuss@lists.argobots.org</a>
<a class="x_moz-txt-link-freetext" href="https://lists.argobots.org/mailman/listinfo/discuss" moz-do-not-send="true">https://lists.argobots.org/mailman/listinfo/discuss</a>
</pre>
</blockquote>
</div>
</blockquote>
</body>
</html>