<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>Confirmed. Thanks again Shintaro!</p>
<p><br>
</p>
<p>I repeated my test case on Summit this morning using an
argobots@main spack build. The suspicious mutex behavior is gone
and the benchmark is hitting its performance target. I'm
optimistic that this will fix some other confusing performance
problems that we've seen there :)<br>
</p>
<p><br>
</p>
<p>We'll adjust our documentation/recommendations for building our
stack on Summit accordingly until there is a number spack version
with the fix.<br>
</p>
<p><br>
</p>
<p>-Phil<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 1/6/21 4:36 PM, Phil Carns via
discuss wrote:<br>
</div>
<blockquote type="cite" cite="mid:536994b4-d2ec-0c8c-e3b7-26c40e4fe913@mcs.anl.gov">
<p>Oh that's interesting! I'm glad I asked about this then :)</p>
<p><br>
</p>
<p>I'm reproducing this on Summit (POWER arch) with Argobots 1.0
and gcc (everything built using spack).<br>
</p>
<p><br>
</p>
<p>I cannot reproduce it on my laptop (x86_64 arch) with Argobots
1.0 and gcc, but there are so many differences between my laptop
and Summit I wasn't sure where to start :)<br>
</p>
<p><br>
</p>
<p>I'll try using the most recent git revision on Summit and see
what that does.</p>
<p><br>
</p>
<p>thanks,</p>
<p>-Phil<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 1/6/21 2:06 PM, Iwasaki, Shintaro
wrote:<br>
</div>
<blockquote type="cite" cite="mid:BY3PR09MB83053A5118D2FC5A0EEB4040D5D00@BY3PR09MB8305.namprd09.prod.outlook.com">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;}</style>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);"> Hi Phil,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);"> Thank you for a good
question! <span style="color: rgb(0, 0, 0); font-family:
Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">I
created an issue: <a class="moz-txt-link-freetext" href="https://github.com/pmodels/argobots/issues/287" moz-do-not-send="true">https://github.com/pmodels/argobots/issues/287</a></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<div><br>
</div>
<div>Yes, what you expect is correct. A ULT (lock/unlock) may
not yield if there is no contention. We guarantee this and
will make it clear in the specification.</div>
<div><br>
</div>
<div>The current Argobots (assuming the current master) should
work as you expect; ULT A should never yield in your case.</div>
<div><br>
</div>
<div>In the case of Argobots 1.0 or 1.0.1, a ULT may yield
because of the following possible reasons, both of which are
fixed in the current master:</div>
<div>1. Lock is not performed atomically "strong" while the
architecture supports weak atomics (e.g., on ARM and POWER)
(fixed by <a class="moz-txt-link-freetext" href="https://github.com/pmodels/argobots/pull/223" moz-do-not-send="true">https://github.com/pmodels/argobots/pull/223</a>)</div>
<div>2. If you explicitly pass `--disable-simple-mutex` at
configuration time, the previous mutex-handover mechanism
may have this issue (fixed by <a class="moz-txt-link-freetext" href="https://github.com/pmodels/argobots/pull/268" moz-do-not-send="true">https://github.com/pmodels/argobots/pull/268</a>)</div>
<div><br>
</div>
<div>Regarding 1., because some atomic instructions spuriously
fail ("weak" <a class="moz-txt-link-freetext" href="https://en.cppreference.com/w/c/atomic/atomic_compare_exchange" moz-do-not-send="true">https://en.cppreference.com/w/c/atomic/atomic_compare_exchange</a>),
maybe the current spinlock implementation in Argobots causes
this issue if you are using non-Intel hardware. I'd be
happy if you could let us know what combination of hardware
and compiler (with a compiler version) you are using. If
you are using Intel hardware, I believe the current Argobots
master work correctly unless you use a not-so-common
compiler (e.g., PGI), but I will check.</div>
<div><br>
</div>
<div>(Note that a priority lock/unlock is just a hint, so it
will not help.)</div>
<div><br>
</div>
<div>Anyway, I should make this point clearer in the
specification. At the same time, I will add a test to see
if this is really the case. If the current mechanism is
broken, I will fix it. Please estimate that this
clarification and fix (if possible) will come this week.</div>
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);"> Thanks,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);"> Shintaro Iwasaki</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);"> <br>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Phil
Carns via discuss <a class="moz-txt-link-rfc2396E" href="mailto:discuss@lists.argobots.org" moz-do-not-send="true"><discuss@lists.argobots.org></a><br>
<b>Sent:</b> Wednesday, January 6, 2021 12:27 PM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:discuss@lists.argobots.org" moz-do-not-send="true">discuss@lists.argobots.org</a> <a class="moz-txt-link-rfc2396E" href="mailto:discuss@lists.argobots.org" moz-do-not-send="true"><discuss@lists.argobots.org></a><br>
<b>Cc:</b> Carns, Philip H. <a class="moz-txt-link-rfc2396E" href="mailto:carns@mcs.anl.gov" moz-do-not-send="true"><carns@mcs.anl.gov></a><br>
<b>Subject:</b> [argobots-discuss] question about ABT_mutex
and ULT scheduling</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">Hi all,<br>
<br>
We've isolated a situation where the ABT_mutex construct
is behaving a <br>
little differently than I expected. We have two ULTs
running on a <br>
single ES. The ULTs are using ABT_mutex_lock/free() to
protect a shared <br>
data structure. This specific configuration will never
have lock <br>
contention (the mutex is really there to protect more
complex <br>
configurations where there are more ESs and ULTs
participating than what <br>
I described above).<br>
<br>
Here is what puzzles me: I'm not 100% sure, but it
really looks like ULT <br>
A yields to ULT B when attempting to lock the mutex
sometimes, even <br>
though there is no contention.<br>
<br>
This is a performance bug for us; ULT B is only supposed
to execute when <br>
ULT A is idle in this configuration. We don't really
want to give up <br>
execution when acquiring an uncontested mutex if we
don't have to.<br>
<br>
I'm sure we could work around it (restructuring our
code, or using a <br>
spinlock or priority mutex or something), but I wanted
to ask on the <br>
list first: is the behavior I described above (a ULT
yielding on a mutex <br>
lock, even if the mutex is available) expected? Or is
it an indication <br>
that we are doing something wrong somewhere? I want to
make sure that I <br>
understand the problem before altering the code.<br>
<br>
thanks!<br>
<br>
-Phil<br>
<br>
<br>
_______________________________________________<br>
discuss mailing list<br>
<a class="moz-txt-link-abbreviated" href="mailto:discuss@lists.argobots.org" moz-do-not-send="true">discuss@lists.argobots.org</a><br>
<a href="https://lists.argobots.org/mailman/listinfo/discuss" moz-do-not-send="true">https://lists.argobots.org/mailman/listinfo/discuss</a><br>
</div>
</span></font></div>
</blockquote>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:discuss@lists.argobots.org">discuss@lists.argobots.org</a>
<a class="moz-txt-link-freetext" href="https://lists.argobots.org/mailman/listinfo/discuss">https://lists.argobots.org/mailman/listinfo/discuss</a>
</pre>
</blockquote>
</body>
</html>