<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Matthieu,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I don't know when such an error typically happens. I'd be happy if you could share your code with me so that I can closely look at it.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;"><br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">This is just my guess, but perhaps ABT_pool_free() is used; in Argobots, primary ES's pool and schedulers may not be freed before ABT_finalize().</span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">If you free that fifo_wait MPMC pool before ABT_finalize(), an already freed pool is accessed in ABT_finalize(), which may cause the error you wrote.</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">(It is related to
<a href="https://github.com/pmodels/argobots/issues/58">https://github.com/pmodels/argobots/issues/58</a>).</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I attached a sample code to explain this issue. Please read the comments in the code if you are interested in it.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div style="margin:0px;font-size:12pt">Thanks,</div>
<span style="margin:0px;font-size:12pt">Shintaro</span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Dorier, Matthieu via discuss <discuss@lists.argobots.org><br>
<b>Sent:</b> Tuesday, March 23, 2021 6:56 AM<br>
<b>To:</b> discuss@argobots.org <discuss@argobots.org><br>
<b>Cc:</b> Dorier, Matthieu <mdorier@anl.gov><br>
<b>Subject:</b> [argobots-discuss] Program aborting in ABT_finalize (ABTI_pool_release failed)</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Calibri",sans-serif}
span.x_EmailStyle17
{font-family:"Calibri",sans-serif;
color:windowtext}
.x_MsoChpDefault
{font-size:12.0pt;
font-family:"Calibri",sans-serif}
@page WordSection1
{margin:72.0pt 72.0pt 72.0pt 72.0pt}
div.x_WordSection1
{}
-->
</style>
<div lang="EN-GB" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="x_WordSection1">
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Hi,</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt">I’m getting this error in a program, when calling ABT_finalize:</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt">../src/include/abti_pool.h:238: ABTI_pool_release: Assertion `ABTD_atomic_acquire_load_int32(&p_pool->num_scheds) > 0' failed.</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt">The program only uses the primary ES, it does not create additional ES, however it does replace the primary ES’s pool and scheduler by creating a fifo_wait MPMC pool and passing it to ABT_xstream_set_main_sched_basic,
with </span><span style="font-size:11.0pt">ABT_SCHED_BASIC_WAIT as sched_predef.</span><span lang="EN-US" style="font-size:11.0pt"></span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt">I’m using Argobots 1.0. I haven’t tested with other versions of Argobots. I’m sure I must be doing something wrong somewhere in my code. Could you tell me what I could look for that would lead
to such an error?</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt">For additional information, here it the stack trace when the error happens:</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">#1 0x00007f56797fe535 in __GI_abort () at abort.c:79</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">#2 0x00007f56797fe40f in __assert_fail_base (fmt=0x7f5679960ee0 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> assertion=0x7f5679d0e4c8 "ABTD_atomic_acquire_load_int32(&p_pool->num_scheds) > 0", file=0x7f5679d0e44b "../src/include/abti_pool.h", line=238,</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> function=<optimized out>) at assert.c:92</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">#3 0x00007f567980c102 in __GI___assert_fail (assertion=0x7f5679d0e4c8 "ABTD_atomic_acquire_load_int32(&p_pool->num_scheds) > 0",</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> file=0x7f5679d0e44b "../src/include/abti_pool.h", line=238, function=0x7f5679d0e660 <__PRETTY_FUNCTION__.6315> "ABTI_pool_release")</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> at assert.c:101</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">#4 0x00007f5679d09e5e in ABTI_sched_free.part.4 ()</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> from /projects/spack/opt/spack/linux-debian10-sandybridge/gcc-8.3.0/argobots-1.0-e4x7h6mgt7igmocc4ajro2g743ej322m/lib/libabt.so.0</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">#5 0x00007f5679cfe2c5 in ABTI_xstream_free ()</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> from /projects/spack/opt/spack/linux-debian10-sandybridge/gcc-8.3.0/argobots-1.0-e4x7h6mgt7igmocc4ajro2g743ej322m/lib/libabt.so.0</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">#6 0x00007f5679cfa080 in ABT_finalize ()</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">Thanks,</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">Matthieu</span></p>
</div>
</div>
</body>
</html>