<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0cm;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">I’m creating the pool using
</span><span style="mso-fareast-language:EN-US">ABT_pool_create_basic and “automatic” set to false, and I do indeed call ABT_pool_free on it.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">I guess Argobots isn’t checking whether a pool is in use before freeing it (or whether it’s used by the first ES)?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">My code is pretty complex so I tried to write a reproducer but I don’t see the error with it. Maybe I’m corrupting memory, somehow, and it just happens to be fine in small code that don’t reuse the
memory after the ABT_pool_free… (though I ran address sanitizer and didn’t see any invalid access).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">I’ll see what happens if I remove the ABT_pool_free.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Matthieu<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">"Iwasaki, Shintaro" <siwasaki@anl.gov><br>
<b>Date: </b>Tuesday, 23 March 2021 at 14:01<br>
<b>To: </b>"discuss@argobots.org" <discuss@argobots.org>, "discuss@lists.argobots.org" <discuss@lists.argobots.org><br>
<b>Cc: </b>"Dorier, Matthieu" <mdorier@anl.gov><br>
<b>Subject: </b>Re: Program aborting in ABT_finalize (ABTI_pool_release failed)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Hi Matthieu,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">I don't know when such an error typically happens. I'd be happy if you could share your code with me so that I can closely look at it.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">This is just my guess, but perhaps ABT_pool_free() is used; in Argobots, primary ES's pool and schedulers may not be freed before ABT_finalize().<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">If you free that fifo_wait MPMC pool before ABT_finalize(), an already freed pool is accessed in ABT_finalize(), which may cause the error you wrote.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">(It is related to
<a href="https://github.com/pmodels/argobots/issues/58">https://github.com/pmodels/argobots/issues/58</a>).<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">I attached a sample code to explain this issue. Please read the comments in the code if you are interested in it.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Thanks,<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Shintaro<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> Dorier, Matthieu via discuss <discuss@lists.argobots.org><br>
<b>Sent:</b> Tuesday, March 23, 2021 6:56 AM<br>
<b>To:</b> discuss@argobots.org <discuss@argobots.org><br>
<b>Cc:</b> Dorier, Matthieu <mdorier@anl.gov><br>
<b>Subject:</b> [argobots-discuss] Program aborting in ABT_finalize (ABTI_pool_release failed)</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="xmsonormal"><span lang="EN-US" style="font-size:11.0pt">Hi,</span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US" style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US" style="font-size:11.0pt">I’m getting this error in a program, when calling ABT_finalize:</span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US" style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US" style="font-size:11.0pt">../src/include/abti_pool.h:238: ABTI_pool_release: Assertion `ABTD_atomic_acquire_load_int32(&p_pool->num_scheds) > 0' failed.</span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US" style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US" style="font-size:11.0pt">The program only uses the primary ES, it does not create additional ES, however it does replace the primary ES’s pool and scheduler by creating a fifo_wait MPMC pool and passing it to ABT_xstream_set_main_sched_basic,
with </span><span style="font-size:11.0pt">ABT_SCHED_BASIC_WAIT as sched_predef.</span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US" style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US" style="font-size:11.0pt">I’m using Argobots 1.0. I haven’t tested with other versions of Argobots. I’m sure I must be doing something wrong somewhere in my code. Could you tell me what I could look for that would lead
to such an error?</span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US" style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US" style="font-size:11.0pt">For additional information, here it the stack trace when the error happens:</span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US" style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">#1 0x00007f56797fe535 in __GI_abort () at abort.c:79</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">#2 0x00007f56797fe40f in __assert_fail_base (fmt=0x7f5679960ee0 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> assertion=0x7f5679d0e4c8 "ABTD_atomic_acquire_load_int32(&p_pool->num_scheds) > 0", file=0x7f5679d0e44b "../src/include/abti_pool.h", line=238,</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> function=<optimized out>) at assert.c:92</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">#3 0x00007f567980c102 in __GI___assert_fail (assertion=0x7f5679d0e4c8 "ABTD_atomic_acquire_load_int32(&p_pool->num_scheds) > 0",</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> file=0x7f5679d0e44b "../src/include/abti_pool.h", line=238, function=0x7f5679d0e660 <__PRETTY_FUNCTION__.6315> "ABTI_pool_release")</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> at assert.c:101</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">#4 0x00007f5679d09e5e in ABTI_sched_free.part.4 ()</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> from /projects/spack/opt/spack/linux-debian10-sandybridge/gcc-8.3.0/argobots-1.0-e4x7h6mgt7igmocc4ajro2g743ej322m/lib/libabt.so.0</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">#5 0x00007f5679cfe2c5 in ABTI_xstream_free ()</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> from /projects/spack/opt/spack/linux-debian10-sandybridge/gcc-8.3.0/argobots-1.0-e4x7h6mgt7igmocc4ajro2g743ej322m/lib/libabt.so.0</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">#6 0x00007f5679cfa080 in ABT_finalize ()</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">Thanks,</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">Matthieu</span><o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>