[argobots-discuss] condition variable and rwlock
Phil Carns
carns at mcs.anl.gov
Tue Aug 10 13:26:06 CDT 2021
Hi Shintaro,
I just confirmed your description with a Margo test program. I've filled
in some description of the test case (and why this is an issue for our
use case) in a new github issue so we can track it and discuss possible
solutions there.
https://github.com/pmodels/argobots/issues/361.
thanks!
-Phil
On 7/23/21 1:36 PM, Iwasaki, Shintaro wrote:
> Hello Phil,
>
> > We use ABT_SCHED_BASIC_WAIT and ABT_POOL_FIFO_WAIT by default in
> Mochi (Margo). Even so, if the default (simple) mutex is
> busy-yielding, that will effectively become a busy spin if the ES has
> no other units to execute, right?
>
> If a ULT is waiting in a lock (e.g., ABT_mutex_lock() or
> ABT_rwlock_rwlock()), yes, you're right. Sorry for the confusing
> behavior. It should have been clarified somewhere.
>
> This "default" behavior is only for a historical reason. If
> `--disable-simple-mutex` solves some performance issues (this
> configuration option works for Argobots 1.1), please let us know. I
> just need to change the default option for this.
>
> Thanks,
> Shintaro
>
> ------------------------------------------------------------------------
> *From:* Phil Carns via discuss <discuss at lists.argobots.org>
> *Sent:* Friday, July 23, 2021 12:25 PM
> *To:* Iwasaki, Shintaro via discuss <discuss at lists.argobots.org>
> *Cc:* Carns, Philip H. <carns at mcs.anl.gov>
> *Subject:* Re: [argobots-discuss] condition variable and rwlock
>
>
> On 7/23/21 11:53 AM, Iwasaki, Shintaro via discuss wrote:
>> Hi Matthieu,
>>
>> By default, ABT_rwlock() and ABT_mutex() waits in a busy-yield loop
>> (keeps yielding until a flag is set). If that configuration is set,
>> waiters will suspend, not yield.
>
>
> I'm confused :)
>
>
> We use ABT_SCHED_BASIC_WAIT and ABT_POOL_FIFO_WAIT by default in Mochi
> (Margo). Even so, if the default (simple) mutex is busy-yielding,
> that will effectively become a busy spin if the ES has no other units
> to execute, right?
>
>
> If so, that wasn't our intention. I thought that the ES would idle
> while trying to acquire an abt mutex if no other ULTs were runnable.
> We might need to do some testing to better understand what's going on
> there.
>
>
> thanks,
>
> -Phil
>
>
>
>>
>> This behavior becomes default because of a historical reason
>> (https://github.com/pmodels/argobots/pull/102
>> <https://github.com/pmodels/argobots/pull/102>). There's a
>> performance trade-off between a simple mutex (current default) and a
>> non-simple mutex, so we hesitate to change it silently.
>>
>> If this default behavior is not what the user expects, I will create
>> a PR to change it (if so I'd be happy if you create a GitHub issue
>> that briefly describes this issue).
>>
>> Thanks,
>> Shintaro
>> ------------------------------------------------------------------------
>> *From:* Dorier, Matthieu <mdorier at anl.gov> <mailto:mdorier at anl.gov>
>> *Sent:* Friday, July 23, 2021 10:34 AM
>> *To:* discuss at lists.argobots.org <mailto:discuss at lists.argobots.org>
>> <discuss at lists.argobots.org> <mailto:discuss at lists.argobots.org>;
>> Iwasaki, Shintaro <siwasaki at anl.gov> <mailto:siwasaki at anl.gov>
>> *Subject:* Re: condition variable and rwlock
>> Thanks, I'm curious about your comment about --disable-simple-mutex.
>> What you describe is what I would expect rwlock to do by default...
>> How do rwlock by default, then?
>>
>> Thanks
>> Matthieu
>>
>> Get Outlook for Android <https://aka.ms/ghei36>
>>
>> ------------------------------------------------------------------------
>> *From:* Iwasaki, Shintaro <siwasaki at anl.gov> <mailto:siwasaki at anl.gov>
>> *Sent:* Friday, July 23, 2021 3:51:21 PM
>> *To:* discuss at lists.argobots.org <mailto:discuss at lists.argobots.org>
>> <discuss at lists.argobots.org> <mailto:discuss at lists.argobots.org>;
>> Dorier, Matthieu <mdorier at anl.gov> <mailto:mdorier at anl.gov>
>> *Subject:* Re: condition variable and rwlock
>> Hello Matthieu,
>>
>> Thanks for your question!
>>
>> > Is there a way of using an Argobots condition variable with an
>> rwlock instead of a mutex?
>> No. The user cannot combine ABT_rwlock with ABT_cond.
>>
>> > my use-case is a structure that receives many read requests and a
>> few writes, clearly justifying using a rwlock instead of a mutex, but
>> I may want some of the reads to block until a write has happened,
>> which means I need a condition variable
>>
>> First, if readers encounter an rwlock locked by a writer, they will
>> suspend until the writer releases the rwlock (if
>> --disable-simple-mutex is set, which is not set by default).
>>
>> For this specific purpose(especially if "some", not "all" of the
>> readers should block), what I came up with in my mind first is the
>> following. It looks fine except for a seemingly complex structure
>> that uses multiple synchronization objects.
>>
>> void reader() {
>> while (1) {
>> if (work_queue.is_empty() and I_AM_SOME_OF_READERS()) {
>> // 1. Internally ABT_self_suspend()-like mechanism needs
>> // to take a lock (even if it is in the readers' lock)
>> // since multiple readers might access the same data
>> // structure (for example, a user-maintained
>> // suspended ULT list).
>> // 2. Anyway this path is not performance sensitive
>> ABT_mutex_lock(mutex);
>> ABT_cond_wait(cond, mutex);
>> ABT_mutex_unlock(mutex);
>> // Now someone woke me up after pushing work.
>> }
>> ABT_rwlock_rdlock(rwlock);
>> if (!work_queue.is_empty())
>> ; // Do real work.
>> ABT_rwlock_unlock(rwlock);
>> }
>> }
>>
>> void writer() {
>> ABT_rwlock_wrlock(rwlock);
>> work_queue.push_work(work);
>> // You do not need to take a mutex to call ABT_cond_broadcast.
>> // It is fine even if there's no waiter.
>> ABT_cond_broadcast(cond);
>> ABT_rwlock_unlock(rwlock);
>> }
>>
>> I would welcome any suggestions! (For example, does
>> ABT_rwlock_trywrlock() help?)
>>
>> Best,
>> Shintaro
>> ------------------------------------------------------------------------
>> *From:* Dorier, Matthieu via discuss <discuss at lists.argobots.org>
>> <mailto:discuss at lists.argobots.org>
>> *Sent:* Friday, July 23, 2021 5:03 AM
>> *To:* discuss at argobots.org <mailto:discuss at argobots.org>
>> <discuss at argobots.org> <mailto:discuss at argobots.org>
>> *Cc:* Dorier, Matthieu <mdorier at anl.gov> <mailto:mdorier at anl.gov>
>> *Subject:* [argobots-discuss] condition variable and rwlock
>>
>> Hi,
>>
>> I suspect the answer is no, but is there a way of using an Argobots
>> condition variable with an rwlock instead of a mutex?
>>
>> (my use-case is a structure that receives many read requests and a
>> few writes, clearly justifying using a rwlock instead of a mutex, but
>> I may want some of the reads to block until a write has happened,
>> which means I need a condition variable).
>>
>> Thanks,
>>
>> Matthieu
>>
>>
>> _______________________________________________
>> discuss mailing list
>> discuss at lists.argobots.org <mailto:discuss at lists.argobots.org>
>> https://lists.argobots.org/mailman/listinfo/discuss <https://lists.argobots.org/mailman/listinfo/discuss>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.argobots.org/pipermail/discuss/attachments/20210810/cf1e20e7/attachment.html>
More information about the discuss
mailing list