[argobots-discuss] condition variable and rwlock

Fri Jul 23 12:36:33 CDT 2021

Hello Phil,

> We use ABT_SCHED_BASIC_WAIT and ABT_POOL_FIFO_WAIT by default in Mochi (Margo).  Even so, if the default (simple) mutex is busy-yielding, that will effectively become a busy spin if the ES has no other units to execute, right?

If a ULT is waiting in a lock (e.g., ABT_mutex_lock() or ABT_rwlock_rwlock()), yes, you're right.  Sorry for the confusing behavior.  It should have been clarified somewhere.

This "default" behavior is only for a historical reason.  If `--disable-simple-mutex` solves some performance issues (this configuration option works for Argobots 1.1), please let us know.  I just need to change the default option for this.

Thanks,
Shintaro

________________________________
From: Phil Carns via discuss <discuss at lists.argobots.org>
Sent: Friday, July 23, 2021 12:25 PM
To: Iwasaki, Shintaro via discuss <discuss at lists.argobots.org>
Cc: Carns, Philip H. <carns at mcs.anl.gov>
Subject: Re: [argobots-discuss] condition variable and rwlock

On 7/23/21 11:53 AM, Iwasaki, Shintaro via discuss wrote:
Hi Matthieu,

By default, ABT_rwlock() and ABT_mutex() waits in a busy-yield loop (keeps yielding until a flag is set). If that configuration is set, waiters will suspend, not yield.

I'm confused :)

We use ABT_SCHED_BASIC_WAIT and ABT_POOL_FIFO_WAIT by default in Mochi (Margo).  Even so, if the default (simple) mutex is busy-yielding, that will effectively become a busy spin if the ES has no other units to execute, right?

If so, that wasn't our intention.  I thought that the ES would idle while trying to acquire an abt mutex if no other ULTs were runnable.  We might need to do some testing to better understand what's going on there.

thanks,

-Phil

This behavior becomes default because of a historical reason (https://github.com/pmodels/argobots/pull/102).  There's a performance trade-off between a simple mutex (current default) and a non-simple mutex, so we hesitate to change it silently.

If this default behavior is not what the user expects, I will create a PR to change it (if so I'd be happy if you create a GitHub issue that briefly describes this issue).

Thanks,
Shintaro
________________________________
From: Dorier, Matthieu <mdorier at anl.gov><mailto:mdorier at anl.gov>
Sent: Friday, July 23, 2021 10:34 AM
To: discuss at lists.argobots.org<mailto:discuss at lists.argobots.org> <discuss at lists.argobots.org><mailto:discuss at lists.argobots.org>; Iwasaki, Shintaro <siwasaki at anl.gov><mailto:siwasaki at anl.gov>
Subject: Re: condition variable and rwlock

Thanks, I'm curious about your comment about  --disable-simple-mutex. What you describe is what I would expect rwlock to do by default... How do rwlock by default, then?

Thanks
Matthieu

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: Iwasaki, Shintaro <siwasaki at anl.gov><mailto:siwasaki at anl.gov>
Sent: Friday, July 23, 2021 3:51:21 PM
To: discuss at lists.argobots.org<mailto:discuss at lists.argobots.org> <discuss at lists.argobots.org><mailto:discuss at lists.argobots.org>; Dorier, Matthieu <mdorier at anl.gov><mailto:mdorier at anl.gov>
Subject: Re: condition variable and rwlock

Hello Matthieu,

Thanks for your question!

> Is there a way of using an Argobots condition variable with an rwlock instead of a mutex?
No. The user cannot combine ABT_rwlock with ABT_cond.

> my use-case is a structure that receives many read requests and a few writes, clearly justifying using a rwlock instead of a mutex, but I may want some of the reads to block until a write has happened, which means I need a condition variable

First, if readers encounter an rwlock locked by a writer, they will suspend until the writer releases the rwlock (if --disable-simple-mutex is set, which is not set by default).

For this specific purpose(especially if "some", not "all" of the readers should block), what I came up with in my mind first is the following.  It looks fine except for a seemingly complex structure that uses multiple synchronization objects.

void reader() {
  while (1) {
    if (work_queue.is_empty() and I_AM_SOME_OF_READERS()) {
      // 1. Internally ABT_self_suspend()-like mechanism needs
      //    to take a lock (even if it is in the readers' lock)
      //    since multiple readers might access the same data
      //    structure (for example, a user-maintained
      //    suspended ULT list).
      // 2. Anyway this path is not performance sensitive
      ABT_mutex_lock(mutex);
      ABT_cond_wait(cond, mutex);
      ABT_mutex_unlock(mutex);
      // Now someone woke me up after pushing work.
    }
    ABT_rwlock_rdlock(rwlock);
    if (!work_queue.is_empty())
      ; // Do real work.
    ABT_rwlock_unlock(rwlock);
  }
}

void writer() {
  ABT_rwlock_wrlock(rwlock);
  work_queue.push_work(work);
  // You do not need to take a mutex to call ABT_cond_broadcast.
  // It is fine even if there's no waiter.
  ABT_cond_broadcast(cond);
  ABT_rwlock_unlock(rwlock);
}

I would welcome any suggestions! (For example, does ABT_rwlock_trywrlock() help?)

Best,
Shintaro
________________________________
From: Dorier, Matthieu via discuss <discuss at lists.argobots.org><mailto:discuss at lists.argobots.org>
Sent: Friday, July 23, 2021 5:03 AM
To: discuss at argobots.org<mailto:discuss at argobots.org> <discuss at argobots.org><mailto:discuss at argobots.org>
Cc: Dorier, Matthieu <mdorier at anl.gov><mailto:mdorier at anl.gov>
Subject: [argobots-discuss] condition variable and rwlock

Hi,

I suspect the answer is no, but is there a way of using an Argobots condition variable with an rwlock instead of a mutex?

(my use-case is a structure that receives many read requests and a few writes, clearly justifying using a rwlock instead of a mutex, but I may want some of the reads to block until a write has happened, which means I need a condition variable).

Thanks,

Matthieu

_______________________________________________
discuss mailing list
discuss at lists.argobots.org<mailto:discuss at lists.argobots.org>
https://lists.argobots.org/mailman/listinfo/discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.argobots.org/pipermail/discuss/attachments/20210723/b0779ab5/attachment-0001.html>