[argobots-discuss] Can I move a static mutex around in memory?

[Dorier, Matthieu]

Hi Shintaro,

Thank you for your answer.

In the example you show, you say “even if you unlock m2, m1 is not unlocked”. That said, if no ULT cares about m1 anymore, and all the ULTs know they should use m2, then why shouldn’t it be fine?

Maybe other objects in Argobots (such as an ES) are keeping pointers to mutexes, and these pointers would be invalidated?

Thanks,

Matthieu

From: "Iwasaki, Shintaro" <siwasaki at anl.gov>
Date: Tuesday, 15 June 2021 at 14:57
To: "discuss at argobots.org" <discuss at argobots.org>, "discuss at lists.argobots.org" <discuss at lists.argobots.org>
Cc: "Dorier, Matthieu" <mdorier at anl.gov>
Subject: Re: Can I move a static mutex around in memory?

Hi Matthieu,

Thanks for the question!
ABT_mutex -> Yes, it's internally just a pointer (more specifically, a handle), so you can copy it.
ABT_mutex_memory -> No, it saves it states in this memory (directly used as ABTI_mutex), so if you copy it, you basically corrupt the ABT_mutex structure.
https://github.com/pmodels/argobots/blob/main/src/include/abt.h.in#L1114-L1115
The same thing goes to ABT_cond_memory: do not move ABT_cond_memory around.

// Example (THE FOLLOWING IS FORBIDDEN)
ABT_mutex_memory m1 = ABT_MUTEX_INITIALIZER;
ABT_mutex_lock(ABT_MUTEX_MEMORY_GET_HANDLE(&m1));
ABT_mutex_memory m2;
memcpy(&m2, &m1, sizeof(ABT_mutex_memory));
// m2 is locked since m1 is locked. The mutex "entity" is copied, so even if you unlock m2, m1 is not unlocked.
// It can cause something even worse since this mutex entity may store the waiters or even a pointer to its own memory.
// Do not use ABT_mutex_memory in this way.
ABT_mutex_lock(ABT_MUTEX_MEMORY_GET_HANDLE(&m2));

In your case, reallocation on resizing std::vector can break the mutex data (even if ABT_mutex_memory is somehow moved "uniquely" ).
Please dynamically allocate each ABT_mutex_memory individually (std::vector<ABT_mutex_memory *>).

Thanks,
Shintaro


________________________________
From: Dorier, Matthieu via discuss <discuss at lists.argobots.org>
Sent: Tuesday, June 15, 2021 7:04 AM
To: discuss at argobots.org <discuss at argobots.org>
Cc: Dorier, Matthieu <mdorier at anl.gov>
Subject: [argobots-discuss] Can I move a static mutex around in memory?


Hi,



I would like to understand statically initialized mutex and condition variables better, in particular: can I move them around in memory (assuming all the ULTs in my code have a consistent way of locating them)?



This would come up in C++ containers, for instance:

std::vector<ABT_mutex_memory>

If I increase the size of the vector, reallocation may take place, and the existing mutexes will be moved somewhere else in memory. Assuming all the ULTs are accessing them by their index in the vector rather than by an ABT_mutex opaque pointer, then this should be fine as long as I am allowed to move ABT_mutex_memory around (regardless of their state). Is that the case?



Same question for condition variables?



Thanks,



Matthieu


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.argobots.org/pipermail/discuss/attachments/20210615/e82374bd/attachment-0003.html>