Nesting of rcu_read_locks

Question

I store a RCU protected pointer MyStruct *, in a RCU protected hashtable MyHash. When reading/updating MyStruct via hashtable, I do as shown below.

rcu_read_lock() /* For hashtable 'MyHash' */
hash_for_each_possible_rcu(MyHash, obj, member, key)
{
    rcu_read_lock(); /* For RCU protected data(MyStruct*) stored in hashtable */
    Mystruct* s = rcu_dereference_pointer(obj->s);
    if(s) 
    {
        s->read++;
    }
    rcu_read_unlock(); /* For RCU protected data(MyStruct*) stored in hashtable */
}
rcu_read_unlock() /* For hashtable 'MyHash'*/

Note that MyStruct is itself part of another RCU protected list ( i.e it is a RCU protected node of another list), which is stored in MyHash for faster lookup.

As I understand, rcu_read_lock's are required to make sure any writer update doesn't free up memory until all read-side critical sections are complete. So, is it really necessary to nest rcu_read_lock's or just having the outer rcu_read_lock/rcu_read_unlock is sufficient?

IOW, since RCU locks are not tied to any single object, do we really need nested rcu locks when accessing multiple RCU protected objects together?

There is only a **single object** corresponded to RCU read lock. So its effect doesn't depend from the number of nested `rcu_read_lock()`. — Tsyvarev, Nov 26 '19 at 08:24
@Tsyvarev so to make it clear I made an edit to my question. `MyStruct` is itself part of another RCU protected list i.e it is a RCU protected node in another list and this node pointer is stored my `MyHash` for fast lookups. Each of `MyHash` node is RCU protected as well. So essentially there are two RCU protected objects here. From my understanding(correct me if wrong), since RCU locks are not tied to any single object, do we need nested rcu locks when accessing multiple RCU protected objects together? — Shyam, Nov 26 '19 at 14:21

score 3 · Accepted Answer · answered Nov 26 '19 at 14:49

No, nested rcu_read_lock() is not required.

Similar to other "nested" critical sections, the only effect of nested rcu_read_lock is increment of the lock level. That is, futher rcu_read_unlock does not immediately end critical section but just revert lock level back.

However, supporting nested locking is treated as an advantage of RCU locking mechanism. Having supported nested operations one may develop components independently one from others.

E.g., you may have object_increment function which can be safely called without RCU lock:

void object_increment(Object obj)
{
    rcu_read_lock();
    Mystruct* s = obj->s;
    if(s) 
    {
        s->read++;
    }
    rcu_read_unlock();
}

Then call this function under RCU lock:

rcu_read_lock(); /* For hashtable 'MyHash' */
hash_for_each_possible_rcu(MyHash, obj, member, key)
{
    // It is perfectly valid to use the function even with RCU lock already taken
    object_increment(obj);
}
rcu_read_unlock(); /* For hashtable 'MyHash'*/

Simple design is almost always outweigh small performance hit from the nested calls to rcu_read_lock.

Without nested calls allowed one would need to implement another component's function for access with RCU lock:

void object_increment_locked(Object obj)
{
    Mystruct* s = obj->s;
    if(s) 
    {
        s->read++;
    }
}

and carefully choose which function - locked or non-locked - to use in concrete situations:

rcu_read_lock(); /* For hashtable 'MyHash' */
hash_for_each_possible_rcu(MyHash, obj, member, key)
{
    // Already have a lock taken, so use _locked version of the function.
    object_increment_locked(obj);
}
rcu_read_unlock(); /* For hashtable 'MyHash'*/

Nesting of rcu_read_locks

1 Answers1