futex and userspace thread syncronization (gnu/linux glibc/nptl) analysis
(arrows mean "uses as backend", e.g. "GThread uses pthread as backend")
Before I start on mutextes and condvars I'd like to say a few words on the whole userspace thread libraries hierarchy.
First of all I'll talk about nptl and futexes mostly. As you can see from the drawing above std::thread, boost::thread and GThread are portability wrappers around pthreads containing glue code designed to duct tape pthreads to their respective platforms so not much interesting there.
Secondly pthread_create() essentially is
void * stack_addr = allocate_stack();
int clone_flags = (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_SIGNAL
| CLONE_SETTLS | CLONE_PARENT_SETTID
| CLONE_CHILD_CLEARTID | CLONE_SYSVSEM
| 0);
/* just see man pthread_create */
sys_clone(start_routine, stack_addr, clone_flags, arg, ...);
so not interesting as well.
Now for the juicy part.
For my analysis I used the condvar.c from Livermore Computing Center pthread tutorial (it's a pity the page does not have the needed anchor so just scroll to "Example Code - Using Condition Variables").
So what does pthread_mutex_lock() do? It's actually a wrapper around a simple futex usecase. And how does futex work? Let's say a thread wants to acquire the lock which is already locked (if the lock is unlocked then thread just atomically locks it and goes on without going to kernel). The thread goes to kernel (FUTEX_WAIT) puts itself to the wait queue (lock is identified by uaddr) and goes to sleep (beautiful, almost no busywaiting and cpu cycles lost). When thread that acquired the lock finishes the work in the critical section it lifts the lock, goes to kernel (FUTEX_WAKE) to wake the first (FIFO) thread waiting in queue and goes on with its work. When the time comes the woken up thread acquires the lock and goes to do its work in the critical section.
(Google Draw is so much fun!)
Now the condvar. Condvar is holding two mutexes (one for safeguarding against races and second for signalling). The funny thing is the futex mechanism makes the pthread_cond_signal() API (and in some cases the condvars themselves) pointless since it does the same FUTEX_WAKE thing as pthread_mutex_unlock() does. And yes, "_signal()" here is just a standard conformant name, it has nothing to do with unix signals. And pthread_cond_broadcast() just wakes everyone from wait queue up starting the wakestorm.
Conclusion:
futex() is a nice, a very very nice userspace mechanism for thread syncronization. If a thread needs the lock it doesn't need to spin (aside from short userspace spin to prevent going to kernel unless absolutely necessary), it just goes to sleep and is woken up when the time is right. And no sys_kill().
Refs:
http://www.kernel.org/doc/ols/2002/ols2002-pages-479-495.pdf
http://lwn.net/Articles/360699/
http://lwn.net/Articles/387246/
http://lwn.net/images/conf/rtlws11/papers/paper.10.html
http://people.redhat.com/drepper/futex.pdf
https://sourceware.org/git/?p=glibc.git;a=tree;h=refs/tags/glibc-2.18 /nptl