Spinlocks

This talk presents an overview of spinlocks. We cover cache coherency models and discover ways we can make spinlocks more efficient with respect to modern hardware. We discuss some potentially undesirable properties of unfairness in spinlocks and look at implementations of fair spinlocks including ticket locks, array locks, MCS locks, and CLH locks.

Presented to the Operating Systems class at Johns Hopkins University as a guest lecture with Samy Al Bahra.

Transcript

1. Spinlocks Spinlocks Samy Al Bahra Devon H. O’Dell Message Systems, Inc. April 8, 2011
2. Spinlocks Introduction Mutexes A mutex is an object which implements acquire and relinquish operations such that the execution following an acquire operation and up to the relinquish operation is executed in a mutually exclusive manner relative to the object implementing a mutex.
3. Spinlocks Introduction Locks Locks are an implementation of a mutex. Sleep lock Any mutex type which deactivates processes that attempt to acquire a mutex that has already been acquired by another process until a relinquish operation on the mutex activates one or more of them. Spinlock Any mutex type which forces callers of an acquire operation to spend an unbounded number of processor cycles re-evaluating the availability of the mutex until it has been acquired. The process that invokes acquire is never deactivated before the completion of the acquire operation. Spinlocks are preferred to sleep mutexes when the waiting time for a resource is less than the time for the scheduling overhead of process activation/deactivation or when scheduling simply is not possible.
4. Spinlocks Non-Arbitrating Spinlocks Naive void lock(uint32_t *mutex) { while (ck_pr_fas_32(mutex, true) != false) ck_pr_stall(); return; } void unlock(uint32_t *mutex) { *mutex = false; return; }
5. Spinlocks Non-Arbitrating Spinlocks Naive 3e+07 4e+07 5e+07 6e+07 7e+07 8e+07 9e+07 1e+08 1.1e+08 1.2e+08 1.3e+08 2 3 4 5 6 7 8 9 10 11 12 Total Acquisitions (in 10 seconds) Number of Cores Naive
6. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
7. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
8. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
9. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
10. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
11. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
12. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
13. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
14. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
15. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
16. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
17. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
18. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
19. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
20. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
21. Spinlocks Non-Arbitrating Spinlocks Cache Coherency
22. Spinlocks Non-Arbitrating Spinlocks TATAS void lock(uint32_t *mutex) { while (ck_pr_fas_32(mutex, true) != false) { while (ck_pr_load_32(mutex) == true) ck_pr_stall(); } return; } void unlock(uint32_t *mutex) { *mutex = false; return; }
23. Spinlocks Non-Arbitrating Spinlocks TATAS 2e+07 4e+07 6e+07 8e+07 1e+08 1.2e+08 1.4e+08 1.6e+08 2 3 4 5 6 7 8 9 10 11 12 Total Acquisitions (in 10 seconds) Number of Cores Naive TATAS
24. Spinlocks Non-Arbitrating Spinlocks Exponential Backoff void lock(uint32_t *mutex) { ck_backoff_t backoff = CK_BACKOFF_INITIALIZER; while (ck_pr_fas_32(mutex, true) != false) ck_backoff_eb(&backoff); return; } void unlock(uint32_t *mutex) { *mutex = false; return; }
25. Spinlocks Non-Arbitrating Spinlocks Exponential Backoff 0 2e+08 4e+08 6e+08 8e+08 1e+09 1.2e+09 1.4e+09 2 3 4 5 6 7 8 9 10 11 12 Total Acquisitions (in 10 seconds) Number of Cores Naive TATAS EB
26. Spinlocks Non-Arbitrating Spinlocks Fairness 0 5e+06 1e+07 1.5e+07 2e+07 2.5e+07 3e+07 3.5e+07 4e+07 4.5e+07 Total Acquisitions (1o seconds) Processor
27. Spinlocks Non-Arbitrating Spinlocks Fairness Non-arbitrating spinlocks do not provide fairness (or starvation-freedom) guarantees.
28. Spinlocks Non-Arbitrating Spinlocks Ticket 0 1 0 2 2 1 3 2 3 Now Serving: 0 Now Serving: 0 Now Serving: 1 Now Serving: 2 Now Serving: 3 LOCK LOCK, LOCK UNLOCK UNLOCK UNLOCK UNLOCK, LOCK 4 Now Serving: 4 UNLOCK
29. Spinlocks Non-Arbitrating Spinlocks Anderson U L L L U L L L U L L L L U L L L U L L P P P P P L L U L P L L U L P LOCK LOCK UNLOCK LOCK UNLOCK LOCK L L L U P U L L L P UNLOCK UNLOCK
30. Spinlocks Non-Arbitrating Spinlocks MCS L L T0 R L T0 R T1 S L T0 R T1 S FAS L T0 N T1 S R = Running S = Spinning N = Notify L T1 R T0 LOCK T1 LOCK T0 UNLOCK
31. Spinlocks Non-Arbitrating Spinlocks MCS L T1 R T2 S L T1 S T2 S L T1 N T2 S L T2 R T2 LOCK T1 UNLOCK
32. Spinlocks Non-Arbitrating Spinlocks CLH Stub Stub T0 Stub T0 T1 T1 LOCK LOCK UNLOCK Stub T0 T1 T0 T1 UNLOCK R R R S S
33. Spinlocks Non-Arbitrating Spinlocks Fast path latency Fast path latency See http://concurrencykit.org/doc/appendixZ.html
34. Spinlocks Non-Arbitrating Spinlocks Limitations Mutexes in general are not composable. Subtle ordering issues can lead to hard-to-detect deadlock conditions. Blocking synchronization is sensitive to preemption.
35. Spinlocks Discussion Questions?