Futexes the good, the bad, the ugly

Futexes! The Good, the Bad and the Ugly!

Futexes! The Good, the Bad and the Ugly! (Part 1)

User space locks • Let’s talk first about locks in
user space

user space • It requires shared memory for the lock

user space • It requires shared memory for the lock ◦ Could be from threads ◦ Could be shared memory between processes

Spinning lock struct lock { long lock; }; void lock(lock
*l) { while (l->lock) ; l->lock = 1; }

*l) { while (l->lock) ; l->lock = 1; } ize pil Com er m ptim ay o

*l) { if (l->lock) { for (;;) ; } l->lock = 1; } tion iler p Com iza optim

*l) { while (READ_ONCE(l->lock)) ; l->lock = 1; }

*l) { while (*(const volatile long *)&(l->lock)) ; l->lock = 1; }

*l) { while (l->lock) barrier(); l->lock = 1; }

*l) { while (l->lock) asm (“” ::: “memory”); l->lock = 1; }

*l) { while (l->lock) ; l->lock = 1; }

*l) { while (l->lock) ; l->lock = 1; } ? Race

Compare and Exchange • Most architectures have a way to
atomically update a variable

atomically update a variable • Compare and exchange is one such way

atomically update a variable • Compare and exchange is one such way ◦ some architectures use Load Link and Store Conditional (ll/sc)

atomically update a variable • Compare and exchange is one such way ◦ some architectures use Load Link and Store Conditional (ll/sc) int cmpxchg(void *ptr, long old, long new);

atomically update a variable • Compare and exchange is one such way ◦ some architectures use Load Link and Store Conditional (ll/sc) d Rea *ptr int cmpxchg(void *ptr, long old, long new);

atomically update a variable • Compare and exchange is one such way ◦ some architectures use Load Link and Store Conditional (ll/sc) h old wit pare com int cmpxchg(void *ptr, long old, long new);

atomically update a variable • Compare and exchange is one such way ◦ some architectures use Load Link and Store Conditional (ll/sc) int cmpxchg(void *ptr, long old, long new); e plac e r , d tche ew a m n if with

atomically update a variable • Compare and exchange is one such way ◦ some architectures use Load Link and Store Conditional (ll/sc) int cmpxchg(void *ptr, long old, long new); retu rn o ld *p tr

atomically update a variable • Compare and exchange is one such way ◦ some architectures use Load Link and Store Conditional (ll/sc) void lock(lock *l) { while (l->lock) ; l->lock = 1; }

atomically update a variable • Compare and exchange is one such way ◦ some architectures use Load Link and Store Conditional (ll/sc) void lock(lock *l) { int old; do { while (l->lock) ; old = cmpxchg(&l->lock, 0, 1); } while (old != 0); }

atomically update a variable • Compare and exchange is one such way ◦ some architectures use Load Link and Store Conditional (ll/sc) void lock(lock *l) { int old; do { while (l->lock) ; old = cmpxchg(&l->lock, 0, 1); } while (old != 0); } l gina i r o rns ck Retu of &l->lo e valu

atomically update a variable • Compare and exchange is one such way ◦ some architectures use Load Link and Store Conditional (ll/sc) void lock(lock *l) { int old; do { while (l->lock) n agai t a ; e rep d, 1); e t a old = cmpxchg(&l->lock, 0, d If up } while (old != 0); }

atomically update a variable • Compare and exchange is one such way ◦ some architectures use Load Link and Store Conditional (ll/sc) void lock(lock *l) { int old; do { while (l->lock) ; old = xchg(&l->lock, 1); } while (old != 0); } This co o rk to o w uld

Mutexes (Sleeping locks) • Mutexes are safer when multiple tasks
share the processor

share the processor • Blocked tasks will “sleep” and wait for the owner to wake it

share the processor • Blocked tasks will “sleep” and wait for the owner to wake it • On Linux, a futex system call is used ◦ Added in 2002, Linux version 2.5.6

share the processor • Blocked tasks will “sleep” and wait for the owner to wake it • On Linux, a futex system call is used ◦ Added in 2002, Linux version 2.5.6 • Avoids a system call in the non-contended case

The Good

futex - “Fast User space mutex” static unsigned int volatile
lock_data; void lock(unsigned int *lock) { int old; do { old = cmpxchg(lock, 0, 1); if (old) futex(lock, FUTEX_WAIT, old, NULL); } while (old != 0); }

lock_data; void lock(unsigned int *lock) { int old; do { oken old = cmpxchg(lock, 0, 1); w e to b if (old) Wait futex(lock, FUTEX_WAIT, old, NULL); } while (old != 0); }

lock_data; void lock(unsigned int *lock) { int old; do { old = cmpxchg(lock, 0, 1); atch m t s Mu if (old) futex(lock, FUTEX_WAIT, old, NULL); } while (old != 0); }

futex - “Fast User space mutex” void unlock(unsigned int *lock)
{ *lock = 0; mb(); futex(lock, FUTEX_WAKE, 1); }

{ s aiter w *lock = 0; p eu Wak mb(); futex(lock, FUTEX_WAKE, 1); }

{ *lock = 0; mb(); futex(lock, FUTEX_WAKE, 1); } wa Only k 1w e up aiter

futex - “Fast User space mutex” Naive implementation void unlock(unsigned
int *lock) ll m ca e t s y { gas n i o *lock = 0; ys d Alwa mb(); futex(lock, FUTEX_WAKE, 1); }

void lock(unsigned int *lock) { int old2, old = 0,
new = 1; for (;;) { old2 = old; old = cmpxchg(lock, old2, new); if (old == old2) return; check: switch (old) { case 0: new = 1; continue; case 2: new = 3; continue; } old2 = old; old = cmpxchg(lock, old2, 3); if (old != old2) goto check; futex(lock, FUTEX_WAIT, 3, NULL); old = *lock & ~1; goto check; } }

new = 1; for (;;) { old2 = old; old = cmpxchg(lock, old2, new); if (old == old2) return; check: switch (old) { case 0: new = 1; continue; case 2: new = 3; continue; } old2 = old; old = cmpxchg(lock, old2, 3); if (old != old2) goto check; futex(lock, FUTEX_WAIT, 3, NULL); old = *lock & ~1; goto check; } } Try k t loc e g to

new = 1; for (;;) { old2 = old; turn old = cmpxchg(lock, old2,esnew); ? Re s Succ if (old == old2) return; check: switch (old) { case 0: new = 1; continue; case 2: new = 3; continue; } old2 = old; old = cmpxchg(lock, old2, 3); if (old != old2) goto check; futex(lock, FUTEX_WAIT, 3, NULL); old = *lock & ~1; goto check; } }

new = 1; for (;;) { old2 = old; old = cmpxchg(lock, old2, new); if (old == old2) k f loc o e return; alu us v o i v check: re est p T switch (old) { case 0: new = 1; continue; case 2: new = 3; continue; } old2 = old; old = cmpxchg(lock, old2, 3); if (old != old2) goto check; futex(lock, FUTEX_WAIT, 3, NULL); old = *lock & ~1; goto check; } }

new = 1; for (;;) { old2 = old; old = cmpxchg(lock, old2, new); if (old == old2) return; n agai k c o l check: ry to t , d switch (old) { locke If un case 0: new = 1; continue; case 2: new = 3; continue; } old2 = old; old = cmpxchg(lock, old2, 3); if (old != old2) goto check; futex(lock, FUTEX_WAIT, 3, NULL); old = *lock & ~1; goto check; } }

new = 1; for (;;) { old2 = old; old = cmpxchg(lock, old2, new); if (old == old2) return; iters s check: a w r h d wit ith waite e switch (old) { k c w lo If un k again case 0: new = 1; continue; c to lo y r t case 2: new = 3; continue; } old2 = old; old = cmpxchg(lock, old2, 3); if (old != old2) goto check; futex(lock, FUTEX_WAIT, 3, NULL); old = *lock & ~1; goto check; } }

new = 1; for (;;) { old2 = old; old = cmpxchg(lock, old2, new); if (old == old2) return; check: switch (old) { case 0: new = 1; continue; case 2: new = 3; continue; iters a w th } ld wi e h ock old2 = old; Set l old = cmpxchg(lock, old2, 3); if (old != old2) goto check; futex(lock, FUTEX_WAIT, 3, NULL); old = *lock & ~1; goto check; } }

new = 1; for (;;) { old2 = old; old = cmpxchg(lock, old2, new); if (old == old2) return; check: switch (old) { case 0: new = 1; continue; case 2: new = 3; continue; } ? old2 = old; cked o l n it u old = cmpxchg(lock, old2, 3); Was if (old != old2) goto check; futex(lock, FUTEX_WAIT, 3, NULL); old = *lock & ~1; goto check; } }

new = 1; for (;;) { old2 = old; old = cmpxchg(lock, old2, new); if (old == old2) return; check: switch (old) { case 0: new = 1; continue; case 2: new = 3; continue; } old2 = old; old = cmpxchg(lock, old2, 3); if (old != old2) goto check; futex(lock, FUTEX_WAIT, 3, NULL); old = *lock & ~1; goto check; } } for Wait lock

new = 1; for (;;) { old2 = old; old = cmpxchg(lock, old2, new); if (old == old2) return; check: switch (old) { case 0: new = 1; continue; case 2: new = 3; continue; } old2 = old; old = cmpxchg(lock, old2, 3); if (old != old2) goto check; n agai k c lo futex(lock, FUTEX_WAIT, 3, NULL); take o t old = *lock & ~1; Try goto check; } }

void unlock(unsigned int *lock) { int old, waiters; old =
cmpxchg(lock, 1, 0); if (old == 1) return; *lock = 2; waiters = futex(lock, FUTEX_WAKE, 1); if (!waiters) cmpxchg(lock, 2, 0); }

void unlock(unsigned int *lock) lock { e s a e
o rel t int old, waiters; y r T old = cmpxchg(lock, 1, 0); if (old == 1) return; *lock = 2; waiters = futex(lock, FUTEX_WAKE, 1); if (!waiters) cmpxchg(lock, 2, 0); }

void unlock(unsigned int *lock) { int old, waiters; eturn R
old = cmpxchg(lock, 1, r0); ? s aite w if (old == 1) o N return; *lock = 2; waiters = futex(lock, FUTEX_WAKE, 1); if (!waiters) cmpxchg(lock, 2, 0); }

cmpxchg(lock, 1, 0); ck to o l t rs if (old == 1) Se ers! ith waite t i a W w return; ked c o l un *lock = 2; waiters = futex(lock, FUTEX_WAKE, 1); if (!waiters) cmpxchg(lock, 2, 0); }

cmpxchg(lock, 1, 0); if (old == 1) return; *lock = 2; waiters = futex(lock, FUTEX_WAKE, 1); if (!waiters) cmpxchg(lock, 2, 0); } Wa pw ke u aiter

cmpxchg(lock, 1, 0); if (old == 1) return; *lock = 2; rs? iteFUTEX_WAKE, a waiters = futex(lock, 1); w No if (!waiters) cmpxchg(lock, 2, 0); }

cmpxchg(lock, 1, 0); if (old == 1) return; *lock = 2; just o t k c waiters = futex(lock, FUTEX_WAKE,et1); o s l ed o t if (!waiters) Try unlock cmpxchg(lock, 2, 0); }

cmpxchg(lock, 1, 0); if (old == 1) return; *lock = 2; waiters = futex(lock, FUTEX_WAKE, 1); if (!waiters) cmpxchg(lock, 2, 0); e}if car not t o r n Do eded o ce suc

Futex system call • Allows user space to synchronize with
other threads or processes

Futex system call • Allows user space to synchronize with
other threads or processes • Threads are easy as they share the same memory space

Thread 1 Thread 2 0x00000000

Lo ck Thread 1 Thread 2 0x00000001

c Lo Thread 1 0x00000001 k? Thread 2

Thread 1 Thread 2 0x00000003 Zzzzzz

Un loc k Thread 1 Thread 2 0x00000002 Zzzzzz

Thread 1 Thread 2 0x00000002 futex(lock, FUTEX_WAKE, 1); Zzzzzz

p Thread 1 w eu ak 0x00000002 futex(lock, FUTEX_WAKE, 1);
Thread 2

k c Lo Thread 1 0x00000003 Thread 2

Futex system call • Processes need to add shared memory
(mmap or shmem)

(mmap or shmem) • This usually means that the memory is not associated with a single task

(mmap or shmem) • This usually means that the memory is not associated with a single task • If that task exits, the memory is still persistent

The Bad

Process 1 Process 2 0x00000000

Lo ck Process 1 Process 2 0x00000001

Process 1 Process 2 0x00000001

Process 2 0x00000001

c Lo 0x00000001 k? Process 2

Process 2 0x00000003 Zzzzzz

Process 2 0x00000001 Zzzzzz

Robust Futexes • Robust futexes solve the problem of an
owner crashing

owner crashing • It has its own API to define a “contract” between user space and the kernel

owner crashing • It has its own API to define a “contract” between user space and the kernel • A per-thread list of locks owned by the thread

owner crashing • It has its own API to define a “contract” between user space and the kernel • A per-thread list of locks owned by the thread ◦ New system call to tell kernel where the head of the list is ◦ set_robust_list(head, len);

owner crashing • It has its own API to define a “contract” between user space and the kernel • A per-thread list of locks owned by the thread ◦ New system call to tell kernel where the head of the list is ◦ set_robust_list(head, len); • The mutex holds the pointer to the next lock

owner crashing • It has its own API to define a “contract” between user space and the kernel • A per-thread list of locks owned by the thread ◦ New system call to tell kernel where the head of the list is ◦ set_robust_list(head, len); • The mutex holds the pointer to the next lock • Only the owner can set and clear that link list

struct robust_list { struct robust_list }; *next; struct robust_list_head {
struct robust_list list; long offset; struct robust_list *list_op_pending; }; struct my_mutex { unsigned int struct robust_list }; #define FUTEX_WAITERS #define FUTEX_OWNER_DIED lock; list; 0x80000000 0x40000000 void init_thread(mythread_t *thread) { struct robust_list_head *rl = &thread->robust_list; memset(rl, 0, sizeof(*rl)); rl->list.next = &rl->list; rl->list.offest = offsetof(struct my_mutex, lock) offsetof(struct my_mutex, list); set_robust_list(rb, sizeof(*rl)); }

struct robust_list list; long offset; struct robust_list *list_op_pending; }; struct my_mutex { unsigned int struct robust_list }; #define FUTEX_WAITERS #define FUTEX_OWNER_DIED lock; list; 0x80000000 0x40000000 d hrea t r e ep niqu U void init_thread(mythread_t *thread) { struct robust_list_head *rl = &thread->robust_list; memset(rl, 0, sizeof(*rl)); rl->list.next = &rl->list; rl->list.offest = offsetof(struct my_mutex, lock) offsetof(struct my_mutex, list); set_robust_list(rb, sizeof(*rl)); }

struct robust_list list; long offset; struct robust_list *list_op_pending; }; struct my_mutex { unsigned int struct robust_list }; #define FUTEX_WAITERS #define FUTEX_OWNER_DIED lock; list; 0x80000000 0x40000000 void init_thread(mythread_t *thread) { struct robust_list_head *rl = &thread->robust_list; memset(rl, 0, sizeof(*rl)); rl->list.next = &rl->list; rl->list.offest = offsetof(struct my_mutex, lock) offsetof(struct my_mutex, list); set_robust_list(rb, sizeof(*rl)); } ist ust l b o R

struct robust_list list; long offset; struct robust_list *list_op_pending; }; struct my_mutex { unsigned int struct robust_list }; #define FUTEX_WAITERS #define FUTEX_OWNER_DIED lock; list; 0x80000000 0x40000000 void init_thread(mythread_t *thread) { struct robust_list_head *rl = &thread->robust_list; ty memset(rl, 0, sizeof(*rl)); Emp rl->list.next = &rl->list; rl->list.offest = offsetof(struct my_mutex, lock) offsetof(struct my_mutex, list); set_robust_list(rb, sizeof(*rl)); }

struct robust_list list; long offset; struct robust_list *list_op_pending; }; struct my_mutex { unsigned int struct robust_list }; #define FUTEX_WAITERS #define FUTEX_OWNER_DIED lock; list; 0x80000000 0x40000000 void init_thread(mythread_t *thread) { struct robust_list_head *rl = &thread->robust_list; memset(rl, 0, sizeof(*rl)); rl->list.next = &rl->list; rl->list.offest = offsetof(struct my_mutex, lock) offsetof(struct my_mutex, list); set_robust_list(rb, sizeof(*rl)); } t m lis o r f t Offse to lock entry

struct robust_list list; long offset; struct robust_list *list_op_pending; }; struct my_mutex { unsigned int struct robust_list }; #define FUTEX_WAITERS #define FUTEX_OWNER_DIED lock; list; 0x80000000 0x40000000 void init_thread(mythread_t *thread) { struct robust_list_head *rl = &thread->robust_list; memset(rl, 0, sizeof(*rl)); l rl->list.next = &rl->list; erne k e rl->list.offest = offsetof(struct my_mutex, lock) ith thw r e ist offsetof(struct my_mutex, Reg list); set_robust_list(rb, sizeof(*rl)); }

struct robust_list list; long offset; struct robust_list *list_op_pending; }; struct my_mutex { unsigned int struct robust_list }; #define FUTEX_WAITERS #define FUTEX_OWNER_DIED lock; list; 0x80000000 0x40000000 b New its void init_thread(mythread_t *thread) { struct robust_list_head *rl = &thread->robust_list; memset(rl, 0, sizeof(*rl)); rl->list.next = &rl->list; rl->list.offest = offsetof(struct my_mutex, lock) offsetof(struct my_mutex, list); set_robust_list(rb, sizeof(*rl)); }

int robust_lock(struct mutex *mutex) { struct my_mutex *rf = (struct
my_mutex *)mutex; int old2, old = 0, id = gettid(), new = id; for (;;) { old2 = old; old = cmpxchg(&rf->lock, old2, new); if (old == old2) { set_robust(rf); return 0; } check: if (old & FUTEX_OWNER_DIED) return -1; switch (old) { case 0: new = id; continue; case FUTEX_WAITERS: new = old | id; continue; } old2 = old; old = cmpxchg(&rf->lock, old2, old2 | id); if (old != old2) goto check; futex(&rf->lock, FUTEX_WAIT, old, NULL); old = rf->lock & FUTEX_WAITERS; goto check; } }

my_mutex *)mutex; int old2, old = 0, id = gettid(), new = id; for (;;) { old2 = old; old = cmpxchg(&rf->lock, old2, new); if (old == old2) { set_robust(rf); return 0; } check: if (old & FUTEX_OWNER_DIED) return -1; switch (old) { case 0: new = id; continue; case FUTEX_WAITERS: new = old | id; continue; } old2 = old; old = cmpxchg(&rf->lock, old2, old2 | id); if (old != old2) goto check; futex(&rf->lock, FUTEX_WAIT, old, NULL); old = rf->lock & FUTEX_WAITERS; goto check; } } Hide robu s mut t within ex

int robust_lock(struct mutex *mutex) ock the l { s i
d ID struct my_mutex *rf = (struct my_mutexT*)mutex; hrea int old2, old = 0, id = gettid(), new = id; for (;;) { old2 = old; old = cmpxchg(&rf->lock, old2, new); if (old == old2) { set_robust(rf); return 0; } check: if (old & FUTEX_OWNER_DIED) return -1; switch (old) { case 0: new = id; continue; case FUTEX_WAITERS: new = old | id; continue; } old2 = old; old = cmpxchg(&rf->lock, old2, old2 | id); if (old != old2) goto check; futex(&rf->lock, FUTEX_WAIT, old, NULL); old = rf->lock & FUTEX_WAITERS; goto check; } }

my_mutex *)mutex; int old2, old = 0, id = gettid(), new = id; for (;;) { old2 = old; ck old = cmpxchg(&rf->lock,Ha old2, ve lo new); if (old == old2) { set_robust(rf); return 0; } check: if (old & FUTEX_OWNER_DIED) return -1; switch (old) { case 0: new = id; continue; case FUTEX_WAITERS: new = old | id; continue; } old2 = old; old = cmpxchg(&rf->lock, old2, old2 | id); if (old != old2) goto check; futex(&rf->lock, FUTEX_WAIT, old, NULL); old = rf->lock & FUTEX_WAITERS; goto check; } }

my_mutex *)mutex; int old2, old = 0, id = gettid(), new = id; for (;;) { old2 = old; old = cmpxchg(&rf->lock, old2, new); ist ate l if (old == old2) { d p U set_robust(rf); return 0; } check: if (old & FUTEX_OWNER_DIED) return -1; switch (old) { case 0: new = id; continue; case FUTEX_WAITERS: new = old | id; continue; } old2 = old; old = cmpxchg(&rf->lock, old2, old2 | id); if (old != old2) goto check; futex(&rf->lock, FUTEX_WAIT, old, NULL); old = rf->lock & FUTEX_WAITERS; goto check; } }

my_mutex *)mutex; int old2, old = 0, id = gettid(), new = id; for (;;) { old2 = old; old = cmpxchg(&rf->lock, old2, new); if (old == old2) { set_robust(rf); return 0; ad 😢 } e d er is check: Own if (old & FUTEX_OWNER_DIED) return -1; switch (old) { case 0: new = id; continue; case FUTEX_WAITERS: new = old | id; continue; } old2 = old; old = cmpxchg(&rf->lock, old2, old2 | id); if (old != old2) goto check; futex(&rf->lock, FUTEX_WAIT, old, NULL); old = rf->lock & FUTEX_WAITERS; goto check; } }

my_mutex *)mutex; int old2, old = 0, id = gettid(), new = id; for (;;) { old2 = old; old = cmpxchg(&rf->lock, old2, new); if (old == old2) { set_robust(rf); return 0; } ow check: k kn s a t e if (old & FUTEX_OWNER_DIED) et th L return -1; switch (old) { case 0: new = id; continue; case FUTEX_WAITERS: new = old | id; continue; } old2 = old; old = cmpxchg(&rf->lock, old2, old2 | id); if (old != old2) goto check; futex(&rf->lock, FUTEX_WAIT, old, NULL); old = rf->lock & FUTEX_WAITERS; goto check; } }

static __thread mythread_t thread; void set_robust(struct my_mutex *rl) { rl->list.next
= thread.robust_list.next; thread.robust_list.next = &rl->list; }

= thread.robust_list.next; thread.robust_list.next = &rl->list; } Lin kt oc urr en th ea d

static __thread mythread_t thread; void set_robust(struct my_mutex *rl) t {
to lis x e t rl->list.next = thread.robust_list.next; d mu d A thread.robust_list.next = &rl->list; }

= thread.robust_list.next; thread.robust_list.next = &rl->list; } Now if we die, the kernel will know!

static __thread mythread_t thread; t if w ha ut, w
void set_robust(struct my_mutex *rl) B { rl->list.next = thread.robust_list.next; thread.robust_list.next = &rl->list; } ere? h e i ed

struct robust_list list; long offset; struct robust_list *list_op_pending; }; struct my_mutex { unsigned int struct robust_list }; #define FUTEX_WAITERS #define FUTEX_OWNER_DIED tor! rotec p e Rac lock; list; 0x80000000 0x40000000 void init_thread(mythread_t *thread) { struct robust_list *rl = &thread->robust_list; memset(rl, 0, sizeof(*rl)); rl->list.next = &rl->list; rl->list.offest = offsetof(struct my_mutex, lock) offsetof(struct my_mutex, list); set_robust_list(rb, sizeof(*rl)); }

list_op_pending • Before locking assign the list_op_pending to the lock
we are about to grab

list_op_pending • Before locking assign the list_op_pending to the lock
we are about to grab • If the task crashes, the kernel will look at this lock ◦ If it holds the TID of the task, it will treat that lock as owner died

static int try_robust_lock(struct my_mutex *rf, int *old, int new) {
int old2; } old2 = old; *old = cmpxchg(&rf->lock, old2, new); if (*old == old2) { set_robust(rf); return 1; } return 0; Move locking into helper function

static __thread mythread_t thread; int robust_lock(struct my_mutex *rf, int *old,
int id) { struct my_mutex *rf = (struct my_mutex *)mutex; int ret, id = gettid(), new = id; for (;;) { thread.robust_list.list_op_pending = &rf->list.next; ret = try_robust_lock(rf, &old, new); thread.robust_list.list_op_pending = NULL; if (ret) return 0; check: if (old & FUTEX_OWNER_DIED) return -1; switch (old) { case 0: new = id; continue; case FUTEX_WAITERS: new = old | id; continue; } old2 = old; old = cmpxchg(&rf->lock, old2, old2 | id); if (old != old2) goto check; futex(&rf->lock, FUTEX_WAIT, old, NULL); old = rf->lock & FUTEX_WAITERS; goto check; } }

int id) { struct my_mutex *rf = (struct my_mutex *)mutex; int ret, id = gettid(), new = id; for (;;) { thread.robust_list.list_op_pending = &rf->list.next; ret = try_robust_lock(rf, &old, new); thread.robust_list.list_op_pending = NULL; if (ret) return 0; check: if (old & FUTEX_OWNER_DIED) return -1; switch (old) { case 0: new = id; continue; case FUTEX_WAITERS: new = old | id; continue; } old2 = old; old = cmpxchg(&rf->lock, old2, old2 | id); if (old != old2) goto check; futex(&rf->lock, FUTEX_WAIT, old, NULL); old = rf->lock & FUTEX_WAITERS; goto check; } } are ck e w lo nel the r e ab lk Tel to gr ut abo

int id) { struct my_mutex *rf = (struct my_mutex *)mutex; int ret, id = gettid(), new = id; k for (;;) { t loc e g o thread.robust_list.list_op_pending = &rf->list.next; Try t ret = try_robust_lock(rf, &old, new); thread.robust_list.list_op_pending = NULL; if (ret) return 0; check: if (old & FUTEX_OWNER_DIED) return -1; switch (old) { case 0: new = id; continue; case FUTEX_WAITERS: new = old | id; continue; } old2 = old; old = cmpxchg(&rf->lock, old2, old2 | id); if (old != old2) goto check; futex(&rf->lock, FUTEX_WAIT, old, NULL); old = rf->lock & FUTEX_WAITERS; goto check; } }

int id) { struct my_mutex *rf = (struct my_mutex *)mutex; int ret, id = gettid(), new = id; for (;;) { lock thread.robust_list.list_op_pending = &rf->list.next; e have rw ret = try_robust_lock(rf, &old, new); Eithe or not thread.robust_list.list_op_pending = NULL; if (ret) return 0; check: if (old & FUTEX_OWNER_DIED) return -1; switch (old) { case 0: new = id; continue; case FUTEX_WAITERS: new = old | id; continue; } old2 = old; old = cmpxchg(&rf->lock, old2, old2 | id); if (old != old2) goto check; futex(&rf->lock, FUTEX_WAIT, old, NULL); old = rf->lock & FUTEX_WAITERS; goto check; } }

static __thread mythread_t thread; void robust_unlock(struct mutex *mutex) { struct
my_mutex *rf = (struct my_mutex *)mutex; struct robust_list **next; int old, waiters, id = gettid(); out: } thread.robust_list.list_op_pending = &rf->list.next; for (next = &thread->robust_list.next; (*next)->next != &rf->list; next = &(*next)->next) ; *next = (*next)->next; rl->list.next = thread.robust_list.next; old = cmpxchg(&rf->lock, id, 0); if (old == id) goto out; *lock = FUTEX_WAITERS; waiters = futex(&rf->lock, FUTEX_WAKE, 1); if (!waiters) cmpxchg(&rf->lock, FUTEX_WAITERS, 0); thread.robust_list.list_op_pending = NULL;

my_mutex *rf = (struct my_mutex *)mutex; struct robust_list **next; int old, waiters, id = gettid(); ck Rea out: } se lo a e l o re dy t thread.robust_list.list_op_pending = &rf->list.next; for (next = &thread->robust_list.next; (*next)->next != &rf->list; next = &(*next)->next) ; *next = (*next)->next; rl->list.next = thread.robust_list.next; old = cmpxchg(&rf->lock, id, 0); if (old == id) goto out; *lock = FUTEX_WAITERS; waiters = futex(&rf->lock, FUTEX_WAKE, 1); if (!waiters) cmpxchg(&rf->lock, FUTEX_WAITERS, 0); thread.robust_list.list_op_pending = NULL;

my_mutex *rf = (struct my_mutex *)mutex; struct robust_list **next; int old, waiters, id = gettid(); out: } thread.robust_list.list_op_pending = &rf->list.next; for (next = &thread->robust_list.next; (*next)->next != &rf->list; next = &(*next)->next) Find this l ; ock *next = (*next)->next; rl->list.next = thread.robust_list.next; old = cmpxchg(&rf->lock, id, 0); if (old == id) goto out; *lock = FUTEX_WAITERS; waiters = futex(&rf->lock, FUTEX_WAKE, 1); if (!waiters) cmpxchg(&rf->lock, FUTEX_WAITERS, 0); thread.robust_list.list_op_pending = NULL;

my_mutex *rf = (struct my_mutex *)mutex; struct robust_list **next; int old, waiters, id = gettid(); out: } thread.robust_list.list_op_pending = &rf->list.next; for (next = &thread->robust_list.next; (*next)->next != &rf->list; list m o r f next = &(*next)->next) k e loc v o ; Rem *next = (*next)->next; rl->list.next = thread.robust_list.next; old = cmpxchg(&rf->lock, id, 0); if (old == id) goto out; *lock = FUTEX_WAITERS; waiters = futex(&rf->lock, FUTEX_WAKE, 1); if (!waiters) cmpxchg(&rf->lock, FUTEX_WAITERS, 0); thread.robust_list.list_op_pending = NULL;

my_mutex *rf = (struct my_mutex *)mutex; struct robust_list **next; int old, waiters, id = gettid(); out: } thread.robust_list.list_op_pending = &rf->list.next; for (next = &thread->robust_list.next; (*next)->next != &rf->list; next = &(*next)->next) ; *next = (*next)->next; rl->list.next = thread.robust_list.next; nlock U old = cmpxchg(&rf->lock, id, 0); if (old == id) goto out; *lock = FUTEX_WAITERS; waiters = futex(&rf->lock, FUTEX_WAKE, 1); if (!waiters) cmpxchg(&rf->lock, FUTEX_WAITERS, 0); thread.robust_list.list_op_pending = NULL;

my_mutex *rf = (struct my_mutex *)mutex; struct robust_list **next; int old, waiters, id = gettid(); out: } thread.robust_list.list_op_pending = &rf->list.next; for (next = &thread->robust_list.next; (*next)->next != &rf->list; next = &(*next)->next) ; *next = (*next)->next; rl->list.next = thread.robust_list.next; old = cmpxchg(&rf->lock, id, 0); if (old == id) goto out; *lock = FUTEX_WAITERS; waiters = futex(&rf->lock, FUTEX_WAKE, 1); nel r e k if (!waiters) l l to te ock d cmpxchg(&rf->lock, FUTEX_WAITERS, 0); e e n sl No thread.robust_list.list_op_pending = NULL; t abou thi

my_mutex *rf = (struct my_mutex *)mutex; struct robust_list **next; int old, waiters, id = gettid(); out: } thread.robust_list.list_op_pending = &rf->list.next; for (next = &thread->robust_list.next; (*next)->next != &rf->list; next = &(*next)->next) ; *next = (*next)->next; rl->list.next = thread.robust_list.next; old = cmpxchg(&rf->lock, id, 0); if (old == id) goto out; *lock = FUTEX_WAITERS; waiters = futex(&rf->lock, FUTEX_WAKE, 1); if (!waiters) cmpxchg(&rf->lock, FUTEX_WAITERS, 0); ace? R thread.robust_list.list_op_pending = NULL;

The Ugly

The Robust Futex Unlock Bug • Has to do with
another task removing the mutex after unlock

The Robust Futex Unlock Bug • • Has to do
with another task removing the mutex after unlock Some data has refcounts and locks to synchronize destruction

The Robust Futex Unlock Bug • • • Has to
do with another task removing the mutex after unlock Some data has refcounts and locks to synchronize destruction When the lock is unlocked, the lock can be freed

do with another task removing the mutex after unlock Some data has refcounts and locks to synchronize destruction When the lock is unlocked, the lock can be freed ◦ Task A unlocks and gets preempted before updating list_op_pending

do with another task removing the mutex after unlock Some data has refcounts and locks to synchronize destruction When the lock is unlocked, the lock can be freed ◦ Task A unlocks and gets preempted before updating list_op_pending ◦ Task B locks sets condition that the lock is not used anymore

do with another task removing the mutex after unlock Some data has refcounts and locks to synchronize destruction When the lock is unlocked, the lock can be freed ◦ Task A unlocks and gets preempted before updating list_op_pending ◦ Task B locks sets condition that the lock is not used anymore ◦ Task B unlocks, frees the mutex, remaps the mutex

do with another task removing the mutex after unlock Some data has refcounts and locks to synchronize destruction When the lock is unlocked, the lock can be freed ◦ Task A unlocks and gets preempted before updating list_op_pending ◦ Task B locks sets condition that the lock is not used anymore ◦ Task B unlocks, frees the mutex, remaps the mutex ◦ Task B adds a value to the lock address that matches Task A’s TID

do with another task removing the mutex after unlock Some data has refcounts and locks to synchronize destruction When the lock is unlocked, the lock can be freed ◦ Task A unlocks and gets preempted before updating list_op_pending ◦ Task B locks sets condition that the lock is not used anymore ◦ Task B unlocks, frees the mutex, remaps the mutex ◦ Task B adds a value to the lock address that matches Task A’s TID ◦ Task A triggers an exception and exits

do with another task removing the mutex after unlock Some data has refcounts and locks to synchronize destruction When the lock is unlocked, the lock can be freed ◦ Task A unlocks and gets preempted before updating list_op_pending ◦ Task B locks sets condition that the lock is not used anymore ◦ Task B unlocks, frees the mutex, remaps the mutex ◦ Task B adds a value to the lock address that matches Task A’s TID ◦ Task A triggers an exception and exits ◦ The kernel sees Task A’s list_op_pending and cleans up the mutex

do with another task removing the mutex after unlock Some data has refcounts and locks to synchronize destruction When the lock is unlocked, the lock can be freed ◦ Task A unlocks and gets preempted before updating list_op_pending ◦ Task B locks sets condition that the lock is not used anymore ◦ Task B unlocks, frees the mutex, remaps the mutex ◦ Task B adds a value to the lock address that matches Task A’s TID ◦ Task A triggers an exception and exits ◦ The kernel sees Task A’s list_op_pending and cleans up the mutex ◦ The kernel adds the FUTEX_OWNER_DIED flag

do with another task removing the mutex after unlock Some data has refcounts and locks to synchronize destruction When the lock is unlocked, the lock can be freed ◦ Task A unlocks and gets preempted before updating list_op_pending ◦ Task B locks sets condition that the lock is not used anymore ◦ Task B unlocks, frees the mutex, remaps the mutex ◦ Task B adds a value to the lock address that matches Task A’s TID ◦ Task A triggers an exception and exits ◦ The kernel sees Task A’s list_op_pending and cleans up the mutex ◦ The kernel adds the FUTEX_OWNER_DIED flag ◦ Task B’s value is now corrupted!

The Robust Futex Unlock Bug • First reported in 2012
◦ https://sourceware.org/bugzilla/show_bug.cgi?id=14485

◦ https://sourceware.org/bugzilla/show_bug.cgi?id=14485 • Partially fixed THIS YEAR!

◦ https://sourceware.org/bugzilla/show_bug.cgi?id=14485 • Partially fixed THIS YEAR! • Mathieu Desnoyers proposed a fix via VDSO ◦ https://lore.kernel.org/all/[email protected]/

◦ https://sourceware.org/bugzilla/show_bug.cgi?id=14485 • Partially fixed THIS YEAR! • Mathieu Desnoyers proposed a fix via VDSO ◦ • https://lore.kernel.org/all/[email protected]/ Thomas Gleixner extended Mathieu’s idea ◦ https://lore.kernel.org/all/[email protected]/

◦ https://sourceware.org/bugzilla/show_bug.cgi?id=14485 • Partially fixed THIS YEAR! • Mathieu Desnoyers proposed a fix via VDSO ◦ • https://lore.kernel.org/all/[email protected]/ Thomas Gleixner extended Mathieu’s idea ◦ https://lore.kernel.org/all/[email protected]/ ◦ Added in 7.2 (the latest Linux release)

◦ https://sourceware.org/bugzilla/show_bug.cgi?id=14485 • Partially fixed THIS YEAR! • Mathieu Desnoyers proposed a fix via VDSO ◦ • https://lore.kernel.org/all/[email protected]/ Thomas Gleixner extended Mathieu’s idea ◦ https://lore.kernel.org/all/[email protected]/ ◦ Added in 7.2 (the latest Linux release) ◦ Introduced FUTEX_ROBUST_UNLOCK and __vdso_futex_robust_try_unlock()

futex_robust_try_unlock() (Pseudo Code) int futex_robust_try_unlock(int *lock, int tid, void *pop)
{ struct my_mutex *rf = (struct my_mutex *)mutex; asm volatile (“ xor %ecx,%ecx // Set ECX to 0 lock cmpxchg %ecx,(%rdi) // Try the TID -> 0 transition .Lstart: jnz .Lend movq %rcx,(%rdx) // Clear list_op_pending .Lend: “ : “+&a”(tid) : “D”(lock), “d”(pop)”); } return tid;

{ struct my_mutex *rf = (struct my_mutex *)mutex; asm volatile (“ xor %ecx,%ecx // Set ECX to t0ask d e h s lock cmpxchg %ecx,(%rdi) // Tryf cthe -> 0 transition i ra TID e t r s e e h T .Lstart: IP is jnz .Lend movq %rcx,(%rdx) // Clear list_op_pending .Lend: “ : “+&a”(tid) : “D”(lock), “d”(pop)”); } return tid;

{ struct my_mutex *rf = (struct my_mutex *)mutex; lag asm volatile (“ ero F Z e th xor %ecx,%ecx Sets// Set ECX to 0 lock cmpxchg %ecx,(%rdi) // Try the TID -> 0 transition .Lstart: jnz .Lend movq %rcx,(%rdx) // Clear list_op_pending .Lend: “ : “+&a”(tid) : “D”(lock), “d”(pop)”); } return tid;

{ struct my_mutex *rf = (struct my_mutex *)mutex; asm volatile (“ xor %ecx,%ecx // Set ECX to 0 lock cmpxchg %ecx,(%rdi) // Try the TID -> 0 transition .Lstart: jnz .Lend movq %rcx,(%rdx) // Clear list_op_pending .Lend: “ : “+&a”(tid) : “D”(lock), “d”(pop)”); } return tid; list_op_pending = regs->flags & X86_EFLAGS_ZF ? NULL : regs->dx;

{ struct my_mutex *rf = (struct my_mutex *)mutex; asm volatile (“ xor %ecx,%ecx // Set ECX to 0 lock cmpxchg %ecx,(%rdi) // Try the TID -> 0 transition .Lstart: jnz .Lend movq %rcx,(%rdx) // Clear list_op_pending .Lend: “ : “+&a”(tid) : “D”(lock), “d”(pop)”); } Kernel Code return tid; list_op_pending = regs->flags & X86_EFLAGS_ZF ? NULL : regs->dx;

my_mutex *rf = (struct my_mutex *)mutex; struct robust_list **next; int old, waiters, id = gettid(); } thread.robust_list.list_op_pending = &rf->list.next; for (next = &thread->robust_list.next; (*next)->next != &rf->list; next = &(*next)->next) ; *next = (*next)->next; rl->list.next = thread.robust_list.next; old = futex_robust_try_unlock(&rf->lock, id, &thread.robust_list.list_op_pending); if (old == id) return; *lock = FUTEX_WAITERS; waiters = futex(&rf->lock, FUTEX_WAKE|FUTEX_ROBUST_UNLOCK, 1); if (!waiters) cmpxchg(&rf->lock, FUTEX_WAITERS, 0);

my_mutex *rf = (struct my_mutex *)mutex; struct robust_list **next; int old, waiters, id = gettid(); } thread.robust_list.list_op_pending = &rf->list.next; for (next = &thread->robust_list.next; (*next)->next != &rf->list; next = &(*next)->next) ) ; chg( x p m *next = (*next)->next; he c t s e rl->list.next = thread.robust_list.next; Do old = futex_robust_try_unlock(&rf->lock, id, &thread.robust_list.list_op_pending); if (old == id) return; *lock = FUTEX_WAITERS; waiters = futex(&rf->lock, FUTEX_WAKE|FUTEX_ROBUST_UNLOCK, 1); if (!waiters) cmpxchg(&rf->lock, FUTEX_WAITERS, 0);

my_mutex *rf = (struct my_mutex *)mutex; struct robust_list **next; int old, waiters, id = gettid(); } thread.robust_list.list_op_pending = &rf->list.next; for (next = &thread->robust_list.next; (*next)->next != &rf->list; next = &(*next)->next) O clea n succe ; rs lis t_op ss, *next = (*next)->next; _pen ding rl->list.next = thread.robust_list.next; old = futex_robust_try_unlock(&rf->lock, id, &thread.robust_list.list_op_pending); if (old == id) return; *lock = FUTEX_WAITERS; waiters = futex(&rf->lock, FUTEX_WAKE|FUTEX_ROBUST_UNLOCK, 1); if (!waiters) cmpxchg(&rf->lock, FUTEX_WAITERS, 0);

my_mutex *rf = (struct my_mutex *)mutex; struct robust_list **next; int old, waiters, id = gettid(); } thread.robust_list.list_op_pending = &rf->list.next; for (next = &thread->robust_list.next; (*next)->next != &rf->list; es o d next = &(*next)->next) e, LCK s ; i rw UNO e *next = (*next)->next; h Ot ST_ rl->list.next = thread.robust_list.next; BU O old = futex_robust_try_unlock(&rf->lock, id, &thread.robust_list.list_op_pending); R X_ if (old == id) E T return; FU *lock = FUTEX_WAITERS; waiters = futex(&rf->lock, FUTEX_WAKE|FUTEX_ROBUST_UNLOCK, 1); if (!waiters) cmpxchg(&rf->lock, FUTEX_WAITERS, 0);

my_mutex *rf = (struct my_mutex *)mutex; struct robust_list **next; int old, waiters, id = gettid(); thread.robust_list.list_op_pending = &rf->list.next; for (next = &thread->robust_list.next; (*next)->next != &rf->list; next = &(*next)->next) ; *next = (*next)->next; rl->list.next = thread.robust_list.next; old = futex_robust_try_unlock(&rf->lock, id, &thread.robust_list.list_op_pending); if (old == id) return; *lock = FUTEX_WAITERS; waiters = futex(&rf->lock, FUTEX_WAKE|FUTEX_ROBUST_UNLOCK, 1); if (!waiters) this! s e o cmpxchg(&rf->lock, FUTEX_WAITERS, 0); ow d n l e // out: Kern // thread.robust_list.list_op_pending = NULL; }

Other ugliness • PI futex ◦ Does priority inheritance ◦
Great for RT tasks

Great for RT tasks ◦ Sucks for non-RT tasks (a fair lock) ▪ A topic for another day

Great for RT tasks ◦ Sucks for non-RT tasks (a fair lock) ▪ A topic for another day ◦ Work to fix this (PI Next Generation)

Great for RT tasks ◦ Sucks for non-RT tasks (a fair lock) ▪ A topic for another day ◦ Work to fix this (PI Next Generation) ▪ PING!

Great for RT tasks ◦ Sucks for non-RT tasks (a fair lock) ▪ A topic for another day ◦ Work to fix this (PI Next Generation) ▪ PING! • Condition variables ◦ This can be a talk on to itself

Questions?

Futexes the good, the bad, the ugly

Futexes the good, the bad, the ugly

More Decks by Kernel Recipes

Other Decks in Technology

Featured

Transcript