Tweet
Tweet

Slide 1

Slide 1 text

Concurrency Patterns in Go [email protected]
 @arnecls

Slide 2

Slide 2 text

Concurrency is about design.

Slide 3

Slide 3 text

Design your program as a collection of independent processes Design these processes to eventually run in parallel Design your code so that the outcome is always the same

Slide 4

Slide 4 text

Concurrency in detail • group code (and data) by identifying independent tasks • no race conditions • no deadlocks • more workers = faster execution

Slide 5

Slide 5 text

Communicating Sequential Processes (CSP) • Tony Hoare, 1978 1. Each process is built for sequential execution 2. Data is communicated between processes via channels.
 No shared state! 3. Scale by adding more of the same

Slide 6

Slide 6 text

Go’s concurrency toolset • go routines • channels • select • sync package

Slide 7

Slide 7 text

Channels • Think of a bucket chain • 3 components: sender, buffer, receiver • The buffer is optional

Slide 8

Slide 8 text

zZz zZz zZz zZz

Slide 9

Slide 9 text

Blocking channels unbuffered := make(chan int) // 1) a := <- unbuffered

Slide 10

Slide 10 text

zZz Blocking channels unbuffered := make(chan int) // 1) blocks a := <- unbuffered

Slide 11

Slide 11 text

Blocking channels unbuffered := make(chan int) // 1) blocks a := <- unbuffered // 2) unbuffered <- 1

Slide 12

Slide 12 text

Blocking channels unbuffered := make(chan int) // 1) blocks a := <- unbuffered // 2) blocks unbuffered <- 1 zZz

Slide 13

Slide 13 text

Blocking channels unbuffered := make(chan int) // 1) blocks a := <- unbuffered // 2) blocks unbuffered <- 1 // 3) go func() { <-unbuffered }() unbuffered <- 1

Slide 14

Slide 14 text

Blocking channels unbuffered := make(chan int) // 1) blocks a := <- unbuffered // 2) blocks unbuffered <- 1 // 3) synchronises go func() { <-unbuffered }() unbuffered <- 1

Slide 15

Slide 15 text

buffered := make(chan int, 1) // 4) a := <- buffered Blocking channels

Slide 16

Slide 16 text

buffered := make(chan int, 1) // 4) still blocks a := <- buffered Blocking channels zZz

Slide 17

Slide 17 text

Blocking channels buffered := make(chan int, 1) // 4) still blocks a := <- buffered // 5) buffered <- 1

Slide 18

Slide 18 text

Blocking channels buffered := make(chan int, 1) // 4) still blocks a := <- buffered // 5) fine buffered <- 1

Slide 19

Slide 19 text

Blocking channels buffered := make(chan int, 1) // 4) still blocks a := <- buffered // 5) fine buffered <- 1 // 6) buffered <- 2

Slide 20

Slide 20 text

Blocking channels buffered := make(chan int, 1) // 4) still blocks a := <- buffered // 5) fine buffered <- 1 // 6) blocks (buffer full) buffered <- 2 zZz

Slide 21

Slide 21 text

Blocking breaks concurrency • Remember? • no deadlocks • more workers = faster execution • Blocking can lead to deadlocks • Blocking can prevent scaling

Slide 22

Slide 22 text

Closing channels • Close sends a special „closed“ message • The receiver will at some point see „closed“. Yay! nothing to do. • If you try to send more: panic! closed closed

Slide 23

Slide 23 text

Closing channels c := make(chan int) close(c) fmt.Println(<-c) // receive and print // What is printed?

Slide 24

Slide 24 text

Closing channels c := make(chan int) close(c) fmt.Println(<-c) // receive and print // What is printed?
 // 0, false

Slide 25

Slide 25 text

Closing channels c := make(chan int) close(c) fmt.Println(<-c) // receive and print // What is printed?
 // 0, false
 // - a receive always returns two values
 // - 0 as it is the zero value of int // - false because „no more data“ or „returned value is not valid“

Slide 26

Slide 26 text

Select • Like a switch statement on channel operations • The order of cases doesn’t matter at all • There is a default case, too • The first non-blocking case is chosen (send and/or receive)

Slide 27

Slide 27 text

Making channels non-blocking func TryReceive(c <-chan int) (data int, more, ok bool) {
 select { case data, more = <-c: return data, more, true
 default: // processed when c is blocking return 0, true, false } }

Slide 28

Slide 28 text

func TryReceiveWithTimeout(c <-chan int, duration time.Duration) (data int, more, ok bool) {
 select { case data, more = <-c: return data, more, true
 case <-time.After(duration): // time.After() returns a channel return 0, true, false } } Making channels non-blocking

Slide 29

Slide 29 text

Shape your data flow • Channels are streams of data • Dealing with multiple streams is the true power of select Fan-out Funnel Turnout

Slide 30

Slide 30 text

Fan-out func Fanout(In <-chan int, OutA, OutB chan int) { 
 for data := range In { // Receive until closed select { // Send to first non-blocking channel case OutA <- data: case OutB <- data: } } }

Slide 31

Slide 31 text

Turnout func Turnout(InA, InB <-chan int, OutA, OutB chan int) {
 // variable declaration left out for readability
 for { select { // Receive from first non-blocking case data, more = <-InA: case data, more = <-InB: } if !more { // ...? return } select { // Send to first non-blocking case OutA <- data: case OutB <- data: } } }

Slide 32

Slide 32 text

Quit channel func Turnout(Quit <-chan int, InA, InB, OutA, OutB chan int) { // variable declaration left out for readability
 for { select { case data = <-InA: case data = <-InB:
 case <-Quit: // remember: close generates a message close(InA) // Actually this is an anti-pattern … close(InB) // … but you can argue that quit acts as a delegate
 Fanout(InA, OutA, OutB) // Flush the remaining data Fanout(InB, OutA, OutB) return }
 
 // ...

Slide 33

Slide 33 text

Where channels fail • You can create deadlocks with channels • Channels pass around copies, which can impact performance • Passing pointers to channels can create race conditions • What about „naturally shared“ structures like caches or registries?

Slide 34

Slide 34 text

Mutexes are not an optimal solution • Mutexes are like toilets.
 The longer you occupy them, the longer the queue gets • Read/write mutexes can only reduce the problem • Using multiple mutexes will cause deadlocks sooner or later • All-in-all not the solution we’re looking for

Slide 35

Slide 35 text

Three shades of code • Blocking = Your program may get locked up (for undefined time) • Lock free = At least one part of your program is always making progress • Wait free = All parts of your program are always making progress

Slide 36

Slide 36 text

Atomic operations • sync.atomic package • Store, Load, Add, Swap and CompareAndSwap • Mapped to thread-safe CPU instructions • These instructions only work on integer types • Only about 10 - 60x slower than their non-atomic counterparts

Slide 37

Slide 37 text

Spinning CAS • You need a state variable and a „free“ constant • Use CAS (CompareAndSwap) in a loop: • If state is not free: try again until it is • If state is free: set it to something else • If you managed to change the state, you „own“ it

Slide 38

Slide 38 text

Spinning CAS type Spinlock struct { state *int32 }
 
 const free = int32(0) func (l *Spinlock) Lock() {
 for !atomic.CompareAndSwapInt32(l.state, free, 42) { // 42 or any other value but 0 runtime.Gosched() // Poke the scheduler } }
 
 func (l *Spinlock) Unlock() { atomic.StoreInt32(l.state, free) // Once atomic, always atomic! }

Slide 39

Slide 39 text

Ticket storage • We need an indexed data structure, a ticket and a done variable • A function draws a new ticket by adding 1 to the ticket • Every ticket number is unique as we never decrement • Treat the ticket as an index to store your data • Increase done to extend the „ready to read“ range

Slide 40

Slide 40 text

Ticket storage type TicketStore struct { ticket *uint64
 done *uint64
 slots []string // for simplicity: imagine this to be infinite }
 func (ts *TicketStore) Put(s string) { t := atomic.AddUint64(ts.ticket, 1) -1 // draw a ticket slots[t] = s // store your data for !atomic.CompareAndSwapUint64(ts.done, t, t+1) { // increase done runtime.Gosched() } } func (ts *TicketStore) GetDone() []string { return ts.slots[:atomic.LoadUint64(ts.done)+1] // read up to done }


Slide 41

Slide 41 text

Ticket storage type TicketStore struct { ticket *uint64
 done *uint64
 slots []string // for simplicity: imagine this to be infinite }
 func (ts *TicketStore) Put(s string) { t := atomic.AddUint64(ts.ticket, 1) -1 // draw a ticket slots[t] = s // store your data for !atomic.CompareAndSwapUint64(ts.done, t, t+1) { // increase done runtime.Gosched() } } func (ts *TicketStore) GetDone() []string { return ts.slots[:atomic.LoadUint64(ts.done)+1] // read up to done }


Slide 42

Slide 42 text

Debugging non-blocking code • I call it „the instruction pointer game“ • The rules: • Pull up two windows (= two go routines) with the same code • You have one instruction pointer that iterates through your code • You may switch windows at any instruction • Watch your variables for race conditions

Slide 43

Slide 43 text

Debugging func (ts *TicketStore) Put(s string) {
 ticket := atomic.AddUint64(ts.next, 1) -1 
 slots[ticket] = s
 
 atomic.AddUint64(ts.done, 1)
 }
 func (ts *TicketStore) Put(s string) {
 ticket := atomic.AddUint64(ts.next, 1) -1 
 slots[ticket] = s
 
 atomic.AddUint64(ts.done, 1)
 }


Slide 44

Slide 44 text

func (ts *TicketStore) Put(s string) {
 ticket := atomic.AddUint64(ts.next, 1) -1 
 slots[ticket] = s
 
 atomic.AddUint64(ts.done, 1)
 }
 func (ts *TicketStore) Put(s string) {
 ticket := atomic.AddUint64(ts.next, 1) -1 
 slots[ticket] = s
 
 atomic.AddUint64(ts.done, 1)
 }
 Debugging ticket: 1

Slide 45

Slide 45 text

func (ts *TicketStore) Put(s string) {
 ticket := atomic.AddUint64(ts.next, 1) -1 
 slots[ticket] = s
 
 atomic.AddUint64(ts.done, 1)
 }
 func (ts *TicketStore) Put(s string) {
 ticket := atomic.AddUint64(ts.next, 1) -1 
 slots[ticket] = s
 
 atomic.AddUint64(ts.done, 1)
 }
 Debugging ticket: 1 ticket: 2

Slide 46

Slide 46 text

func (ts *TicketStore) Put(s string) {
 ticket := atomic.AddUint64(ts.next, 1) -1 
 slots[ticket] = s
 
 atomic.AddUint64(ts.done, 1)
 }
 func (ts *TicketStore) Put(s string) {
 ticket := atomic.AddUint64(ts.next, 1) -1 
 slots[ticket] = s
 
 atomic.AddUint64(ts.done, 1)
 }
 Debugging ticket: 1 ticket: 2

Slide 47

Slide 47 text

func (ts *TicketStore) Put(s string) {
 ticket := atomic.AddUint64(ts.next, 1) -1 
 slots[ticket] = s
 
 atomic.AddUint64(ts.done, 1)
 }
 func (ts *TicketStore) Put(s string) {
 ticket := atomic.AddUint64(ts.next, 1) -1 
 slots[ticket] = s
 
 atomic.AddUint64(ts.done, 1)
 }
 Debugging ticket: 1 ticket: 2 done: 1

Slide 48

Slide 48 text

func (ts *TicketStore) Put(s string) {
 ticket := atomic.AddUint64(ts.next, 1) -1 
 slots[ticket] = s
 
 atomic.AddUint64(ts.done, 1)
 }
 func (ts *TicketStore) Put(s string) {
 ticket := atomic.AddUint64(ts.next, 1) -1 
 slots[ticket] = s
 
 atomic.AddUint64(ts.done, 1)
 }
 Debugging ticket: 1 ticket: 2 done: 1

Slide 49

Slide 49 text

Guidelines for non-blocking code • Don’t switch between atomic and non-atomic functions • Target and exploit situations which enforce uniqueness • Avoid changing two things at a time • Sometimes you can exploit bit operations • Sometimes intelligent ordering can do the trick • Sometimes it’s just not possible at all

Slide 50

Slide 50 text

Concurrency in practice • Avoid blocking, avoid race conditions • Use channels to avoid shared state.
 Use select to manage channels. • Where channels don’t work: • Try to use tools from the sync package first • In simple cases or when really needed: try lockless code

Slide 51

Slide 51 text

Thank you for listening! [email protected]
 @arnecls slides