Simulating a real-world system in Go Sameer Ajmani Go team lead, Google dotGo, Paris November 2017

By Sajibabu79 (Own work) [CC BY-SA 3.0 (https://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons

By Mtattrain (Own work) [CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0)], via Wikimedia Commons

By Eden, Janine and Jim from New York City (Voting Line IX) [CC BY 2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons

https://www.goodfreephotos.com/people/people-in-a-coffee-shop.jpg.php

The simulator Generate customers Collect latency distribution Measure throughput and utilization Order coffee & wait Order coffee & wait Order coffee & wait Order coffee & wait Order coffee & wait Order coffee & wait Customers Wait times

An ideal latte Grind beans 1ms Make espresso 1ms Steam milk 1ms Make latte 1ms Time

The ideal function Grind beans 1ms Make espresso 1ms Steam milk 1ms Make latte 1ms func idealBrew() latte { grounds := grindCoffee(grinder) coffee := makeEspresso(espressoMachine, grounds) milk := steamMilk(steamer) return makeLatte(coffee, milk) } Time

Ideal performance Ideal Ideal Better Better

Reality strikes func grindCoffee(grinder *machine) grounds { grinder.add(runPhase(*grindTime)) return grounds(0) } func makeEspresso(espressoMachine *machine, grounds grounds) coffee { espressoMachine.add(runPhase(*pressTime)) return coffee(grounds) } func steamMilk(steamer *machine) milk { steamer.add(runPhase(*steamTime)) return milk(0) } By Tijuana Brass at en.wikipedia [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY- SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/)], from Wikimedia Commons

Lesson: Use the race detector ================== WARNING: DATA RACE Read at 0x00c42009c328 by goroutine 9: main.(*sampler).add() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/perf.go:53 +0x72 main.grindCoffee() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/main.go:151 +0x99 main.idealBrew() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/main.go:130 +0x4e main.main.func2() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/main.go:596 +0x3d main.perfTest.func2() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/perf.go:182 +0xe4 Previous write at 0x00c42009c328 by goroutine 8: main.(*sampler).add() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/perf.go:57 +0xba main.grindCoffee() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/main.go:151 +0x99 main.idealBrew() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/main.go:130 +0x4e main.main.func2() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/main.go:596 +0x3d main.perfTest.func2() /usr/local/google/home/sameer/gocode/src/github.com/Sajmani/dotgo/coffee/perf.go:182 +0xe4

var kitchen sync.Mutex func lockingBrew() latte { kitchen.Lock() defer kitchen.Unlock() grounds := grindCoffee(grinder) coffee := makeEspresso(espressoMachine, grounds) milk := steamMilk(steamer) return makeLatte(coffee, milk) } By David Adam Kess (Own work) [CC BY-SA 4.0], via Wikimedia Commons

Locking performance Ideal (uncontended) Ideal (uncontended) Whole-kitchen locking Whole-kitchen locking

Fine-grain locking func lockingGrind() grounds { grinder.Lock() defer grinder.Unlock() return grindCoffee(grinder) } func lockingPress(grounds grounds) coffee { espressoMachine.Lock() defer espressoMachine.Unlock() return makeEspresso(espressoMachine, grounds) } func lockingSteam() milk { steamer.Lock() defer steamer.Unlock() return steamMilk(steamer) } By Tijuana Brass at en.wikipedia [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY- SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/)], from Wikimedia Commons

Fine-grain locking performance Fine-grain locking Fine-grain locking Ideal (uncontended) Ideal (uncontended) W hole-kitchen locking Whole-kitchen locking

Lesson: Minimize locked duration Fine-grain locking Fine-grain locking Ideal (uncontended) Ideal (uncontended) W hole-kitchen locking Whole-kitchen locking

Structural limits Grind beans 1ms Make espresso 1ms Steam milk 1ms Make latte 1ms Grind beans 1ms Make espresso 1ms Steam milk 1ms Make latte 1ms CPU1 Time Grind beans 1ms Make espresso 1ms Steam milk 1ms Make latte 1ms Grind beans 1ms Make espresso 1ms Steam milk 1ms CPU2 CPU3 CPU4 CPU5 CPU6 Grind beans 1ms Make espresso 1ms Grind beans 1ms

By dumbonyc (Brooklyn Ice Cream Factory) [CC BY-SA 2.0 (https://creativecommons.org/licenses/by-sa/2.0)], via Wikimedia Commons

Two sets of machines func multiGrind() grounds { m := <-grinders grounds := grindCoffee(m) grinders <- m return grounds } func multiPress(grounds grounds) coffee { m := <-espressoMachines coffee := makeEspresso(m, grounds) espressoMachines <- m return coffee } func multiSteam() milk { m := <-steamers milk := steamMilk(m) steamers <- m return milk } By Tijuana Brass at en.wikipedia [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY- SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/)], from Wikimedia Commons

Multi machine performance Ideal & Multiple machines Fine-grain locking Ideal & Multiple machines Fine-grain locking W hole-kitchen locking Whole-kitchen locking

Lesson: Identify the limiting factor

\ By HAO XING (BIZ INDIA-STARBUCKS-BIZPLUS 2 SE) [CC BY 2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons

A coffee pipeline func (p *linearPipeline) grinder() { for o := range p.orders { o.grounds = grindCoffee(p.grinderMachine) p.ordersWithGrounds <- o } close(p.ordersWithGrounds) } func (p *linearPipeline) presser() { for o := range p.ordersWithGrounds { o.coffee = makeEspresso( p.espressoMachine, o.grounds) p.ordersWithCoffee <- o } close(p.ordersWithCoffee) } func (p *linearPipeline) steamer() { for o := range p.ordersWithCoffee { o.milk <- steamMilk(p.steamerMachine) } close(p.done) } By Tijuana Brass at en.wikipedia [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY- SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/)], from Wikimedia Commons

Pipeline vs. locking Unbuffered pipeline Fine-grain locking & Unbuffered pipeline Fine-grain locking Ideal (uncontended) Ideal (uncontended) W hole-kitchen locking Whole-kitchen locking

Buffered vs. unbuffered Buffered pipelines Buffered pipelines Unbuffered pipeline Unbuffered pipeline Ideal (uncontended) Ideal (uncontended) W hole-kitchen locking Whole-kitchen locking

Major vs. minor optimizations

Many more scenarios

Remove structural barriers to parallelism Study real-world systems for inspiration

Get the simulator More simulation modes! More controls! More data! • Americano (no “steam” stage) & Espresso (no “steam” or “make” stage) • Steam milk in parallel with grinding beans & making espresso • Controls: stage durations, jitter, bounded request queues, arrival rate • Data: latency distributions, CPU utilization, queue drop rate go get github.com/Sajmani/dotgo/coffee