Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Golang race detection - Neven Miculinic, KrakenSystems

GoDays
January 30, 2019

Golang race detection - Neven Miculinic, KrakenSystems

Golang race detection - Neven Miculinic, KrakenSystems

GoDays

January 30, 2019
Tweet

More Decks by GoDays

Other Decks in Technology

Transcript

  1. Counter example var Cnt int func Run(amount int) { for

    i := 0; i < amount; i++ { Cnt++ } }
  2. Counter example func main() { wg := &sync.WaitGroup{} for i

    := 0; i < 10; i++ { wg.Add(1) go func() { Run(1000); wg.Done() }() } wg.Wait(); fmt.Println(Cnt) }
  3. Therac-25 • Radiotherapy machine • Race condition between user input

    and system state • Consequences: 6+ accidents, 2+ deaths
  4. Happens before G1: 0 G2: 0 G1: 1 G1: 2

    G2: 3 G2: 1 G2: 2 G1: 3 G2: 4 CHAN RECV CHAN SEND
  5. Happens before G1: 0 G2: 0 G1: 1 G1: 2

    G2: 3 G2: 1 G2: 2 G1: 3 G2: 4 MUTEX LOCK MUTEX UNLOCK
  6. Race on loop counter func main() { var wg sync.WaitGroup

    wg.Add(5) for i := 0; i < 5; i++ { go func() { fmt.Println(i) // Not the 'i' you are looking for. wg.Done() }() } wg.Wait() }
  7. Race on loop counter func main() { var wg sync.WaitGroup

    wg.Add(5) for i := 0; i < 5; i++ { go func(j int) { fmt.Println(j) wg.Done() }(i) } wg.Wait() }
  8. Unprotected global var Cnt int func Inc(amount int) { Cnt

    += amount } func main() { go Inc(10) go Int(100) }
  9. Unprotected global var Cnt int var m = &sync.Mutex{} func

    Inc(amount int) { m.Lock() defer m.Unlock() Cnt += amount }
  10. Violating go memory model (aka being smart) var a string

    var done bool func setup() { a = "hello, world" done = true } func main() { go setup() for !done { } fmt.Print(a) }
  11. ThreadSanitizer v2 • Battle proven library • Runtime slowdown 2x-20x.

    • Memory overhead 5x-10x. • Requires CGO, 64bit OS • No false positives
  12. func Inc(x *int) { *x++ } Compiler instrumentation movq "".x+32(SP),

    AX movq AX, (SP) call runtime.raceread(SB) movq "".x+32(SP), AX movq (AX), CX movq CX, ""..autotmp_4+8(SP) movq AX, (SP) call runtime.racewrite(SB) incq AX movq "".x+32(SP), CX movq AX, (CX)
  13. func Inc(x *int) { *x++ } Compiler instrumentation ... call

    runtime.raceread(SB) ... call runtime.racewrite(SB) ...
  14. Vector clocks G1: 0 G2: 0 G1: 0 G2: 0

    G1: 1 G2: 0 G1: 2 G2: 0 G1: 0 G2: 3 G1: 0 G2: 1 G1: 0 G2: 2
  15. Vector clocks G1: 0 G2: 0 G1: 0 G2: 0

    G1: 1 G2: 0 G1: 2 G2: 0 G1: 0 G2: 3 G1: 0 G2: 1 G1: 0 G2: 2 G1: 3 G2: 3 G1: 0 G2: 4 MUTEX LOCK MUTEX UNLOCK
  16. Vector clocks • Partially ordered set • (4, 11) <

    (10, 12) → happens before • (12, 5) is concurrent to (7, 12)
  17. DJIT+ We remember vector clocks for: • Each lock release

    L m = (t 0 , ... , t n ) • Last memory read, R x = (t 0 , …, t n ) • Last memory write W x = (t 0 , …, t n ) • Goroutine vector clock, C x = (t 0 , …, t n )
  18. DJIT+ 4 0 4 0 5 0 5 0 5

    0 0 8 0 8 0 8 4 8 4 8 0 0 0 0 4 0 4 0 4 0 0 0 4 0 4 0 4 0 4 8 C 0 C 1 L m W x w(x) w(x) unlock (m) lock (m)
  19. DJIT+ 4 0 4 0 4 0 4 0 4

    0 0 8 0 8 0 8 0 8 0 8 0 0 0 0 0 0 0 0 0 0 0 0 4 0 4 0 4 0 RACE C 0 C 1 L m W x w(x) w(x)
  20. FastTrack 4 0 4 0 5 0 5 0 5

    0 0 8 0 8 0 8 4 8 4 8 0 0 0 0 4 0 4 0 4 0 0@0 C 0 C 1 L m W x w(x) w(x) unlock (m) lock (m) 4@0 4@0 4@0 8@1
  21. FastTrack 4 0 4 0 4 0 4 0 4

    0 0 8 0 8 0 8 4 8 4 8 0 0 0 0 0 0 0 0 0 0 0@0 C 0 C 1 L m R x r(x) r(x) 4@0 4@0 4@0 4 8
  22. Shadow word pool Write goroutine 1 10@1 0:2 W 10

    0 0 Memory mapped Fixed size pool 64 bits
  23. Shadow word pool Read goroutine 3 10@1 0:2 W 20@2

    4:8 R 40@3 0:4 R RACE found 5 0 40
  24. Summary var Cnt int64 func Run(amount int) { for i

    := 0; i < amount; i++ { val := atomic.LoadInt64(&Cnt) val ++ atomic.StoreInt64(&Cnt, val) } }
  25. References • https://github.com/google/sanitizers/wiki/ThreadSanitizerAlgorithm • https://golang.org/doc/articles/race_detector.html • https://golang.org/ref/mem • ""go test

    -race" Under the Hood" by Kavya Joshi • https://blog.golang.org/race-detector • https://danluu.com/concurrency-bugs/ • ThreadSanitizer – data race detection in practice google paper • FastTrack: Efficient and Precise Dynamic Race Detection • AddressSanitizer/ThreadSanitizer for Linux Kernel and userspace. • [DJIT+ algo] MultiRace: efficient on-the-fly data race detection in multithreaded C++ programs • On the Complexity of Event Ordering for Shared-Memory Parallel Program Executions (Netzer, Miller, 1990)