Sanitizing Threads for Fun & Profit

Transcript

1. Sanitizing
   Threads
   For Fun
   And Profit
   iOS Conf SG
   @gregheo
   

   View Slide

2. Uncertainty
   
   

   View Slide

3. View Slide

4. View Slide

5. View Slide

6. Uncertainty
   
   

   View Slide

7. View Slide

8. What?
   Why?
   How?
   

   View Slide

9. What?
   Why?
   How?
   

   View Slide

10. a = 10
    a = 42
    print(a)
    

    View Slide

11. DispatchQueue.global().async { a = 10 }
    DispatchQueue.main.async { a = 42 }
    print(a)
    

    View Slide

12. a = 10
    a = 42
    print(a)
    

    View Slide

13. a = 10
    a = 42
    print(a)
    

    View Slide

14. Complexity Level
    This Example Your Code
    

    View Slide

15. var a = 88

    

    DispatchQueue.global().async { print(a) }
    DispatchQueue.main.async { print(a) }
    

    View Slide

16. 42
    reads
    reads
    Thread 1
    Thread 2
    Thread 3
    

    View Slide

17. 88
    42 reads
    Thread 1
    Thread 2
    Thread 3
    Thread 4
    writes
    

    View Slide

18. View Slide

19. $ swiftc -sanitize=thread

    

    $ clang -fsanitize=thread
    

    View Slide

20. DispatchQueue.global().async { a = 10 }
    DispatchQueue.main.async { a = 42 }
    print(a)
    

    View Slide

21. WARNING: ThreadSanitizer: data race (pid=12968)
    Write of size 8 at 0x0001015c51b0 by thread T1:
    #0 closure #1 in assign.swift:5 (assign:x86_64+0x1000019cc)
    #1 thunk for @callee_owned () $-> () assign.swift (assign:x86_64+0x100001a4c)
    #2 $__tsan$::invoke_and_release_block(void*) :1063616
    (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x6657b)
    #3 _dispatch_client_callout :1063616 (libdispatch.dylib:x86_64+0x1f63)
    Previous read of size 8 at 0x0001015c51b0 by main thread:
    #0 main assign.swift:9 (assign:x86_64+0x1000017ef)
    Location is global 'a' at 0x0001015c51b0 (assign+0x0001000021b0)
    Thread T1 (tid=4890272, running) is a GCD worker thread
    

    View Slide

22. View Slide

23. thread
    another
    thread
    where?
    over here?
    thread
    race?
    

    View Slide

24. var a = 10
    print(a)
    a = 20
    print(a)
    

    View Slide

25. 0000000 cf fa ed fe 07 00 00 01 03 00 00 80 02 00 00 00
    0000010 19 00 00 00 28 0d 00 00 85 00 20 00 00 00 00 00
    0000020 19 00 00 00 48 00 00 00 5f 5f 50 41 47 45 5a 45
    0000030 52 4f 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0000040 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
    0000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0000060 00 00 00 00 00 00 00 00 19 00 00 00 b8 03 00 00
    0000070 5f 5f 54 45 58 54 00 00 00 00 00 00 00 00 00 00
    0000080 00 00 00 00 01 00 00 00 00 40 00 00 00 00 00 00
    0000090 00 00 00 00 00 00 00 00 00 40 00 00 00 00 00 00
    00000a0 07 00 00 00 05 00 00 00 0b 00 00 00 00 00 00 00
    00000b0 5f 5f 74 65 78 74 00 00 00 00 00 00 00 00 00 00
    00000c0 5f 5f 54 45 58 54 00 00 00 00 00 00 00 00 00 00
    00000d0 40 10 00 00 01 00 00 00 d5 23 00 00 00 00 00 00
    00000e0 40 10 00 00 04 00 00 00 00 00 00 00 00 00 00 00
    00000f0 00 04 00 80 00 00 00 00 00 00 00 00 00 00 00 00
    0000100 5f 5f 73 74 75 62 73 00 00 00 00 00 00 00 00 00
    0000110 5f 5f 54 45 58 54 00 00 00 00 00 00 00 00 00 00
    0000120 16 34 00 00 01 00 00 00 6e 01 00 00 00 00 00 00
    0000130 16 34 00 00 01 00 00 00 00 00 00 00 00 00 00 00
    0000140 08 04 00 80 00 00 00 00 06 00 00 00 00 00 00 00
    0000150 5f 5f 73 74 75 62 5f 68 65 6c 70 65 72 00 00 00
    0000160 5f 5f 54 45 58 54 00 00 00 00 00 00 00 00 00 00
    0000170 84 35 00 00 01 00 00 00 72 02 00 00 00 00 00 00
    0000180 84 35 00 00 02 00 00 00 00 00 00 00 00 00 00 00
    0000190 00 04 00 80 00 00 00 00 00 00 00 00 00 00 00 00
    00001a0 5f 5f 67 63 63 5f 65 78 63 65 70 74 5f 74 61 62
    00001b0 5f 5f 54 45 58 54 00 00 00 00 00 00 00 00 00 00
    00001c0 f8 37 00 00 01 00 00 00 70 00 00 00 00 00 00 00
    00001d0 f8 37 00 00 02 00 00 00 00 00 00 00 00 00 00 00
    00001e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    00001f0 5f 5f 63 6f 6e 73 74 00 00 00 00 00 00 00 00 00
    0000200 5f 5f 54 45 58 54 00 00 00 00 00 00 00 00 00 00
    0000210 68 38 00 00 01 00 00 00 10 00 00 00 00 00 00 00
    0000220 68 38 00 00 03 00 00 00 00 00 00 00 00 00 00 00
    0000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0000240 5f 5f 6f 62 6a 63 5f 63 6c 61 73 73 6e 61 6d 65
    0000250 5f 5f 54 45 58 54 00 00 00 00 00 00 00 00 00 00
    0000260 78 38 00 00 01 00 00 00 48 00 00 00 00 00 00 00
    0000270 78 38 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0000280 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0000290 5f 5f 6f 62 6a 63 5f 6d 65 74 68 6e 61 6d 65 00
    00002a0 5f 5f 54 45 58 54 00 00 00 00 00 00 00 00 00 00
    00002b0 c0 38 00 00 01 00 00 00 28 02 00 00 00 00 00 00
    00002c0 c0 38 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    00002d0 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    00002e0 5f 5f 6f 62 6a 63 5f 6d 65 74 68 74 79 70 65 00
    00002f0 5f 5f 54 45 58 54 00 00 00 00 00 00 00 00 00 00
    0000300 e8 3a 00 00 01 00 00 00 36 00 00 00 00 00 00 00
    0000310 e8 3a 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0000320 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0000330 5f 5f 63 73 74 72 69 6e 67 00 00 00 00 00 00 00
    0000340 5f 5f 54 45 58 54 00 00 00 00 00 00 00 00 00 00
    0000350 1e 3b 00 00 01 00 00 00 61 03 00 00 00 00 00 00
    0000360 1e 3b 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0000370 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0000380 5f 5f 75 6e 77 69 6e 64 5f 69 6e 66 6f 00 00 00
    0000390 5f 5f 54 45 58 54 00 00 00 00 00 00 00 00 00 00
    00003a0 80 3e 00 00 01 00 00 00 04 01 00 00 00 00 00 00
    00003b0 80 3e 00 00 02 00 00 00 00 00 00 00 00 00 00 00
    00003c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    00003d0 5f 5f 65 68 5f 66 72 61 6d 65 00 00 00 00 00 00
    00003e0 5f 5f 54 45 58 54 00 00 00 00 00 00 00 00 00 00
    00003f0 88 3f 00 00 01 00 00 00 70 00 00 00 00 00 00 00
    
    

    View Slide

26. SIL IR Object
    LLVM
    IRGen
    AST
    Sema
    SILGen
    Parse
    

    View Slide

27. $ swiftc -emit-sil simpleprint.swift

    > simpleprint.orig.sil
    $ swiftc -emit-sil -sanitize=thread simpleprint.swift

    > simpleprint.tsan.sil
    

    -rw-r"--r"-- 1 gth staff 61813 simpleprint.orig.sil

    -rw-r"--r"-- 1 gth staff 61813 simpleprint.tsan.sil
    

    View Slide

28. SIL IR Object
    LLVM
    IRGen
    AST Sema
    SILGen
    Parse
    

    View Slide

29. $ swiftc -emit-ir simpleprint.swift

    > simpleprint.orig.ir
    $ swiftc -emit-ir -sanitize=thread simpleprint.swift

    > simpleprint.tsan.ir
    

    -rw-r"--r"-- 1 gth staff 9443 simpleprint.orig.ir

    -rw-r"--r"-- 1 gth staff 19009 simpleprint.tsan.ir
    

    View Slide

30. define i32 @main(i32, i8"**) #0 {
    entry:
    %access-scratch = alloca [24 x i8], align 8
    %access-scratch1 = alloca [24 x i8], align 8
    %access-scratch2 = alloca [24 x i8], align 8
    %2 = bitcast i8"** %1 to i8*
    store i64 10, i64* getelementptr inbounds (%TSi, %TSi*
    @_T011simpleprint1aSiv, i32 0, i32 0), align 8
    %3 = call %swift.type* @_T0ypMa() #5
    %4 = call swiftcc { %swift.bridge*, i8* }
    @_T0s27_allocateUninitializedArraySayxG_BptBwlF(i64 1,
    %swift.type* %3)
    %5 = extractvalue { %swift.bridge*, i8* } %4, 0
    %6 = extractvalue { %swift.bridge*, i8* } %4, 1
    %7 = call %swift.bridge*
    @swift_bridgeObjectRetain(%swift.bridge* %5) #6
    call void @swift_bridgeObjectRelease(%swift.bridge* %5) #6
    
    

    View Slide

31. define i32 @main(i32, i8"**) #0 {
    entry:
    %access-scratch = alloca [24 x i8], align 8
    %access-scratch1 = alloca [24 x i8], align 8
    %access-scratch2 = alloca [24 x i8], align 8
    %2 = bitcast i8"** %1 to i8*
    store i64 10, i64* getelementptr inbounds (%TSi, %TSi*
    @_T011simpleprint1aSiv, i32 0, i32 0), align 8
    %3 = call %swift.type* @_T0ypMa() #5
    %4 = call swiftcc { %swift.bridge*, i8* }
    @_T0s27_allocateUninitializedArraySayxG_BptBwlF(i64 1,
    %swift.type* %3)
    %5 = extractvalue { %swift.bridge*, i8* } %4, 0
    %6 = extractvalue { %swift.bridge*, i8* } %4, 1
    %7 = call %swift.bridge*
    @swift_bridgeObjectRetain(%swift.bridge* %5) #6
    call void @swift_bridgeObjectRelease(%swift.bridge* %5) #6
    

    View Slide

32. define i32 @main(i32, i8"**) #0 {
    entry:
    %access-scratch = alloca [24 x i8], align 8
    %access-scratch1 = alloca [24 x i8], align 8
    %access-scratch2 = alloca [24 x i8], align 8
    %2 = bitcast i8"** %1 to i8*
    store i64 10, i64* getelementptr inbounds (%TSi, %TSi*
    @_T011simpleprint1aSiv, i32 0, i32 0), align 8
    %3 = call %swift.type* @_T0ypMa() #5
    %4 = call swiftcc { %swift.bridge*, i8* }
    @_T0s27_allocateUninitializedArraySayxG_BptBwlF(i64 1,
    %swift.type* %3)
    %5 = extractvalue { %swift.bridge*, i8* } %4, 0
    %6 = extractvalue { %swift.bridge*, i8* } %4, 1
    %7 = call %swift.bridge*
    @swift_bridgeObjectRetain(%swift.bridge* %5) #6
    call void @swift_bridgeObjectRelease(%swift.bridge* %5) #6
    

    View Slide

33. define i32 @main(i32, i8"**) #0 {
    entry:
    %access-scratch = alloca [24 x i8], align 8
    %access-scratch1 = alloca [24 x i8], align 8
    %access-scratch2 = alloca [24 x i8], align 8
    %2 = bitcast i8"** %1 to i8*

    call void @"__tsan_write8(i8* bitcast

    (%TSi* @_T011simpleprint1aSiv to i8*))

    store i64 10, i64* getelementptr inbounds (%TSi, %TSi*
    @_T011simpleprint1aSiv, i32 0, i32 0), align 8
    %3 = call %swift.type* @_T0ypMa() #5
    %4 = call swiftcc { %swift.bridge*, i8* }
    @_T0s27_allocateUninitializedArraySayxG_BptBwlF(i64 1,
    %swift.type* %3)
    %5 = extractvalue { %swift.bridge*, i8* } %4, 0
    %6 = extractvalue { %swift.bridge*, i8* } %4, 1
    TSan addition!
    

    View Slide

34. call void @llvm.lifetime.start(i64 -1, i8* %13)
    call void @swift_beginAccess(i8* bitcast (%TSi*
    @_T011simpleprint1aSiv to i8*), [24 x i8]* %access-scratch, i64
    0, i8* null) #6
    %14 = load i64, i64* getelementptr inbounds (%TSi, %TSi*
    @_T011simpleprint1aSiv, i32 0, i32 0), align 8
    call void @swift_endAccess([24 x i8]* %access-scratch) #6
    %15 = bitcast [24 x i8]* %access-scratch to i8*
    call void @llvm.lifetime.end(i64 -1, i8* %15)
    %._value = getelementptr inbounds %TSi, %TSi* %12, i32 0, i32
    0
    store i64 %14, i64* %._value, align 8
    %16 = call swiftcc { i64, i64, i64 }
    @_T0s5printySayypGd_SS9separatorSS10terminatortFfA0_()
    %17 = extractvalue { i64, i64, i64 } %16, 0
    %18 = extractvalue { i64, i64, i64 } %16, 1
    %19 = extractvalue { i64, i64, i64 } %16, 2
    

    View Slide

35. call void @llvm.lifetime.start(i64 -1, i8* %13)
    call void @swift_beginAccess(i8* bitcast (%TSi*
    @_T011simpleprint1aSiv to i8*), [24 x i8]* %access-scratch, i64
    0, i8* null) #6

    

    call void @"__tsan_read8(i8* bitcast (%TSi*
    @_T011simpleprint1aSiv to i8*))
    %14 = load i64, i64* getelementptr inbounds (%TSi, %TSi*
    @_T011simpleprint1aSiv, i32 0, i32 0), align 8
    call void @swift_endAccess([24 x i8]* %access-scratch) #6
    %15 = bitcast [24 x i8]* %access-scratch to i8*
    call void @llvm.lifetime.end(i64 -1, i8* %15)
    %._value = getelementptr inbounds %TSi, %TSi* %12, i32 0, i32
    0
    store i64 %14, i64* %._value, align 8
    %16 = call swiftcc { i64, i64, i64 }
    @_T0s5printySayypGd_SS9separatorSS10terminatortFfA0_()
    TSan addition!
    

    View Slide

36. void "__tsan_write8(void *addr) {
    MemoryWrite(cur_thread(), CALLERPC, (uptr)addr, kSizeLog8);
    }
    

    View Slide

37. Thread ID Memory Address Size Read or Write?
    1 0x083817a81 8 read
    4 0xdeadbeef 4 read
    7 0x4c898a40 8 write
    2 0x738d4778 1 read
    11 0xc020a1f0 4 write
    1 0x5b430838 2 write
    3 0x80dff8db 8 read
    1 0x2f471de2 8 write
    

    View Slide

38. 1. Memory addresses intersect
    2. Different threads
    3. At least one write
    

    View Slide

39. Uncertainty
    Threads & Race Conditions
    Thread Sanitizer?
    SIL & LLVM IR
    Thread Sanitizer Analysis
    

    View Slide

40. Uncertainty
    ☺
    

    View Slide

41. Lessons

    Learned
    using Thread Sanitizer
    analyzing Thread Sanitizer
    

    View Slide

42. Thread sanitize
    all the things
    (tests too!)
    1
    

    View Slide

43. Serial queues,
    Concurrent queues,

    Target queues,
    Dispatch barriers
    2
    

    View Slide

44. 3Collect the right
    data
    

    View Slide

45. 4Go one more
    level down
    

    View Slide

46. threads
    synchronized!!!
    success
    iOS Conf SG
    @gregheo
    

    View Slide