Level up your CI for iOS and macOS

Developers need a fast, reliable CI to be productive. An excellent CI helps you quickly find issues in your code and confidently ship new code, reducing stress and helping you sleep better at night. But setting it up properly takes a lot of time. This talk isn’t about showing you how to set up a CI from scratch. Instead, you’ll get advanced tips and tricks for creating and maintaining an awesome CI for your iOS or macOS app. It’s time to level up your CI.

First, we’ll look at tips and tricks for unit, integration, and UI tests. When tests fail in CI, it’s vital to be able to tell at a glance why they failed — flaky tests can drive developers crazy! We’ll show you a few ways to investigate and fix flaky tests, and even flaky CI.

Linters and code analyzers are often overlooked, although they can help you identify bugs early. Along with code formatting tools and Xcode Thread Sanitizer, they bring your CI to the next level.

Next, we’ll cover how to properly validate your libraries before publishing to CocoaPods, Swift Package Manager, or Carthage. Say goodbye to developers pinging you that they can’t build the latest version of your SDK.

Some code is very hard to test and risky to change without introducing new bugs. You’re going to see a tactic that ensures nobody changes code like that by accident.

Transcript

1. Level up your CI for
   iOS and macOS
   Philipp
   Hofmann
   

   View Slide

2. View Slide

3. Photo by Sean Benesh on Unsplash
   

   View Slide

4. Photo by Patrick on Unsplash
   

   View Slide

5. Tips and Tricks
   1. Better Test Logs
   2. Flaky Tests
   3. Flaky CI
   4. SwiftLint & Clang-Format
   5. Xcode Thread Sanitizer
   6. Xcode Analyze
   7. Validate your Library
   8. Borrowing Tests
   9. High Risk Files
   

   View Slide

6. Level Up Your CI.
   Photo by Denys Sudilkovsky on Unsplash
   

   View Slide

7. Photo by Wesley Tingey on Unsplash
   1. Better Test Logs
   

   View Slide

8. Command line invocation:
   
   
   /Applications/Xcode_12.5.1.app/Contents/Developer/usr/bin/xcodebuild -workspace Sentry.xcworkspace -scheme Sentry
   -configuration Test GCC_GENERATE_TEST_COVERAGE_FILES=YES GCC_INSTRUMENT_PROGRAM_FLOW_ARCS=YES -destination "platform=iOS
   Simulator,OS=latest,name=iPhone 8" test
   
   
   User defaults from command line:
   
   
   IDEPackageSupportUseBuiltinSCM = YES
   
   
   Build settings from command line:
   
   
   GCC_GENERATE_TEST_COVERAGE_FILES = YES
   
   
   GCC_INSTRUMENT_PROGRAM_FLOW_ARCS = YES
   
   
   note: Using new build system
   
   
   note: Building targets in parallel
   
   
   note: Planning build
   
   
   note: Analyzing workspace
   
   
   note: Constructing build description
   
   
   note: Build preparation complete
   
   
   CreateBuildDirectory /Users/runner/Library/Developer/Xcode/DerivedData/Sentry-azcqyxgbmmrdtdegukmbgarszteu/Build/
   Intermediates.noindex
   
   
   cd /Users/runner/work/sentry-cocoa/sentry-cocoa
   
   
   builtin-create-build-directory /Users/runner/Library/Developer/Xcode/DerivedData/Sentry-azcqyxgbmmrdtdegukmbgarszteu/Build/
   Intermediates.noindex
   
   
   CreateBuildDirectory /Users/runner/Library/Developer/Xcode/DerivedData/Sentry-azcqyxgbmmrdtdegukmbgarszteu/Build/Products
   
   
   cd /Users/runner/work/sentry-cocoa/sentry-cocoa
   
   
   builtin-create-build-directory /Users/runner/Library/Developer/Xcode/DerivedData/Sentry-azcqyxgbmmrdtdegukmbgarszteu/Build/
   Products
   
   
   CreateBuildDirectory /Users/runner/Library/Developer/Xcode/DerivedData/Sentry-azcqyxgbmmrdtdegukmbgarszteu/Build/Products/Test-
   iphonesimulator
   
   
   cd /Users/runner/work/sentry-cocoa/sentry-cocoa
   
   
   builtin-create-build-directory /Users/runner/Library/Developer/Xcode/DerivedData/Sentry-azcqyxgbmmrdtdegukmbgarszteu/Build/
   Products/Test-iphonesimulator
   

   View Slide

9. Test Suite 'SentryCrashFileUtils_Tests' started at 2022-10-25 23:28:08.671
   
   
   Test Case '-[SentryCrashFileUtils_Tests testLastPathEntry]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testLastPathEntry]' passed (0.002 seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBuffered_EmptyFile]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBuffered_EmptyFile]' passed (0.005 seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBuffered_FileIsBigger]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBuffered_FileIsBigger]' passed (0.004 seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBuffered_ReadBufferIsMuchSmaller]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBuffered_ReadBufferIsMuchSmaller]' passed (0.004 seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBuffered_ReadBufferIsSmaller]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBuffered_ReadBufferIsSmaller]' passed (0.003 seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBuffered_SameSize]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBuffered_SameSize]' passed (0.006 seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_Beginning]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_Beginning]' passed (0.005 seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_End]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_End]' passed (0.003 seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_Halfway]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_Halfway]' passed (0.004 seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_NotFound]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_NotFound]' passed (0.004 seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_NotFound_LargeFile]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_NotFound_LargeFile]' passed (0.003
   seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_SmallDstBuffer]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_SmallDstBuffer]' passed (0.003 seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_SmallReadBuffer]' started.
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBufferedUntilChar_SmallReadBuffer]' passed (0.026 seconds).
   
   
   Test Case '-[SentryCrashFileUtils_Tests testReadBytesFromFD]' started.
   

   View Slide

10. Test Case '-[SentryTests.SentryCrashIntegrationTests testEndSessionAsCrashed_NoCrashLastLaunch]' started.
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests testEndSessionAsCrashed_NoCrashLastLaunch]' passed
    (0.011 seconds).
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests testEndSessionAsCrashed_NoCurrentSession]' started.
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests testEndSessionAsCrashed_NoCurrentSession]' passed
    (0.004 seconds).
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests testEndSessionAsCrashed_WhenOOM_WithCurrentSession]'
    started.
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests testEndSessionAsCrashed_WhenOOM_WithCurrentSession]'
    passed (0.023 seconds).
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests testEndSessionAsCrashed_WithCurrentSession]' started.
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests testEndSessionAsCrashed_WithCurrentSession]' passed
    (0.008 seconds).
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests
    testInstall_WhenStitchAsyncCallsDisabled_DoesNotCallInstallAsyncHooks]' started.
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests
    testInstall_WhenStitchAsyncCallsDisabled_DoesNotCallInstallAsyncHooks]' passed (0.003 seconds).
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests
    testInstall_WhenStitchAsyncCallsEnabled_CallsInstallAsyncHooks]' started.
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests
    testInstall_WhenStitchAsyncCallsEnabled_CallsInstallAsyncHooks]' passed (0.003 seconds).
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests testLocaleChanged_DifferentLocale_SetsCurrentLocale]'
    started.
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests testLocaleChanged_DifferentLocale_SetsCurrentLocale]'
    passed (0.007 seconds).
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests testLocaleChanged_NoDeviceContext_SetsCurrentLocale]'
    started.
    
    
    Test Case '-[SentryTests.SentryCrashIntegrationTests testLocaleChanged_NoDeviceContext_SetsCurrentLocale]'
    passed (0.003 seconds).
    

    View Slide

11. Test Suite 'Selected tests' failed at 2022-10-27 11:29:58.069.
    
    
    Executed 1564 tests, with 1 test skipped and 3 failures (0 unexpected) in 46.706
    (47.486) seconds
    
    
    2022-10-27 11:30:07.091 xcodebuild[48576:4363253] [MT]
    IDETestOperationsObserverDebug: 57.629 elapsed -- Testing started completed.
    
    
    2022-10-27 11:30:07.091 xcodebuild[48576:4363253] [MT]
    IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
    
    
    2022-10-27 11:30:07.091 xcodebuild[48576:4363253] [MT]
    IDETestOperationsObserverDebug: 57.629 sec, +57.629 sec -- end
    
    
    Test session results, code coverage, and logs:
    
    
    /Users/philipphofmann/Library/Developer/Xcode/DerivedData/Sentry-
    gcimrafeikdpcwaanncxmwrieqhi/Logs/Test/Test-Sentry-2022.10.27_11-29-08-+0200.xcresult
    
    
    Failing tests:
    
    
    SentryClientTest.testCaptureCrash_Culture()
    
    
    -[SentryDeviceTests testDeviceModel]
    
    
    SentryNetworkTrackerIntegrationTests.testGetRequest_CompareSentryTraceHeader()
    
    
    ** TEST FAILED **
    
    
    

    View Slide

12. xcbeautify
    

    View Slide

13. xcodebuild [flags]
    

    View Slide

14. xcodebuild [flags] | xcbeautify
    

    View Slide

15. SentryTests.SentryNetworkTrackerIntegrationTests
    
    
    testGetRequest_CompareSentryTraceHeader, XCTAssertEqual failed:
    ("Optional("4f6049d2753e484f81ee859aa8c88c7e-bbc6d1f19e76479e-1")") is not equal to
    ("Optional("")")
    
    
    /Users/philipphofmann/git-repos/sentry-cocoa/Tests/SentryTests/Integrations/Performance/Network/
    SentryNetworkTrackerIntegrationTests.swift:201
    
    
    ```
    
    
    let expectedTraceHeader = networkSpan.toTraceHeader().value()
    
    
    XCTAssertEqual(expectedTraceHeader, response)
    
    
    }
    
    
    ```
    
    
    Executed 1567 tests, with 1 test skipped and 3 failures (0 unexpected) in 63.980 (64.542) seconds
    
    
    2022-10-28 09:03:34.884 xcodebuild[5492:40161] [MT] IDETestOperationsObserverDebug: 70.839 elapsed
    -- Testing started completed.
    
    
    2022-10-28 09:03:34.884 xcodebuild[5492:40161] [MT] IDETestOperationsObserverDebug: 0.000 sec,
    +0.000 sec -- start
    
    
    2022-10-28 09:03:34.884 xcodebuild[5492:40161] [MT] IDETestOperationsObserverDebug: 70.839 sec,
    +70.839 sec -- end
    
    
    Failing tests:
    
    
    SentryClientTest.testCaptureCrash_Culture()
    
    
    -[SentryDeviceTests testDeviceModel]
    
    
    SentryNetworkTrackerIntegrationTests.testGetRequest_CompareSentryTraceHeader()
    
    
    ** TEST FAILED **
    
    
    

    View Slide

16. Fastlane uses xcbeautify.
    

    View Slide

17. Raw test output log
    

    View Slide

18. xcodebuild [flags] | xcbeautify
    

    View Slide

19. xcodebuild [flags] |
    
    
    tee raw-test-output.log |
    
    
    xcbeautify
    

    View Slide

20. View Slide

21. Derived Data Logs
    

    View Slide

22. /Xcode/DerivedData/YourProject/Logs/**
    

    View Slide

23. View Slide

24. YourTests.xcresult
    

    View Slide

25. View Slide

26. View Slide

27. 2. Flaky Tests
    Photo by Jaakko Kemppainen on Unsplash
    

    View Slide

28. Reproduce them locally.
    

    View Slide

29. Run tests repeatedly.
    

    View Slide

30. View Slide

31. View Slide

32. View Slide

33. View Slide

34. XCTestObservationCenter
    

    View Slide

35. XCTestObservationCenter.shared
    
    
    .addTestObserver(TestObserver())
    

    View Slide

36. class TestObserver: XCTestObservation {
    
    
    
    
    }
    

    View Slide

37. class TestObserver : XCTestObservation {
    
    
    
    
    func testCase(_ testCase: XCTestCase,
    
    
    didRecord issue: XCTIssue) {
    
    
    }
    
    
    
    
    }
    

    View Slide

38. class TestObserver : XCTestObservation {
    
    
    
    
    func testCase(_ testCase: XCTestCase,
    
    
    didRecord issue: XCTIssue) {
    
    
    let exception =
    
    
    NSException(name: testCase.name,
    
    
    reason: issue.description)
    
    
    }
    
    
    
    
    }
    

    View Slide

39. class TestObserver : XCTestObservation {
    
    
    
    
    func testCase(_ testCase: XCTestCase,
    
    
    didRecord issue: XCTIssue) {
    
    
    let exception =
    
    
    NSException(name: testCase.name,
    
    
    reason: issue.description)
    
    
    SentrySDK.capture(exception: exception)
    
    
    }
    
    
    
    
    }
    

    View Slide

40. class TestObserver : XCTestObservation {
    
    
    
    
    func testCaseWillStart(_ testCase: XCTestCase) {
    
    
    }
    
    
    
    
    }
    

    View Slide

41. class TestObserver : XCTestObservation {
    
    
    
    
    func testCaseWillStart(_ testCase: XCTestCase) {
    
    
    let crumb = Breadcrumb(level: .debug,
    
    
    category: "test.started")
    
    
    }
    
    
    
    
    }
    

    View Slide

42. class TestObserver : XCTestObservation {
    
    
    
    
    func testCaseWillStart(_ testCase: XCTestCase) {
    
    
    let crumb = Breadcrumb(level: .debug,
    
    
    category: "test.started")
    
    
    crumb.message = testCase.name
    
    
    }
    
    
    
    
    }
    

    View Slide

43. class TestObserver : XCTestObservation {
    
    
    
    
    func testCaseWillStart(_ testCase: XCTestCase) {
    
    
    let crumb = Breadcrumb(level: .debug,
    
    
    category: "test.started")
    
    
    crumb.message = testCase.name
    
    
    SentrySDK.addBreadcrumb(crumb: crumb)
    
    
    }
    
    
    
    
    }
    

    View Slide

44. View Slide

45. View Slide

46. Use CI for validation.
    

    View Slide

47. matrix:
    
    
    ios-version: ["16.1", "15.7", "14.0"]
    
    
    device: ["iPhone", "iPad"]
    

    View Slide

48. iPhone 16.1
    
    
    iPhone 15.7
    
    
    iPhone 14.0
    matrix:
    
    
    ios-version: ["16.1", "15.7", "14.0"]
    
    
    device: ["iPhone", "iPad"]
    iPad 16.1
    
    
    iPad 15.7
    
    
    iPad 14.0
    

    View Slide

49. matrix:
    
    
    a: [1,2,3,4,5]
    
    
    b: [1,2,3,4,5]
    

    View Slide

50. 11
    
    
    12
    
    
    13
    matrix:
    
    
    a: [1,2,3,4,5]
    
    
    b: [1,2,3,4,5]
    …
    
    
    54
    
    
    55
    

    View Slide

51. 3. Flaky CI
    Photo by Pawel Czerwinski on Unsplash
    

    View Slide

52. Photo by Nathan Anderson on Unsplash
    

    View Slide

53. for i in {1..5}; do flaky-command && break ; done
    

    View Slide

54. 4. SwiftLint & Clang-Format
    Photo by Cesar Carlevarino Aragon on Unsplash
    

    View Slide

55. 5. Xcode Thread Sanitizer
    Photo by Nick Fewings on Unsplash
    

    View Slide

56. class Counter {
    
    
    
    
    }
    

    View Slide

57. class Counter {
    
    
    func increment() {
    
    
    }
    
    
    
    
    var count: Int {
    
    
    }
    
    
    }
    

    View Slide

58. class Counter {
    
    
    
    
    private var internalCount = 0
    
    
    func increment() {
    
    
    internalCount += 1
    
    
    }
    
    
    
    
    var count: Int {
    
    
    return internalCount
    
    
    }
    
    
    }
    

    View Slide

59. class CounterTests: XCTestCase {
    
    
    
    
    func testIncrementConcurrent() {
    
    
    
    
    }
    
    
    }
    

    View Slide

60. class CounterTests: XCTestCase {
    
    
    
    
    func testIncrementConcurrent() {
    
    
    let expectedCounts = 100
    
    
    let counter = Counter()
    
    
    }
    
    
    }
    

    View Slide

61. class CounterTests: XCTestCase {
    
    
    
    
    func testIncrementConcurrent() {
    
    
    let expectedCounts = 100
    
    
    let counter = Counter()
    
    
    
    
    let queue = DispatchQueue(label: "CounterTest")
    
    
    }
    
    
    }
    

    View Slide

62. class CounterTests: XCTestCase {
    
    
    
    
    func testIncrementConcurrent() {
    
    
    let expectedCounts = 100
    
    
    let counter = Counter()
    
    
    
    
    let queue = DispatchQueue(label: "CounterTest")
    
    
    
    
    for _ in 0..
    
    }
    
    
    }
    
    
    }
    

    View Slide

63. class CounterTests: XCTestCase {
    
    
    
    
    func testIncrementConcurrent() {
    
    
    let expectedCounts = 100
    
    
    let counter = Counter()
    
    
    
    
    let queue = DispatchQueue(label: "CounterTest")
    
    
    
    
    for _ in 0..
    
    queue.async {
    
    
    counter.increment()
    
    
    }
    
    
    }
    
    
    }
    
    
    }
    

    View Slide

64. class CounterTests: XCTestCase {
    
    
    
    
    func testIncrementConcurrent() {
    
    
    let expectedCounts = 100
    
    
    let counter = Counter()
    
    
    
    
    let queue = DispatchQueue(label: "CounterTest")
    
    
    
    
    for _ in 0..
    
    queue.async {
    
    
    counter.increment()
    
    
    }
    
    
    }
    
    
    XCTAssertEqual(expectedCounts, counter.count)
    
    
    }
    
    
    }
    

    View Slide

65. View Slide

66. View Slide

67. View Slide

68. #!/bin/bash
    
    
    set -euo pipefail
    
    
    xcodebuild [flags] test
    
    
    

    View Slide

69. #!/bin/bash
    
    
    set -euo pipefail
    
    
    xcodebuild [flags] -enableThreadSanitizer YES test
    
    
    

    View Slide

70. ==================
    
    
    WARNING: ThreadSanitizer: data race (pid=61245)
    
    
    Write of size 8 at 0x7b080029af50 by thread T8:
    
    
    #0 Counter.increment() Counter.swift:8 (MobileDevSummit:x86_64+0x100007052)
    
    
    #1 closure #1 in CounterTests.testIncrementConcurrent() CounterTests.swift:14 (MobileDevSummitTests:x86_64+0x1e7c)
    
    
    #2 partial apply for closure #1 in CounterTests.testIncrementConcurrent() (MobileDevSummitTests:x86_64+0x332d)
    
    
    #3 thunk for @escaping @callee_guaranteed () -> () (MobileDevSummitTests:x86_64+0x1ef2)
    
    
    #4 __tsan::invoke_and_release_block(void*) :2 (libclang_rt.tsan_iossim_dynamic.dylib:x86_64+0x7f3eb)
    
    
    #5 _dispatch_client_callout :2 (libdispatch.dylib:x86_64+0x2a39)
    
    
    Previous write of size 8 at 0x7b080029af50 by thread T9:
    
    
    #0 Counter.increment() Counter.swift:8 (MobileDevSummit:x86_64+0x100007052)
    
    
    #1 closure #1 in CounterTests.testIncrementConcurrent() CounterTests.swift:14 (MobileDevSummitTests:x86_64+0x1e7c)
    
    
    #2 partial apply for closure #1 in CounterTests.testIncrementConcurrent() (MobileDevSummitTests:x86_64+0x332d)
    
    
    #3 thunk for @escaping @callee_guaranteed () -> () (MobileDevSummitTests:x86_64+0x1ef2)
    
    
    #4 __tsan::invoke_and_release_block(void*) :2 (libclang_rt.tsan_iossim_dynamic.dylib:x86_64+0x7f3eb)
    
    
    #5 _dispatch_client_callout :2 (libdispatch.dylib:x86_64+0x2a39)
    
    
    Location is heap block of size 24 at 0x7b080029af40 allocated by main thread:
    
    
    #0 __sanitizer_mz_malloc :2 (libclang_rt.tsan_iossim_dynamic.dylib:x86_64+0x5583c)
    
    
    #1 _malloc_zone_malloc_instrumented_or_legacy :2 (libsystem_malloc.dylib:x86_64+0x1742f)
    
    
    #2 CounterTests.testIncrementConcurrent() CounterTests.swift:8 (MobileDevSummitTests:x86_64+0x18fa)
    
    
    #3 @objc CounterTests.testIncrementConcurrent() (MobileDevSummitTests:x86_64+0x2151)
    
    
    #4 __invoking___ :2 (CoreFoundation:x86_64+0x12c31b)
    
    
    #5 main MobileDevSummitApp.swift (MobileDevSummit:x86_64+0x100006db5)
    
    
    Thread T8 (tid=2045181, running) is a GCD worker thread
    
    
    Thread T9 (tid=2045180, running) is a GCD worker thread
    
    
    SUMMARY: ThreadSanitizer: data race Counter.swift:8 in Counter.increment()
    
    
    ==================
    
    
    Test session results, code coverage, and logs:
    
    
    /Users/philipphofmann/Library/Developer/Xcode/DerivedData/MobileDevSummit-cqfkhtwtenpewocwjipnmnsajuwd/Logs/Test/Test-
    MobileDevSummit-2022.11.03_15-35-43-+0100.xcresult
    
    
    ** TEST SUCCEEDED **
    
    
    

    View Slide

71. WARNING: ThreadSanitizer: data race (pid=61245)
    
    
    SUMMARY: ThreadSanitizer: data race Counter.swift:8 in Counter.increment()
    
    
    ** TEST SUCCEEDED **
    

    View Slide

72. #!/bin/bash
    
    
    set -euo pipefail
    
    
    xcodebuild [flags] -enableThreadSanitizer YES test
    
    
    

    View Slide

73. #!/bin/bash
    
    
    set -euo pipefail
    
    
    xcodebuild [flags] -enableThreadSanitizer YES test | \
    
    
    tee thread-sanitizer.log
    
    
    

    View Slide

74. #!/bin/bash
    
    
    set -euo pipefail
    
    
    xcodebuild [flags] -enableThreadSanitizer YES test | \
    
    
    tee thread-sanitizer.log
    
    
    if grep -Fq "WARNING: ThreadSanitizer:" thread-sanitizer.log ; then
    
    
    else
    
    
    fi
    
    
    

    View Slide

75. #!/bin/bash
    
    
    set -euo pipefail
    
    
    xcodebuild [flags] -enableThreadSanitizer YES test | \
    
    
    tee thread-sanitizer.log
    
    
    if grep -Fq "WARNING: ThreadSanitizer:" thread-sanitizer.log ; then
    
    
    message="ThreadSanitizer found problems.
    
    
    Search for \"ThreadSanitizer\" in logs for more details."
    
    
    echo "$message"
    
    
    exit 1
    
    
    else
    
    
    fi
    
    
    

    View Slide

76. #!/bin/bash
    
    
    set -euo pipefail
    
    
    xcodebuild [flags] -enableThreadSanitizer YES test | \
    
    
    tee thread-sanitizer.log
    
    
    if grep -Fq "WARNING: ThreadSanitizer:" thread-sanitizer.log ; then
    
    
    message="ThreadSanitizer found problems.
    
    
    Search for \"ThreadSanitizer\" in logs for more details."
    
    
    echo "$message"
    
    
    exit 1
    
    
    else
    
    
    echo "ThreadSanitizer didn't find problems."
    
    
    exit 0
    
    
    fi
    
    
    

    View Slide

77. Adds a 2x to 20x slowdown of your code.
    

    View Slide

78. 6. Xcode Analyze
    Photo by Markus Spiske on Unsplash
    

    View Slide

79. Works for Objective-C, C, and C++.
    

    View Slide

80. @implementation YourClass
    
    
    - (instancetype)initWithNSString:(NSString *)string
    
    
    {
    
    
    if ([super init]) {
    
    
    _string = string;
    
    
    }
    
    
    return self;
    
    
    }
    
    
    @end
    

    View Slide

81. View Slide

82. xcodebuild analyze [flags]
    

    View Slide

83. xcodebuild analyze [flags] | xcbeautify
    

    View Slide

84. ▸ Analyzing YourClass.m
    
    
    ⚠ /Users/philipphofmann/git-repos/sample/Sources/YourClass.m:11:5:
    Returning 'self' while it is not set to the result of '[(super or
    self) init...]' [osx.cocoa.SelfInit]
    
    
    return self;
    
    
    ^~~~~~~~~~~
    
    
    ▸ Analyze Succeeded
    

    View Slide

85. ▸ Analyzing YourClass.m
    
    
    ⚠ /Users/philipphofmann/git-repos/sample/Sources/YourClass.m:11:5:
    Returning 'self' while it is not set to the result of '[(super or
    self) init...]' [osx.cocoa.SelfInit]
    
    
    return self;
    
    
    ^~~~~~~~~~~
    
    
    ▸ Analyze Succeeded
    

    View Slide

86. xcodebuild analyze [flags] | xcbeautify
    

    View Slide

87. xcodebuild analyze [flags]
    
    
    CLANG_ANALYZER_OUTPUT=html |
    
    
    xcbeautify
    

    View Slide

88. xcodebuild analyze [flags]
    
    
    CLANG_ANALYZER_OUTPUT=html
    
    
    CLANG_ANALYZER_OUTPUT_DIR=analyzer |
    
    
    xcpretty
    

    View Slide

89. xcodebuild analyze [flags]
    
    
    CLANG_ANALYZER_OUTPUT=html
    
    
    CLANG_ANALYZER_OUTPUT_DIR=analyzer |
    
    
    xcpretty && [[ -z `find analyzer ` ]]
    

    View Slide

90. xcodebuild analyze [flags]
    
    
    CLANG_ANALYZER_OUTPUT=html
    
    
    CLANG_ANALYZER_OUTPUT_DIR=analyzer |
    
    
    xcpretty && [[ -z `find analyzer -name "*.html"` ]]
    

    View Slide

91. ▸ Analyzing YourClass.m
    
    
    ⚠ /Users/philipphofmann/git-repos/sample/Sources/YourClass.m:11:5:
    Returning 'self' while it is not set to the result of '[(super or
    self) init...]' [osx.cocoa.SelfInit]
    
    
    return self;
    
    
    ^~~~~~~~~~~
    
    
    ▸ Analyze Succeeded
    
    
    Error 1
    

    View Slide

92. 7. Validate your Library
    Photo by Gabriel Sollmann on Unsplash
    

    View Slide

93. CocoaPods
    

    View Slide

94. pod lib lint
    

    View Slide

95. 12 minutes
    

    View Slide

96. pod lib lint -—platforms=iOS
    
    
    pod lib lint -—platforms=macOS
    
    
    pod lib lint -—platforms=tvOS
    
    
    pod lib lint --platforms=watchos
    Run in parallel
    

    View Slide

97. 3 minutes
    

    View Slide

98. Swift Package Manager
    

    View Slide

99. Add a sample app.
    

    View Slide

100. let package = Package(
     
     
     name: "macOS-SPM-CommandLine",
     
     
     dependencies: [
     
     
     .package(
     
     
     name: "Sentry",
     
     
     url: "https://github.com/getsentry/sentry-cocoa",
     
     
     .branch(“main")
     
     
     )
     
     
     ],
     
     
     targets: [
     
     
     .target(
     
     
     name: "macOS-SPM-CommandLine",
     
     
     dependencies: ["Sentry"],
     
     
     swiftSettings: [
     
     
     .unsafeFlags(["-warnings-as-errors"])
     
     
     ])
     
     
     ]
     
     
     )
     
     
     

     View Slide

101. dependencies: [
     
     
     .package(
     
     
     name: "Sentry",
     
     
     url: "https://github.com/getsentry/sentry-cocoa",
     
     
     .branch("main")
     
     
     )
     
     
     ]
     

     View Slide

102. dependencies: [
     
     
     .package(
     
     
     name: "Sentry",
     
     
     url: "https://github.com/getsentry/sentry-cocoa",
     
     
     .revision("4e037c")
     
     
     )
     
     
     ]
     

     View Slide

103. sed
     

     View Slide

104. sed -i '' \
     
     
     ’s/.branch("main")/
     

     View Slide

105. sed -i '' \
     
     
     ’s/.branch("main")/.revision("${{ github.sha }}")/g'
     

     View Slide

106. sed -i '' \
     
     
     ’s/.branch("main")/.revision("${{ github.sha }}")/g' \
     
     
     Samples/macOS-SPM-CommandLine/Package.swift
     

     View Slide

107. dependencies: [
     
     
     .package(
     
     
     name: "Sentry",
     
     
     url: "https://github.com/getsentry/sentry-cocoa",
     
     
     .branch("main")
     
     
     )
     
     
     ]
     

     View Slide

108. dependencies: [
     
     
     .package(
     
     
     name: "Sentry",
     
     
     url: "https://github.com/getsentry/sentry-cocoa",
     
     
     .revision("4e037c")
     
     
     )
     
     
     ]
     

     View Slide

109. swift build
     

     View Slide

110. Carthage
     

     View Slide

111. github "getsentry/sentry-cocoa" "main"
     Cartfile
     

     View Slide

112. github "getsentry/sentry-cocoa" "4e037c"
     Cartfile
     

     View Slide

113. Prebuilt XCFramework
     

     View Slide

114. 8. Borrowing Tests
     Photo by Louis Reed on Unsplash
     

     View Slide

115. Tests from VLC, Alamofire, HomeKit.
     

     View Slide

116. git clone https://github.com/other/repo.git
     

     View Slide

117. Add your library to tests.
     

     View Slide

118. Create a git patch.
     

     View Slide

119. + repositoryURL = "https://github.com/org/repo";
     
     
     + requirement = {
     
     
     + kind = revision;
     
     
     + revision = __GITHUB_REVISION_PLACEHOLDER__;
     
     
     + };
     
     
     Patchfile
     

     View Slide

120. git clone https://github.com/other/repo.git
     
     
     CI
     

     View Slide

121. git clone https://github.com/other/repo.git
     
     
     git checkout 4e23cas
     
     
     CI
     

     View Slide

122. git clone https://github.com/other/repo.git
     
     
     git checkout 4e23cas
     
     
     curl “https://raw.githubusercontent.com/org/repo/${SHA}/your.patch"
     
     
     --output your.patch
     
     
     CI
     

     View Slide

123. git clone https://github.com/other/repo.git
     
     
     git checkout 4e23cas
     
     
     curl “https://raw.githubusercontent.com/org/repo/${SHA}/your.patch"
     
     
     --output your.patch
     
     
     # Replace revision with SHA
     
     
     REPLACE="s/__GITHUB_REVISION_PLACEHOLDER__/${SHA}/g"
     
     
     sed -i '' $REPLACE your.patch
     
     
     CI
     

     View Slide

124. git clone https://github.com/other/repo.git
     
     
     git checkout 4e23cas
     
     
     curl “https://raw.githubusercontent.com/org/repo/${SHA}/your.patch"
     
     
     --output your.patch
     
     
     # Replace revision with SHA
     
     
     REPLACE="s/__GITHUB_REVISION_PLACEHOLDER__/${SHA}/g"
     
     
     sed -i '' $REPLACE your.patch
     
     
     git apply your.patch
     
     
     CI
     

     View Slide

125. git clone https://github.com/other/repo.git
     
     
     git checkout 4e23cas
     
     
     curl “https://raw.githubusercontent.com/org/repo/${SHA}/your.patch"
     
     
     --output your.patch
     
     
     # Replace revision with SHA
     
     
     REPLACE="s/__GITHUB_REVISION_PLACEHOLDER__/${SHA}/g"
     
     
     sed -i '' $REPLACE your.patch
     
     
     git apply your.patch
     
     
     xcodebuild [flags] test
     
     
     CI
     

     View Slide

126. 9. High-Risk Files
     

     View Slide

127. // WARNING START
     
     
     // This code is bulletproof.
     
     
     // Don't think you are smart and can
     
     
     // improve it.
     
     
     …
     
     
     // WARNING END
     

     View Slide

128. ./no-changes-in-high-risk-files.sh
     

     View Slide

129. #!/bin/bash
     
     
     set -euo pipefail
     
     
     ACTUAL=$(shasum -a 256 ./Sources/BulletProof.swift)
     

     View Slide

130. #!/bin/bash
     
     
     set -euo pipefail
     
     
     ACTUAL=$(shasum -a 256 ./Sources/BulletProof.swift)
     
     
     EXPECTED="54a41f19... ./Sources/BulletProof.swift"
     

     View Slide

131. #!/bin/bash
     
     
     set -euo pipefail
     
     
     ACTUAL=$(shasum -a 256 ./Sources/BulletProof.swift)
     
     
     EXPECTED="54a41f19... ./Sources/BulletProof.swift"
     
     
     if [ "$ACTUAL" = "$EXPECTED" ]; then
     
     
     else
     
     
     fi
     

     View Slide

132. #!/bin/bash
     
     
     set -euo pipefail
     
     
     ACTUAL=$(shasum -a 256 ./Sources/BulletProof.swift)
     
     
     EXPECTED="54a41f19... ./Sources/BulletProof.swift"
     
     
     if [ "$ACTUAL" = "$EXPECTED" ]; then
     
     
     echo "No changes in high risk files."
     
     
     exit 0
     
     
     else
     
     
     fi
     

     View Slide

133. #!/bin/bash
     
     
     set -euo pipefail
     
     
     ACTUAL=$(shasum -a 256 ./Sources/BulletProof.swift)
     
     
     EXPECTED="54a41f19... ./Sources/BulletProof.swift"
     
     
     if [ "$ACTUAL" = "$EXPECTED" ]; then
     
     
     echo "No changes in high risk files."
     
     
     exit 0
     
     
     else
     
     
     text="Changes in high risk files.
     
     
     If your changes are intended please
     
     
     update the sha in
     
     
     ./no-changes-in-high-risk-files.sh."
     
     
     echo "$text"
     
     
     exit 1
     
     
     fi
     

     View Slide

134. Tips and Tricks
     1. Better Test Logs
     2. Flaky Tests
     3. Flaky CI
     4. SwiftLint & Clang-Format
     5. Xcode Thread Sanitizer
     6. Xcode Analyze
     7. Validate your Library
     8. Borrowing Tests
     9. High-Risk Files
     

     View Slide

135. Checkout sentry-cocoa for
     seeing the CI tips in action.
     Level up your CI for
     iOS and macOS
     Philipp
     Hofmann
     1. Better Test Logs
     2. Flaky Tests
     3. Flaky CI
     4. SwiftLint & Clang-Format
     5. Xcode Thread Sanitizer
     6. Xcode Analyze
     7. Validate your Library
     8. Borrowing Tests
     9. High-Risk Files
     

     View Slide