Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Working at Scale

Working at Scale

Working with a large code base in a large distributed team involves a lot of challenges. You need to deal with complex workflows, slow build times, etc.

In this talk, Yusei will share how development teams can tackle these issues and speed up daily development. This talk will also cover the following topics:

- Workflow automation with Fastlane
- Code review with Danger and SwiftLint
- Collecting and visualizing code metrics with InfluxDB and Grafana
Build time reduction
- Code modularization

Yusei Nishiyama

September 23, 2017
Tweet

More Decks by Yusei Nishiyama

Other Decks in Technology

Transcript

  1. Working at scale
    How to save time with CI and build time reduction
    Yusei Nishiyama
    @yuseinishiyama

    View Slide

  2. • iOS, Android, Web
    • 67 countries
    • 21 languages
    • 60M monthly users (Japan)
    • 30M+ monthly users 

    (Outside Japan)
    https://info.cookpad.com/en
    https://cookpad.com/en

    View Slide

  3. Distributed Team

    View Slide

  4. Working at Cookpad is …
    • Working in a large team
    • Working with large code base

    View Slide

  5. Working at Cookpad is …
    • Working in a large team
    • Working with a large code base

    View Slide

  6. • Someone broke the build
    • We can’t release today because {name} is on
    vacation
    • Code review never finishes

    View Slide

  7. Automation with CI
    • Testing
    • Nitpicking
    • Beta distribution
    • Release management

    View Slide

  8. CI Server

    View Slide

  9. CI Server

    View Slide

  10. CI Server

    View Slide

  11. CI Server

    View Slide

  12. Fastlane in Cookpad
    • 7 Fastfiles
    • 28 lanes
    • 740 lines
    • 5 plugins

    View Slide

  13. Sync dSYMs
    desc "Refresh dSYMs"
    lane :refresh_dsyms do
    version = get_version_number
    build_number = get_build_number
    # Download dSYM files from iTC into a temporary directory
    download_dsyms(version: version, build_number: build_number)
    # Upload them to Crashlytics
    upload_symbols_to_crashlytics
    # Delete the temporary dSYM files
    clean_build_artifacts
    end

    View Slide

  14. Release the App
    Spaceship::Tunes.login
    app_identifier =
    CredentialsManager::AppfileConfig.try_fetch_value(:app_identifier)
    edit_version = Spaceship::Application.find(app_identifier).edit_version
    version_to_release = edit_version.version
    UI.user_error! "No pending verion to release" unless
    edit_version.app_status == "Pending Developer Release"
    edit_version.release!
    UI.success "Version #{version_to_release} is released "
    slack(
    channel: "#release",
    attachment_properties: {
    pretext: ":appleinc: *Cookpad iOS #{version_to_release} - Released*”
    })
    refresh_dsyms

    View Slide

  15. Triggers beta distribution

    View Slide

  16. Check Labels
    desc "Check if pr is marked as beta distribution"
    private_lane :beta_distribution do
    labels.any? { |label| label == 'beta distribution' }
    end
    desc "Get labels of pull request"
    private_lane :labels do
    pr_number = ENV['PR_NUMBER']
    repo = ENV['GITHUB_PULL_REQUEST_REPO']
    result = github_api(
    http_method: 'GET',
    path: "/repos/#{repo}/issues/#{pr_number}/labels"
    )
    result[:json].map { |label| label['name'] }
    end

    View Slide

  17. Notify a New Build
    desc "Notify new beta distribution"
    private_lane :notify_build do
    messages = "A new version is available"
    pr_number = ENV['PR_NUMBER']
    repo = ENV['GITHUB_PULL_REQUEST_REPO']
    github_api(
    http_method: 'POST',
    path: "/repos/#{repo}/issues/#{pr_number}/comments",
    body: { "body": message }
    )
    end

    View Slide

  18. View Slide

  19. CI Server

    View Slide

  20. Dangerfile
    github.dismiss_out_of_range_messages({
    error: false,
    warning: true,
    message: true,
    markdown: true
    })
    xcode_summary.inline_mode = true
    xcode_summary.ignored_files = ['Pods/**']
    xcode_summary.ignored_results { |result|
    result.message.start_with? 'ld'
    }
    log = File.join('logs', 'test.json')
    xcode_summary.report log if File.exist?(log)
    swiftlint.binary_path = './Pods/SwiftLint/swiftlint'
    swiftlint.lint_files inline_mode: true

    View Slide

  21. View Slide

  22. CI Server

    View Slide

  23. InfluxDB
    • Only purpose-built platform for any time-based data
    • Written in Go
    • Built-in HTTP API
    • SQL-like query language
    • Answer queries in real-time
    • Rich client libraries

    View Slide

  24. Post Code Coverage
    lane :post_coverage do
    rate = export_coverage
    if rate
    influxdb(table_name: table_name, values: {coverage: rate})
    slack(message: "Code coverage: #{rate.to_f.round(4) * 100}%", channel: “#ios-
    notifications")
    end
    end
    private_lane :export_coverage do
    fastlane_require 'nokogiri'
    output_directory = 'coverage_report'
    reports = Dir.glob(File.join(ROOT_PATH, output_directory, '*.xml'))
    slather(use_bundle_exec: true, output_directory: output_directory)
    if reports.empty?
    nil
    else
    xml = Nokogiri::XML(File.open(reports.first))
    coverage = xml.xpath('//coverage').first
    coverage.attr("line-rate").to_f
    end
    end

    View Slide

  25. CI Server

    View Slide

  26. Grafana
    • Rich metrics dashboard and editor
    • Fast rendering
    • Support InfluxDB as a data source
    • Query editor

    View Slide

  27. Visualize Code Coverage

    View Slide

  28. Other metrics

    View Slide

  29. Working at Cookpad is …
    • Working in a large team
    • Working with a large code base

    View Slide

  30. How many times 

    do you build every day?

    View Slide

  31. Reduce Build Time
    • Measurement
    • Improvement

    View Slide

  32. What I will “not” talk about
    • Annotate all types explicitly
    • Write as much as code in a single file
    • Get your Mac Pro

    View Slide

  33. Reduce Build Time
    • Measurement
    • Improvement

    View Slide

  34. “Premature optimization is
    the root of all evil.”


    — Donald Knuth —

    View Slide

  35. Display build duration
    $ defaults write com.apple.dt.Xcode ShowBuildOperationDuration YES

    View Slide

  36. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Run tests

    View Slide

  37. fastlane/report.xml





    time="8.59854">



    time="0.001982">








    View Slide

  38. View Slide

  39. View Slide

  40. View Slide

  41. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Run tests

    View Slide

  42. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Function A
    • Function B
    • …
    • Runs tests

    View Slide

  43. Warn long function bodies

    View Slide

  44. Time function bodies
    $ pbpaste | grep '.[0-9]ms' | sort -nr | head -20

    View Slide

  45. xcactivitylog
    • You can access build logs under DerivedData
    • xcactivitylog is just gzip. You can check the
    content as a plain text just by unzipping it

    View Slide

  46. giginet/danger-xcprofiler
    (in Dangerfile)


    xcprofiler.thresholds = {
    warn: 50,
    fail: 500
    }
    xcprofiler.report ‘ProductName'

    View Slide

  47. Reduce Build Time
    • Measurement
    • Improvement

    View Slide

  48. Test with Real Data
    $ cloc


    github.com/AlDanial/cloc v 1.72 T=3.40 s (153.3 files/s, 16646.6 lines/s)
    -------------------------------------------------------------------------------
    Language files blank comment code
    -------------------------------------------------------------------------------
    Swift 497 8944 3122 43831
    JSON 18 0 0 522
    Objective C 3 37 5 166
    C/C++ Header 4 19 6 27
    -------------------------------------------------------------------------------
    SUM: 522 9000 3133 44546
    -------------------------------------------------------------------------------

    View Slide

  49. Test with Real Data
    • 40,000 lines of Swift code
    • CocoaPods
    • 38 direct dependencies and 60 libraries
    • Linter in build phases
    • CI archives the app and distributes it for internal
    testing

    View Slide

  50. (Local clean build: 330s)

    View Slide

  51. View Slide

  52. Whole Module Optimization

    View Slide

  53. WMO without Optimization

    View Slide

  54. (Local clean build: 160s)

    View Slide

  55. Time Function Bodies

    View Slide

  56. F

    View Slide

  57. Carthage
    + github "ReactiveX/RxSwift" ~> 3.4
    + github "RxSwiftCommunity/RxRealm"
    + github "realm/realm-cocoa" ~> 2.4
    + github "danielgindi/Charts" ~> 3.0.2
    - pod 'RxSwift', '3.3.1'
    - pod 'RxCocoa', '3.3.1'
    - pod 'RxRealm', '0.5.2'
    - pod 'RealmSwift', '2.4.4'
    Cartfile
    Podfile

    View Slide

  58. https://github.com/apple/swift-evolution#development-major-version--swift-50

    View Slide

  59. (Local clean build: 68s)

    View Slide

  60. Xcode 9 New Build System

    View Slide

  61. • Xcode 9
    • Swift 3.2
    • New build system
    • No significant improvement in build time

    View Slide

  62. Do you really need to build
    your entire app?

    View Slide

  63. Code Modularization
    Feature A
    Design
    Feature B Feature C
    Network Persistence
    Core

    View Slide

  64. Benefits
    • Faster incremental build/indexing
    • Strict access control
    • Clear separation

    View Slide

  65. Lessons Learned
    • Breaking existing implicit dependencies is hard
    • Starting from the single repository
    • Once modules become stable, consider
    • managing them in separated repositories
    • using pre-build versions

    View Slide

  66. Pitfall

    View Slide

  67. dylib Loading Time
    Total pre-main time: 4.2 seconds (100.0%)
    dylib loading time: 4.0 seconds (94.4%)
    rebase/binding time: 93.32 milliseconds (2.1%)
    ObjC setup time: 44.64 milliseconds (1.0%)
    initializer time: 98.30 milliseconds (2.3%)

    View Slide

  68. Pitfall

    View Slide

  69. Xcode 9 supports
    Swift Static Library
    https://download.developer.apple.com/Developer_Tools/Xcode_9_beta_4/
    Xcode_9_beta_4_Release_Notes.pdf

    View Slide

  70. Recap
    • Automate daily workflows with CI and Fastlane
    • Reduce workload of code review with Danger
    • Visualize code metrics you want to improve
    • Revising project settings may improve build time
    dramatically
    • Code modularization improves build/index time and
    helps you get well structured code/teams

    View Slide

  71. The presentation is available on


    https://speakerdeck.com/yuseinishiyama/working-at-scale

    View Slide

  72. Thank you!
    Yusei Nishiyama
    @yuseinishiyama

    View Slide