Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building iOS apps at scale (Mobilization)

Building iOS apps at scale (Mobilization)

Working with a large code base in a large distributed team involves a lot of challenges. You need to deal with complex workflows, slow build times, communications across different time zones, etc.

In this talk, Yusei will share how development teams can tackle these issues and speed up daily development. This talk will also cover the following topics:

- Workflow automation with Fastlane
- Code review with Danger and SwiftLint
- Collecting and visualizing code metrics with InfluxDB and Grafana
- Build time reduction
- Code modularization

Yusei Nishiyama

October 21, 2017
Tweet

More Decks by Yusei Nishiyama

Other Decks in Technology

Transcript

  1. Building iOS apps
    at scale
    Yusei Nishiyama
    @yuseinishiyama

    View Slide

  2. Introduction

    View Slide

  3. • iOS, Android, Web
    • 67 countries
    • 21 languages
    • 60M monthly users (Japan)
    • 30M+ monthly users 

    (Outside Japan)
    https://info.cookpad.com/en
    https://cookpad.com/en

    View Slide

  4. Distributed Team

    View Slide

  5. Working at Cookpad is …
    • Working in a large team
    • Working with large code base

    View Slide

  6. Agenda
    • Working in a large team
    • Automation with Fastlane
    • Code review with Danger
    • Code metrics collection
    • Working with a large code base
    • Build time reduction
    • Code modularization

    View Slide

  7. Agenda
    • Working in a large team
    • Automation with Fastlane
    • Code review with Danger
    • Code metrics collection
    • Working with a large code base
    • Build time reduction
    • Code modularization

    View Slide

  8. • Working in a large team
    • Automation with Fastlane
    • Code review with Danger
    • Code metrics collection
    • Working with a large code base
    • Build time reduction
    • Code modularization

    View Slide

  9. • Someone broke the build
    • We can’t release today because {name} is on
    vacation
    • Code review never finishes

    View Slide

  10. Automation with CI
    • Testing
    • Nitpicking
    • Beta distribution
    • Release management

    View Slide

  11. CI Server

    View Slide

  12. CI Server

    View Slide

  13. CI Server

    View Slide

  14. CI Server

    View Slide

  15. • Working in a large team
    • Automation with Fastlane
    • Code review with Danger
    • Code metrics collection
    • Working with a large code base
    • Build time reduction
    • Code modularization

    View Slide

  16. Fastlane in Cookpad
    • 7 Fastfiles
    • 28 lanes
    • 740 lines
    • 5 plugins

    View Slide

  17. Sync dSYMs
    version = get_version_number
    build_number = get_build_number
    # Download dSYM files from iTC into a temporary directory
    download_dsyms(version: version, build_number: build_number)
    # Upload them to Crashlytics
    upload_symbols_to_crashlytics
    # Delete the temporary dSYM files
    clean_build_artifacts

    View Slide

  18. Release the App
    Spaceship::Tunes.login
    app_identifier =
    CredentialsManager::AppfileConfig.try_fetch_value(:app_identifier)
    edit_version = Spaceship::Application.find(app_identifier).edit_version
    version_to_release = edit_version.version
    UI.user_error! "No pending verion to release" unless
    edit_version.app_status == "Pending Developer Release"
    edit_version.release!
    UI.success "Version #{version_to_release} is released "
    slack(
    channel: "#release",
    attachment_properties: {
    pretext: ":appleinc: *Cookpad iOS #{version_to_release} - Released*”
    })
    refresh_dsyms

    View Slide

  19. Triggers beta distribution

    View Slide

  20. Check Labels
    desc "Check if pr is marked as beta distribution"
    private_lane :beta_distribution do
    labels.any? { |label| label == 'beta distribution' }
    end
    desc "Get labels of pull request"
    private_lane :labels do
    pr_number = ENV['PR_NUMBER']
    repo = ENV['GITHUB_PULL_REQUEST_REPO']
    result = github_api(
    http_method: 'GET',
    path: "/repos/#{repo}/issues/#{pr_number}/labels"
    )
    result[:json].map { |label| label['name'] }
    end

    View Slide

  21. Internal testers
    Developers
    Beta apps
    Feedback

    View Slide

  22. Internal testers
    Developers
    v 1.0.0.1
    Feedback
    v 1.0.0.2
    v 2.0.0.0
    v 2.1.0.2
    Which version should I download?
    and what needs to be tested?

    View Slide

  23. Append badge on icon
    if is_release?
    next
    elsif is_rc?
    title = 'RC'
    number = get_version_number
    elsif is_beta?
    title = 'Beta'
    number = sh('git rev-parse --short=7 HEAD').strip
    elsif is_pr?
    title = 'PR'
    number = ENV.fetch('PR_NUMBER')
    end
    badge(shield: “#{title}-#{number}-blue",
    dark: true,
    no_badge: true)

    View Slide

  24. We are going to release a new version soon.
    Please check the latest RC.

    View Slide

  25. • Working in a large team
    • Automation with Fastlane
    • Code review with Danger
    • Code metrics collection
    • Working with a large code base
    • Build time reduction
    • Code modularization

    View Slide

  26. View Slide

  27. View Slide

  28. CI Server

    View Slide

  29. Danger
    Danger runs during your CI process, and gives teams the chance to automate
    common code review chores.
    This provides another logical step in your build, through this Danger can help
    lint your rote tasks in daily code review.
    You can use Danger to codify your teams norms. Leaving humans to think
    about harder problems.
    She does this by leaving messages inside your PRs based on rules that you
    create with the Ruby scripting language.
    Over time, as rules are adhered to, the message is amended to reflect the
    current state of the code review
    http://danger.systems/ruby/

    View Slide

  30. Dangerfile
    github.dismiss_out_of_range_messages({
    error: false,
    warning: true,
    message: true,
    markdown: true
    })
    xcode_summary.inline_mode = true
    xcode_summary.ignored_files = ['Pods/**']
    xcode_summary.ignored_results { |result|
    result.message.start_with? 'ld'
    }
    log = File.join('logs', 'test.json')
    xcode_summary.report log if File.exist?(log)
    swiftlint.binary_path = './Pods/SwiftLint/swiftlint'
    swiftlint.lint_files inline_mode: true

    View Slide

  31. View Slide

  32. • Working in a large team
    • Automation with Fastlane
    • Code review with Danger
    • Code metrics collection
    • Working with a large code base
    • Build time reduction
    • Code modularization

    View Slide

  33. CI Server

    View Slide

  34. InfluxDB
    • Only purpose-built platform for any time-based data
    • Written in Go
    • Built-in HTTP API
    • SQL-like query language
    • Answer queries in real-time
    • Rich client libraries

    View Slide

  35. Post Code Coverage
    lane :post_coverage do
    rate = export_coverage
    if rate
    influxdb(table_name: table_name, values: {coverage: rate})
    slack(message: "Code coverage: #{rate.to_f.round(4) * 100}%", channel: “#ios-
    notifications")
    end
    end
    private_lane :export_coverage do
    fastlane_require 'nokogiri'
    output_directory = 'coverage_report'
    reports = Dir.glob(File.join(ROOT_PATH, output_directory, '*.xml'))
    slather(use_bundle_exec: true, output_directory: output_directory)
    if reports.empty?
    nil
    else
    xml = Nokogiri::XML(File.open(reports.first))
    coverage = xml.xpath('//coverage').first
    coverage.attr("line-rate").to_f
    end
    end

    View Slide

  36. CI Server

    View Slide

  37. Grafana
    • Rich metrics dashboard and editor
    • Fast rendering
    • Support InfluxDB as a data source
    • Query editor

    View Slide

  38. Visualize Code Coverage

    View Slide

  39. Other metrics

    View Slide

  40. View Slide

  41. • Working in a large team
    • Automation with Fastlane
    • Code review with Danger
    • Code metrics collection
    • Working with a large code base
    • Build time reduction
    • Code modularization

    View Slide

  42. Let’s improve the registration
    screen…

    View Slide

  43. Let’s improve the registration
    screen…
    • Implement

    View Slide

  44. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints

    View Slide

  45. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints
    • Test with older iOS
    versions

    View Slide

  46. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints
    • Test with older iOS
    versions
    • Fix tests

    View Slide

  47. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints
    • Test with older iOS
    versions
    • Fix tests
    • Wait for CI

    View Slide

  48. How many times 

    did you build so far?

    View Slide

  49. • Working in a large team
    • Automation with Fastlane
    • Code review with Danger
    • Code metrics collection
    • Working with a large code base
    • Build time reduction
    • Code modularization

    View Slide

  50. Reduce Build Time
    • Measurement
    • Improvement

    View Slide

  51. What I will “not” talk about
    • Annotate all types explicitly
    • Write as much as code in a single file
    • Get your Mac Pro

    View Slide

  52. Reduce Build Time
    • Measurement
    • Improvement

    View Slide

  53. “Premature optimization is
    the root of all evil.”


    — Donald Knuth —

    View Slide

  54. Display build duration
    $ defaults write com.apple.dt.Xcode ShowBuildOperationDuration YES

    View Slide

  55. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Run Scripts (Linter, Formatter)

    View Slide

  56. fastlane/report.xml





    time="8.59854">



    time="0.001982">








    View Slide

  57. View Slide

  58. View Slide

  59. View Slide

  60. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Run Scripts (Linter, Formatter)

    View Slide

  61. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Function A
    • Function B
    • …
    • Runs Scripts (Linter, Formatter)

    View Slide

  62. Warn long function bodies

    View Slide

  63. Time function bodies
    $ pbpaste | grep '.[0-9]ms' | sort -nr | head -20

    View Slide

  64. xcactivitylog
    • You can access build logs under DerivedData
    • xcactivitylog is just gzip. You can check the
    content as a plain text just by unzipping it

    View Slide

  65. giginet/danger-xcprofiler
    (in Dangerfile)


    xcprofiler.thresholds = {
    warn: 50,
    fail: 500
    }
    xcprofiler.report ‘ProductName'

    View Slide

  66. Reduce Build Time
    • Measurement
    • Improvement

    View Slide

  67. Test with Real Data
    $ cloc


    github.com/AlDanial/cloc v 1.72 T=3.40 s (153.3 files/s, 16646.6 lines/s)
    -------------------------------------------------------------------------------
    Language files blank comment code
    -------------------------------------------------------------------------------
    Swift 497 8944 3122 43831
    JSON 18 0 0 522
    Objective C 3 37 5 166
    C/C++ Header 4 19 6 27
    -------------------------------------------------------------------------------
    SUM: 522 9000 3133 44546
    -------------------------------------------------------------------------------

    View Slide

  68. Test with Real Data
    • 40,000 lines of Swift code
    • CocoaPods
    • 38 direct dependencies and 60 libraries
    • Linter in build phases
    • CI archives the app and distributes it for internal
    testing

    View Slide

  69. (Local clean build: 330s)

    View Slide

  70. View Slide

  71. Whole Module Optimization

    View Slide

  72. WMO without Optimization

    View Slide

  73. (Local clean build: 160s)

    View Slide

  74. Time Function Bodies

    View Slide

  75. F

    View Slide

  76. Carthage
    + github "ReactiveX/RxSwift" ~> 3.4
    + github "RxSwiftCommunity/RxRealm"
    + github "realm/realm-cocoa" ~> 2.4
    + github "danielgindi/Charts" ~> 3.0.2
    - pod 'RxSwift', '3.3.1'
    - pod 'RxCocoa', '3.3.1'
    - pod 'RxRealm', '0.5.2'
    - pod 'RealmSwift', '2.4.4'
    Cartfile
    Podfile

    View Slide

  77. (Local clean build: 68s)

    View Slide

  78. Xcode 9 New Build System

    View Slide

  79. • Xcode 9
    • Swift 3.2
    • New build system
    • No significant improvement in build time

    View Slide

  80. Do you really need to build
    your entire app?

    View Slide

  81. View Slide

  82. Prototyping with Playground
    • Test your idea in Playground 

    (interface, design, reproducing a bug, etc.)
    • Once your idea becomes concrete, move
    those code into the application project

    View Slide

  83. Playground

    View Slide

  84. Playground

    View Slide

  85. You need “existing” code 

    for prototyping
    class ViewController: UITableViewController {
    let myView = MyView()
    override func viewDidLoad() {
    super.viewDidLoad()
    view.backgroundColor = .myColor
    view.addSubview(myView)
    }
    }

    View Slide

  86. Build your own UI library
    import UIKit
    @testable import YourUIKit
    let color = UIColor.myGreen

    View Slide

  87. • Working in a large team
    • Automation with Fastlane
    • Code review with Danger
    • Code metrics collection
    • Working with a large code base
    • Build time reduction
    • Code modularization

    View Slide

  88. Microservice Architecture
    https://martinfowler.com/articles/microservices.html

    View Slide

  89. Code Modularization
    Feature A
    Design
    Feature B Feature C
    Network Persistence
    Core

    View Slide

  90. Benefits
    • Strict access control
    • Faster incremental build/indexing

    View Slide

  91. Autonomous teams
    https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1/

    View Slide

  92. Repository Strategy
    • Single Repository
    • Multiple Repositories

    View Slide

  93. Single repository
    • Simple workflow
    • Can’t open-source

    View Slide

  94. Multiple Repositories
    • Complex workflow
    • Code Review Process
    • Easy to open-source

    View Slide

  95. Dependency Management
    • CocoaPods
    • git submodule
    • Carthage
    • git subtree

    View Slide

  96. CocoaPods
    • Easy in use
    • Black box in nature

    View Slide

  97. git submodule
    • Simple (!= Easy)
    • Full control
    • Manual works
    • Complex workflow

    View Slide

  98. Carthage
    • Flexible
    • Pre-build
    • Not all libraries support it
    • ABI stability

    View Slide

  99. https://github.com/apple/swift-evolution#development-major-version--swift-50

    View Slide

  100. git subtree
    • An alternative to git submodule
    • Clones a subproject and merge it into the
    parent project
    • `git subtree push` goes through the commits
    and picking the changes that should go to a
    subproject

    View Slide

  101. git subtree

    View Slide

  102. git subtree
    • Requires no change to workflow
    • Need to learn a new merge strategy
    • Contributing back to upstream is a bit tricky
    https://medium.com/@v/git-subtrees-a-tutorial-6ff568381844

    View Slide

  103. Lesson Learned
    • Breaking existing implicit dependencies is hard
    • Starting from the single repository
    • Once modules become stable, consider
    • managing them in separated repositories
    • using pre-build versions

    View Slide

  104. Pitfall

    View Slide

  105. dylib Loading Time
    Total pre-main time: 4.2 seconds (100.0%)
    dylib loading time: 4.0 seconds (94.4%)
    rebase/binding time: 93.32 milliseconds (2.1%)
    ObjC setup time: 44.64 milliseconds (1.0%)
    initializer time: 98.30 milliseconds (2.3%)

    View Slide

  106. Pitfall

    View Slide

  107. Xcode 9 supports
    Swift Static Library
    https://download.developer.apple.com/Developer_Tools/Xcode_9_beta_4/
    Xcode_9_beta_4_Release_Notes.pdf

    View Slide

  108. Recap
    • Automate daily workflows with CI and Fastlane
    • Reduce workload of code review with Danger
    • Visualize code metrics you want to improve
    • Revising project settings may improve build time
    dramatically
    • Code modularization improves build/index time and
    helps you getting well structured code/teams

    View Slide

  109. Thank you!
    Yusei Nishiyama
    @yuseinishiyama
    https://speakerdeck.com/yuseinishiyama/building-ios-apps-at-scale-mobilization

    View Slide