$30 off During Our Annual Pro Sale. View Details »

Building iOS apps at scale

Yusei Nishiyama
September 06, 2017

Building iOS apps at scale

Working with a large code base in a large distributed team involves a lot of challenges. You need to deal with complex workflows, slow build times, communications across different time zones, etc.

In this talk, Yusei will share how development teams can tackle these issues and speed up daily development. This talk will also cover the following topics:

- Workflow automation with Fastlane
- Code review with Danger and SwiftLint
- Collecting and visualizing code metrics with InfluxDB and Grafana
- Build time reduction
- Code modularization

Yusei Nishiyama

September 06, 2017
Tweet

More Decks by Yusei Nishiyama

Other Decks in Technology

Transcript

  1. Building iOS apps
    at scale
    Yusei Nishiyama
    @yuseinishiyama

    View Slide

  2. • iOS, Android, Web
    • 67 countries
    • 21 languages
    • 60M monthly users (Japan)
    • 30M+ monthly users 

    (Outside Japan)
    https://info.cookpad.com/en
    https://cookpad.com/en

    View Slide

  3. Distributed Team

    View Slide

  4. Working at Cookpad is …
    • Working in a large team
    • Working with large code base

    View Slide

  5. Working at Cookpad is …
    • Working in a large team
    • Working with a large code base

    View Slide

  6. • Someone broke the build
    • We can’t release today because {name} is on
    vacation
    • Code review never finishes

    View Slide

  7. Automation with CI
    • Testing
    • Nitpicking
    • Beta distribution
    • Release management

    View Slide

  8. CI Server

    View Slide

  9. CI Server

    View Slide

  10. CI Server

    View Slide

  11. CI Server

    View Slide

  12. View Slide

  13. Fastlane in Cookpad
    • 7 Fastfiles
    • 28 lanes
    • 740 lines
    • 5 plugins

    View Slide

  14. Sync dSYMs
    desc "Refresh dSYMs"
    lane :refresh_dsyms do
    version = get_version_number
    build_number = get_build_number
    # Download dSYM files from iTC into a temporary directory
    download_dsyms(version: version, build_number: build_number)
    # Upload them to Crashlytics
    upload_symbols_to_crashlytics
    # Delete the temporary dSYM files
    clean_build_artifacts
    end

    View Slide

  15. Release the App
    Spaceship::Tunes.login
    app_identifier =
    CredentialsManager::AppfileConfig.try_fetch_value(:app_identifier)
    edit_version = Spaceship::Application.find(app_identifier).edit_version
    version_to_release = edit_version.version
    UI.user_error! "No pending verion to release" unless
    edit_version.app_status == "Pending Developer Release"
    edit_version.release!
    UI.success "Version #{version_to_release} is released "
    slack(
    channel: "#release",
    attachment_properties: {
    pretext: ":appleinc: *Cookpad iOS #{version_to_release} - Released*”
    })
    refresh_dsyms

    View Slide

  16. Triggers beta distribution

    View Slide

  17. Check Labels
    desc "Check if pr is marked as beta distribution"
    private_lane :beta_distribution do
    labels.any? { |label| label == 'beta distribution' }
    end
    desc "Get labels of pull request"
    private_lane :labels do
    pr_number = ENV['PR_NUMBER']
    repo = ENV['GITHUB_PULL_REQUEST_REPO']
    result = github_api(
    http_method: 'GET',
    path: "/repos/#{repo}/issues/#{pr_number}/labels"
    )
    result[:json].map { |label| label['name'] }
    end

    View Slide

  18. Notify a New Build
    desc "Notify new beta distribution"
    private_lane :notify_build do
    messages = "A new version is available"
    pr_number = ENV['PR_NUMBER']
    repo = ENV['GITHUB_PULL_REQUEST_REPO']
    github_api(
    http_method: 'POST',
    path: "/repos/#{repo}/issues/#{pr_number}/comments",
    body: { "body": message }
    )
    end

    View Slide

  19. View Slide

  20. View Slide

  21. View Slide

  22. CI Server

    View Slide

  23. Danger
    Danger runs during your CI process, and gives teams the chance to automate
    common code review chores.
    This provides another logical step in your build, through this Danger can help
    lint your rote tasks in daily code review.
    You can use Danger to codify your teams norms. Leaving humans to think
    about harder problems.
    She does this by leaving messages inside your PRs based on rules that you
    create with the Ruby scripting language.
    Over time, as rules are adhered to, the message is amended to reflect the
    current state of the code review
    http://danger.systems/ruby/

    View Slide

  24. Dangerfile
    github.dismiss_out_of_range_messages({
    error: false,
    warning: true,
    message: true,
    markdown: true
    })
    xcode_summary.inline_mode = true
    xcode_summary.ignored_files = ['Pods/**']
    xcode_summary.ignored_results { |result|
    result.message.start_with? 'ld'
    }
    log = File.join('logs', 'test.json')
    xcode_summary.report log if File.exist?(log)
    swiftlint.binary_path = './Pods/SwiftLint/swiftlint'
    swiftlint.lint_files inline_mode: true

    View Slide

  25. View Slide

  26. CI Server

    View Slide

  27. InfluxDB
    • Only purpose-built platform for any time-based data
    • Written in Go
    • Built-in HTTP API
    • SQL-like query language
    • Answer queries in real-time
    • Rich client libraries

    View Slide

  28. Post Code Coverage
    lane :post_coverage do
    rate = export_coverage
    if rate
    influxdb(table_name: table_name, values: {coverage: rate})
    slack(message: "Code coverage: #{rate.to_f.round(4) * 100}%", channel: “#ios-
    notifications")
    end
    end
    private_lane :export_coverage do
    fastlane_require 'nokogiri'
    output_directory = 'coverage_report'
    reports = Dir.glob(File.join(ROOT_PATH, output_directory, '*.xml'))
    slather(use_bundle_exec: true, output_directory: output_directory)
    if reports.empty?
    nil
    else
    xml = Nokogiri::XML(File.open(reports.first))
    coverage = xml.xpath('//coverage').first
    coverage.attr("line-rate").to_f
    end
    end

    View Slide

  29. CI Server

    View Slide

  30. Grafana
    • Rich metrics dashboard and editor
    • Fast rendering
    • Support InfluxDB as a data source
    • Query editor

    View Slide

  31. Visualize Code Coverage

    View Slide

  32. Other metrics

    View Slide

  33. Working at Cookpad is …
    • Working in a large team
    • Working with a large code base

    View Slide

  34. Let’s improve the registration
    screen…

    View Slide

  35. Let’s improve the registration
    screen…
    • Implement

    View Slide

  36. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints

    View Slide

  37. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints
    • Test with older iOS
    versions

    View Slide

  38. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints
    • Test with older iOS
    versions
    • Fix tests

    View Slide

  39. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints
    • Test with older iOS
    versions
    • Fix tests
    • Wait for CI

    View Slide

  40. How many times 

    did you build so far?

    View Slide

  41. Reduce Build Time
    • Measurement
    • Improvement

    View Slide

  42. What I will “not” talk about
    • Annotate all types explicitly
    • Write as much as code in a single file
    • Get your Mac Pro

    View Slide

  43. Reduce Build Time
    • Measurement
    • Improvement

    View Slide

  44. “Premature optimization is
    the root of all evil.”


    — Donald Knuth —

    View Slide

  45. Display build duration
    $ defaults write com.apple.dt.Xcode ShowBuildOperationDuration YES

    View Slide

  46. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Run Scripts (Linter, Formatter)

    View Slide

  47. fastlane/report.xml





    time="8.59854">



    time="0.001982">








    View Slide

  48. View Slide

  49. View Slide

  50. View Slide

  51. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Run Scripts (Linter, Formatter)

    View Slide

  52. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Function A
    • Function B
    • …
    • Runs Scripts (Linter, Formatter)

    View Slide

  53. Warn long function bodies

    View Slide

  54. Time function bodies
    $ pbpaste | grep '.[0-9]ms' | sort -nr | head -20

    View Slide

  55. xcactivitylog
    • You can access build logs under DerivedData
    • xcactivitylog is just gzip. You can check the
    content as a plain text just by unzipping it

    View Slide

  56. giginet/danger-xcprofiler
    (in Dangerfile)


    xcprofiler.thresholds = {
    warn: 50,
    fail: 500
    }
    xcprofiler.report ‘ProductName'

    View Slide

  57. Reduce Build Time
    • Measurement
    • Improvement

    View Slide

  58. Test with Real Data
    $ cloc


    github.com/AlDanial/cloc v 1.72 T=3.40 s (153.3 files/s, 16646.6 lines/s)
    -------------------------------------------------------------------------------
    Language files blank comment code
    -------------------------------------------------------------------------------
    Swift 497 8944 3122 43831
    JSON 18 0 0 522
    Objective C 3 37 5 166
    C/C++ Header 4 19 6 27
    -------------------------------------------------------------------------------
    SUM: 522 9000 3133 44546
    -------------------------------------------------------------------------------

    View Slide

  59. Test with Real Data
    • 40,000 lines of Swift code
    • CocoaPods
    • 38 direct dependencies and 60 libraries
    • Linter in build phases
    • CI archives the app and distributes it for internal
    testing

    View Slide

  60. (Local clean build: 330s)

    View Slide

  61. View Slide

  62. Whole Module Optimization

    View Slide

  63. WMO without Optimization

    View Slide

  64. (Local clean build: 160s)

    View Slide

  65. https://swift.org/blog/whole-module-optimizations/

    View Slide

  66. Time Function Bodies

    View Slide

  67. F

    View Slide

  68. Carthage
    + github "ReactiveX/RxSwift" ~> 3.4
    + github "RxSwiftCommunity/RxRealm"
    + github "realm/realm-cocoa" ~> 2.4
    + github "danielgindi/Charts" ~> 3.0.2
    - pod 'RxSwift', '3.3.1'
    - pod 'RxCocoa', '3.3.1'
    - pod 'RxRealm', '0.5.2'
    - pod 'RealmSwift', '2.4.4'
    Cartfile
    Podfile

    View Slide

  69. (Local clean build: 68s)

    View Slide

  70. Do you really need to build
    your entire app?

    View Slide

  71. View Slide

  72. Prototyping with Playground
    • Test your idea in Playground 

    (interface, design, reproducing a bug, etc.)
    • Once your idea becomes concrete, move
    those code into the application project

    View Slide

  73. Playground

    View Slide

  74. Playground

    View Slide

  75. You need “existing” code 

    for prototyping
    class ViewController: UITableViewController {
    let myView = MyView()
    override func viewDidLoad() {
    super.viewDidLoad()
    view.backgroundColor = .myColor
    view.addSubview(myView)
    }
    }

    View Slide

  76. Code Modularization
    import UIKit
    @testable import YourUIKit
    let color = UIColor.myGreen

    View Slide

  77. Code Modularization
    Feature A
    Design
    Feature B Feature C
    Network Persistence
    Core

    View Slide

  78. Microservice Architecture
    https://martinfowler.com/articles/microservices.html

    View Slide

  79. Benefits
    • Strict access control
    • Faster incremental build/indexing
    • Clear separation

    View Slide

  80. Repository Strategy
    • Single Repository
    • Multiple Repositories

    View Slide

  81. Single repository
    • Simple workflow
    • Can’t open-source

    View Slide

  82. Multiple Repositories
    • Complex workflow
    • Code Review Process
    • Easy to open-source

    View Slide

  83. Dependency Management
    • CocoaPods
    • git submodule
    • Carthage
    • git subtree

    View Slide

  84. CocoaPods
    • Easy in use
    • Black box in nature

    View Slide

  85. git submodule
    • Simple (!= Easy)
    • Full control
    • Manual works
    • Complex workflow

    View Slide

  86. Carthage
    • Flexible
    • Pre-build
    • Not all libraries support it
    • ABI stability

    View Slide

  87. https://github.com/apple/swift-evolution#development-major-version--swift-50

    View Slide

  88. git subtree
    • An alternative to git submodule
    • Clones a subproject and merge it into the
    parent project
    • `git subtree push` goes through the commits
    and picking the changes that should go to a
    subproject

    View Slide

  89. git subtree

    View Slide

  90. git subtree
    • Requires no change to workflow
    • Need to learn a new merge strategy
    • Contributing back to upstream is a bit tricky

    View Slide

  91. Lesson Learned
    • Breaking existing implicit dependencies is hard
    • Starting from the single repository
    • Once modules become stable, consider
    • managing them in separated repositories
    • using pre-build versions

    View Slide

  92. Case Study:
    Single repo + CocoaPods

    View Slide

  93. Directory Structure
    .
    "## MainApp
    $ "## MainApp
    $ $ "## AppDelegate.swift
    $ $ "## Info.plist
    $ $ # ViewController.swift
    $ "## MainApp.xcodeproj
    $ # MainAppTests
    $ "## Info.plist
    $ # MainAppTests.swift
    "## Library
    $ "## Library
    $ $ "## Info.plist
    $ $ "## Library.h
    $ $ # Library.swift
    $ "## Library.xcodeproj
    $ # LibraryTests
    $ "## Info.plist
    $ # LibraryTests.swift
    "## MainApp.xcworkspace
    "## Podfile
    "## Podfile.lock
    "## Pods
    # README.md

    View Slide

  94. Podfile
    source 'https://github.com/CocoaPods/Specs.git'
    use_frameworks!
    platform :ios, '9.3'
    workspace 'MainApp.xcworkspace'
    target 'MainApp' do
    project 'MainApp/MainApp.xcodeproj'
    # Dependencies of the MainApp
    end
    target 'Library' do
    project 'Library/Library.xcodeproj'
    # Dependencies of the Library
    end

    View Slide

  95. Workspace Structure

    View Slide

  96. Link it!

    View Slide

  97. Find Implicit Dependencies

    View Slide

  98. Pitfall

    View Slide

  99. dylib Loading Time
    Total pre-main time: 4.2 seconds (100.0%)
    dylib loading time: 4.0 seconds (94.4%)
    rebase/binding time: 93.32 milliseconds (2.1%)
    ObjC setup time: 44.64 milliseconds (1.0%)
    initializer time: 98.30 milliseconds (2.3%)

    View Slide

  100. Pitfall

    View Slide

  101. Xcode 9 supports
    Swift Static Library
    https://download.developer.apple.com/Developer_Tools/Xcode_9_beta_4/
    Xcode_9_beta_4_Release_Notes.pdf

    View Slide

  102. Recap
    • Automate daily workflows with CI and Fastlane
    • Reduce workload of code review with Danger
    • Visualize code metrics you want to improve
    • Revising project settings may improve build time
    dramatically
    • Code modularization improves build/index time and
    helps you getting well structured code/teams

    View Slide

  103. The presentation is available on


    https://speakerdeck.com/yuseinishiyama/building-ios-apps-at-scale

    View Slide

  104. Thank you!
    Yusei Nishiyama
    @yuseinishiyama

    View Slide