Building iOS apps at scale

Working with a large code base in a large distributed team involves a lot of challenges. You need to deal with complex workflows, slow build times, communications across different time zones, etc.

In this talk, Yusei will share how development teams can tackle these issues and speed up daily development. This talk will also cover the following topics:

- Workflow automation with Fastlane
- Code review with Danger and SwiftLint
- Collecting and visualizing code metrics with InfluxDB and Grafana
- Build time reduction
- Code modularization

Transcript

1. Building iOS apps
   at scale
   Yusei Nishiyama
   @yuseinishiyama
   

   View Slide

2. • iOS, Android, Web
   • 67 countries
   • 21 languages
   • 60M monthly users (Japan)
   • 30M+ monthly users 

   (Outside Japan)
   https://info.cookpad.com/en
   https://cookpad.com/en
   

   View Slide

3. Distributed Team
   

   View Slide

4. Working at Cookpad is …
   • Working in a large team
   • Working with large code base
   

   View Slide

5. Working at Cookpad is …
   • Working in a large team
   • Working with a large code base
   

   View Slide

6. • Someone broke the build
   • We can’t release today because {name} is on
   vacation
   • Code review never finishes
   

   View Slide

7. Automation with CI
   • Testing
   • Nitpicking
   • Beta distribution
   • Release management
   

   View Slide

8. CI Server
   

   View Slide

9. CI Server
   

   View Slide

10. CI Server
    

    View Slide

11. CI Server
    

    View Slide

12. View Slide

13. Fastlane in Cookpad
    • 7 Fastfiles
    • 28 lanes
    • 740 lines
    • 5 plugins
    

    View Slide

14. Sync dSYMs
    desc "Refresh dSYMs"
    lane :refresh_dsyms do
    version = get_version_number
    build_number = get_build_number
    # Download dSYM files from iTC into a temporary directory
    download_dsyms(version: version, build_number: build_number)
    # Upload them to Crashlytics
    upload_symbols_to_crashlytics
    # Delete the temporary dSYM files
    clean_build_artifacts
    end
    

    View Slide

15. Release the App
    Spaceship::Tunes.login
    app_identifier =
    CredentialsManager::AppfileConfig.try_fetch_value(:app_identifier)
    edit_version = Spaceship::Application.find(app_identifier).edit_version
    version_to_release = edit_version.version
    UI.user_error! "No pending verion to release" unless
    edit_version.app_status == "Pending Developer Release"
    edit_version.release!
    UI.success "Version #{version_to_release} is released "
    slack(
    channel: "#release",
    attachment_properties: {
    pretext: ":appleinc: *Cookpad iOS #{version_to_release} - Released*”
    })
    refresh_dsyms
    

    View Slide

16. Triggers beta distribution
    

    View Slide

17. Check Labels
    desc "Check if pr is marked as beta distribution"
    private_lane :beta_distribution do
    labels.any? { |label| label == 'beta distribution' }
    end
    desc "Get labels of pull request"
    private_lane :labels do
    pr_number = ENV['PR_NUMBER']
    repo = ENV['GITHUB_PULL_REQUEST_REPO']
    result = github_api(
    http_method: 'GET',
    path: "/repos/#{repo}/issues/#{pr_number}/labels"
    )
    result[:json].map { |label| label['name'] }
    end
    

    View Slide

18. Notify a New Build
    desc "Notify new beta distribution"
    private_lane :notify_build do
    messages = "A new version is available"
    pr_number = ENV['PR_NUMBER']
    repo = ENV['GITHUB_PULL_REQUEST_REPO']
    github_api(
    http_method: 'POST',
    path: "/repos/#{repo}/issues/#{pr_number}/comments",
    body: { "body": message }
    )
    end
    

    View Slide

19. View Slide

20. View Slide

21. View Slide

22. CI Server
    

    View Slide

23. Danger
    Danger runs during your CI process, and gives teams the chance to automate
    common code review chores.
    This provides another logical step in your build, through this Danger can help
    lint your rote tasks in daily code review.
    You can use Danger to codify your teams norms. Leaving humans to think
    about harder problems.
    She does this by leaving messages inside your PRs based on rules that you
    create with the Ruby scripting language.
    Over time, as rules are adhered to, the message is amended to reflect the
    current state of the code review
    http://danger.systems/ruby/
    

    View Slide

24. Dangerfile
    github.dismiss_out_of_range_messages({
    error: false,
    warning: true,
    message: true,
    markdown: true
    })
    xcode_summary.inline_mode = true
    xcode_summary.ignored_files = ['Pods/**']
    xcode_summary.ignored_results { |result|
    result.message.start_with? 'ld'
    }
    log = File.join('logs', 'test.json')
    xcode_summary.report log if File.exist?(log)
    swiftlint.binary_path = './Pods/SwiftLint/swiftlint'
    swiftlint.lint_files inline_mode: true
    

    View Slide

25. View Slide

26. CI Server
    

    View Slide

27. InfluxDB
    • Only purpose-built platform for any time-based data
    • Written in Go
    • Built-in HTTP API
    • SQL-like query language
    • Answer queries in real-time
    • Rich client libraries
    

    View Slide

28. Post Code Coverage
    lane :post_coverage do
    rate = export_coverage
    if rate
    influxdb(table_name: table_name, values: {coverage: rate})
    slack(message: "Code coverage: #{rate.to_f.round(4) * 100}%", channel: “#ios-
    notifications")
    end
    end
    private_lane :export_coverage do
    fastlane_require 'nokogiri'
    output_directory = 'coverage_report'
    reports = Dir.glob(File.join(ROOT_PATH, output_directory, '*.xml'))
    slather(use_bundle_exec: true, output_directory: output_directory)
    if reports.empty?
    nil
    else
    xml = Nokogiri::XML(File.open(reports.first))
    coverage = xml.xpath('//coverage').first
    coverage.attr("line-rate").to_f
    end
    end
    

    View Slide

29. CI Server
    

    View Slide

30. Grafana
    • Rich metrics dashboard and editor
    • Fast rendering
    • Support InfluxDB as a data source
    • Query editor
    

    View Slide

31. Visualize Code Coverage
    

    View Slide

32. Other metrics
    

    View Slide

33. Working at Cookpad is …
    • Working in a large team
    • Working with a large code base
    

    View Slide

34. Let’s improve the registration
    screen…
    

    View Slide

35. Let’s improve the registration
    screen…
    • Implement
    

    View Slide

36. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints
    

    View Slide

37. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints
    • Test with older iOS
    versions
    

    View Slide

38. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints
    • Test with older iOS
    versions
    • Fix tests
    

    View Slide

39. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints
    • Test with older iOS
    versions
    • Fix tests
    • Wait for CI
    

    View Slide

40. How many times 

    did you build so far?
    

    View Slide

41. Reduce Build Time
    • Measurement
    • Improvement
    

    View Slide

42. What I will “not” talk about
    • Annotate all types explicitly
    • Write as much as code in a single file
    • Get your Mac Pro
    

    View Slide

43. Reduce Build Time
    • Measurement
    • Improvement
    

    View Slide

44. “Premature optimization is
    the root of all evil.”

    

    — Donald Knuth —
    

    View Slide

45. Display build duration
    $ defaults write com.apple.dt.Xcode ShowBuildOperationDuration YES
    

    View Slide

46. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Run Scripts (Linter, Formatter)
    

    View Slide

47. fastlane/report.xml
    
    
    
    
    
    time="8.59854">
    
    
    
    time="0.001982">
    
    …
    
    
    
    
    
    
    

    View Slide

48. View Slide

49. View Slide

50. View Slide

51. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Run Scripts (Linter, Formatter)
    

    View Slide

52. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Function A
    • Function B
    • …
    • Runs Scripts (Linter, Formatter)
    

    View Slide

53. Warn long function bodies
    

    View Slide

54. Time function bodies
    $ pbpaste | grep '.[0-9]ms' | sort -nr | head -20
    

    View Slide

55. xcactivitylog
    • You can access build logs under DerivedData
    • xcactivitylog is just gzip. You can check the
    content as a plain text just by unzipping it
    

    View Slide

56. giginet/danger-xcprofiler
    (in Dangerfile)

    

    xcprofiler.thresholds = {
    warn: 50,
    fail: 500
    }
    xcprofiler.report ‘ProductName'
    

    View Slide

57. Reduce Build Time
    • Measurement
    • Improvement
    

    View Slide

58. Test with Real Data
    $ cloc

    

    github.com/AlDanial/cloc v 1.72 T=3.40 s (153.3 files/s, 16646.6 lines/s)
    -------------------------------------------------------------------------------
    Language files blank comment code
    -------------------------------------------------------------------------------
    Swift 497 8944 3122 43831
    JSON 18 0 0 522
    Objective C 3 37 5 166
    C/C++ Header 4 19 6 27
    -------------------------------------------------------------------------------
    SUM: 522 9000 3133 44546
    -------------------------------------------------------------------------------
    

    View Slide

59. Test with Real Data
    • 40,000 lines of Swift code
    • CocoaPods
    • 38 direct dependencies and 60 libraries
    • Linter in build phases
    • CI archives the app and distributes it for internal
    testing
    

    View Slide

60. (Local clean build: 330s)
    

    View Slide

61. View Slide

62. Whole Module Optimization
    

    View Slide

63. WMO without Optimization
    

    View Slide

64. (Local clean build: 160s)
    

    View Slide

65. https://swift.org/blog/whole-module-optimizations/
    

    View Slide

66. Time Function Bodies
    

    View Slide

67. F
    

    View Slide

68. Carthage
    + github "ReactiveX/RxSwift" ~> 3.4
    + github "RxSwiftCommunity/RxRealm"
    + github "realm/realm-cocoa" ~> 2.4
    + github "danielgindi/Charts" ~> 3.0.2
    - pod 'RxSwift', '3.3.1'
    - pod 'RxCocoa', '3.3.1'
    - pod 'RxRealm', '0.5.2'
    - pod 'RealmSwift', '2.4.4'
    Cartfile
    Podfile
    

    View Slide

69. (Local clean build: 68s)
    

    View Slide

70. Do you really need to build
    your entire app?
    

    View Slide

71. View Slide

72. Prototyping with Playground
    • Test your idea in Playground 

    (interface, design, reproducing a bug, etc.)
    • Once your idea becomes concrete, move
    those code into the application project
    

    View Slide

73. Playground
    

    View Slide

74. Playground
    

    View Slide

75. You need “existing” code 

    for prototyping
    class ViewController: UITableViewController {
    let myView = MyView()
    override func viewDidLoad() {
    super.viewDidLoad()
    view.backgroundColor = .myColor
    view.addSubview(myView)
    }
    }
    

    View Slide

76. Code Modularization
    import UIKit
    @testable import YourUIKit
    let color = UIColor.myGreen
    

    View Slide

77. Code Modularization
    Feature A
    Design
    Feature B Feature C
    Network Persistence
    Core
    

    View Slide

78. Microservice Architecture
    https://martinfowler.com/articles/microservices.html
    

    View Slide

79. Benefits
    • Strict access control
    • Faster incremental build/indexing
    • Clear separation
    

    View Slide

80. Repository Strategy
    • Single Repository
    • Multiple Repositories
    

    View Slide

81. Single repository
    • Simple workflow
    • Can’t open-source
    

    View Slide

82. Multiple Repositories
    • Complex workflow
    • Code Review Process
    • Easy to open-source
    

    View Slide

83. Dependency Management
    • CocoaPods
    • git submodule
    • Carthage
    • git subtree
    

    View Slide

84. CocoaPods
    • Easy in use
    • Black box in nature
    

    View Slide

85. git submodule
    • Simple (!= Easy)
    • Full control
    • Manual works
    • Complex workflow
    

    View Slide

86. Carthage
    • Flexible
    • Pre-build
    • Not all libraries support it
    • ABI stability
    

    View Slide

87. https://github.com/apple/swift-evolution#development-major-version--swift-50
    

    View Slide

88. git subtree
    • An alternative to git submodule
    • Clones a subproject and merge it into the
    parent project
    • `git subtree push` goes through the commits
    and picking the changes that should go to a
    subproject
    

    View Slide

89. git subtree
    

    View Slide

90. git subtree
    • Requires no change to workflow
    • Need to learn a new merge strategy
    • Contributing back to upstream is a bit tricky
    

    View Slide

91. Lesson Learned
    • Breaking existing implicit dependencies is hard
    • Starting from the single repository
    • Once modules become stable, consider
    • managing them in separated repositories
    • using pre-build versions
    

    View Slide

92. Case Study:
    Single repo + CocoaPods
    

    View Slide

93. Directory Structure
    .
    "## MainApp
    $ "## MainApp
    $ $ "## AppDelegate.swift
    $ $ "## Info.plist
    $ $ # ViewController.swift
    $ "## MainApp.xcodeproj
    $ # MainAppTests
    $ "## Info.plist
    $ # MainAppTests.swift
    "## Library
    $ "## Library
    $ $ "## Info.plist
    $ $ "## Library.h
    $ $ # Library.swift
    $ "## Library.xcodeproj
    $ # LibraryTests
    $ "## Info.plist
    $ # LibraryTests.swift
    "## MainApp.xcworkspace
    "## Podfile
    "## Podfile.lock
    "## Pods
    # README.md
    

    View Slide

94. Podfile
    source 'https://github.com/CocoaPods/Specs.git'
    use_frameworks!
    platform :ios, '9.3'
    workspace 'MainApp.xcworkspace'
    target 'MainApp' do
    project 'MainApp/MainApp.xcodeproj'
    # Dependencies of the MainApp
    end
    target 'Library' do
    project 'Library/Library.xcodeproj'
    # Dependencies of the Library
    end
    

    View Slide

95. Workspace Structure
    

    View Slide

96. Link it!
    

    View Slide

97. Find Implicit Dependencies
    

    View Slide

98. Pitfall
    

    View Slide

99. dylib Loading Time
    Total pre-main time: 4.2 seconds (100.0%)
    dylib loading time: 4.0 seconds (94.4%)
    rebase/binding time: 93.32 milliseconds (2.1%)
    ObjC setup time: 44.64 milliseconds (1.0%)
    initializer time: 98.30 milliseconds (2.3%)
    

    View Slide

100. Pitfall
     

     View Slide

101. Xcode 9 supports
     Swift Static Library
     https://download.developer.apple.com/Developer_Tools/Xcode_9_beta_4/
     Xcode_9_beta_4_Release_Notes.pdf
     

     View Slide

102. Recap
     • Automate daily workflows with CI and Fastlane
     • Reduce workload of code review with Danger
     • Visualize code metrics you want to improve
     • Revising project settings may improve build time
     dramatically
     • Code modularization improves build/index time and
     helps you getting well structured code/teams
     

     View Slide

103. The presentation is available on

     
     https://speakerdeck.com/yuseinishiyama/building-ios-apps-at-scale
     

     View Slide

104. Thank you!
     Yusei Nishiyama
     @yuseinishiyama
     

     View Slide