Reduce Build Times and Get Home Eariler

Swift is a robust language, but at the same time, it means that many things are done at build time. Swift is now the standard for iOS development, but as a result, we suffer longer builds and it deprives us of our precious time. Time is more valuable than anything.

In this session, I will show you how to improve Swift’s build times and gain productivity in three parts:

- Measurement
- Improvement
- Avoid building all together!

Transcript

1. Reduce Build Times
   and Get Home Earlier
   Yusei Nishiyama
   @yuseinishiyama
   

   View Slide

2. • iOS, Android, Web
   • 63 countries
   • 17 languages
   • 63M+ MAU (Japan)
   • 35M+ MAU (Outside Japan)
   https://info.cookpad.com/en
   https://cookpad.com/en
   

   View Slide

3. Distributed Team
   

   View Slide

4. Try
   Error
   

   View Slide

5. Try
   Error
   

   View Slide

6. Let’s improve the registration
   screen…
   

   View Slide

7. Let’s improve the registration
   screen…
   • Implement
   

   View Slide

8. Let’s improve the registration
   screen…
   • Implement
   • Tweak constraints
   

   View Slide

9. Let’s improve the registration
   screen…
   • Implement
   • Tweak constraints
   • Test with older iOS
   versions
   

   View Slide

10. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints
    • Test with older iOS
    versions
    • Fix tests
    

    View Slide

11. Let’s improve the registration
    screen…
    • Implement
    • Tweak constraints
    • Test with older iOS
    versions
    • Fix tests
    • Wait for CI
    

    View Slide

12. How many times 

    did you build so far?
    

    View Slide

13. View Slide

14. View Slide

15. View Slide

16. You are not efficient
    

    View Slide

17. View Slide

18. ?
    

    View Slide

19. 3 steps to improve
    • Measurement
    • Visualization
    • Improvement
    

    View Slide

20. What I will “not” talk about
    • Annotate all types explicitly
    • Write as much as code in a single file
    • Get your Mac Pro
    

    View Slide

21. 3 steps to improve
    • Measurement
    • Visualization
    • Improvement
    

    View Slide

22. “Premature optimization is
    the root of all evil.”

    

    — Donald Knuth —
    

    View Slide

23. Display build duration
    $ defaults write com.apple.dt.Xcode ShowBuildOperationDuration YES
    

    View Slide

24. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Run Scripts (Linter, Formatter)
    

    View Slide

25. cookpad/dokumi
    • Open source
    • Advanced Lint tool
    • Comments issues on a pull request
    • Runs on CI server
    

    View Slide

26. Violated linter rules
    Failed tests
    

    View Slide

27. require 'benchmark'

    

    def measure(*args, &block)
    start_time = Time.now
    tms = Benchmark.measure(args, &block)
    @data << ExecutionLog.new(args, tms, start_time)
    end
    Benchmark
    

    View Slide

28. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Runs Scripts (Linter, Formatter)
    

    View Slide

29. Build Process
    • Dependency Manager (CocoaPods, Carthage)
    • Compile source files
    • Function A
    • Function B
    • …
    • Runs Scripts (Linter, Formatter)
    

    View Slide

30. Warn long function bodies
    

    View Slide

31. Time function bodies
    $ pbpaste | grep '.[0-9]ms' | sort -nr | head -20
    

    View Slide

32. RobertGummesson/
    BuildTimeAnalyzer-for-Xcode
    

    View Slide

33. Is that data easy to manipulate?
    • Integrate data collection into our build
    process
    • Flexible format which doesn’t not restrict
    how we use it
    

    View Slide

34. giginet/xcprofiler
    • A CLI tool to time body functions
    • Easy to integrate into build process
    • Flexible output (stdout, JSON, Custom)
    

    View Slide

35. View Slide

36. require 'xcprofiler'
    profiler = Xcprofiler::Profiler.by_product_name('MyApp')
    profiler.reporters = [
    Xcprofiler::StandardOutputReporter.new(limit: 20, order: :time),
    Xcprofiler::JSONReporter.new(output_path: 'result.json'),
    Xcprofiler::BlockReporter.new do |executions|
    do_something(executions)
    end,
    ]
    profiler.report!
    Custom Reporter
    

    View Slide

37. xactivitylog
    • You can access build logs under DerivedData
    • xactivitylog is just gzip. You can check the
    content as a plain text just by unzipping it
    

    View Slide

38. 3 steps to improve
    • Measurement
    • Visualization
    • Improvement
    

    View Slide

39. CI Server
    cookpad/dokumi
    

    View Slide

40. CI Server
    cookpad/dokumi
    

    View Slide

41. CI Server
    cookpad/dokumi
    

    View Slide

42. CI Server
    cookpad/dokumi
    

    View Slide

43. CI Server
    cookpad/dokumi
    

    View Slide

44. InfluxDB
    • Only purpose-built platform for any time-based data
    • Written in Go
    • Built-in HTTP API
    • SQL-like query language
    • Answer queries in real-time
    • Rich client libraries
    

    View Slide

45. CI Server
    cookpad/dokumi
    

    View Slide

46. Grafana
    • Rich metrics dashboard and editor
    • Fast rendering
    • Support InfluxDB as a data source
    • Query editor
    

    View Slide

47. Build time visualization
    

    View Slide

48. Other metrics
    

    View Slide

49. 3 steps to improve
    • Measurement
    • Visualization
    • Improvement
    

    View Slide

50. Test with Real Data
    $ cloc

    

    github.com/AlDanial/cloc v 1.72 T=3.40 s (153.3 files/s, 16646.6 lines/s)
    -------------------------------------------------------------------------------
    Language files blank comment code
    -------------------------------------------------------------------------------
    Swift 497 8944 3122 43831
    JSON 18 0 0 522
    Objective C 3 37 5 166
    C/C++ Header 4 19 6 27
    -------------------------------------------------------------------------------
    SUM: 522 9000 3133 44546
    -------------------------------------------------------------------------------
    

    View Slide

51. Test with Real Data
    • 40,000 lines of Swift code
    • CocoaPods
    • 38 direct dependencies and 60 libraries
    • Linter in build phases
    • CI archives the app and distributes it for internal
    testing
    

    View Slide

52. Machine specs
    • CI
    • Mac mini (Late 2014)
    • 3 GHz Intel Core i7
    • Memory 16GB
    • 256GB SSD
    • Local
    • MBP (Retina, 15-inch, Late 2013)
    • 2.3 GHz Intel Core i7
    • Memory 16 GB
    • 256GB SSD
    

    View Slide

53. (Local clean build: 330s)
    

    View Slide

54. View Slide

55. Whole Module Optimization
    

    View Slide

56. WMO without Optimization
    

    View Slide

57. (Local clean build: 160s)
    

    View Slide

58. https://swift.org/blog/whole-module-optimizations/
    

    View Slide

59. post_install do |installer|
    installer.pods_project.targets.each do |target|
    target.build_configurations.each do |config|
    config.build_settings['SWIFT_OPTIMIZATION_LEVEL'] = '-Owholemodule'
    unless config.name == 'Release'
    other_swift_flags = config.build_settings['OTHER_SWIFT_FLAGS'] || ['$(inherited)']
    other_swift_flags << '-Onone'
    config.build_settings['OTHER_SWIFT_FLAGS'] = other_swift_flags
    end
    end
    end
    end
    

    View Slide

60. (Local clean build: 118s)
    

    View Slide

61. Time Function Bodies
    

    View Slide

62. View Slide

63. post_install do |installer|
    installer.pods_project.targets.each do |target|
    target.build_configurations.each do |config|
    config.build_settings['SWIFT_OPTIMIZATION_LEVEL'] = '-Owholemodule'
    unless config.name == 'Release'
    other_swift_flags = config.build_settings['OTHER_SWIFT_FLAGS'] || ['$(inherited)']
    other_swift_flags << '-Onone'
    other_swift_flags << '-Xfrontend'
    other_swift_flags << '-debug-time-function-bodies'
    config.build_settings['OTHER_SWIFT_FLAGS'] = other_swift_flags
    end
    end
    end
    end
    

    View Slide

64. (Local clean build: 130s)
    

    View Slide

65. F
    

    View Slide

66. Carthage
    + github "ReactiveX/RxSwift" ~> 3.4
    + github "RxSwiftCommunity/RxRealm"
    + github "realm/realm-cocoa" ~> 2.4
    + github "danielgindi/Charts" ~> 3.0.2
    - pod 'RxSwift', '3.3.1'
    - pod 'RxCocoa', '3.3.1'
    - pod 'RxRealm', '0.5.2'
    - pod 'RealmSwift', '2.4.4'
    Cartfile
    Podfile
    

    View Slide

67. View Slide

68. View Slide

69. (Local clean build: 68s)
    

    View Slide

70. Partial Lint
    lint() {
    while read filename; do
    if [[ "${filename##*.}" == "swift" ]]; then
    ${SWIFT_LINT} lint --path "${filename}" --quiet
    fi
    done
    }
    FORK_POINT=`git merge-base --fork-point origin/develop`
    git diff $FORK_POINT --name-only | lint
    

    View Slide

71. (Local clean build: 55s)
    

    View Slide

72. Do you really need to build
    your entire app?
    

    View Slide

73. View Slide

74. Playground
    

    View Slide

75. Playground
    

    View Slide

76. Real World Prototyping
    class ViewController: UITableViewController {
    let myView = MyView()
    override func viewDidLoad() {
    super.viewDidLoad()
    view.backgroundColor = .myColor
    view.addSubview(myView)
    }
    }
    

    View Slide

77. You need “existing” code
    for prototyping
    

    View Slide

78. Embedded Playground
    

    View Slide

79. View Slide

80. View Slide

81. View Slide

82. View Slide

83. Framework for Playground
    

    View Slide

84. View Slide

85. View Slide

86. View Slide

87. View Slide

88. View Slide

89. How to use it in Playground
    import UIKit
    @testable import MyPlayground
    let color = UIColor.myGreen
    

    View Slide

90. View Slide

91. Code Modularization
    import UIKit
    @testable import YourUIKit
    let color = UIColor.myGreen
    

    View Slide

92. Framework Oriented Architecture
    Feature A
    Design
    Feature B Feature C
    Network Persistence
    Core
    

    View Slide

93. Microservice Architecture
    https://martinfowler.com/articles/microservices.html
    

    View Slide

94. Easy to prototype
    

    View Slide

95. High testability
    

    View Slide

96. Autonomous Teams
    

    View Slide

97. Independent Technology Stack 

    (Swift, Objc, React Native, Flux)
    

    View Slide

98. Complex Workflow
    

    View Slide

99. Dynamic Frameworks make
    your app slow to launch
    

    View Slide

100. dylib Loading Time
     Total pre-main time: 4.2 seconds (100.0%)
     dylib loading time: 4.0 seconds (94.4%)
     rebase/binding time: 93.32 milliseconds (2.1%)
     ObjC setup time: 44.64 milliseconds (1.0%)
     initializer time: 98.30 milliseconds (2.3%)
     

     View Slide

101. WWDC16 406 

     “Optimizing App Startup Time"
     

     View Slide

102. Recap
     • You can get detailed build time information
     • Visualization helps you to identify the
     bottleneck
     • You can improve build time in many ways
     (WMO and Carthage are highly effective)
     • Modularization makes your prototyping easier
     

     View Slide

103. References
     • http://techlife.cookpad.com/entry/2015/06/04/dokumi-en
     • http://developear.com/blog/2016/12/30/Speed-Swift-
     Compilation.html
     • https://swift.org/blog/whole-module-optimizations/
     • https://medium.com/@LogMaestro/adding-playgrounds-to-
     your-xcode-project-79d5ea0c7087
     • http://frameworkoriented.io/
     • https://developer.apple.com/videos/play/wwdc2016/406/
     

     View Slide

104. I’ll summarize this talk on
     our engineering blog

     https://sourcediving.com/
     

     View Slide

105. The presentation is available on

     
     https://speakerdeck.com/yuseinishiyama/reduce-build-times-
     and-get-home-eariler
     

     View Slide

106. Any Question?
     

     View Slide