Reduce Build Times and Get Home Eariler

Reduce Build Times and Get Home Eariler

Swift is a robust language, but at the same time, it means that many things are done at build time. Swift is now the standard for iOS development, but as a result, we suffer longer builds and it deprives us of our precious time. Time is more valuable than anything.

In this session, I will show you how to improve Swift’s build times and gain productivity in three parts:

- Measurement
- Improvement
- Avoid building all together!

562e29ba057361b2b944bd7bbd274887?s=128

Yusei Nishiyama

April 20, 2017
Tweet

Transcript

  1. Reduce Build Times and Get Home Earlier Yusei Nishiyama @yuseinishiyama

  2. • iOS, Android, Web • 63 countries • 17 languages

    • 63M+ MAU (Japan) • 35M+ MAU (Outside Japan) https://info.cookpad.com/en https://cookpad.com/en
  3. Distributed Team

  4. Try Error

  5. Try Error

  6. Let’s improve the registration screen…

  7. Let’s improve the registration screen… • Implement

  8. Let’s improve the registration screen… • Implement • Tweak constraints

  9. Let’s improve the registration screen… • Implement • Tweak constraints

    • Test with older iOS versions
  10. Let’s improve the registration screen… • Implement • Tweak constraints

    • Test with older iOS versions • Fix tests
  11. Let’s improve the registration screen… • Implement • Tweak constraints

    • Test with older iOS versions • Fix tests • Wait for CI
  12. How many times 
 did you build so far?

  13. None
  14. None
  15. None
  16. You are not efficient

  17. None
  18. ?

  19. 3 steps to improve • Measurement • Visualization • Improvement

  20. What I will “not” talk about • Annotate all types

    explicitly • Write as much as code in a single file • Get your Mac Pro
  21. 3 steps to improve • Measurement • Visualization • Improvement

  22. “Premature optimization is the root of all evil.”
 
 —

    Donald Knuth —
  23. Display build duration $ defaults write com.apple.dt.Xcode ShowBuildOperationDuration YES

  24. Build Process • Dependency Manager (CocoaPods, Carthage) • Compile source

    files • Run Scripts (Linter, Formatter)
  25. cookpad/dokumi • Open source • Advanced Lint tool • Comments

    issues on a pull request • Runs on CI server
  26. Violated linter rules Failed tests

  27. require 'benchmark'
 
 def measure(*args, &block) start_time = Time.now tms

    = Benchmark.measure(args, &block) @data << ExecutionLog.new(args, tms, start_time) end Benchmark
  28. Build Process • Dependency Manager (CocoaPods, Carthage) • Compile source

    files • Runs Scripts (Linter, Formatter)
  29. Build Process • Dependency Manager (CocoaPods, Carthage) • Compile source

    files • Function A • Function B • … • Runs Scripts (Linter, Formatter)
  30. Warn long function bodies

  31. Time function bodies $ pbpaste | grep '.[0-9]ms' | sort

    -nr | head -20
  32. RobertGummesson/ BuildTimeAnalyzer-for-Xcode

  33. Is that data easy to manipulate? • Integrate data collection

    into our build process • Flexible format which doesn’t not restrict how we use it
  34. giginet/xcprofiler • A CLI tool to time body functions •

    Easy to integrate into build process • Flexible output (stdout, JSON, Custom)
  35. None
  36. require 'xcprofiler' profiler = Xcprofiler::Profiler.by_product_name('MyApp') profiler.reporters = [ Xcprofiler::StandardOutputReporter.new(limit: 20,

    order: :time), Xcprofiler::JSONReporter.new(output_path: 'result.json'), Xcprofiler::BlockReporter.new do |executions| do_something(executions) end, ] profiler.report! Custom Reporter
  37. xactivitylog • You can access build logs under DerivedData •

    xactivitylog is just gzip. You can check the content as a plain text just by unzipping it
  38. 3 steps to improve • Measurement • Visualization • Improvement

  39. CI Server cookpad/dokumi

  40. CI Server cookpad/dokumi

  41. CI Server cookpad/dokumi

  42. CI Server cookpad/dokumi

  43. CI Server cookpad/dokumi

  44. InfluxDB • Only purpose-built platform for any time-based data •

    Written in Go • Built-in HTTP API • SQL-like query language • Answer queries in real-time • Rich client libraries
  45. CI Server cookpad/dokumi

  46. Grafana • Rich metrics dashboard and editor • Fast rendering

    • Support InfluxDB as a data source • Query editor
  47. Build time visualization

  48. Other metrics

  49. 3 steps to improve • Measurement • Visualization • Improvement

  50. Test with Real Data $ cloc
 
 github.com/AlDanial/cloc v 1.72

    T=3.40 s (153.3 files/s, 16646.6 lines/s) ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- Swift 497 8944 3122 43831 JSON 18 0 0 522 Objective C 3 37 5 166 C/C++ Header 4 19 6 27 ------------------------------------------------------------------------------- SUM: 522 9000 3133 44546 -------------------------------------------------------------------------------
  51. Test with Real Data • 40,000 lines of Swift code

    • CocoaPods • 38 direct dependencies and 60 libraries • Linter in build phases • CI archives the app and distributes it for internal testing
  52. Machine specs • CI • Mac mini (Late 2014) •

    3 GHz Intel Core i7 • Memory 16GB • 256GB SSD • Local • MBP (Retina, 15-inch, Late 2013) • 2.3 GHz Intel Core i7 • Memory 16 GB • 256GB SSD
  53. (Local clean build: 330s)

  54. None
  55. Whole Module Optimization

  56. WMO without Optimization

  57. (Local clean build: 160s)

  58. https://swift.org/blog/whole-module-optimizations/

  59. post_install do |installer| installer.pods_project.targets.each do |target| target.build_configurations.each do |config| config.build_settings['SWIFT_OPTIMIZATION_LEVEL']

    = '-Owholemodule' unless config.name == 'Release' other_swift_flags = config.build_settings['OTHER_SWIFT_FLAGS'] || ['$(inherited)'] other_swift_flags << '-Onone' config.build_settings['OTHER_SWIFT_FLAGS'] = other_swift_flags end end end end
  60. (Local clean build: 118s)

  61. Time Function Bodies

  62. None
  63. post_install do |installer| installer.pods_project.targets.each do |target| target.build_configurations.each do |config| config.build_settings['SWIFT_OPTIMIZATION_LEVEL']

    = '-Owholemodule' unless config.name == 'Release' other_swift_flags = config.build_settings['OTHER_SWIFT_FLAGS'] || ['$(inherited)'] other_swift_flags << '-Onone' other_swift_flags << '-Xfrontend' other_swift_flags << '-debug-time-function-bodies' config.build_settings['OTHER_SWIFT_FLAGS'] = other_swift_flags end end end end
  64. (Local clean build: 130s)

  65. F

  66. Carthage + github "ReactiveX/RxSwift" ~> 3.4 + github "RxSwiftCommunity/RxRealm" +

    github "realm/realm-cocoa" ~> 2.4 + github "danielgindi/Charts" ~> 3.0.2 - pod 'RxSwift', '3.3.1' - pod 'RxCocoa', '3.3.1' - pod 'RxRealm', '0.5.2' - pod 'RealmSwift', '2.4.4' Cartfile Podfile
  67. None
  68. None
  69. (Local clean build: 68s)

  70. Partial Lint lint() { while read filename; do if [[

    "${filename##*.}" == "swift" ]]; then ${SWIFT_LINT} lint --path "${filename}" --quiet fi done } FORK_POINT=`git merge-base --fork-point origin/develop` git diff $FORK_POINT --name-only | lint
  71. (Local clean build: 55s)

  72. Do you really need to build your entire app?

  73. None
  74. Playground

  75. Playground

  76. Real World Prototyping class ViewController: UITableViewController { let myView =

    MyView() override func viewDidLoad() { super.viewDidLoad() view.backgroundColor = .myColor view.addSubview(myView) } }
  77. You need “existing” code for prototyping

  78. Embedded Playground

  79. None
  80. None
  81. None
  82. None
  83. Framework for Playground

  84. None
  85. None
  86. None
  87. None
  88. None
  89. How to use it in Playground import UIKit @testable import

    MyPlayground let color = UIColor.myGreen
  90. None
  91. Code Modularization import UIKit @testable import YourUIKit let color =

    UIColor.myGreen
  92. Framework Oriented Architecture Feature A Design Feature B Feature C

    Network Persistence Core
  93. Microservice Architecture https://martinfowler.com/articles/microservices.html

  94. Easy to prototype

  95. High testability

  96. Autonomous Teams

  97. Independent Technology Stack 
 (Swift, Objc, React Native, Flux)

  98. Complex Workflow

  99. Dynamic Frameworks make your app slow to launch

  100. dylib Loading Time Total pre-main time: 4.2 seconds (100.0%) dylib

    loading time: 4.0 seconds (94.4%) rebase/binding time: 93.32 milliseconds (2.1%) ObjC setup time: 44.64 milliseconds (1.0%) initializer time: 98.30 milliseconds (2.3%)
  101. WWDC16 406 
 “Optimizing App Startup Time"

  102. Recap • You can get detailed build time information •

    Visualization helps you to identify the bottleneck • You can improve build time in many ways (WMO and Carthage are highly effective) • Modularization makes your prototyping easier
  103. References • http://techlife.cookpad.com/entry/2015/06/04/dokumi-en • http://developear.com/blog/2016/12/30/Speed-Swift- Compilation.html • https://swift.org/blog/whole-module-optimizations/ • https://medium.com/@LogMaestro/adding-playgrounds-to-

    your-xcode-project-79d5ea0c7087 • http://frameworkoriented.io/ • https://developer.apple.com/videos/play/wwdc2016/406/
  104. I’ll summarize this talk on our engineering blog
 https://sourcediving.com/

  105. The presentation is available on
 https://speakerdeck.com/yuseinishiyama/reduce-build-times- and-get-home-eariler

  106. Any Question?