Story of Migrating a Large iOS Codebase to Bazel

Thi Doan
LINE App Dev Team4 Software Engineer
https://linedevday.linecorp.com/2020/ja/sessions/6718
https://linedevday.linecorp.com/2020/en/sessions/6718

Transcript

2. Agenda › Why Migrate? › Migration Best Practices › Bazel for iOS – Tips & Tricks
3. Why Migrate to Bazel? Services growing led to project growing › LINE iOS has inevitable grown to a huge codebase over the years
4. Why Migrate to Bazel? › Build times increased for all developers Developer productivity degraded Services growing led to project growing › LINE iOS has inevitable grown to a huge codebase over the years
5. Why Migrate to Bazel? › Build times increased for all developers Developer productivity degraded Services growing led to project growing › LINE iOS has inevitable grown to a huge codebase over the years Bazel › Fast and reproducible › Advanced caching
6. Why Migrate to Bazel? › Build times increased for all developers Developer productivity degraded Services growing led to project growing › LINE iOS has inevitable grown to a huge codebase over the years Bazel › Fast and reproducible › Advanced caching Growing LINE for iOS LINE DEVELOPER DAY 2019 Faster iOS Builds With Bazel LINE DEVELOPER DAY 2019
7. 0 5 10 15 20 25 30 April 16 April 23 May 1 May 8 May 16 May 24 June 1 June 8 June 16 Build Times https://engineering.linecorp.com/en/blog/improving-build-performance-line-ios-bazel
8. Migration Approaches
9. Keep Project Structure Simple › Helps reduce complexity of synchronizing between Xcode and Bazel
10. Keep Project Structure Simple › Helps reduce complexity of synchronizing between Xcode and Bazel › One module per directory, named after module name, all flatten
11. Keep Project Structure Simple › Helps reduce complexity of synchronizing between Xcode and Bazel › One module per directory, named after module name, all flatten › Bonus: Swift generated headers get into search paths for free
12. Support Both Xcode or Bazel – Choose One
13. Support Both Xcode or Bazel – Choose One › Helps measure both build systems side by side
14. Support Both Xcode or Bazel – Choose One › Easier to try new Xcode features › Helps measure both build systems side by side
15. Support Both Xcode or Bazel – Choose One › Easier to try new Xcode features › Xcode still provides better debugging experience › Helps measure both build systems side by side
16. Tips & Tricks
17. Avoid Mix and Match Mixed Objective-C and Swift modules lead to many problems
18. Avoid Mix and Match Mixed Objective-C and Swift modules lead to many problems May slow down your builds › Changes to code in a language may trigger a full rebuild of the other
19. Avoid Mix and Match Mixed Objective-C and Swift modules lead to many problems › Needs a custom rule › In case you need one: https://github.com/line/rules_apple_line No official supports from Bazel May slow down your builds › Changes to code in a language may trigger a full rebuild of the other
20. Avoid Mix and Match Mixed Objective-C and Swift modules lead to many problems › Needs a custom rule › In case you need one: https://github.com/line/rules_apple_line No official supports from Bazel Clang modules can’t be cached by Bazel › Module maps rely on implicit module caching that clashes with Bazel’s hermeticity model › Slow down Obj-C builds on remote execution May slow down your builds › Changes to code in a language may trigger a full rebuild of the other
21. Provide Faster Remote Cache Provide a way to disable remote cache › Remote cache may be slower than building locally depending on network speed › Most developers work from home now, not everyone has a fast connection
22. Provide Faster Remote Cache › If VPN is a bottleneck, provide remote cache without VPN › If you have a distributed team, distribute remote cache via CDN Provide faster remote cache Provide a way to disable remote cache › Remote cache may be slower than building locally depending on network speed › Most developers work from home now, not everyone has a fast connection
23. Optimize for Reproducible Builds Identify cache misses › More reproducible builds = Less cache misses
24. Optimize for Reproducible Builds Identify cache misses # Re-run the builds without caching options to obtain execution logs $ bazel clean $ bazel build --disk_cache= --remote_cache= --execution_log_json_file=exec1.json //your:target $ bazel clean $ bazel build --disk_cache= --remote_cache= --execution_log_json_file=exec1.json //your:target # Compare the execution logs between two runs $ diff -u exec1.json exec2.json › More reproducible builds = Less cache misses › Re-run the builds w/o cache to identify non-reproducible actions
25. Optimize for Reproducible Builds Reproducible object files build --swiftcopt=-Xfrontend build --swiftcopt=-no-clang-module-breadcrumbs › Remove Clang module breadcrumbs from debug information
26. Optimize for Reproducible Builds Reproducible object files build --swiftcopt=-Xfrontend build --swiftcopt=-no-clang-module-breadcrumbs # https://github.com/bazelbuild/rules_swift/pull/456 build --features=swift.deterministic_module_breadcrumbs › Remove Clang module breadcrumbs from debug information › Make Clang module breadcrumbs deterministic
27. Bazel’s default settings for optimized builds produce larger binary size compared to Xcode’s. Binary Size build:release --features=dead_strip build:release --features=swift.opt_uses_osize build:release --objc_enable_binary_stripping build:release --objccopt=-Oz
28. Apply LTO flags to release builds Binary Size build:release --features=dead_strip build:release --features=swift.opt_uses_osize build:release --objc_enable_binary_stripping build:release --objccopt=-Oz build:release --objccopt=-flto build:release --linkopt=-flto
29. Apply LTO flags to release builds only Binary Size build:release --features=dead_strip build:release --features=swift.opt_uses_osize build:release --objc_enable_binary_stripping build:release --objccopt=-Oz build:release --objccopt=-flto build:release --linkopt=-flto
30. Conclusion
31. Conclusion › Simple structure
32. Conclusion › Simple structure › Support both
33. Conclusion › Simple structure › Support both › Avoid mix and match
34. Conclusion › Simple structure › Support both › Avoid mix and match › Provide faster cache
35. Conclusion › Simple structure › Support both › Avoid mix and match › Provide faster cache › Optimize for reproducible builds
36. Conclusion › Simple structure › Support both › Avoid mix and match › Provide faster cache › Optimize for reproducible builds › Keep binary size manageable
37. Thank you