Postmortem for switching Lyft's iOS app to Bazel

Details of how we migrated Lyft's iOS codebase to build entirely with bazel presented at BazelCon 2019

Transcript

1. Postmortem for switching Lyft's iOS app to Bazel
2. Lyft - 70+ iOS engineers - 2 iOS tooling engineers - 800k+ lines of Swift, 0 lines of Objective-C - 5500+ bazel targets
3. Why switch?
4. Standard iOS tooling
5. Scalable, flexible, open source
6. Shared tooling
7. Remote caching / execution
8. How we migrated
9. 1. Lyft.xcodeproj
10. 2. Reign in your build
11. 3. Monorepo
12. 4. Ramp up with bazel
13. 5. Prototype your build
14. 6. Build it for real
15. 7. Some IDE integration
16. 8. Documentation
17. 9.
18. 10. Iterate
19. What went well
20. Iterating with starlark
21. Simple project means simpler migration
22. No forks
23. Staying up to date
24. Keeping up with the community
25. What could have gone better
26. Disclaimer: bazel is great
27. Adoption cost is very high
28. Google's defaults aren't always right for the iOS community
29. Transitive dependency management
30. Stay up to date!
31. Hermiticity is hard.
32. iOS is second class
33. ❤ Bazel iOS
34. IDE Integration
35. Xcode Bazel Integration • Tulsi – Google • XCHammer – Pinterest
36. Like Google and Pinterest, we made our own
37. Generating Projects
38. Generating Projects gen_project(b: Bazel) -> XcodeProject
39. Generating Projects gen_project(b: Bazel) -> XcodeProject
40. Exporting Bazel Projects • bazel build --nobuild --aspects <aspect> • bazel query --output=xml <query> • bazel cquery --output=proto <query>
41. Exporting Bazel Projects • bazel build --nobuild --aspects <aspect> • bazel query --output=xml <query> • bazel cquery --output=proto <query>
42. Generating Projects gen_project(Bazel) -> XcodeProject
43. Generating Projects XcodeProject[XcodeBuild] XcodeProject[BazelBuild]
44. Generating Projects XcodeProject[XcodeBuild] XcodeProject[BazelBuild]
45. Incremental Migration 1A. Convert to Bazel as the build source of truth and build on CI 1B. Generate Xcode projects from Bazel, and build with Xcode
46. Generating Projects XcodeProject[XcodeBuild] XcodeProject[BazelBuild]
47. This is two operations: add Bazel, also remove xcodebuild
48. Integrating bazel
49. Integrating bazel in Xcode • "External Build System" target
50. Integrating bazel in Xcode • "External Build System" target • "Run Script" phase
51. Running & Testing
52. Running & Testing • Copy .app or .xctest to Xcode DerivedData (ie "bazel-out") • Xcode does the rest, which helps preserves many IDE features
53. More to Integrating bazel in Xcode • Installing apps and tests • Appease the Xcode build, by copying build artifacts • Supporting the IDE by importing build outputs • Generating debugger settings
54. Basically, an ad-hoc, informally specified, installer
55. Hyrum's Law "With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody." - Hyrum Wright, Googler
56. Decoupling xcodebuild
57. Decoupling xcodebuild • CC=/usr/bin/true • LD=/usr/bin/true • LIBTOOL=/usr/bin/true • SWIFT_EXEC=/usr/bin/true
58. Decoupling xcodebuild • CC=/usr/bin/true • LD=/usr/bin/true • LIBTOOL=/usr/bin/true • SWIFT_EXEC=./tools/swiftc-stub.sh
59. Indexing
60. Indexing Facts of Life • Backbone of important IDE features • Common developer slowdown • Usually proprietary • Out of band
61. Indexing Integration • swiftc & clang can produce indexes while building • Indexes embed some absolute paths • swiftc & clang and their component libraries are open source • index-import: Purpose built tool for installing indexes into Xcode • https://github.com/lyft/index-import
62. Debugging
63. Debugging, with lldb • Paths have been at the core of every challenge • Source paths, binary paths, absolute paths, relative paths, project paths, build paths • Fix by removing paths, or making them relative
64. Debugging, with lldb • Compiler flags, with the help of Bazel wrapped clang, rules_swift worker • Linking locally to ensure the binary has local paths • Empty swift module, locally compiled and bearing embedded debugging options • Generating lldb settings and wiring them into developer's global settings
65. Finally, everything works
66. Stepping Back
67. Stepping Back 1. Build with Bazel 2. Customizable project generation 3. Taken the build away from Xcode
68. Focused Subprojects
69. Pick any targets, dependencies not required
70. Focused Subprojects • Generating projects is faster • Launching Xcode and time to usability is faster • Provide natural scopes for teams/projects • Performance scales by developer scope, not repo
71. IDE Finer Details
72. IDE Finer Details • Native Xcode test experience • Augmented project search • Compiler fixits • Full index, not focused index • Refactoring • Remote cache!
73. Unresolved
74. Unresolved • Progress bar is useless, build log looks different • No background indexing • bazel×swiftc parallelism mismatch • Code coverage features don't work
75. Questions! @SmileyKeith, @kastiglione