Slide 1

Slide 1 text

Postmortem for switching Lyft's iOS app to Bazel

Slide 2

Slide 2 text

Lyft - 70+ iOS engineers - 2 iOS tooling engineers - 800k+ lines of Swift, 0 lines of Objective-C - 5500+ bazel targets

Slide 3

Slide 3 text

Why switch?

Slide 4

Slide 4 text

Standard iOS tooling

Slide 5

Slide 5 text

Scalable, flexible, open source

Slide 6

Slide 6 text

Shared tooling

Slide 7

Slide 7 text

Remote caching / execution

Slide 8

Slide 8 text

How we migrated

Slide 9

Slide 9 text

1. Lyft.xcodeproj

Slide 10

Slide 10 text

2. Reign in your build

Slide 11

Slide 11 text

3. Monorepo

Slide 12

Slide 12 text

4. Ramp up with bazel

Slide 13

Slide 13 text

5. Prototype your build

Slide 14

Slide 14 text

6. Build it for real

Slide 15

Slide 15 text

7. Some IDE integration

Slide 16

Slide 16 text

8. Documentation

Slide 17

Slide 17 text

9.

Slide 18

Slide 18 text

10. Iterate

Slide 19

Slide 19 text

What went well

Slide 20

Slide 20 text

Iterating with starlark

Slide 21

Slide 21 text

Simple project means simpler migration

Slide 22

Slide 22 text

No forks

Slide 23

Slide 23 text

Staying up to date

Slide 24

Slide 24 text

Keeping up with the community

Slide 25

Slide 25 text

What could have gone better

Slide 26

Slide 26 text

Disclaimer: bazel is great

Slide 27

Slide 27 text

Adoption cost is very high

Slide 28

Slide 28 text

Google's defaults aren't always right for the iOS community

Slide 29

Slide 29 text

Transitive dependency management

Slide 30

Slide 30 text

Stay up to date!

Slide 31

Slide 31 text

Hermiticity is hard.

Slide 32

Slide 32 text

iOS is second class

Slide 33

Slide 33 text

❤ Bazel iOS

Slide 34

Slide 34 text

IDE Integration

Slide 35

Slide 35 text

Xcode Bazel Integration ● Tulsi – Google ● XCHammer – Pinterest

Slide 36

Slide 36 text

Like Google and Pinterest, we made our own

Slide 37

Slide 37 text

Generating Projects

Slide 38

Slide 38 text

Generating Projects gen_project(b: Bazel) -> XcodeProject

Slide 39

Slide 39 text

Generating Projects gen_project(b: Bazel) -> XcodeProject

Slide 40

Slide 40 text

Exporting Bazel Projects ● bazel build --nobuild --aspects ● bazel query --output=xml ● bazel cquery --output=proto

Slide 41

Slide 41 text

Exporting Bazel Projects ● bazel build --nobuild --aspects ● bazel query --output=xml ● bazel cquery --output=proto

Slide 42

Slide 42 text

Generating Projects gen_project(Bazel) -> XcodeProject

Slide 43

Slide 43 text

Generating Projects XcodeProject[XcodeBuild] XcodeProject[BazelBuild]

Slide 44

Slide 44 text

Generating Projects XcodeProject[XcodeBuild] XcodeProject[BazelBuild]

Slide 45

Slide 45 text

Incremental Migration 1A. Convert to Bazel as the build source of truth and build on CI 1B. Generate Xcode projects from Bazel, and build with Xcode

Slide 46

Slide 46 text

Generating Projects XcodeProject[XcodeBuild] XcodeProject[BazelBuild]

Slide 47

Slide 47 text

This is two operations: add Bazel, also remove xcodebuild

Slide 48

Slide 48 text

Integrating bazel

Slide 49

Slide 49 text

Integrating bazel in Xcode ● "External Build System" target

Slide 50

Slide 50 text

Integrating bazel in Xcode ● "External Build System" target ● "Run Script" phase

Slide 51

Slide 51 text

Running & Testing

Slide 52

Slide 52 text

Running & Testing ● Copy .app or .xctest to Xcode DerivedData (ie "bazel-out") ● Xcode does the rest, which helps preserves many IDE features

Slide 53

Slide 53 text

More to Integrating bazel in Xcode ● Installing apps and tests ● Appease the Xcode build, by copying build artifacts ● Supporting the IDE by importing build outputs ● Generating debugger settings

Slide 54

Slide 54 text

Basically, an ad-hoc, informally specified, installer

Slide 55

Slide 55 text

Hyrum's Law "With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody." - Hyrum Wright, Googler

Slide 56

Slide 56 text

Decoupling xcodebuild

Slide 57

Slide 57 text

Decoupling xcodebuild ● CC=/usr/bin/true ● LD=/usr/bin/true ● LIBTOOL=/usr/bin/true ● SWIFT_EXEC=/usr/bin/true

Slide 58

Slide 58 text

Decoupling xcodebuild ● CC=/usr/bin/true ● LD=/usr/bin/true ● LIBTOOL=/usr/bin/true ● SWIFT_EXEC=./tools/swiftc-stub.sh

Slide 59

Slide 59 text

Indexing

Slide 60

Slide 60 text

Indexing Facts of Life ● Backbone of important IDE features ● Common developer slowdown ● Usually proprietary ● Out of band

Slide 61

Slide 61 text

Indexing Integration ● swiftc & clang can produce indexes while building ● Indexes embed some absolute paths ● swiftc & clang and their component libraries are open source ● index-import: Purpose built tool for installing indexes into Xcode ● https://github.com/lyft/index-import

Slide 62

Slide 62 text

Debugging

Slide 63

Slide 63 text

Debugging, with lldb ● Paths have been at the core of every challenge ● Source paths, binary paths, absolute paths, relative paths, project paths, build paths ● Fix by removing paths, or making them relative

Slide 64

Slide 64 text

Debugging, with lldb ● Compiler flags, with the help of Bazel wrapped clang, rules_swift worker ● Linking locally to ensure the binary has local paths ● Empty swift module, locally compiled and bearing embedded debugging options ● Generating lldb settings and wiring them into developer's global settings

Slide 65

Slide 65 text

Finally, everything works

Slide 66

Slide 66 text

Stepping Back

Slide 67

Slide 67 text

Stepping Back 1. Build with Bazel 2. Customizable project generation 3. Taken the build away from Xcode

Slide 68

Slide 68 text

Focused Subprojects

Slide 69

Slide 69 text

Pick any targets, dependencies not required

Slide 70

Slide 70 text

Focused Subprojects ● Generating projects is faster ● Launching Xcode and time to usability is faster ● Provide natural scopes for teams/projects ● Performance scales by developer scope, not repo

Slide 71

Slide 71 text

IDE Finer Details

Slide 72

Slide 72 text

IDE Finer Details ● Native Xcode test experience ● Augmented project search ● Compiler fixits ● Full index, not focused index ● Refactoring ● Remote cache!

Slide 73

Slide 73 text

Unresolved

Slide 74

Slide 74 text

Unresolved ● Progress bar is useless, build log looks different ● No background indexing ● bazel×swiftc parallelism mismatch ● Code coverage features don't work

Slide 75

Slide 75 text

Questions! @SmileyKeith, @kastiglione