Using Bazel for iOS Development

Once iOS projects get large enough, Xcode and its default build system become a bottleneck. Bazel, a build system by Google, is able to address some of these shortcomings.

Transcript

1. Using Bazel for iOS Development Brentley Jones contact@brentleyjones.com
2. Hi ! I’m Brentley Jones I work on the iOS Platform team at Target I contribute to Open Source when I can (e.g. XcodeGen) I love ! and "
3. What is Bazel? Bazel is an open-source build and test tool similar to Make, Maven, and Gradle. It uses a human-readable, high-level build language. Bazel supports projects in multiple languages and builds outputs for multiple platforms. Bazel supports large codebases across multiple repositories, and large numbers of users. — Bazel documentation
4. Why use Bazel? — Declarative build language — Reliable, correct, and reproducible — Fast — Multiplatform (monorepo ! ) — Very extensible
5. Why use Bazel for iOS development? — Long compile times (Swift ) decreased by caching — Cached test results speed up CI — First class code generation — Hermiticity
6. Target Flagship app development ~40 iOS engineers >500,000 lines of code 90 “modules” (238 targets) CI on every commit of every PR
7. Target Flagship app development
8. How Bazel has helped us Duration 0 2m 4m 6m 8m 10m 12m Classic Bazel
9. Let's write some Bazel . .bazelrc BUILD Resources HelloWorld.png Sources ApplicationDelegate.swift Info.plist Tests Application_Tests.swift WORKSPACE
10. WORKSPACE file The WORKSPACE file defines the root of your workspace Contents declare external dependencies iOS needs the apple_support and rules_apple rules You probably also want rules_swift
11. Minimum Swi! iOS WORKSPACE file load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "build_bazel_apple_support", sha256 = "bdbc3f426be3d0fa6489a3b5cb6b7c1af689215a19bfa1abbaaf3cb3280ed58b", strip_prefix = "apple_support-9605c3da1c5bcdddc20d1704b52415a6f3a5f422", url = "https://github.com/bazelbuild/apple_support/archive/9605c3da1c5bcdddc20d1704b52415a6f3a5f422.tar.gz", ) http_archive( name = "build_bazel_rules_apple", sha256 = "1b1984c6c93b1be3e251bc18be5475d6db65e2c808fa71885ce6f07be530c8b9", strip_prefix = "rules_apple-f6a95e8d0c2bd6fa9f0a6280ef3c4d34c9594513", url = "https://github.com/bazelbuild/rules_apple/archive/f6a95e8d0c2bd6fa9f0a6280ef3c4d34c9594513.tar.gz", ) http_archive( name = "build_bazel_rules_swift", sha256 = "77bdb0a5dbe9a5722b2760ccf6bfd943a739b030de7235bdbbe89f295b551d33", strip_prefix = "rules_swift-69b0848979c51352ef022ba4f312c912c2ffcac1", url = "https://github.com/bazelbuild/rules_swift/archive/69b0848979c51352ef022ba4f312c912c2ffcac1.tar.gz", ) load("@build_bazel_apple_support//lib:repositories.bzl", "apple_support_dependencies") load("@build_bazel_rules_apple//apple:repositories.bzl", "apple_rules_dependencies") load("@build_bazel_rules_swift//swift:repositories.bzl", "swift_rules_dependencies") load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") apple_support_dependencies() apple_rules_dependencies() swift_rules_dependencies() protobuf_deps()
12. BUILD files BUILD files define packages Their path relative to the WORKSPACE defines their name: A package defined at ./X/Y/BUILD has the name X/Y Contents declare rules, a type of target
13. Targets The elements of a package are called targets Targets are primarily of two types: files and rules All files in the directory of/below the package's BUILD, excluding those in sub-packages, are contained in the package Target labels include package names: App defined in ./X/Y/BUILD has the label //X/Y:App
14. Labels Targets are referenced via labels Canonical form: @myrepo//my/app/main:main Label in the same repo: //my/app/main:main Target name == last component of the package name: //my/app/main
15. Relative labels Inside //my/app:BUILD all of these are the same: //my/app:app //my/app :app app
16. Minimum iOS Application BUILD file load("@build_bazel_rules_apple//apple:ios.bzl", "ios_application") load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library") swift_library( name = "Application.library", data = glob(["Resources/**"]), srcs = glob(["Sources/**/*.swift"]), ) ios_application( name = "Application", bundle_id = "com.example.Application", families = ["iphone"], infoplists = ["Sources/Info.plist"], minimum_os_version = "13.0", deps = [":Application.library"], ) # TODO: Tests !
17. Building bazel build //:Application bazel run --ios_simulator_version=13.3 --ios_simulator_device="iPhone 11" //:Application bazel test --ios_simulator_version=13.3 //:Tests
18. Querying bazel query 'deps(//:Application)' --output graph //:Application //:Application_entitlements //:Application.__internal__.apple_binary //:Application.library //:Application.swift_runtime_linkopts
19. Querying bazel test -- $(bazel query 'tests(//...)')` bazel test -- //:Tests
20. Starlark files Specified in .bzl files Allows extending Bazel Used to define constants, macros, and custom rules Starlark is a dialect of Python
21. Caching //:A.swift Compile Hash(//:A.swift) //:B.swift Hash(//:B.swift) //:Foo.swiftmodule Link Hash(//:Foo.swiftmodule) //:Bar.swiftmodule Hash(//:Bar.swiftmodule)
22. Local cache Typical build artifacts, equivalent to Derived Data Most recent build will overwrite some outputs Sometimes cleaned up automatically (resides in /tmp) bazel clean deletes it bazel clean --expunge deletes external cache
23. Remote Cache "Remote" meaning HTTP or gRPC, which technically can be on your local machine Largest benefit: pulling master is no longer a full rebuild Smaller benefit: same thing for running someone’s PR Warning: it's possible to poison the cache, best if only CI writes, but everyone reads
24. Disk Cache Local version of the Remote Cache, built into bazel Largest benefit: switching branches/stashes Currently has no automatic cleanup, so has to be periodically cleaned or it gets too large
25. Remote Execution Distribute your build to many machines Populates the remote cache with outputs Also does a “local” build, in case certain actions complete locally first
26. .bazelrc file Specifies default command line flags Each entry is scoped to a command: build, test, run, query, startup, etc. Commands inherit flags: common < build < test
27. .bazelrc baseline example # Don't nest .app in .ipa build --define=apple.experimental.tree_artifact_outputs=1 # Don't generate Objective-C Interface Headers build --features=swift.no_generated_header # Enable Whole Module Optimization for all builds build --features=swift.opt_uses_wmo build --swiftcopt=-whole-module-optimization # LLDB, but also cachability build --features=debug_prefix_map_pwd_is_dot # Fast Swift compiles build --features=swift.use_global_module_cache
28. .bazelrc caching example # Cache repository downloads locally build --repository_cache=~/.cache/bzl-repository/flagship build --experimental_repository_cache_hardlinks # Cache action graph and CAS in a local disk cache build --disk_cache=~/.cache/bazel-target # Don't download cached items # Drastically speeds up 100% cached situation build --remote_download_minimal
29. .bazelrc improved cache hits example # Reduce cache misses from environment variables changing build --incompatible_strict_action_env --action_env=PATH=/usr/bin:/bin # Try to prevent cache poisoning build --experimental_guard_against_concurrent_changes # Cache checks/downloads require more jobs to be efficient build --jobs=200 # Faster remote cache hashing build --experimental_multi_threaded_digest # Keep more of the cache digests in memory build --cache_computed_file_digests=500000
30. .bazelrc configs Can specify groups of settings called configs Settings are applied by using the --config=NAME flag
31. .bazelrc Debug config example # Debug configuration == dbg compilation build:Debug --compilation_mode=dbg # Disable sandboxing (for SPEED) build:Debug --spawn_strategy=local # Fix Debugging build:Debug --strategy=SwiftCompile=worker # Xcode Indexing support build:Debug --features=swift.index_while_building build:Debug --swiftcopt=-index-ignore-system-modules
32. .bazelrc Release config example # Release configuration == opt compilation build:Release --compilation_mode=opt # Release builds should use -0size build:Release --features=swift.opt_uses_osize # Nest .app in .ipa build:Release --define=apple.experimental.tree_artifact_outputs=0 # Strip symbols (large size savings because of our static linking) build:Release --objc_enable_binary_stripping=true --features=dead_strip # Generate dSYMs build:Release --apple_generate_dsym --output_groups=+dsyms
33. .bazelrc flavors Configs can use other configs, since --config=NAME is just another flag We use these sorts of configs as “flavors” of a build, such as “CI” or “App Store”
34. .bazelrc CI flavor example # CI uses Debug config build:ci --config=Debug # Build, but don't run, manual tests test:ci --test_tag_filters=-manual # Point out, but still continue after, flakey tests test:ci --flaky_test_attempts=3 # Collect all test results, even if some fail test:ci --test_keep_going
35. .bazelrc Distribution flavors example # Bullflight distribution is a Release build build:bullflight --config=Release # Allow code to condition on Bullflight build:bullflight --swiftcopt=-DBULLFLIGHT # SwiftSupport is only needed for the App Store build:bullflight --define=apple.package_swift_support=no # App Store distribution is a Release build build:appstore --config=Release
36. Xcode integration Currently 3 options: — Tulsi — XCHammer — Custom
37. Custom Xcode project generation bazel query --output=xml 'deps(//:Application)' > /tmp/query.xml bazel2xcg /tmp/query.xml > /tmp/generated_project.json xcodegen --spec /tmp/generated_project.json
38. Links Bazel: bazel.build Xcode + Bazel example: github.com/kastiglione/bazel-xcode-demo-swift-driver Lyft Bazel talks: youtu.be/-u40WqvdRsg (Swiftable:BA 2019) youtu.be/NAPeWoimGx8 (BazelCon 2019) These slides: tinyurl.com/tc-bazel-ios
40. Macro example def tgt_unit_test(srcs, target_under_test, test_host = None, name = None, module_name = None, minimum_os_version = MIN_OS_VERSION, data = [], deps = [], visibility = None): name = name if name else "Tests" module_name = module_name if module_name else "{package_basename}Tests" .format(package_basename = paths.basename(native.package_name())) library_name = "{name}.__tgt__.library".format(name = name) swift_library( name = library_name, module_name = module_name, srcs = srcs, data = data, deps = depset([target_under_test] + deps), ) ios_unit_test( name = name, minimum_os_version = minimum_os_version, test_host = test_host, deps = [library_name], )
41. Custom rule example generate_intent_classes = rule( implementation = _generate_intent_classes_impl, attrs = { 'intent_definitions': attr.label_list( mandatory = True, allow_files = [".intentdefinition"] ), '_generator': attr.label( executable = True, cfg = "host", default = Label("//tools/IntentDefinitionCodegen"), ), }, )
42. Custom rule example def _generate_intent_classes_impl(ctx): outs = [] for intent_definition in ctx.files.intent_definitions: output_path = "{name}.{intent_definition}.swift".format( name = ctx.label.name, intent_definition = intent_definition.basename ) out = ctx.actions.declare_file(output_path) ctx.actions.run( progress_message = "Generating classes for {intent_definition}".format( intent_definition = intent_definition.basename ), executable = ctx.executable._generator, arguments = [intent_definition.path, out.path], inputs = [intent_definition], outputs = [out], ) outs.append(out) return [DefaultInfo(files = depset(outs))]
43. Building Run script before “Compile Sources”, invokes bazel build //:Application Stub tools so “Compile Sources” doesn’t do anything: CC: /usr/bin/true LD: /usr/bin/true LIBTOOL: /usr/bin/true SWIFT_EXEC: /usr/bin/true
44. Running/Testing cp bazel-bin/Application.app $TARGET_BUILD_DIR/$FULL_PRODUCT_NAME cp bazel-bin/Tests.xctest $TARGET_BUILD_DIR/$FULL_PRODUCT_NAME
45. Indexing Generated while building (a flag in .bazelrc) Copied into correct location with index-import xcode-index-preferences.json: { "EnableFullStoreVisibility": true }
46. Fixing lldb settings set target.source-map ./external/ "$SRCROOT/$build_link/external" settings append target.source-map ./ "$SRCROOT"