Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Bazel

 Bazel

Talk was presented at Droidcon NYC 2019.

Why Bazel worth taking a look at and how it can help you scale development of your Android and iOS apps while (mostly) outperforming standard tooling like Gradle and xcodebuild.

In the talk we'll discuss following major topics:

- Multi-module setup vs build system
- Why Gradle is slow and how they're fixing it
- Why Gradle is fast and how they're fixing it
- ABI vs API vs implementation
- Compile avoidance
- Remote caching
- Remote execution
- Build configuration DSL (Groovy/Kotlin vs Starlark)
- IDE support
- CI/CD support
- Testing

1669655d7494301079bbae17c15675b7?s=128

Artem Zinnatullin

August 27, 2019
Tweet

Transcript

  1. Bazel

  2. Bazel

  3. None
  4. Tacking a look at Bazel

  5. Build System Use Cases

  6. Build System Use Cases: User

  7. Build System Use Cases: User

  8. Build System Use Cases: User

  9. Build System Use Cases: Automation

  10. Build System Use Cases: Automation

  11. Build System Use Cases: Plugins/Extensions

  12. Build System Use Cases -Resolve dependencies

  13. Build System Use Cases -Resolve dependencies -Build (compile, process resources,

    etc)
  14. Build System Use Cases -Resolve dependencies -Build (compile, process resources,

    etc) -Run apps/tests
  15. Build System Use Cases -Resolve dependencies -Build (compile, process resources,

    etc) -Run apps/tests -Integrate with IDE (modules, actions, etc)
  16. Build System Use Cases -Resolve dependencies -Build (compile, process resources,

    etc) -Run apps/tests -Integrate with IDE (modules, actions, etc) -Integrate with scripts & CI
  17. Build System Use Cases -Resolve dependencies -Build (compile, process resources,

    etc) -Run apps/tests -Integrate with IDE (modules, actions, etc) -Integrate with scripts & CI -Plugins/Extensions API
  18. Build System Use Cases -Resolve dependencies -Build (compile, process resources,

    etc) -Run apps/tests -Integrate with IDE (modules, actions, etc) -Integrate with scripts & CI -Plugins/Extensions API -etc
  19. Design of Bazel: Configuration

  20. Design of Bazel: Configuration java_library( name = "a", srcs =

    [ "A.java" ], deps = [ "//:b" ], ) java_library( name = "b", srcs = [ "B.java" ], ) Bazel BUILD file
  21. Design of Bazel: Configuration ‣Readable

  22. Design of Bazel: Configuration ‣Readable ‣Writeable

  23. Design of Bazel: Configuration ‣Readable ‣Writeable ‣Maintainable

  24. Design of Bazel: Configuration ‣Static Markdown Language

  25. Design of Bazel: Configuration ‣Static Markdown Language -XML -JSON -YAML

    (no God please no) -TOML -etc
  26. Design of Bazel: Configuration ‣Static Markdown Language -XML -JSON -YAML

    (no God please no) -TOML -etc ‣Programming Language
  27. Design of Bazel: Configuration ‣Static Markdown Language -XML -JSON -YAML

    (no God please no) -TOML -etc ‣Programming Language -Dynamically Typed -Statically Typed -etc
  28. Design of Bazel: Configuration <project xmlns="…" xmlns:xsi="…" xsi:schemaLocation="…"> <modelVersion>4.0.0</modelVersion> <groupId>com.lyft.dagger1</groupId>

    <artifactId>dagger-parent</artifactId> <packaging>pom</packaging> <version>1.3.0</version> <name>Dagger (Parent)</name> <description>A fast dependency injector for Android and Java.</description> <url>https://github.com/square/dagger</url> <modules> <module>compiler</module> <module>core</module> </modules> Maven pom.xml
  29. Design of Bazel: Configuration apply plugin: 'com.android.library' apply plugin: 'kotlin-android'

    android { defaultConfig { minSdkVersion versions.minSdk compileSdkVersion versions.compileSdk buildToolsVersion versions.buildTools } } dependencies { api project(':redom:api') api androidXLibraries.recyclerView } Gradle build.gradle
  30. Design of Bazel: Configuration java_library( name = "lib", srcs =

    glob(["src/main/java/**/*.java"]), deps = [ "//:b", ] ) java_test( name = "test", srcs = glob(["src/test/java/**/*.java"]), deps = [ ":lib", ] ) Bazel BUILD file
  31. Design of Bazel: Configuration ‣Reproducibility

  32. Design of Bazel: Configuration ‣Reproducibility -Limited stdlib

  33. Design of Bazel: Configuration ‣Reproducibility -Limited stdlib -No untracked IO

  34. Design of Bazel: Configuration ‣Reproducibility -Limited stdlib -No untracked IO

    -External deps need sha256
  35. Design of Bazel: Configuration apply plugin: 'java-library' apply plugin: 'kotlin'

    // yoooooo don't do this version = Calendar.instance.get(Calendar.DAY_OF_WEEK) == 2 ? System.currentTimeMillis() : new Random().nextInt()
  36. Design of Bazel: Configuration ‣Gradle: Groovy

  37. Design of Bazel: Configuration ‣Gradle: Groovy ‣Groovy syntax and dynamic

    type system ()
  38. Design of Bazel: Configuration ‣Gradle: Groovy ‣Groovy syntax and dynamic

    type system () ‣Unlimited mutability
  39. Design of Bazel: Configuration ‣Gradle: Groovy ‣Groovy syntax and dynamic

    type system () ‣Unlimited mutability ‣Unlimited IO
  40. Design of Bazel: Configuration ‣Gradle: Groovy ‣Groovy syntax and dynamic

    type system () ‣Unlimited mutability ‣Unlimited IO ‣Full JDK stdlib
  41. Design of Bazel: Configuration ‣Gradle: Groovy ‣Groovy syntax and dynamic

    type system () ‣Unlimited mutability ‣Unlimited IO ‣Full JDK stdlib ‣Can write Java in Groovy
  42. None
  43. Design of Bazel: Configuration ‣Gradle: Kotlin

  44. Design of Bazel: Configuration ‣Gradle: Kotlin ‣Kotlin syntax and static

    type system ()
  45. Design of Bazel: Configuration ‣Gradle: Kotlin ‣Kotlin syntax and static

    type system () ‣Unlimited mutability
  46. Design of Bazel: Configuration ‣Gradle: Kotlin ‣Kotlin syntax and static

    type system () ‣Unlimited mutability ‣Unlimited IO
  47. Design of Bazel: Configuration ‣Gradle: Kotlin ‣Kotlin syntax and static

    type system () ‣Unlimited mutability ‣Unlimited IO ‣Full JDK stdlib
  48. Design of Bazel: Configuration See “Fundamental Design Issues of Gradle

    Build System” https:!//artemzin.com/blog/fundamental-design-issues-of-gradle-build-system/ *includes proposals to solve the issues
  49. Design of Bazel: Configuration Enter: Skylark Starlark

  50. Design of Bazel: Configuration ‣Bazel Starlark Evaluation Model https:"//docs.bazel.build/versions/master/skylark/concepts.html#evaluation-model

  51. Design of Bazel: Configuration ‣Bazel Starlark Evaluation Model -Loading Phase

    -Read BUILD and .bzl files required for given build -Bazel targets are directly mapped to file paths in the project https:"//docs.bazel.build/versions/master/skylark/concepts.html#evaluation-model
  52. Design of Bazel: Configuration ‣Bazel Starlark Evaluation Model -Loading Phase

    -Read BUILD and .bzl files required for given build -Bazel targets are directly mapped to file paths in the project -Analysis Phase -Rules are executed -Rules instantiate actual Actions -Action graph is composed and validated https:"//docs.bazel.build/versions/master/skylark/concepts.html#evaluation-model
  53. Design of Bazel: Configuration ‣Bazel Starlark Evaluation Model -Loading Phase

    -Read BUILD and .bzl files required for given build -Bazel targets are directly mapped to file paths in the project -Analysis Phase -Rules are executed -Rules instantiate actual Actions -Action graph is composed and validated -Execution Phase -Action graph is executed https:"//docs.bazel.build/versions/master/skylark/concepts.html#evaluation-model
  54. Design of Bazel: Configuration ‣Starlark https:"//docs.bazel.build/versions/master/skylark/language.html

  55. Design of Bazel: Configuration ‣Starlark ‣Python 3 syntax and dynamic

    type system () https:"//docs.bazel.build/versions/master/skylark/language.html
  56. Design of Bazel: Configuration ‣Starlark ‣Python 3 syntax and dynamic

    type system () ‣Custom runtime https:"//docs.bazel.build/versions/master/skylark/language.html
  57. Design of Bazel: Configuration ‣Starlark ‣Python 3 syntax and dynamic

    type system () ‣Custom runtime ‣Custom stdlib https:"//docs.bazel.build/versions/master/skylark/language.html
  58. Design of Bazel: Configuration ‣Starlark ‣Python 3 syntax and dynamic

    type system () ‣Custom runtime ‣Custom stdlib ‣Extremely limited mutability https:"//docs.bazel.build/versions/master/skylark/language.html
  59. Design of Bazel: Configuration ‣Starlark ‣Python 3 syntax and dynamic

    type system () ‣Custom runtime ‣Custom stdlib ‣Extremely limited mutability ‣No IO https:"//docs.bazel.build/versions/master/skylark/language.html
  60. Design of Bazel: Configuration ‣Starlark ‣Python 3 syntax and dynamic

    type system () ‣Custom runtime ‣Custom stdlib ‣Extremely limited mutability ‣No IO ‣Bazel provides build & IO specific APIs https:"//docs.bazel.build/versions/master/skylark/language.html
  61. Design of Bazel: Configuration ‣Starlark ‣Python 3 syntax and dynamic

    type system () ‣Custom runtime ‣Custom stdlib ‣Extremely limited mutability ‣No IO ‣Bazel provides build & IO specific APIs ‣Can be used outside of Bazel! https:"//docs.bazel.build/versions/master/skylark/language.html
  62. Design of Bazel: Configuration java_library( name = "lib", srcs =

    glob(["src/main/java/**/*.java"]), deps = [ "//:b", ] ) java_test( name = "test", srcs = glob(["src/test/java/**/*.java"]), deps = [ ":lib", ] ) Bazel BUILD file
  63. Design of Bazel: Configuration java_library( name = "lib", srcs =

    glob(["src/main/java/**/*.java"]), deps = [ "//:b", ] ) java_test( name = "test", srcs = glob(["src/test/java/**/*.java"]), deps = [ ":lib", ] ) Bazel BUILD file Target with name “lib”
  64. Design of Bazel: Configuration ‣Starlark https:"//docs.bazel.build/versions/master/skylark/language.html

  65. Design of Bazel: Configuration ‣Starlark -Functions https:"//docs.bazel.build/versions/master/skylark/language.html

  66. Design of Bazel: Configuration ‣Starlark -Functions - Rules https:"//docs.bazel.build/versions/master/skylark/language.html

  67. Design of Bazel: Configuration ‣Starlark -Functions - Rules - Macros

    https:"//docs.bazel.build/versions/master/skylark/language.html
  68. Design of Bazel: Configuration ‣Starlark -Functions - Rules - Macros

    -Variables - Mutable within a “context” - Frozen once context has been evaluated https:"//docs.bazel.build/versions/master/skylark/language.html
  69. Design of Bazel: Configuration ‣Rule https:"//docs.bazel.build/versions/master/skylark/rules.html

  70. Design of Bazel: Configuration ‣Rule -Function that performs reproducible IO

    for given Inputs https:"//docs.bazel.build/versions/master/skylark/rules.html
  71. Design of Bazel: Configuration ‣Rule -Function that performs reproducible IO

    for given Inputs -Declares set of public and private attributes (parameters) https:"//docs.bazel.build/versions/master/skylark/rules.html
  72. Design of Bazel: Configuration ‣Rule -Function that performs reproducible IO

    for given Inputs -Declares set of public and private attributes (parameters) -Examples: kotlin_library, java_test, etc https:"//docs.bazel.build/versions/master/skylark/rules.html
  73. Design of Bazel: Configuration ‣Rule -Function that performs reproducible IO

    for given Inputs -Declares set of public and private attributes (parameters) -Examples: kotlin_library, java_test, etc -Cannot perform IO during configuration/analysis https:"//docs.bazel.build/versions/master/skylark/rules.html
  74. Design of Bazel: Configuration ‣Rule -Function that performs reproducible IO

    for given Inputs -Declares set of public and private attributes (parameters) -Examples: kotlin_library, java_test, etc -Cannot perform IO during configuration/analysis -Declares list of actions to perform IO during execution https:"//docs.bazel.build/versions/master/skylark/rules.html
  75. Design of Bazel: Configuration ‣Rule -Function that performs reproducible IO

    for given Inputs -Declares set of public and private attributes (parameters) -Examples: kotlin_library, java_test, etc -Cannot perform IO during configuration/analysis -Declares list of actions to perform IO during execution -Exposes Provider(s) — rule’s public API for other rules https:"//docs.bazel.build/versions/master/skylark/rules.html
  76. Design of Bazel: Configuration android_instrumentation_test = rule( attrs = attrs.add(

    ANDROID_INSTRUMENTATION_TEST_ATTRS, ADDON_ATTRS ), fragments = ["android"], implementation = _impl, outputs = _outputs, test = True, ) https:"//github.com/bazelbuild/rules_android
  77. Design of Bazel: Configuration https:"//github.com/bazelbuild/rules_android def _impl(ctx): # Main entry

    point into the Android test runner. test_runner = get_test_runner(ctx) ……… # TODO(str): remove after android_test migration if ctx.attr._android_test_migration: # Test APK is a predeclared output on android_test ctx.actions.run_shell( inputs = [ctx.attr.test_app[ApkInfo].signed_apk], outputs = [ctx.outputs.test_app], command = "cp %s %s" % (ctx.attr.test_app[ApkInfo].signed_apk.path, ctx.outputs.test_app.path), ) runfiles = runfiles.merge(ctx.runfiles(files = [ctx.outputs.test_app])) apks_to_install = []
  78. Design of Bazel: Configuration ‣Macro https:"//docs.bazel.build/versions/master/skylark/macros.html

  79. Design of Bazel: Configuration ‣Macro -Can instantiate rule(s) https:"//docs.bazel.build/versions/master/skylark/macros.html

  80. Design of Bazel: Configuration ‣Macro -Can instantiate rule(s) -Useful for

    code reuse, composing logic, default values https:"//docs.bazel.build/versions/master/skylark/macros.html
  81. Design of Bazel: Configuration ‣Macro -Can instantiate rule(s) -Useful for

    code reuse, composing logic, default values -Present only during Loading phase https:"//docs.bazel.build/versions/master/skylark/macros.html
  82. Design of Bazel: Configuration ‣Macro -Can instantiate rule(s) -Useful for

    code reuse, composing logic, default values -Present only during Loading phase -Examples: lyft_kotlin_library, lyft_android_binary, etc https:"//docs.bazel.build/versions/master/skylark/macros.html
  83. Design of Bazel: Configuration def lyft_android_library(**kwargs):

  84. Design of Bazel: Configuration def lyft_android_library(**kwargs): if kwargs.get("language") == "kotlin":

    _apply_detekt(**kwargs)
  85. Design of Bazel: Configuration def lyft_android_library(**kwargs): if kwargs.get("language") == "kotlin":

    _apply_detekt(**kwargs) _apply_pmd(**kwargs)
  86. Design of Bazel: Configuration def lyft_android_library(**kwargs): if kwargs.get("language") == "kotlin":

    _apply_detekt(**kwargs) _apply_pmd(**kwargs) kwargs = _apply_error_prone(**kwargs)
  87. Design of Bazel: Configuration def lyft_android_library(**kwargs): if kwargs.get("language") == "kotlin":

    _apply_detekt(**kwargs) _apply_pmd(**kwargs) kwargs = _apply_error_prone(**kwargs) kwargs = _apply_java_null_pkg_info_gen( **kwargs )
  88. Design of Bazel: Configuration def lyft_android_library(**kwargs): if kwargs.get("language") == "kotlin":

    _apply_detekt(**kwargs) _apply_pmd(**kwargs) kwargs = _apply_error_prone(**kwargs) kwargs = _apply_java_nullability_pkg_info_generator(**kwargs) android_library(**kwargs)
  89. Design of Bazel: Configuration Starlark vs multi-module projects

  90. Design of Bazel: Configuration ‣Parallelism

  91. Design of Bazel: Configuration ‣Parallelism -Limited mutability lets configure packages

    independently
  92. Design of Bazel: Configuration ‣Parallelism -Limited mutability lets configure packages

    independently -Lack of IO makes configuration CPU bound
  93. Design of Bazel: Configuration ‣Cache-ability -Lack of mutability and IO

    makes configuration cacheable
  94. Design of Bazel: Configuration ‣Configuration Avoidance -Bazel only evaluates BUILD

    and .bzl files required for the build command
  95. Design of Bazel: Configuration https:"//artemzin.com/blog/fundamental-design-issues-of-gradle-build-system/ Gradle: config time / number

    of modules Seconds 0 6.5 13 19.5 26 Number of modules 400 600 800 1000 10 15 20 25
  96. Design of Bazel: Configuration Gradle is working on Configuration Caching!

  97. Design of Bazel: Configuration ‣Visibility -Defines what packages or targets

    are allowed to directly depend on this target https:"//docs.bazel.build/versions/master/be/common-definitions.html#common-attributes
  98. Design of Bazel: Configuration java_library( name = "a", srcs =

    [ "A.java" ], deps = [ "//:b" ], visibility = [ "//visibility:public" ] ) java_library( name = "b", srcs = [ "B.java" ], visibility = [ "//myfriendlypkg:lib" ] )
  99. Design of Bazel: Configuration ‣Maintainability

  100. Design of Bazel: Configuration ‣Maintainability ‣Buildifier — tool for formatting

    & linting BUILD and .bzl files https:"//github.com/bazelbuild/buildtools
  101. Design of Bazel: Configuration ‣Maintainability ‣Buildifier — tool for formatting

    & linting BUILD and .bzl files ‣Buildozer — tool for mass refactoring of BUILD files https:"//github.com/bazelbuild/buildtools
  102. Design of Bazel: Configuration ‣Maintainability ‣Buildifier — tool for formatting

    & linting BUILD and .bzl files ‣Buildozer — tool for mass refactoring of BUILD files ‣Both written in Go, run pretty fast https:"//github.com/bazelbuild/buildtools
  103. Design of Bazel: Configuration java_library( name = "a", srcs =

    ["A.java"], deps = ["//:b"], ) Before:
  104. Design of Bazel: Configuration java_library( name = "a", srcs =

    ["A.java"], deps = ["//:b"], ) Before:
  105. Design of Bazel: Configuration java_library( name = "a", srcs =

    ["A.java"], deps = ["//:b"], ) java_library( name = "a", srcs = ["A.java"], deps = ["//:b"], ) Before: After:
  106. Design of Bazel: Configuration java_library( name = "a", srcs =

    ["A.java"], deps = ["//:b"], ) https:"//github.com/bazelbuild/buildtools/blob/master/buildozer/README.md
  107. Design of Bazel: Configuration java_library( name = "a", srcs =

    ["A.java"], deps = ["//:b"], ) https:"//github.com/bazelbuild/buildtools/blob/master/buildozer/README.md
  108. Design of Bazel: Configuration java_library( name = "a", srcs =

    ["A.java"], deps = ["//:b"], ) lyft_java_library( name = "a", srcs = ["A.java"], deps = ["//:b"], ) https:"//github.com/bazelbuild/buildtools/blob/master/buildozer/README.md
  109. Design of Bazel: Execution

  110. Design of Bazel: Execution

  111. Design of Bazel: Execution ‣Commands Command Purpose bazel query What

    Targets are there https:"//docs.bazel.build/versions/master/command-line-reference.html
  112. Design of Bazel: Execution ‣Commands Command Purpose bazel query What

    Targets are there bazel build Build one or more Targets https:"//docs.bazel.build/versions/master/command-line-reference.html
  113. Design of Bazel: Execution ‣Commands Command Purpose bazel query What

    Targets are there bazel build Build one or more Targets bazel test Run one or more Test Targets https:"//docs.bazel.build/versions/master/command-line-reference.html
  114. Design of Bazel: Execution ‣Commands Command Purpose bazel query What

    Targets are there bazel build Build one or more Targets bazel test Run one or more Test Targets bazel run Run one or more Run Targets https:"//docs.bazel.build/versions/master/command-line-reference.html
  115. Design of Bazel: Execution ‣Incrementability

  116. Design of Bazel: Execution ‣Incrementability ‣Bazel won’t do what has

    already been done (duh)
  117. Design of Bazel: Execution ‣Incrementability ‣Bazel won’t do what has

    already been done (duh) ‣Compilation is not incremental
  118. Design of Bazel: Execution ‣Incrementability ‣Bazel won’t do what has

    already been done (duh) ‣Compilation is not incremental ‣Tests results are also cached (you can opt out)
  119. Design of Bazel: Execution

  120. Design of Bazel: Execution ‣Parallelism ‣Bazel by default runs what

    it can in parallel
  121. Design of Bazel: Execution ‣ABI/header Extraction

  122. Design of Bazel: Execution ‣ABI/header Extraction -Bazel computes ABI for

    supported languages and uses it
  123. Design of Bazel: Execution

  124. Design of Bazel: Execution

  125. Design of Bazel: Execution

  126. Design of Bazel: Execution public class MyClass { private String

    x; public void a() { this.x = b(); } private String b() { Random random = new Random(); StringBuilder str = new StringBuilder(); for (int i = 0; i < random.nextInt(1024); i++) { str.append(i); } return str.toString(); } public static Runnable c(String s) { return () -> { new MyClass().b(); }; } } MyClass.java
  127. Design of Bazel: Execution public class MyClass { private String

    x; public void a() { this.x = b(); } private String b() { Random random = new Random(); StringBuilder str = new StringBuilder(); for (int i = 0; i < random.nextInt(1024); i++) { str.append(i); } return str.toString(); } public static Runnable c(String s) { return () -> { new MyClass().b(); }; } MyClass.java
  128. Design of Bazel: Execution public class MyClass { private String

    x; public void a() { this.x = b(); } private String b() { Random random = new Random(); StringBuilder str = new StringBuilder(); for (int i = 0; i < random.nextInt(1024); i++) { str.append(i); } return str.toString(); } public static Runnable c(String s) { return () -> { new MyClass().b(); }; } MyClass.java
  129. Design of Bazel: Execution public class MyClass { private String

    x; public void a() { this.x = b(); } private String b() { Random random = new Random(); StringBuilder str = new StringBuilder(); for (int i = 0; i < random.nextInt(1024); i++) { str.append(i); } return str.toString(); } public static Runnable c(String s) { return () -> { new MyClass().b(); }; } MyClass.java
  130. Design of Bazel: Execution public class MyClass { private String

    x; public void a() { this.x = b(); } private String b() { Random random = new Random(); StringBuilder str = new StringBuilder(); for (int i = 0; i < random.nextInt(1024); i++) { str.append(i); } return str.toString(); } public static Runnable c(String s) { return () -> { new MyClass().b(); }; } MyClass.java
  131. Design of Bazel: Execution public class MyClass { public void

    a() { this.x = b(); } private String b() { Random random = new Random(); StringBuilder str = new StringBuilder(); for (int i = 0; i < random.nextInt(1024); i++) { str.append(i); } return str.toString(); } public static Runnable c(String s) { return () -> { new MyClass().b(); }; } } MyClass.java
  132. Design of Bazel: Execution public class MyClass { public void

    a() { this.x = b(); } private String b() { Random random = new Random(); StringBuilder str = new StringBuilder(); for (int i = 0; i < random.nextInt(1024); i++) { str.append(i); } return str.toString(); } public static Runnable c(String s) { return () -> { new MyClass().b(); }; } MyClass.java
  133. Design of Bazel: Execution public class MyClass { public void

    a() { this.x = b(); } private String b() { Random random = new Random(); StringBuilder str = new StringBuilder(); for (int i = 0; i < random.nextInt(1024); i++) { str.append(i); } return str.toString(); } public static Runnable c(String s) { return () -> { new MyClass().b(); }; } MyClass.java
  134. Design of Bazel: Execution public class MyClass { public void

    a(); private String b() { Random random = new Random(); StringBuilder str = new StringBuilder(); for (int i = 0; i < random.nextInt(1024); i++) { str.append(i); } return str.toString(); } public static Runnable c(String s) { return () -> { new MyClass().b(); }; } } MyClass.java
  135. Design of Bazel: Execution public class MyClass { public void

    a(); private String b() { Random random = new Random(); StringBuilder str = new StringBuilder(); for (int i = 0; i < random.nextInt(1024); i++) { str.append(i); } return str.toString(); } public static Runnable c(String s) { return () -> { new MyClass().b(); }; } } MyClass.java
  136. Design of Bazel: Execution public class MyClass { public void

    a(); private String b() { Random random = new Random(); StringBuilder str = new StringBuilder(); for (int i = 0; i < random.nextInt(1024); i++) { str.append(i); } return str.toString(); } public static Runnable c(String s) { return () -> { new MyClass().b(); }; } } MyClass.java
  137. Design of Bazel: Execution public class MyClass { public void

    a(); private String b() { Random random = new Random(); StringBuilder str = new StringBuilder(); for (int i = 0; i < random.nextInt(1024); i++) { str.append(i); } return str.toString(); } public static Runnable c(String s) { return () -> { new MyClass().b(); }; } } MyClass.java
  138. Design of Bazel: Execution public class MyClass { public void

    a(); public static Runnable c(String s) { return () -> { new MyClass().b(); }; } } MyClass.java
  139. Design of Bazel: Execution public class MyClass { public void

    a(); public static Runnable c(String s) { return () -> { new MyClass().b(); }; } } MyClass.java
  140. Design of Bazel: Execution public class MyClass { public void

    a(); public static Runnable c(String s) { return () -> { new MyClass().b(); }; } } MyClass.java
  141. Design of Bazel: Execution public class MyClass { public void

    a(); public static Runnable c(String s); }
  142. Design of Bazel: Execution public class MyClass { public void

    a(); public static Runnable c(String s); } MyClass_Header.java (actually, it’s .class right away)
  143. Design of Bazel: Execution

  144. Design of Bazel: Execution ABI/Header extraction Works for entire action

    graph
  145. Design of Bazel: Execution

  146. Design of Bazel: Execution ‣Persistent Workers https:"//blog.bazel.build/2015/12/10/java-workers.html

  147. Design of Bazel: Execution ‣Persistent Workers ‣Simple Request/Response Protocol between

    Bazel and a Worker
  148. Design of Bazel: Execution ‣Persistent Workers ‣Simple Request/Response Protocol between

    Bazel and a Worker ‣Bazel sends requests as Protobuf messages to Worker’s stdin
  149. Design of Bazel: Execution ‣Persistent Workers ‣Simple Request/Response Protocol between

    Bazel and a Worker ‣Bazel sends requests as Protobuf messages to Worker’s stdin ‣Bazel reads responses as Protobuf messages from Worker’s stdout
  150. Design of Bazel: Execution ‣Persistent Workers ‣Simple Request/Response Protocol between

    Bazel and a Worker ‣Bazel sends requests as Protobuf messages to Worker’s stdin ‣Bazel reads responses as Protobuf messages from Worker’s stdout ‣PW is useful for JVM-based or other tools that run faster when warmed up
  151. Design of Bazel: Execution ‣Persistent Workers ‣Simple Request/Response Protocol between

    Bazel and a Worker ‣Bazel sends requests as Protobuf messages to Worker’s stdin ‣Bazel reads responses as Protobuf messages from Worker’s stdout ‣PW is useful for JVM-based or other tools that run faster when warmed up ‣Expect changes in parallelism and maybe +JSON
  152. Design of Bazel: Execution ‣Persistent Workers ‣Simple Request/Response Protocol between

    Bazel and a Worker ‣Bazel sends requests as Protobuf messages to Worker’s stdin ‣Bazel reads responses as Protobuf messages from Worker’s stdout ‣PW is useful for JVM-based or other tools that run faster when warmed up ‣Expect changes in parallelism and maybe +JSON ‣In theory, PW can be used for actual incremental compilation since it keeps state
  153. Design of Bazel: Execution ‣Persistent Workers https:"//blog.bazel.build/2015/12/10/java-workers.html

  154. Design of Bazel: Execution https:"//github.com/artem-zinnatullin/error-prone-performance ‣Persistent Workers Gradle with Javac

    accidentally class loaded each time
  155. Design of Bazel: Execution ‣Sandboxing https:"//blog.bazel.build/2015/09/11/sandboxing.html

  156. Design of Bazel: Execution ‣Sandboxing ‣Another important piece of reproducible

    builds
  157. Design of Bazel: Execution ‣Sandboxing ‣Another important piece of reproducible

    builds ‣Isolates filesystem and process environment for build actions similarly to containers
  158. Design of Bazel: Execution ‣Sandboxing ‣Another important piece of reproducible

    builds ‣Isolates filesystem and process environment for build actions similarly to containers ‣Supports Linux and macOS, Windows WIP
  159. Design of Bazel: Execution ‣Sandboxing ‣Another important piece of reproducible

    builds ‣Isolates filesystem and process environment for build actions similarly to containers ‣Supports Linux and macOS, Windows WIP ‣macOS performance is not great yet (we saw 30% degradation)
  160. Design of Bazel: Execution Remote Cache https:"//docs.bazel.build/versions/master/remote-caching.html

  161. Design of Bazel: Execution

  162. Design of Bazel: Execution ‣Remote Cache ‣Action Cache ‣Output Cache

    ‣Has amazing optimization “Remote Builds without the Bytes”
  163. Design of Bazel: Execution Remote Execution https:"//docs.bazel.build/versions/master/remote-execution.html

  164. Design of Bazel: Execution Remote Execution https:"//docs.bazel.build/versions/master/remote-execution.html

  165. Design of Bazel: Execution Remote Execution https:"//docs.bazel.build/versions/master/remote-execution.html

  166. Design of Bazel: Execution Remote Execut

  167. Design of Bazel: Execution

  168. Design of Bazel: Execution ‣Remote Bazel Execution

  169. Design of Bazel: Execution ‣Remote Bazel Execution -Acts as Remote

    Cache too
  170. Design of Bazel: Execution ‣Remote Bazel Execution -Acts as Remote

    Cache too -Automatic selection of execution strategy (local vs remote)
  171. Design of Bazel: Execution ‣Remote Bazel Execution -Acts as Remote

    Cache too -Automatic selection of execution strategy (local vs remote) -Remote environment for actions with Docker images
  172. Design of Bazel: Execution ‣Remote Bazel Execution -Acts as Remote

    Cache too -Automatic selection of execution strategy (local vs remote) -Remote environment for actions with Docker images -Virtually indefinitely parallelism
  173. Design of Bazel: Execution ‣Remote Bazel Execution -Acts as Remote

    Cache too -Automatic selection of execution strategy (local vs remote) -Remote environment for actions with Docker images -Virtually indefinitely parallelism -Google Cloud provides it as service yo (): RBE
  174. Design of Bazel: Execution ‣Mobile Install https:"//docs.bazel.build/versions/master/mobile-install.html

  175. Design of Bazel: Execution ‣Mobile Install -Like “Apply Changes” but

    made years ago https:"//docs.bazel.build/versions/master/mobile-install.html
  176. Design of Bazel: Execution ‣Mobile Install -Like “Apply Changes” but

    made years ago -Works by installing 2 apps: • Stub app with custom application class to load code & resources from “Contents app” • "Contents app" with actual code and resources https:"//docs.bazel.build/versions/master/mobile-install.html
  177. Design of Bazel: Execution ‣Mobile Install -Like “Apply Changes” but

    made years ago -Works by installing 2 apps: • Stub app with custom application class to load code & resources from “Contents app” • "Contents app" with actual code and resources -Incrementally updates dex files and resources in “Contents app” https:"//docs.bazel.build/versions/master/mobile-install.html
  178. Design of Bazel: Execution ‣Mobile Install -Like “Apply Changes” but

    made years ago -Works by installing 2 apps: • Stub app with custom application class to load code & resources from “Contents app” • "Contents app" with actual code and resources -Incrementally updates dex files and resources in “Contents app” -Has few corner cases and limitations https:"//docs.bazel.build/versions/master/mobile-install.html
  179. Design of Bazel: Execution https:"//docs.bazel.build/versions/master/mobile-install.html

  180. Design of Bazel: IDE Integration

  181. Design of Bazel: IDE Integration

  182. Design of Bazel: IDE Integration

  183. Design of Bazel: IDE Integration

  184. Design of Bazel: IDE Integration -IDE integration uses Bazel CLI

    commands
  185. Design of Bazel: IDE Integration -IDE integration uses Bazel CLI

    commands -IDE queries Bazel about the project (bazel query …)
  186. Design of Bazel: IDE Integration -IDE integration uses Bazel CLI

    commands -IDE queries Bazel about the project (bazel query …) -Supports Partial Sync for specific files, dirs
  187. Design of Bazel: IDE Integration TODO: screenshot of Android Studio

    Bazel Sync
  188. Design of Bazel: IDE Integration ‣IntelliJ Family -IntelliJ -Android Studio

    -CLion https:"//docs.bazel.build/versions/master/ide.html
  189. Design of Bazel: IDE Integration ‣Xcode -Tulsi and XCHammer to

    generate Xcode project from Bazel project -Lyft is working on Focus tool to limit number of loaded modules -lyft/index-import to import swift and clang index files produced as side effect of compilation https:"//docs.bazel.build/versions/master/ide.html
  190. Design of Bazel: Integration w/ Other Tools

  191. Design of Bazel: Integration w/ Other Tools ‣Genrule -Run arbitrary

    CLI tool, Bash, Powershell, etc -As any other rule, inputs and outputs are tracked -Sandboxing makes it easier to achieve reproducibility https:"//docs.bazel.build/versions/master/be/general.html#genrule
  192. Design of Bazel: Integration w/ Other Tools genrule( name =

    "mygenrule", cmd = "mkdir $(RULEDIR)/mydir; javac -d $(OUTS) $(SRCS)", outs = [ "mydir" ], srcs = [ "B.java" ], ) https:"//docs.bazel.build/versions/master/be/general.html#genrule
  193. Design of Bazel: Integration w/ Other Tools ‣Toolchains and Platforms

    -Bazel has notion of Target and Execution Platforms -Toolchain defines key/value pairs per Platform -Toolchain typically represents a set of Tooling for a language -Compiler -Linker -etc https:"//docs.bazel.build/versions/master/toolchains.html
  194. Design of Bazel: Integration w/ Other Tools ‣Extensions -.bzl files

    written in Starlark with macros, rules, constants -load(“#//mybazelextensions:file.bzl”, “myrule_or_macro”) -One of main Bazel 1.0 goals is to extract impls from core to external rules repos https:"//docs.bazel.build/versions/master/skylark/concepts.html
  195. Design of Bazel: Integration w/ Other Tools ‣Extensions

  196. Design of Bazel: Integration w/ Other Tools ‣Extensions github.com/bazelbuild/

  197. Design of Bazel: Integration w/ Other Tools ‣Extensions github.com/bazelbuild/ -rules_java

  198. Design of Bazel: Integration w/ Other Tools ‣Extensions github.com/bazelbuild/ -rules_java

    -rules_android
  199. Design of Bazel: Integration w/ Other Tools ‣Extensions github.com/bazelbuild/ -rules_java

    -rules_android -rules_kotlin
  200. Design of Bazel: Integration w/ Other Tools ‣Extensions github.com/bazelbuild/ -rules_java

    -rules_android -rules_kotlin -rules_jvm_external
  201. Design of Bazel: Integration w/ Other Tools ‣Extensions github.com/bazelbuild/ -rules_java

    -rules_android -rules_kotlin -rules_jvm_external -rules_apple
  202. Design of Bazel: Integration w/ Other Tools ‣Extensions github.com/bazelbuild/ -rules_java

    -rules_android -rules_kotlin -rules_jvm_external -rules_apple -etc
  203. Design of Bazel: Integration w/ Other Tools ‣Extensions github.com/bazelbuild/ -rules_java

    -rules_android -rules_kotlin -rules_jvm_external -rules_apple -etc ‣github.com/you/rules_x !
  204. Bazel Q&A: Open https:!//sli.do and enter code “bazel”

  205. Artem Zinnatullin ‣ @artem_zin ‣ artemzin.com Come work with me!

    Also: Lyft Mobile Podcast tools, libraries, CI, builds @