Android Builds at Speed and Scale

How do Android engineers at Facebook "move fast on stable infrastructure"? How do we make local builds stupendously fast? How do ensure that there are no regressions in functionality? As our codebase grows, how do we make sharing code easy and lightweight? Answers to all these questions can be found within….

Transcript

1. Building Android Apps At Scale and Speed Simon Stewart @shs96c Software Engineer, Facebook
2. One Guiding Principle
3. We don't want people waiting on computers <
4. We want computers waiting on people <
5. Scalable Source Organisation
6. Server Sharing Code Today Android c++ iOS
7. Server Sharing Code Today Android iOS c++ c++
8. Server Sharing Code Today Android iOS c++ c++ c++
9. Sharing Code Tomorrow fbsource c++
10. All our source in a single repository (a "monorepo")
11. Why a monorepo? Easier: code sharing, versioning, contributing Simpler: large scale changes Consistent: uniform tooling
12. Scalable Source Control
13. Mercurial >1M >100K Source control commands per day commits per week
14. Large Repos Are Slow ▪Lots of commits a day Pulls are slow Checkouts are slow ▪Years of history Clones are slow Requires lots of disk space
15. We want computers waiting on people <
16. Shallow Checkouts Clone just the latest version Work locally without complete history Need more history? Download on demand HEAD HEAD-1 HEAD-2 HEAD-3
17. Sparse Checkouts Work on just the files you need Build system integrations knows how to check out more ~/fbsource ios/... common/... ~/fbsource/.hg
18. Watchman OS independent filesystem change notification Integrates with mercurial File checking operations scale linearly with number of files changed
19. Remote Filelog Clone and pull just download metadata Download files when required Clones and pulls: 10x faster
20. Large Repos Are Fast 50 times faster than git 2000 hg improvements Up to More than
21. Scalable Builds
22. Buck ▪Next generation build system ▪Designed for modern hardware ▪SSD ▪Plentiful RAM ▪Many cores ▪Automatically parallelizes builds ▪And it's Open Source
24. Build Files java_binary( name = 'example-binary', deps = [ ':example', ], ) java_library( name = 'example', srcs = glob(['*.java']), deps = [ '//third-party/guava:guava', ], visibility = [ '//javatests/...', ], ) Build file. Typically named BUCK
25. Build Files java_binary( name = 'example-binary', deps = [ ':example', ], ) java_library( name = 'example', srcs = glob(['*.java']), deps = [ '//third-party/guava:guava', ], visibility = [ '//javatests/...', ], ) Build file. Typically named BUCK A build rule of type "java_binary"
26. Build Files java_binary( name = 'example-binary', deps = [ ':example', ], ) java_library( name = 'example', srcs = glob(['*.java']), deps = [ '//third-party/guava:guava', ], visibility = [ '//javatests/...', ], ) Build file. Typically named BUCK A build rule of type "java_binary" A target defined in the same build file
27. Build Files java_binary( name = 'example-binary', deps = [ ':example', ], ) java_library( name = 'example', srcs = glob(['*.java']), deps = [ '//third-party/guava:guava', ], visibility = [ '//javatests/...', ], ) Build file. Typically named BUCK A build rule of type "java_binary" A target defined in the same build file Build rule of type "java_library"
28. Build Files java_binary( name = 'example-binary', deps = [ ':example', ], ) java_library( name = 'example', srcs = glob(['*.java']), deps = [ '//third-party/guava:guava', ], visibility = [ '//javatests/...', ], ) Build file. Typically named BUCK A build rule of type "java_binary" A target defined in the same build file A target defined in another build file Build rule of type "java_library"
29. Build Files java_binary( name = 'example-binary', deps = [ ':example', ], ) java_library( name = 'example', srcs = glob(['*.java']), deps = [ '//third-party/guava:guava', ], visibility = [ '//javatests/...', ], ) Build file. Typically named BUCK A build rule of type "java_binary" A target defined in the same build file A target defined in another build file Build rule of type "java_library" All targets have visibility
30. Target and Action Graph Target Graph: a naive representation of targets defined in build files. Action Graph: an optimized graph of actions to perform as the build
31. Language-Aware Smarts public class Foo { private String someField; public void doSomething(int times) { System.out.println("Wow! " + times); } } Buck calculates the ABI, and keys rebuilds of dependencies off that Packaging rules that depend on this will always be re-run.
32. Language-Aware Smarts public class Foo { private boolean theCakeIsALie = true; public void doSomething(int times) { System.out.println("Wow! " + times); } } Private field changed: ABI remains the same as before
33. Language-Aware Smarts public class Foo { private boolean theCakeIsALie = true; public void doSomething(int num) { System.out.println("Wow! " + num); } } Parameter name changed: ABI remains the same as before.
34. Language-Aware Smarts public class Foo { private boolean theCakeIsALie = true; public void doSomething(String num) { System.out.println("Wow! " + num); } } Parameter type changed: ABI changes too!
35. Language-Aware Smarts public class Foo { private boolean theCakeIsALie = true; public void doSomething(String num) { System.err.println("Wow! " + num); } } Body of method changed: ABI remains the same.
36. Pervasive Caching • Only 3% of each build is unique. • Local and distributed caches slash build times.
37. Exopackage ▪Exopackage is an optimization for building and installing development builds of Android apps.
38. Buck Smarts Build rule knows all of the inputs that can affect its output Changes to dx and proguard for faster compilation Java ABI smarts Android resource smarts Daemon to watch file changes
39. Scalable Continuous Integration
40. <10 Minutes Feedback to Developers >1000 test results per second 5 Years machine work per day
41. Life Cycle of a Diff Write Code Test Locally Diff and review Sandcastle Land
42. Sandcastle Rapid High-Signal Frequent
43. Sandcastle Design Single Master Master Worker Worker Worker Worker
44. Sandcastle Design Single Master Distributed Queue Master Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Distributed Queue
46. Test Runs Flakiness Speed Coverage
47. Landcastle Land Target Determinator Test Test Test Push
48. We don't want people waiting on computers <
49. We want computers waiting on people <
50. Summary Tool OSS? Link! Buck Yes https://facebook.github.io/buck/ Mercurial Yes http://mercurial.selenic.com/ Watchman Yes https://facebook.github.io/watchman/ hgwatchman Yes https://bitbucket.org/facebook/hgwatchman remote filelog Yes https://bitbucket.org/facebook/remotefilelog Sandcastle No