Isolated Development

With the growth of codebases come more challenges: applications become larger, build times longer, the iteration speed for developers decreases. To scale our Android and iOS codebase at Square horizontally and keep engineers productive we started to invest in Isolated Development three years ago. The idea to run single features in a sandbox environment was a big shift on the engineering side but also enabled many opportunities. This talk discusses some of our learnings, how we transformed the idea into reality, how teams adopted development apps, how we reduced build and IDE sync times by 10X and where the journey is going next.

Transcript

1. November 02, 2022 Isolated Development Ralf Wondratschek @vRallev
2. Challenges in 2018
3. Challenges in 2018: lines of code growth
4. Challenges in 2018: lines of code growth
5. Challenges in 2018: build times are growing
6. :hairball :app2 :app1 :lib1 :lib2 Challenges in 2018: module structure
7. :apps :feature :common :api Challenges in 2018: module structure
8. :hairball :app2 :app1 :lib1 :lib2 Challenges in 2018: multiple apps with different flavors
9. :hairball :app2 :app1 :lib1 :lib2 Challenges in 2018: multiple apps with different flavors
13. https://bit.ly/3D8SsUl
14. Dependency inversion
15. class Feature(s: LoginStrategy) interface LoginStrategy class SmsLoginStrategy() : LoginStrategy Dependency inversion
16. class SmsLoginStrategy @Inject constructor( private val helper: Helper, private val helper: Helper1, private val helper: Helper2, private val helper: Helper3, private val helper: Helper4, private val helper: Helper5, private val helper: Helper6, private val helper: Helper7, private val helper: Helper8 ) : LoginStrategy Dependency inversion
17. dependencies { api project(':helper1') api project(':helper2') api project(':helper3') api project(':helper4') api project(':helper5') api project(':helper6') api project(':helper7') api project(':helper8') } Dependency inversion
18. dependencies { api project(':login-strategy') } Dependency inversion
19. Module Structure :public :impl :impl-wiring
20. :login-strategy Module Structure :public :impl :impl-wiring
21. Module Structure :login-strategy:public :login-strategy:impl :login-strategy:impl-wiring
22. Module Structure
23. :login-strategy Module Structure :public :impl :impl-wiring :feature :public :impl :impl-wiring X
24. Development Apps :public :impl :impl-wiring
25. Development Apps :public :impl :impl-wiring :demo
26. Development Apps
27. Development Apps • Install a feature with an isolated application on the device ◦ Launch the feature directly ◦ Shorter build times • Easy way to experiment and merge code
28. :login-screen Development Apps :public :impl :impl-wiring :demo
29. :login-screen Development Apps :public :impl :impl-wiring :demo :development-app :public :impl :impl-wiring
30. :login-screen Development Apps :public :impl :impl-wiring :demo :development-app :public :impl :impl-wiring :ui-engine :public :impl
31. :login-screen Development Apps :public :impl :impl-wiring :demo
32. :login-screen Development Apps :public :impl :impl-wiring :demo :account-screen :public :impl
33. https://square.github.io/workflow/
34. :account-screen:public interface AccountScreenWorkflow : Workflow<Unit, Unit, LayeredScreen<PosLayering>>
35. :account-screen:impl class RealAccountScreenWorkflow @Inject constructor( … ) : AccountScreenWorkflow
36. :login-screen:public interface LoginScreenWorkflow : Workflow<Unit, Unit, LayeredScreen<PosLayering>>
37. :login-screen:impl class RealLoginScreenWorkflow @Inject constructor( private val accountScreenWorkflow: Provider<AccountScreenWorkflow> ) : LoginScreenWorkflow
38. :login-screen Development Apps :public :impl :impl-wiring :demo :account-screen :public :impl
39. :login-screen Development Apps :public :impl :impl-wiring :demo :account-screen :public :impl
40. :login-screen Development Apps :public :impl :impl-wiring :demo :account-screen :public :fake
41. :account-screen:fake class FakeAccountScreenWorkflow @Inject constructor() : AccountScreenWorkflow { // ... }
42. Module Structure :public :impl :impl-wiring :demo
43. Module Structure :public :impl :impl-wiring :demo :fake
44. :login-screen:demo android { defaultConfig { applicationId "com.squareup.sample.login.screen.demo" } } dependencies { // … implementation project(':account-screen:fake') implementation project(':development-app:impl-wiring) }
45. :login-screen:demo @DevelopmentAppComponent class DemoLoginApp : DevelopmentApplication() @Module @ContributesTo(ActivityScope::class) object DemoLoginMainActivityModule { @Provides fun provideFeatureWorkflowProvider( workflow: LoginScreenWorkflow ): FeatureProvider = FeatureWorkflowProvider( workflow = workflow, props = Unit, ) }
46. :login-screen:demo @DevelopmentAppComponent class DemoLoginApp : DevelopmentApplication() @Module @ContributesTo(ActivityScope::class) object DemoLoginMainActivityModule { @Provides fun provideFeatureWorkflowProvider( workflow: LoginScreenWorkflow ): FeatureProvider = FeatureWorkflowProvider( workflow = workflow, props = Unit, ) }
47. :login-screen:demo @DevelopmentAppComponent class DemoLoginApp : DevelopmentApplication() @Module @ContributesTo(ActivityScope::class) object DemoLoginMainActivityModule { @Provides fun provideFeatureWorkflowProvider( workflow: LoginScreenWorkflow ): FeatureProvider = FeatureWorkflowProvider( workflow = workflow, props = Unit, ) }
48. • Part of the development apps → Faster builds → Tend to be very stable → More build cache hits → Automatic test shardening • Integration tests live in the final applications → Test robots are shared across all apps UI Tests
49. Mechanisms
50. plugins { id 'com.android.library id 'org.jetbrains.kotlin.android' id 'org.jetbrains.kotlin.kapt' id 'com.squareup.anvil' } dependencies { api project(':account-screen:public') ... } tasks.withType(KotlinCompile).configureEach { kotlinOptions { jvmTarget = JavaVersion.VERSION_11 } } Mechanisms: module structure through convention plugins
51. // :login-screen:impl plugins { id 'com.squareup.android.lib } // :login-screen:demo plugins { id 'com.squareup.demo-app } Mechanisms: module structure through convention plugins https://developer.squareup.com/blog/herding-elephants/
52. class NoImplDependencyModuleCheck : ModuleCheck { override fun runCheck(project: Project) { … fail( "No :impl dependency is allowed. Requested: " + "${implDependencies.joinToString { it.dependencyName }} in project: $project." ) } } Mechanisms: module structure enforced through Lint rules
53. Mechanisms: library generator
54. Mechanisms: library generator
55. Mechanisms: library generator
56. Mechanisms: design patterns & frameworks • Workflow → Composition was key → https://square.github.io/workflow/ • Dependency inversion → Enforced through the module structure • Dependency Injection → Anvil: https://github.com/square/anvil/ → Dagger 2: https://dagger.dev/
57. class RealLoginScreenWorkflow @Inject constructor( private val accountScreenWorkflow: Provider<AccountScreenWorkflow> ) : LoginScreenWorkflow Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
58. class RealLoginScreenWorkflow @Inject constructor( private val accountScreenWorkflow: Provider<AccountScreenWorkflow> ) : LoginScreenWorkflow @Module abstract class LoginScreenWorkflowModule { @Binds abstract fun bindLoginScreenWorkflow( workflow: RealLoginScreenWorkflow ) : LoginScreenWorkflow } Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
59. class RealLoginScreenWorkflow @Inject constructor( private val accountScreenWorkflow: Provider<AccountScreenWorkflow> ) : LoginScreenWorkflow @Module abstract class LoginScreenWorkflowModule { @Binds abstract fun bindLoginScreenWorkflow( workflow: RealLoginScreenWorkflow ) : LoginScreenWorkflow } @Component( modules = { LoginScreenWorkflowModule::class, } ) interface AppComponent Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
60. @ContributesBinding(ActivityScope::class) class RealLoginScreenWorkflow @Inject constructor( private val accountScreenWorkflow: Provider<AccountScreenWorkflow> ) : LoginScreenWorkflow Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
61. @ContributesBinding(ActivityScope::class) class RealLoginScreenWorkflow @Inject constructor( private val accountScreenWorkflow: Provider<AccountScreenWorkflow> ) : LoginScreenWorkflow Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
62. @DevelopmentAppComponent class DemoLoginApp : DevelopmentApplication() @Module @ContributesTo(ActivityScope::class) object DemoLoginMainActivityModule { @Provides fun provideFeatureWorkflowProvider( workflow: LoginScreenWorkflow ): FeatureProvider = FeatureWorkflowProvider( workflow = workflow, props = Unit, ) } Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
63. @DevelopmentAppComponent class DemoLoginApp : DevelopmentApplication() @Module @ContributesTo(ActivityScope::class) object DemoLoginMainActivityModule { @Provides fun provideFeatureWorkflowProvider( workflow: LoginScreenWorkflow ): FeatureProvider = FeatureWorkflowProvider( workflow = workflow, props = Unit, ) } Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
64. https://bit.ly/3WbPHdm
65. Mechanisms: design patterns & frameworks: Anvil https://bit.ly/3WbPHdm Try to build :demo module Add missing dependencies to build.gradle file Sync :demo module in Android Studio Add Dagger modules to Dagger components
66. Mechanisms: design patterns & frameworks: Anvil https://bit.ly/3WbPHdm Try to build :demo module Add missing dependencies to build.gradle file Sync :demo module in Android Studio Add Dagger modules to Dagger components
67. Mechanisms: dashboards
68. Mechanisms: dashboards
69. Mechanisms: dashboards
70. Mechanisms: dashboards
71. Mechanisms: dashboards
72. Mechanisms: dashboards
73. Mechanisms: dashboards
74. Mechanisms: dashboards
75. Mechanisms: benchmarks https://developer.squareup.com/blog/measure-measure-measure/
76. Mechanisms: benchmarks https://developer.squareup.com/blog/measure-measure-measure/
77. Mechanisms: benchmarks https://developer.squareup.com/blog/measure-measure-measure/
78. Mechanisms: migrations • We were a small team and relied on the help of feature teams to contribute to common goals → Legacy module migration → Adopt Anvil → Extract UI tests into :demo apps https://www.droidcon.com/2022/09/29/migration-without-migraines-automatic-migration-at-scale/
79. Challenges
80. Challenges: module count
81. Solution: • Convention plugins • Benchmarks • Anvil • Configuration caching • Partial IDE sync → https://github.com/dropbox/focus Challenges: module count
82. Legacy modules break our dependency rules Solution: • Finish migration Challenges: legacy modules
83. Anti-pattern Challenges: granular modules
84. Challenges: granular modules :login-screen
85. Challenges: granular modules :login-screen :feature-a
86. Challenges: granular modules :login-screen :feature-a :feature-b
87. Challenges: granular modules :login-screen :feature-a :feature-b :login-strategy
88. Challenges: granular modules :login-screen :public :impl :impl-wiring
89. Challenges: granular modules :login-screen :public :impl-a :impl-a-wiring :impl-b :impl-b-wiring
90. Challenges: granular modules :login-screen :public :impl-a :impl-a-wiring :impl-b :impl-c :impl-b-wiring :impl-c-wiring
91. Challenges: development app shell
92. Challenges: extract UI tests
93. November 02, 2022 Isolated Development Ralf Wondratschek @vRallev