Isolated Development

With the growth of codebases come more challenges: applications become larger, build times longer, the iteration speed for developers decreases. To scale our Android and iOS codebase at Square horizontally and keep engineers productive we started to invest in Isolated Development three years ago. The idea to run single features in a sandbox environment was a big shift on the engineering side but also enabled many opportunities. This talk discusses some of our learnings, how we transformed the idea into reality, how teams adopted development apps, how we reduced build and IDE sync times by 10X and where the journey is going next.

Transcript

1. November 02, 2022
   Isolated Development
   Ralf Wondratschek
   @vRallev
   

   View Slide

2. Challenges in 2018
   

   View Slide

3. Challenges in 2018: lines of code growth
   

   View Slide

4. Challenges in 2018: lines of code growth
   

   View Slide

5. Challenges in 2018: build times are growing
   

   View Slide

6. :hairball
   :app2
   :app1
   :lib1 :lib2
   Challenges in 2018: module structure
   

   View Slide

7. :apps
   :feature
   :common
   :api
   Challenges in 2018: module structure
   

   View Slide

8. :hairball
   :app2
   :app1
   :lib1 :lib2
   Challenges in 2018: multiple apps with different flavors
   

   View Slide

9. :hairball
   :app2
   :app1
   :lib1 :lib2
   Challenges in 2018: multiple apps with different flavors
   

   View Slide

10. View Slide

11. View Slide

12. View Slide

13. https://bit.ly/3D8SsUl
    

    View Slide

14. Dependency inversion
    

    View Slide

15. class Feature(s: LoginStrategy) interface LoginStrategy
    class SmsLoginStrategy() : LoginStrategy
    Dependency inversion
    

    View Slide

16. class SmsLoginStrategy @Inject constructor(
    private val helper: Helper,
    private val helper: Helper1,
    private val helper: Helper2,
    private val helper: Helper3,
    private val helper: Helper4,
    private val helper: Helper5,
    private val helper: Helper6,
    private val helper: Helper7,
    private val helper: Helper8
    ) : LoginStrategy
    Dependency inversion
    

    View Slide

17. dependencies {
    api project(':helper1')
    api project(':helper2')
    api project(':helper3')
    api project(':helper4')
    api project(':helper5')
    api project(':helper6')
    api project(':helper7')
    api project(':helper8')
    }
    Dependency inversion
    

    View Slide

18. dependencies {
    api project(':login-strategy')
    }
    Dependency inversion
    

    View Slide

19. Module Structure
    :public
    :impl
    :impl-wiring
    

    View Slide

20. :login-strategy
    Module Structure
    :public
    :impl
    :impl-wiring
    

    View Slide

21. Module Structure
    :login-strategy:public
    :login-strategy:impl
    :login-strategy:impl-wiring
    

    View Slide

22. Module Structure
    

    View Slide

23. :login-strategy
    Module Structure
    :public
    :impl
    :impl-wiring
    :feature
    :public
    :impl
    :impl-wiring
    X
    

    View Slide

24. Development Apps
    :public
    :impl
    :impl-wiring
    

    View Slide

25. Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    

    View Slide

26. Development Apps
    

    View Slide

27. Development Apps
    ● Install a feature with an isolated application on the device
    ○ Launch the feature directly
    ○ Shorter build times
    ● Easy way to experiment and merge code
    

    View Slide

28. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    

    View Slide

29. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    :development-app
    :public
    :impl
    :impl-wiring
    

    View Slide

30. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    :development-app
    :public
    :impl
    :impl-wiring
    :ui-engine
    :public
    :impl
    

    View Slide

31. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    

    View Slide

32. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    :account-screen
    :public
    :impl
    

    View Slide

33. https://square.github.io/workflow/
    

    View Slide

34. :account-screen:public
    interface AccountScreenWorkflow : Workflow>
    

    View Slide

35. :account-screen:impl
    class RealAccountScreenWorkflow @Inject constructor(
    …
    ) : AccountScreenWorkflow
    

    View Slide

36. :login-screen:public
    interface LoginScreenWorkflow : Workflow>
    

    View Slide

37. :login-screen:impl
    class RealLoginScreenWorkflow @Inject constructor(
    private val accountScreenWorkflow: Provider
    ) : LoginScreenWorkflow
    

    View Slide

38. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    :account-screen
    :public
    :impl
    

    View Slide

39. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    :account-screen
    :public
    :impl
    

    View Slide

40. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    :account-screen
    :public
    :fake
    

    View Slide

41. :account-screen:fake
    class FakeAccountScreenWorkflow @Inject constructor() : AccountScreenWorkflow {
    // ...
    }
    

    View Slide

42. Module Structure
    :public
    :impl
    :impl-wiring
    :demo
    

    View Slide

43. Module Structure
    :public
    :impl
    :impl-wiring
    :demo
    :fake
    

    View Slide

44. :login-screen:demo
    android {
    defaultConfig {
    applicationId "com.squareup.sample.login.screen.demo"
    }
    }
    dependencies {
    // …
    implementation project(':account-screen:fake')
    implementation project(':development-app:impl-wiring)
    }
    

    View Slide

45. :login-screen:demo
    @DevelopmentAppComponent
    class DemoLoginApp : DevelopmentApplication()
    @Module
    @ContributesTo(ActivityScope::class)
    object DemoLoginMainActivityModule {
    @Provides
    fun provideFeatureWorkflowProvider(
    workflow: LoginScreenWorkflow
    ): FeatureProvider =
    FeatureWorkflowProvider(
    workflow = workflow,
    props = Unit,
    )
    }
    

    View Slide

46. :login-screen:demo
    @DevelopmentAppComponent
    class DemoLoginApp : DevelopmentApplication()
    @Module
    @ContributesTo(ActivityScope::class)
    object DemoLoginMainActivityModule {
    @Provides
    fun provideFeatureWorkflowProvider(
    workflow: LoginScreenWorkflow
    ): FeatureProvider =
    FeatureWorkflowProvider(
    workflow = workflow,
    props = Unit,
    )
    }
    

    View Slide

47. :login-screen:demo
    @DevelopmentAppComponent
    class DemoLoginApp : DevelopmentApplication()
    @Module
    @ContributesTo(ActivityScope::class)
    object DemoLoginMainActivityModule {
    @Provides
    fun provideFeatureWorkflowProvider(
    workflow: LoginScreenWorkflow
    ): FeatureProvider =
    FeatureWorkflowProvider(
    workflow = workflow,
    props = Unit,
    )
    }
    

    View Slide

48. ● Part of the development apps
    → Faster builds
    → Tend to be very stable
    → More build cache hits
    → Automatic test shardening
    ● Integration tests live in the final applications
    → Test robots are shared across all apps
    UI Tests
    

    View Slide

49. Mechanisms
    

    View Slide

50. plugins {
    id 'com.android.library
    id 'org.jetbrains.kotlin.android'
    id 'org.jetbrains.kotlin.kapt'
    id 'com.squareup.anvil'
    }
    dependencies {
    api project(':account-screen:public')
    ...
    }
    tasks.withType(KotlinCompile).configureEach {
    kotlinOptions {
    jvmTarget = JavaVersion.VERSION_11
    }
    }
    Mechanisms: module structure through convention plugins
    

    View Slide

51. // :login-screen:impl
    plugins {
    id 'com.squareup.android.lib
    }
    // :login-screen:demo
    plugins {
    id 'com.squareup.demo-app
    }
    Mechanisms: module structure through convention plugins
    https://developer.squareup.com/blog/herding-elephants/
    

    View Slide

52. class NoImplDependencyModuleCheck : ModuleCheck {
    override fun runCheck(project: Project) {
    …
    fail(
    "No :impl dependency is allowed. Requested: " +
    "${implDependencies.joinToString { it.dependencyName }} in project: $project."
    )
    }
    }
    Mechanisms: module structure enforced through Lint rules
    

    View Slide

53. Mechanisms: library generator
    

    View Slide

54. Mechanisms: library generator
    

    View Slide

55. Mechanisms: library generator
    

    View Slide

56. Mechanisms: design patterns & frameworks
    ● Workflow
    → Composition was key
    → https://square.github.io/workflow/
    ● Dependency inversion
    → Enforced through the module structure
    ● Dependency Injection
    → Anvil: https://github.com/square/anvil/
    → Dagger 2: https://dagger.dev/
    

    View Slide

57. class RealLoginScreenWorkflow @Inject constructor(
    private val accountScreenWorkflow: Provider
    ) : LoginScreenWorkflow
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/
    

    View Slide

58. class RealLoginScreenWorkflow @Inject constructor(
    private val accountScreenWorkflow: Provider
    ) : LoginScreenWorkflow
    @Module
    abstract class LoginScreenWorkflowModule {
    @Binds abstract fun bindLoginScreenWorkflow(
    workflow: RealLoginScreenWorkflow
    ) : LoginScreenWorkflow
    }
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/
    

    View Slide

59. class RealLoginScreenWorkflow @Inject constructor(
    private val accountScreenWorkflow: Provider
    ) : LoginScreenWorkflow
    @Module
    abstract class LoginScreenWorkflowModule {
    @Binds abstract fun bindLoginScreenWorkflow(
    workflow: RealLoginScreenWorkflow
    ) : LoginScreenWorkflow
    }
    @Component(
    modules = {
    LoginScreenWorkflowModule::class,
    }
    )
    interface AppComponent
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/
    

    View Slide

60. @ContributesBinding(ActivityScope::class)
    class RealLoginScreenWorkflow @Inject constructor(
    private val accountScreenWorkflow: Provider
    ) : LoginScreenWorkflow
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/
    

    View Slide

61. @ContributesBinding(ActivityScope::class)
    class RealLoginScreenWorkflow @Inject constructor(
    private val accountScreenWorkflow: Provider
    ) : LoginScreenWorkflow
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/
    

    View Slide

62. @DevelopmentAppComponent
    class DemoLoginApp : DevelopmentApplication()
    @Module
    @ContributesTo(ActivityScope::class)
    object DemoLoginMainActivityModule {
    @Provides
    fun provideFeatureWorkflowProvider(
    workflow: LoginScreenWorkflow
    ): FeatureProvider =
    FeatureWorkflowProvider(
    workflow = workflow,
    props = Unit,
    )
    }
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/
    

    View Slide

63. @DevelopmentAppComponent
    class DemoLoginApp : DevelopmentApplication()
    @Module
    @ContributesTo(ActivityScope::class)
    object DemoLoginMainActivityModule {
    @Provides
    fun provideFeatureWorkflowProvider(
    workflow: LoginScreenWorkflow
    ): FeatureProvider =
    FeatureWorkflowProvider(
    workflow = workflow,
    props = Unit,
    )
    }
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/
    

    View Slide

64. https://bit.ly/3WbPHdm
    

    View Slide

65. Mechanisms: design patterns & frameworks: Anvil
    https://bit.ly/3WbPHdm
    Try to build :demo module
    Add missing dependencies
    to build.gradle file
    Sync :demo module in
    Android Studio
    Add Dagger modules to
    Dagger components
    

    View Slide

66. Mechanisms: design patterns & frameworks: Anvil
    https://bit.ly/3WbPHdm
    Try to build :demo module
    Add missing dependencies
    to build.gradle file
    Sync :demo module in
    Android Studio
    Add Dagger modules to
    Dagger components
    

    View Slide

67. Mechanisms: dashboards
    

    View Slide

68. Mechanisms: dashboards
    

    View Slide

69. Mechanisms: dashboards
    

    View Slide

70. Mechanisms: dashboards
    

    View Slide

71. Mechanisms: dashboards
    

    View Slide

72. Mechanisms: dashboards
    

    View Slide

73. Mechanisms: dashboards
    

    View Slide

74. Mechanisms: dashboards
    

    View Slide

75. Mechanisms: benchmarks
    https://developer.squareup.com/blog/measure-measure-measure/
    

    View Slide

76. Mechanisms: benchmarks
    https://developer.squareup.com/blog/measure-measure-measure/
    

    View Slide

77. Mechanisms: benchmarks
    https://developer.squareup.com/blog/measure-measure-measure/
    

    View Slide

78. Mechanisms: migrations
    ● We were a small team and relied on the help of feature teams to contribute to common goals
    → Legacy module migration
    → Adopt Anvil
    → Extract UI tests into :demo apps
    https://www.droidcon.com/2022/09/29/migration-without-migraines-automatic-migration-at-scale/
    

    View Slide

79. Challenges
    

    View Slide

80. Challenges: module count
    

    View Slide

81. Solution:
    ● Convention plugins
    ● Benchmarks
    ● Anvil
    ● Configuration caching
    ● Partial IDE sync
    → https://github.com/dropbox/focus
    Challenges: module count
    

    View Slide

82. Legacy modules break our dependency rules
    Solution:
    ● Finish migration
    Challenges: legacy modules
    

    View Slide

83. Anti-pattern
    Challenges: granular modules
    

    View Slide

84. Challenges: granular modules
    :login-screen
    

    View Slide

85. Challenges: granular modules
    :login-screen
    :feature-a
    

    View Slide

86. Challenges: granular modules
    :login-screen
    :feature-a :feature-b
    

    View Slide

87. Challenges: granular modules
    :login-screen
    :feature-a :feature-b
    :login-strategy
    

    View Slide

88. Challenges: granular modules
    :login-screen
    :public
    :impl
    :impl-wiring
    

    View Slide

89. Challenges: granular modules
    :login-screen
    :public
    :impl-a
    :impl-a-wiring
    :impl-b
    :impl-b-wiring
    

    View Slide

90. Challenges: granular modules
    :login-screen
    :public
    :impl-a
    :impl-a-wiring
    :impl-b :impl-c
    :impl-b-wiring :impl-c-wiring
    

    View Slide

91. Challenges: development app shell
    

    View Slide

92. Challenges: extract UI tests
    

    View Slide

93. November 02, 2022
    Isolated Development
    Ralf Wondratschek
    @vRallev
    

    View Slide