$30 off During Our Annual Pro Sale. View Details »

Isolated Development

Ralf
November 02, 2022

Isolated Development

With the growth of codebases come more challenges: applications become larger, build times longer, the iteration speed for developers decreases. To scale our Android and iOS codebase at Square horizontally and keep engineers productive we started to invest in Isolated Development three years ago. The idea to run single features in a sandbox environment was a big shift on the engineering side but also enabled many opportunities. This talk discusses some of our learnings, how we transformed the idea into reality, how teams adopted development apps, how we reduced build and IDE sync times by 10X and where the journey is going next.

Ralf

November 02, 2022
Tweet

More Decks by Ralf

Other Decks in Programming

Transcript

  1. November 02, 2022
    Isolated Development
    Ralf Wondratschek
    @vRallev

    View Slide

  2. Challenges in 2018

    View Slide

  3. Challenges in 2018: lines of code growth

    View Slide

  4. Challenges in 2018: lines of code growth

    View Slide

  5. Challenges in 2018: build times are growing

    View Slide

  6. :hairball
    :app2
    :app1
    :lib1 :lib2
    Challenges in 2018: module structure

    View Slide

  7. :apps
    :feature
    :common
    :api
    Challenges in 2018: module structure

    View Slide

  8. :hairball
    :app2
    :app1
    :lib1 :lib2
    Challenges in 2018: multiple apps with different flavors

    View Slide

  9. :hairball
    :app2
    :app1
    :lib1 :lib2
    Challenges in 2018: multiple apps with different flavors

    View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. https://bit.ly/3D8SsUl

    View Slide

  14. Dependency inversion

    View Slide

  15. class Feature(s: LoginStrategy) interface LoginStrategy
    class SmsLoginStrategy() : LoginStrategy
    Dependency inversion

    View Slide

  16. class SmsLoginStrategy @Inject constructor(
    private val helper: Helper,
    private val helper: Helper1,
    private val helper: Helper2,
    private val helper: Helper3,
    private val helper: Helper4,
    private val helper: Helper5,
    private val helper: Helper6,
    private val helper: Helper7,
    private val helper: Helper8
    ) : LoginStrategy
    Dependency inversion

    View Slide

  17. dependencies {
    api project(':helper1')
    api project(':helper2')
    api project(':helper3')
    api project(':helper4')
    api project(':helper5')
    api project(':helper6')
    api project(':helper7')
    api project(':helper8')
    }
    Dependency inversion

    View Slide

  18. dependencies {
    api project(':login-strategy')
    }
    Dependency inversion

    View Slide

  19. Module Structure
    :public
    :impl
    :impl-wiring

    View Slide

  20. :login-strategy
    Module Structure
    :public
    :impl
    :impl-wiring

    View Slide

  21. Module Structure
    :login-strategy:public
    :login-strategy:impl
    :login-strategy:impl-wiring

    View Slide

  22. Module Structure

    View Slide

  23. :login-strategy
    Module Structure
    :public
    :impl
    :impl-wiring
    :feature
    :public
    :impl
    :impl-wiring
    X

    View Slide

  24. Development Apps
    :public
    :impl
    :impl-wiring

    View Slide

  25. Development Apps
    :public
    :impl
    :impl-wiring
    :demo

    View Slide

  26. Development Apps

    View Slide

  27. Development Apps
    ● Install a feature with an isolated application on the device
    ○ Launch the feature directly
    ○ Shorter build times
    ● Easy way to experiment and merge code

    View Slide

  28. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo

    View Slide

  29. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    :development-app
    :public
    :impl
    :impl-wiring

    View Slide

  30. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    :development-app
    :public
    :impl
    :impl-wiring
    :ui-engine
    :public
    :impl

    View Slide

  31. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo

    View Slide

  32. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    :account-screen
    :public
    :impl

    View Slide

  33. https://square.github.io/workflow/

    View Slide

  34. :account-screen:public
    interface AccountScreenWorkflow : Workflow>

    View Slide

  35. :account-screen:impl
    class RealAccountScreenWorkflow @Inject constructor(

    ) : AccountScreenWorkflow

    View Slide

  36. :login-screen:public
    interface LoginScreenWorkflow : Workflow>

    View Slide

  37. :login-screen:impl
    class RealLoginScreenWorkflow @Inject constructor(
    private val accountScreenWorkflow: Provider
    ) : LoginScreenWorkflow

    View Slide

  38. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    :account-screen
    :public
    :impl

    View Slide

  39. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    :account-screen
    :public
    :impl

    View Slide

  40. :login-screen
    Development Apps
    :public
    :impl
    :impl-wiring
    :demo
    :account-screen
    :public
    :fake

    View Slide

  41. :account-screen:fake
    class FakeAccountScreenWorkflow @Inject constructor() : AccountScreenWorkflow {
    // ...
    }

    View Slide

  42. Module Structure
    :public
    :impl
    :impl-wiring
    :demo

    View Slide

  43. Module Structure
    :public
    :impl
    :impl-wiring
    :demo
    :fake

    View Slide

  44. :login-screen:demo
    android {
    defaultConfig {
    applicationId "com.squareup.sample.login.screen.demo"
    }
    }
    dependencies {
    // …
    implementation project(':account-screen:fake')
    implementation project(':development-app:impl-wiring)
    }

    View Slide

  45. :login-screen:demo
    @DevelopmentAppComponent
    class DemoLoginApp : DevelopmentApplication()
    @Module
    @ContributesTo(ActivityScope::class)
    object DemoLoginMainActivityModule {
    @Provides
    fun provideFeatureWorkflowProvider(
    workflow: LoginScreenWorkflow
    ): FeatureProvider =
    FeatureWorkflowProvider(
    workflow = workflow,
    props = Unit,
    )
    }

    View Slide

  46. :login-screen:demo
    @DevelopmentAppComponent
    class DemoLoginApp : DevelopmentApplication()
    @Module
    @ContributesTo(ActivityScope::class)
    object DemoLoginMainActivityModule {
    @Provides
    fun provideFeatureWorkflowProvider(
    workflow: LoginScreenWorkflow
    ): FeatureProvider =
    FeatureWorkflowProvider(
    workflow = workflow,
    props = Unit,
    )
    }

    View Slide

  47. :login-screen:demo
    @DevelopmentAppComponent
    class DemoLoginApp : DevelopmentApplication()
    @Module
    @ContributesTo(ActivityScope::class)
    object DemoLoginMainActivityModule {
    @Provides
    fun provideFeatureWorkflowProvider(
    workflow: LoginScreenWorkflow
    ): FeatureProvider =
    FeatureWorkflowProvider(
    workflow = workflow,
    props = Unit,
    )
    }

    View Slide

  48. ● Part of the development apps
    → Faster builds
    → Tend to be very stable
    → More build cache hits
    → Automatic test shardening
    ● Integration tests live in the final applications
    → Test robots are shared across all apps
    UI Tests

    View Slide

  49. Mechanisms

    View Slide

  50. plugins {
    id 'com.android.library
    id 'org.jetbrains.kotlin.android'
    id 'org.jetbrains.kotlin.kapt'
    id 'com.squareup.anvil'
    }
    dependencies {
    api project(':account-screen:public')
    ...
    }
    tasks.withType(KotlinCompile).configureEach {
    kotlinOptions {
    jvmTarget = JavaVersion.VERSION_11
    }
    }
    Mechanisms: module structure through convention plugins

    View Slide

  51. // :login-screen:impl
    plugins {
    id 'com.squareup.android.lib
    }
    // :login-screen:demo
    plugins {
    id 'com.squareup.demo-app
    }
    Mechanisms: module structure through convention plugins
    https://developer.squareup.com/blog/herding-elephants/

    View Slide

  52. class NoImplDependencyModuleCheck : ModuleCheck {
    override fun runCheck(project: Project) {

    fail(
    "No :impl dependency is allowed. Requested: " +
    "${implDependencies.joinToString { it.dependencyName }} in project: $project."
    )
    }
    }
    Mechanisms: module structure enforced through Lint rules

    View Slide

  53. Mechanisms: library generator

    View Slide

  54. Mechanisms: library generator

    View Slide

  55. Mechanisms: library generator

    View Slide

  56. Mechanisms: design patterns & frameworks
    ● Workflow
    → Composition was key
    → https://square.github.io/workflow/
    ● Dependency inversion
    → Enforced through the module structure
    ● Dependency Injection
    → Anvil: https://github.com/square/anvil/
    → Dagger 2: https://dagger.dev/

    View Slide

  57. class RealLoginScreenWorkflow @Inject constructor(
    private val accountScreenWorkflow: Provider
    ) : LoginScreenWorkflow
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/

    View Slide

  58. class RealLoginScreenWorkflow @Inject constructor(
    private val accountScreenWorkflow: Provider
    ) : LoginScreenWorkflow
    @Module
    abstract class LoginScreenWorkflowModule {
    @Binds abstract fun bindLoginScreenWorkflow(
    workflow: RealLoginScreenWorkflow
    ) : LoginScreenWorkflow
    }
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/

    View Slide

  59. class RealLoginScreenWorkflow @Inject constructor(
    private val accountScreenWorkflow: Provider
    ) : LoginScreenWorkflow
    @Module
    abstract class LoginScreenWorkflowModule {
    @Binds abstract fun bindLoginScreenWorkflow(
    workflow: RealLoginScreenWorkflow
    ) : LoginScreenWorkflow
    }
    @Component(
    modules = {
    LoginScreenWorkflowModule::class,
    }
    )
    interface AppComponent
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/

    View Slide

  60. @ContributesBinding(ActivityScope::class)
    class RealLoginScreenWorkflow @Inject constructor(
    private val accountScreenWorkflow: Provider
    ) : LoginScreenWorkflow
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/

    View Slide

  61. @ContributesBinding(ActivityScope::class)
    class RealLoginScreenWorkflow @Inject constructor(
    private val accountScreenWorkflow: Provider
    ) : LoginScreenWorkflow
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/

    View Slide

  62. @DevelopmentAppComponent
    class DemoLoginApp : DevelopmentApplication()
    @Module
    @ContributesTo(ActivityScope::class)
    object DemoLoginMainActivityModule {
    @Provides
    fun provideFeatureWorkflowProvider(
    workflow: LoginScreenWorkflow
    ): FeatureProvider =
    FeatureWorkflowProvider(
    workflow = workflow,
    props = Unit,
    )
    }
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/

    View Slide

  63. @DevelopmentAppComponent
    class DemoLoginApp : DevelopmentApplication()
    @Module
    @ContributesTo(ActivityScope::class)
    object DemoLoginMainActivityModule {
    @Provides
    fun provideFeatureWorkflowProvider(
    workflow: LoginScreenWorkflow
    ): FeatureProvider =
    FeatureWorkflowProvider(
    workflow = workflow,
    props = Unit,
    )
    }
    Mechanisms: design patterns & frameworks: Anvil
    https://github.com/square/anvil/

    View Slide

  64. https://bit.ly/3WbPHdm

    View Slide

  65. Mechanisms: design patterns & frameworks: Anvil
    https://bit.ly/3WbPHdm
    Try to build :demo module
    Add missing dependencies
    to build.gradle file
    Sync :demo module in
    Android Studio
    Add Dagger modules to
    Dagger components

    View Slide

  66. Mechanisms: design patterns & frameworks: Anvil
    https://bit.ly/3WbPHdm
    Try to build :demo module
    Add missing dependencies
    to build.gradle file
    Sync :demo module in
    Android Studio
    Add Dagger modules to
    Dagger components

    View Slide

  67. Mechanisms: dashboards

    View Slide

  68. Mechanisms: dashboards

    View Slide

  69. Mechanisms: dashboards

    View Slide

  70. Mechanisms: dashboards

    View Slide

  71. Mechanisms: dashboards

    View Slide

  72. Mechanisms: dashboards

    View Slide

  73. Mechanisms: dashboards

    View Slide

  74. Mechanisms: dashboards

    View Slide

  75. Mechanisms: benchmarks
    https://developer.squareup.com/blog/measure-measure-measure/

    View Slide

  76. Mechanisms: benchmarks
    https://developer.squareup.com/blog/measure-measure-measure/

    View Slide

  77. Mechanisms: benchmarks
    https://developer.squareup.com/blog/measure-measure-measure/

    View Slide

  78. Mechanisms: migrations
    ● We were a small team and relied on the help of feature teams to contribute to common goals
    → Legacy module migration
    → Adopt Anvil
    → Extract UI tests into :demo apps
    https://www.droidcon.com/2022/09/29/migration-without-migraines-automatic-migration-at-scale/

    View Slide

  79. Challenges

    View Slide

  80. Challenges: module count

    View Slide

  81. Solution:
    ● Convention plugins
    ● Benchmarks
    ● Anvil
    ● Configuration caching
    ● Partial IDE sync
    → https://github.com/dropbox/focus
    Challenges: module count

    View Slide

  82. Legacy modules break our dependency rules
    Solution:
    ● Finish migration
    Challenges: legacy modules

    View Slide

  83. Anti-pattern
    Challenges: granular modules

    View Slide

  84. Challenges: granular modules
    :login-screen

    View Slide

  85. Challenges: granular modules
    :login-screen
    :feature-a

    View Slide

  86. Challenges: granular modules
    :login-screen
    :feature-a :feature-b

    View Slide

  87. Challenges: granular modules
    :login-screen
    :feature-a :feature-b
    :login-strategy

    View Slide

  88. Challenges: granular modules
    :login-screen
    :public
    :impl
    :impl-wiring

    View Slide

  89. Challenges: granular modules
    :login-screen
    :public
    :impl-a
    :impl-a-wiring
    :impl-b
    :impl-b-wiring

    View Slide

  90. Challenges: granular modules
    :login-screen
    :public
    :impl-a
    :impl-a-wiring
    :impl-b :impl-c
    :impl-b-wiring :impl-c-wiring

    View Slide

  91. Challenges: development app shell

    View Slide

  92. Challenges: extract UI tests

    View Slide

  93. November 02, 2022
    Isolated Development
    Ralf Wondratschek
    @vRallev

    View Slide