$30 off During Our Annual Pro Sale. View Details »

Isolated Development

Ralf
November 02, 2022

Isolated Development

With the growth of codebases come more challenges: applications become larger, build times longer, the iteration speed for developers decreases. To scale our Android and iOS codebase at Square horizontally and keep engineers productive we started to invest in Isolated Development three years ago. The idea to run single features in a sandbox environment was a big shift on the engineering side but also enabled many opportunities. This talk discusses some of our learnings, how we transformed the idea into reality, how teams adopted development apps, how we reduced build and IDE sync times by 10X and where the journey is going next.

Ralf

November 02, 2022
Tweet

More Decks by Ralf

Other Decks in Programming

Transcript

  1. November 02, 2022 Isolated Development Ralf Wondratschek @vRallev

  2. Challenges in 2018

  3. Challenges in 2018: lines of code growth

  4. Challenges in 2018: lines of code growth

  5. Challenges in 2018: build times are growing

  6. :hairball :app2 :app1 :lib1 :lib2 Challenges in 2018: module structure

  7. :apps :feature :common :api Challenges in 2018: module structure

  8. :hairball :app2 :app1 :lib1 :lib2 Challenges in 2018: multiple apps

    with different flavors
  9. :hairball :app2 :app1 :lib1 :lib2 Challenges in 2018: multiple apps

    with different flavors
  10. None
  11. None
  12. None
  13. https://bit.ly/3D8SsUl

  14. Dependency inversion

  15. class Feature(s: LoginStrategy) interface LoginStrategy class SmsLoginStrategy() : LoginStrategy Dependency

    inversion
  16. class SmsLoginStrategy @Inject constructor( private val helper: Helper, private val

    helper: Helper1, private val helper: Helper2, private val helper: Helper3, private val helper: Helper4, private val helper: Helper5, private val helper: Helper6, private val helper: Helper7, private val helper: Helper8 ) : LoginStrategy Dependency inversion
  17. dependencies { api project(':helper1') api project(':helper2') api project(':helper3') api project(':helper4')

    api project(':helper5') api project(':helper6') api project(':helper7') api project(':helper8') } Dependency inversion
  18. dependencies { api project(':login-strategy') } Dependency inversion

  19. Module Structure :public :impl :impl-wiring

  20. :login-strategy Module Structure :public :impl :impl-wiring

  21. Module Structure :login-strategy:public :login-strategy:impl :login-strategy:impl-wiring

  22. Module Structure

  23. :login-strategy Module Structure :public :impl :impl-wiring :feature :public :impl :impl-wiring

    X
  24. Development Apps :public :impl :impl-wiring

  25. Development Apps :public :impl :impl-wiring :demo

  26. Development Apps

  27. Development Apps • Install a feature with an isolated application

    on the device ◦ Launch the feature directly ◦ Shorter build times • Easy way to experiment and merge code
  28. :login-screen Development Apps :public :impl :impl-wiring :demo

  29. :login-screen Development Apps :public :impl :impl-wiring :demo :development-app :public :impl

    :impl-wiring
  30. :login-screen Development Apps :public :impl :impl-wiring :demo :development-app :public :impl

    :impl-wiring :ui-engine :public :impl
  31. :login-screen Development Apps :public :impl :impl-wiring :demo

  32. :login-screen Development Apps :public :impl :impl-wiring :demo :account-screen :public :impl

  33. https://square.github.io/workflow/

  34. :account-screen:public interface AccountScreenWorkflow : Workflow<Unit, Unit, LayeredScreen<PosLayering>>

  35. :account-screen:impl class RealAccountScreenWorkflow @Inject constructor( … ) : AccountScreenWorkflow

  36. :login-screen:public interface LoginScreenWorkflow : Workflow<Unit, Unit, LayeredScreen<PosLayering>>

  37. :login-screen:impl class RealLoginScreenWorkflow @Inject constructor( private val accountScreenWorkflow: Provider<AccountScreenWorkflow> )

    : LoginScreenWorkflow
  38. :login-screen Development Apps :public :impl :impl-wiring :demo :account-screen :public :impl

  39. :login-screen Development Apps :public :impl :impl-wiring :demo :account-screen :public :impl

  40. :login-screen Development Apps :public :impl :impl-wiring :demo :account-screen :public :fake

  41. :account-screen:fake class FakeAccountScreenWorkflow @Inject constructor() : AccountScreenWorkflow { // ...

    }
  42. Module Structure :public :impl :impl-wiring :demo

  43. Module Structure :public :impl :impl-wiring :demo :fake

  44. :login-screen:demo android { defaultConfig { applicationId "com.squareup.sample.login.screen.demo" } } dependencies

    { // … implementation project(':account-screen:fake') implementation project(':development-app:impl-wiring) }
  45. :login-screen:demo @DevelopmentAppComponent class DemoLoginApp : DevelopmentApplication() @Module @ContributesTo(ActivityScope::class) object DemoLoginMainActivityModule

    { @Provides fun provideFeatureWorkflowProvider( workflow: LoginScreenWorkflow ): FeatureProvider = FeatureWorkflowProvider( workflow = workflow, props = Unit, ) }
  46. :login-screen:demo @DevelopmentAppComponent class DemoLoginApp : DevelopmentApplication() @Module @ContributesTo(ActivityScope::class) object DemoLoginMainActivityModule

    { @Provides fun provideFeatureWorkflowProvider( workflow: LoginScreenWorkflow ): FeatureProvider = FeatureWorkflowProvider( workflow = workflow, props = Unit, ) }
  47. :login-screen:demo @DevelopmentAppComponent class DemoLoginApp : DevelopmentApplication() @Module @ContributesTo(ActivityScope::class) object DemoLoginMainActivityModule

    { @Provides fun provideFeatureWorkflowProvider( workflow: LoginScreenWorkflow ): FeatureProvider = FeatureWorkflowProvider( workflow = workflow, props = Unit, ) }
  48. • Part of the development apps → Faster builds →

    Tend to be very stable → More build cache hits → Automatic test shardening • Integration tests live in the final applications → Test robots are shared across all apps UI Tests
  49. Mechanisms

  50. plugins { id 'com.android.library id 'org.jetbrains.kotlin.android' id 'org.jetbrains.kotlin.kapt' id 'com.squareup.anvil'

    } dependencies { api project(':account-screen:public') ... } tasks.withType(KotlinCompile).configureEach { kotlinOptions { jvmTarget = JavaVersion.VERSION_11 } } Mechanisms: module structure through convention plugins
  51. // :login-screen:impl plugins { id 'com.squareup.android.lib } // :login-screen:demo plugins

    { id 'com.squareup.demo-app } Mechanisms: module structure through convention plugins https://developer.squareup.com/blog/herding-elephants/
  52. class NoImplDependencyModuleCheck : ModuleCheck { override fun runCheck(project: Project) {

    … fail( "No :impl dependency is allowed. Requested: " + "${implDependencies.joinToString { it.dependencyName }} in project: $project." ) } } Mechanisms: module structure enforced through Lint rules
  53. Mechanisms: library generator

  54. Mechanisms: library generator

  55. Mechanisms: library generator

  56. Mechanisms: design patterns & frameworks • Workflow → Composition was

    key → https://square.github.io/workflow/ • Dependency inversion → Enforced through the module structure • Dependency Injection → Anvil: https://github.com/square/anvil/ → Dagger 2: https://dagger.dev/
  57. class RealLoginScreenWorkflow @Inject constructor( private val accountScreenWorkflow: Provider<AccountScreenWorkflow> ) :

    LoginScreenWorkflow Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
  58. class RealLoginScreenWorkflow @Inject constructor( private val accountScreenWorkflow: Provider<AccountScreenWorkflow> ) :

    LoginScreenWorkflow @Module abstract class LoginScreenWorkflowModule { @Binds abstract fun bindLoginScreenWorkflow( workflow: RealLoginScreenWorkflow ) : LoginScreenWorkflow } Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
  59. class RealLoginScreenWorkflow @Inject constructor( private val accountScreenWorkflow: Provider<AccountScreenWorkflow> ) :

    LoginScreenWorkflow @Module abstract class LoginScreenWorkflowModule { @Binds abstract fun bindLoginScreenWorkflow( workflow: RealLoginScreenWorkflow ) : LoginScreenWorkflow } @Component( modules = { LoginScreenWorkflowModule::class, } ) interface AppComponent Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
  60. @ContributesBinding(ActivityScope::class) class RealLoginScreenWorkflow @Inject constructor( private val accountScreenWorkflow: Provider<AccountScreenWorkflow> )

    : LoginScreenWorkflow Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
  61. @ContributesBinding(ActivityScope::class) class RealLoginScreenWorkflow @Inject constructor( private val accountScreenWorkflow: Provider<AccountScreenWorkflow> )

    : LoginScreenWorkflow Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
  62. @DevelopmentAppComponent class DemoLoginApp : DevelopmentApplication() @Module @ContributesTo(ActivityScope::class) object DemoLoginMainActivityModule {

    @Provides fun provideFeatureWorkflowProvider( workflow: LoginScreenWorkflow ): FeatureProvider = FeatureWorkflowProvider( workflow = workflow, props = Unit, ) } Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
  63. @DevelopmentAppComponent class DemoLoginApp : DevelopmentApplication() @Module @ContributesTo(ActivityScope::class) object DemoLoginMainActivityModule {

    @Provides fun provideFeatureWorkflowProvider( workflow: LoginScreenWorkflow ): FeatureProvider = FeatureWorkflowProvider( workflow = workflow, props = Unit, ) } Mechanisms: design patterns & frameworks: Anvil https://github.com/square/anvil/
  64. https://bit.ly/3WbPHdm

  65. Mechanisms: design patterns & frameworks: Anvil https://bit.ly/3WbPHdm Try to build

    :demo module Add missing dependencies to build.gradle file Sync :demo module in Android Studio Add Dagger modules to Dagger components
  66. Mechanisms: design patterns & frameworks: Anvil https://bit.ly/3WbPHdm Try to build

    :demo module Add missing dependencies to build.gradle file Sync :demo module in Android Studio Add Dagger modules to Dagger components
  67. Mechanisms: dashboards

  68. Mechanisms: dashboards

  69. Mechanisms: dashboards

  70. Mechanisms: dashboards

  71. Mechanisms: dashboards

  72. Mechanisms: dashboards

  73. Mechanisms: dashboards

  74. Mechanisms: dashboards

  75. Mechanisms: benchmarks https://developer.squareup.com/blog/measure-measure-measure/

  76. Mechanisms: benchmarks https://developer.squareup.com/blog/measure-measure-measure/

  77. Mechanisms: benchmarks https://developer.squareup.com/blog/measure-measure-measure/

  78. Mechanisms: migrations • We were a small team and relied

    on the help of feature teams to contribute to common goals → Legacy module migration → Adopt Anvil → Extract UI tests into :demo apps https://www.droidcon.com/2022/09/29/migration-without-migraines-automatic-migration-at-scale/
  79. Challenges

  80. Challenges: module count

  81. Solution: • Convention plugins • Benchmarks • Anvil • Configuration

    caching • Partial IDE sync → https://github.com/dropbox/focus Challenges: module count
  82. Legacy modules break our dependency rules Solution: • Finish migration

    Challenges: legacy modules
  83. Anti-pattern Challenges: granular modules

  84. Challenges: granular modules :login-screen

  85. Challenges: granular modules :login-screen :feature-a

  86. Challenges: granular modules :login-screen :feature-a :feature-b

  87. Challenges: granular modules :login-screen :feature-a :feature-b :login-strategy

  88. Challenges: granular modules :login-screen :public :impl :impl-wiring

  89. Challenges: granular modules :login-screen :public :impl-a :impl-a-wiring :impl-b :impl-b-wiring

  90. Challenges: granular modules :login-screen :public :impl-a :impl-a-wiring :impl-b :impl-c :impl-b-wiring

    :impl-c-wiring
  91. Challenges: development app shell

  92. Challenges: extract UI tests

  93. November 02, 2022 Isolated Development Ralf Wondratschek @vRallev