Modularization - Siggi Jonsson

Since we took the stage at Droidcon NYC 2019 to talk about modularization at Tinder, our team and codebase have grown a lot and so has our project and build complexity. This talk will look back at some of the mistakes we made with our original modularization strategy and the impact that had on our engineering teams. We will look at the tools we used to analyze our dependency and build graphs, and how we identified bottlenecks in our processes. Finally, we will illustrate how we pivoted our strategy and restructured the project in a way that optimizes for the independence and efficiency of our feature teams and automates key decisions when it comes to dependency management, all without being disruptive to our engineering teams and ensuring we can continue evolving with the changing landscape of Android development.

Transcript

1. Siggi Jonsson - @siggijons Modularization Untangling the dependency graph
2. app
3. domain data app
4. app c:ui b:ui a:ui
5. app c:domain c:data c:ui b:domain b:data b:ui a:domain a:data a:ui
7. Modularization Module Growth 0 150 300 450 600 2018 2019 2020 2021 Modules
8. Modularization Module Dependency Growth 0 1500 3000 4500 6000 2018 2019 2020 2021 Modules Dependencies
9. Revisited strategy
10. Three Layered Structure app feature library
11. Three+Two Layered Structure app feature:public library:public feature:internal library:internal
12. Three+Two Layered Structure app feature:public library:public feature:internal library:internal
13. Three+Two Layered Structure app feature:public library:public feature:internal library:internal
14. Now what?
15. Graph Assert Gradle Plugin github.com/jraska/modules-graph-assert
16. Graph Assert Allowed Dependencies restricted = [ ':library:.* -> :feature:.*', '.*:public -> .*:internal’, '.*:internal -> .*:internal' ] moduleGraphAssert { maxHeight = 24 }
17. Graph Assert Allowed Dependencies moduleGraphAssert { maxHeight = 24 restricted = [ ':library:.* -> :feature:.*', '.*:public -> .*:internal’, '.*:internal -> .*:internal' ] }
18. Graph Assert Statistics GraphStatistics( modulesCount=598, edgesCount=4389, height=24, longestPath=':Tinder -> [...] -> :common:kotlinx-coroutines’ ) $ gw generateModulesGraphStatistics
19. Graph Assert Graph Generation digraph G { … “:chat:ui” -> “:common:datetime" “:chat:ui” -> ":common:dialog" ":chat:ui" -> ":chat:domain" ":chat:ui" -> “:safety-center:ui" ":chat:ui" -> ":profile:domain" … } $ gw generateModulesGraphvizText
20. github.com/jraska/modules-graph-assert Graph Assert Graph Generation
21. Graph Assert Graphviz $ dot -Tpng
22. Gephi
23. Gephi Out of the box
30. Graph Theory
31. In-Degree
32. Gephi In-Degree
33. Out-Degree
34. Gephi Out-Degree
35. Degree
36. Gephi Degree
37. Gephi Modularity Class
38. Betweenness Centrality
39. Graph Theory Betweenness centrality • Betweenness centrality is a way of detecting the amount of influence a node has over the flow of information in a graph. It is often used to find nodes that serve as a bridge from one part of a graph to another.
40. Betweenness Centrality
41. Gephi Betweenness centrality
42. Gephi Putting the metrics into action • Color: Modularity class • Node Size: Betweenness centrality • Layout: Force Atlas
45. Putting graph theory to use Betweenness Centrality Correlations • Modules with too many responsibilities • Modules with unclear boundaries • Modules which had an outsized impact on build times
47. Putting graph theory to use Capturing Metrics and Acting • Capture and report height • Capture and report betweenness centrality • Work towards reducing both, prioritizing the top ones
48. Putting graph theory to use Reduce Height
49. Putting graph theory to use Reduce Betweenness Centrality
50. Putting graph theory to use Reduce Betweenness Centrality
51. • Simplified build graph • Improved feature and library isolation • Faster builds • Happier engineers Putting graph theory to use Results
52. OK?
53. Case Study
54. Case Study Step 1: Add Graph Assert Plugin // app/build.gradle.kts plugins { ... id("com.jraska.module.graph.assertion") version “2.2.0” }
55. Case Study Step 2: Generate Statistics $ gw generateModulesGraphStatistics GraphStatistics( modulesCount=14, edgesCount=39, height=4, longestPath=':app -> :feature-author -> :core-data -> :core-database -> :core-model’ )
56. Case Study Step 3: Generate Graph File $ gw generateModulesGraphvizText -Pmodules.graph.output.gv=graph.gv GraphViz saved to graph.gv digraph G { ":app" -> ":feature-author" ":app" -> ":feature-interests" ":app" -> ":feature-foryou" ":app" -> ":feature-topic" ":app" -> ":core-ui" ":app" -> ":core-navigation" ... }
57. Case Study Step 4: Generate Graph $ dot -Tpng graph.gv -o graph.png
58. Case Study Step 5: Open in Gephi
59. Case Study Step 6: Run stats
60. Case Study Step 6: Run stats
61. Case Study Step 6: Run stats
62. Case Study Step 6: Run stats
63. • Node Color: Modularity class • Node Size: Betweenness centrality • Layout: Force Atlas Case Study Step 7: Adjust Appearance
66. Case Study Now in Android
67. Case Study Now in Android [ ] Extract core-result out of core-common
68. Case Study Now in Android
69. Case Study Now in Android
70. Case Study Now in Android
71. Case Study Now in Android
72. Case Study Now in Android [ ] Extract core-result out of core-common [ ] Extract repository interfaces out of core-data
73. Splitting a library
74. Splitting a library
75. Splitting a library
76. Case Study Now in Android [ ] Extract core-result out of core-common [ ] Extract repository interfaces out of core-data
77. Case Study Now in Android [x] Extract core-result out of core-common [x] Extract repository interfaces out of core-data
78. Case Study Now in Android
79. Case Study Now in Android
80. Case Study Now in Android
81. Case Study Now in Android
82. Case Study Now in Android Before After
83. One more thing
84. Case Study Step 3*: Skip to the insights $ gw generateModulesGraphNodeStatistics node betweennessCentrality degree inDegree outDegree height :core-data 18.00 10 5 5 2 :sync 2.10 5 1 4 3 :feature-author 1.10 6 1 5 3 :feature-topic 1.10 6 1 5 3 :feature-interests 0.77 5 1 4 3 :feature-foryou 0.77 5 1 4 3 :core-ui 0.17 6 5 1 1 :app 0.00 7 0 7 4 :core-model 0.00 9 9 0 0 :core-common 0.00 6 6 0 0 :core-database 0.00 2 1 1 1 :core-datastore 0.00 3 2 1 1 :core-network 0.00 3 1 2 1 :core-navigation 0.00 5 5 0 0 PR: github.com/jraska/modules-graph-assert/pull/176
85. Case Study $ gw generateModulesGraphNodeStatistics node betweennessCentrality degree inDegree outDegree height :sync 5.17 5 1 4 3 :core-data 5.00 7 1 6 2 :feature-author 0.92 6 1 5 2 :feature-topic 0.92 6 1 5 2 :feature-interests 0.42 5 1 4 2 :feature-foryou 0.42 5 1 4 2 :core-ui 0.17 6 5 1 1 :app 0.00 7 0 7 4 :core-model 0.00 10 10 0 0 :core-data-public 0.00 6 5 1 1 :core-result 0.00 2 2 0 0 :core-navigation 0.00 5 5 0 0 :core-common 0.00 4 4 0 0 :core-database 0.00 2 1 1 1 :core-datastore 0.00 3 2 1 1 :core-network 0.00 3 1 2 1 Step 3*: Skip to the insights PR: github.com/jraska/modules-graph-assert/pull/176
86. Summary Flatten the graph • Visualize dependencies using Graph Assert, GraphViz and Gephi • Capture and reduce graph height • Capture and reduce the betweenness centrality of module
87. Thanks • Iñaki Villar - Tinder • Joseph Raska - Graph Assert • Gephi • JGraphT
88. Thank You! Come see me at our booth for questions! Also, we’re hiring! lifeattinder.com @siggijons