One for all, all for one- The Journey to Android Monorepo at Uber

Managing code across multiple repositories at scale impedes developer productivity, comes with a cost of duplication of tooling, processes and dealing with breaking api changes in non atomic change-sets. At Uber, the Android codebase moved from 100+ repositories into a single monolithic repository. This massive migration was done while ensuring VCS history is preserved, breaking API changes across repositories are mitigated, build tooling and infrastructure are scalable and developer workflows are minimally impacted. In this talk, Gautam will deep dive into why the decision was made, the tooling needed to support it, how it was executed and the developer productivity wins that came as a result.

Transcript

1. One for all, all for one The Journey to Android Monorepo at Uber Gautam Korlam Senior Software Engineer Android Developer Experience @kageiit
2. Your Codebase
3. Uber’s Old Android Codebase
4. Uber’s Old Android Codebase Android Rider First came Rider
5. Uber’s Old Android Codebase Android Rider Android Driver Then Driver
6. Uber’s Old Android Codebase Android Rider Android Driver Android Library Shared Library
7. Uber’s Old Android Codebase Android Rider Android Driver Network Storage UI Utils Core Architectural Components
8. Uber’s Old Android Codebase Proliferation of Repositories
9. Uber’s Old Android Codebase Clear Ownership The Good
10. Uber’s Old Android Codebase Clear Ownership Less Toe Stepping The Good
11. Uber’s Old Android Codebase Clear Ownership Less Toe Stepping Better Performance The Good
12. Uber’s Old Android Codebase The Bad Architectural Silos
13. Uber’s Old Android Codebase The Bad Architectural Silos Duplication Of Effort
14. Uber’s Old Android Codebase The Bad Architectural Silos Duplication Of Effort Never up-to-date
15. Uber’s Old Android Codebase Non Atomic Breaking Changes Lib D Lib A Lib B Lib C
16. Uber’s Old Android Codebase Dealing with Dependency Hell
17. Developer Efficiency Decreased
18. Evolution into Monorepo
19. One for all, all for one Alexandre Dumas
20. One for all, all for one
21. One for all, all for one
22. Monorepo in the Wild
25. Will ? Scale?
26. Will VCS Scale?
27. Will Build Times Scale? https://github.com/uber/okbuck https://buckbuild.com
28. Will Development Scale?
29. Uber’s iOS Monorepo https://eng.uber.com/ios-monorepo
30. Mini Monorepo
31. Never Rewrite History
32. ~/Uber/android ├── apps │ ├── android-driver │ │ └── .git │ ├── android-eats │ │ └── .git │ └── android-rider │ └── .git └── libraries ├── android-partner-funnel │ └── .git └── android-help └── .git ~/Uber/android ├── .git ├── apps │ ├── driver │ ├── eats │ └── rider └── libraries ├── partner-funnel └── help https://github.com/unravelin/tomono
33. Evolution into Monorepo Snapshot Regularly Staging the Monolith
34. Evolution into Monorepo Snapshot Regularly Build/Test/Fix Staging the Monolith
35. Evolution into Monorepo Snapshot Regularly Build/Test/Commit Fix Breaking API Changes Staging the Monolith
36. Evolution into Monorepo Breaking API Changes https://github.com/siom79/japicmp
37. The Rollout
38. Code freeze
39. git rm -rf *
40. Lots and Lots of Documentation
42. No. of Files No. of Commits Diffs for Migration Size of Repository No. of Modules 75000 44500 800 350 MB 680 Migration Stats Today
43. Impact on Developer Productivity
44. One Repository
45. One Version
46. One Changeset
47. Faster Builds
48. Always up-to-date
49. Master is always greener on the other side
52. Downsides
53. The Journey is Not Over Yet Tooling Improvements
54. The Journey is Not Over Yet Tooling Improvements Distributed builds
55. The Journey is Not Over Yet Tooling Improvements Distributed builds Automatic dependency cleanup
56. The Journey is Not Over Yet Tooling Improvements Distributed builds Automatic dependency cleanup Duplicate/dead code detection
57. Do Try This at Home!
58. Thank you Proprietary and confidential © 2016 Uber Technologies, Inc. All rights reserved. No part of this document may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval systems, without permission in writing from Uber. This document is intended only for the use of the individual or entity to whom it is addressed and contains information that is privileged, confidential or otherwise exempt from disclosure under applicable law. All recipients of this document are notified that the information contained herein includes proprietary and confidential information of Uber, and recipient may not make use of, disseminate, or in any way disclose this document or any of the enclosed information to any person other than employees of addressee to the extent necessary for consultations with authorized personnel of Uber. Questions?
59. Proprietary and confidential © 2017 Uber Technologies, Inc. All rights reserved. No part of this document may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval systems, without permission in writing from Uber. This document is intended only for the use of the individual or entity to whom it is addressed and contains information that is privileged, confidential or otherwise exempt from disclosure under applicable law. All recipients of this document are notified that the information contained herein includes proprietary and confidential information of Uber, and recipient may not make use of, disseminate, or in any way disclose this document or any of the enclosed information to any person other than employees of addressee to the extent necessary for consultations with authorized personnel of Uber.
60. Attribution for Images used in this deck • Slide 16 - http://tfwiki.net/wiki/File:WFC_OmegaSupreme.jpg • Slide 23 - https://media.giphy.com/media/5H7pDI6uF7ZNm/giphy.gif • Slide 24 - https://imgflip.com/memetemplate/One-Does-Not-Simply • Slide 35 - http://replycandy.com/ew/yao-ming-scared-rage-face • Slide 38 - https://upload.wikimedia.org/wikipedia/commons/0/0e/Matanuska_Glacier_mouth.jpg • Slide 41 - https://giphy.com/gifs/wLOk1aJX9Mis0/ • Slide 42 - https://imgflip.com/memegenerator/1489292/Vegeta-over-9000 • Slide 49 - https://az616578.vo.msecnd.net/files/2016/03/24/6359443246595527281092097913_grass_is_greener.jpg