Save 37% off PRO during our Black Friday Sale! »

One for all, all for one- The Journey to Android Monorepo at Uber

3f9ebfb254c56d281d91bbee960219a8?s=47 Gautam Korlam
September 26, 2017

One for all, all for one- The Journey to Android Monorepo at Uber

Managing code across multiple repositories at scale impedes developer productivity, comes with a cost of duplication of tooling, processes and dealing with breaking api changes in non atomic change-sets. At Uber, the Android codebase moved from 100+ repositories into a single monolithic repository. This massive migration was done while ensuring VCS history is preserved, breaking API changes across repositories are mitigated, build tooling and infrastructure are scalable and developer workflows are minimally impacted. In this talk, Gautam will deep dive into why the decision was made, the tooling needed to support it, how it was executed and the developer productivity wins that came as a result.

3f9ebfb254c56d281d91bbee960219a8?s=128

Gautam Korlam

September 26, 2017
Tweet

Transcript

  1. One for all, all for one The Journey to Android

    Monorepo at Uber Gautam Korlam Senior Software Engineer Android Developer Experience @kageiit
  2. Your Codebase

  3. Uber’s Old Android Codebase

  4. Uber’s Old Android Codebase Android Rider First came Rider

  5. Uber’s Old Android Codebase Android Rider Android Driver Then Driver

  6. Uber’s Old Android Codebase Android Rider Android Driver Android Library

    Shared Library
  7. Uber’s Old Android Codebase Android Rider Android Driver Network Storage

    UI Utils Core Architectural Components
  8. Uber’s Old Android Codebase Proliferation of Repositories

  9. Uber’s Old Android Codebase Clear Ownership The Good

  10. Uber’s Old Android Codebase Clear Ownership Less Toe Stepping The

    Good
  11. Uber’s Old Android Codebase Clear Ownership Less Toe Stepping Better

    Performance The Good
  12. Uber’s Old Android Codebase The Bad Architectural Silos

  13. Uber’s Old Android Codebase The Bad Architectural Silos Duplication Of

    Effort
  14. Uber’s Old Android Codebase The Bad Architectural Silos Duplication Of

    Effort Never up-to-date
  15. Uber’s Old Android Codebase Non Atomic Breaking Changes Lib D

    Lib A Lib B Lib C
  16. Uber’s Old Android Codebase Dealing with Dependency Hell

  17. Developer Efficiency Decreased

  18. Evolution into Monorepo

  19. One for all, all for one Alexandre Dumas

  20. One for all, all for one

  21. One for all, all for one

  22. Monorepo in the Wild

  23. None
  24. None
  25. Will ? Scale?

  26. Will VCS Scale?

  27. Will Build Times Scale? https://github.com/uber/okbuck https://buckbuild.com

  28. Will Development Scale?

  29. Uber’s iOS Monorepo https://eng.uber.com/ios-monorepo

  30. Mini Monorepo

  31. Never Rewrite History

  32. ~/Uber/android ├── apps │ ├── android-driver │ │ └── .git

    │ ├── android-eats │ │ └── .git │ └── android-rider │ └── .git └── libraries ├── android-partner-funnel │ └── .git └── android-help └── .git ~/Uber/android ├── .git ├── apps │ ├── driver │ ├── eats │ └── rider └── libraries ├── partner-funnel └── help https://github.com/unravelin/tomono
  33. Evolution into Monorepo Snapshot Regularly Staging the Monolith

  34. Evolution into Monorepo Snapshot Regularly Build/Test/Fix Staging the Monolith

  35. Evolution into Monorepo Snapshot Regularly Build/Test/Commit Fix Breaking API Changes

    Staging the Monolith
  36. Evolution into Monorepo Breaking API Changes https://github.com/siom79/japicmp

  37. The Rollout

  38. Code freeze

  39. git rm -rf *

  40. Lots and Lots of Documentation

  41. None
  42. No. of Files No. of Commits Diffs for Migration Size

    of Repository No. of Modules 75000 44500 800 350 MB 680 Migration Stats Today
  43. Impact on Developer Productivity

  44. One Repository

  45. One Version

  46. One Changeset

  47. Faster Builds

  48. Always up-to-date

  49. Master is always greener on the other side

  50. None
  51. None
  52. Downsides

  53. The Journey is Not Over Yet Tooling Improvements

  54. The Journey is Not Over Yet Tooling Improvements Distributed builds

  55. The Journey is Not Over Yet Tooling Improvements Distributed builds

    Automatic dependency cleanup
  56. The Journey is Not Over Yet Tooling Improvements Distributed builds

    Automatic dependency cleanup Duplicate/dead code detection
  57. Do Try This at Home!

  58. Thank you Proprietary and confidential © 2016 Uber Technologies, Inc.

    All rights reserved. No part of this document may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval systems, without permission in writing from Uber. This document is intended only for the use of the individual or entity to whom it is addressed and contains information that is privileged, confidential or otherwise exempt from disclosure under applicable law. All recipients of this document are notified that the information contained herein includes proprietary and confidential information of Uber, and recipient may not make use of, disseminate, or in any way disclose this document or any of the enclosed information to any person other than employees of addressee to the extent necessary for consultations with authorized personnel of Uber. Questions?
  59. Proprietary and confidential © 2017 Uber Technologies, Inc. All rights

    reserved. No part of this document may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval systems, without permission in writing from Uber. This document is intended only for the use of the individual or entity to whom it is addressed and contains information that is privileged, confidential or otherwise exempt from disclosure under applicable law. All recipients of this document are notified that the information contained herein includes proprietary and confidential information of Uber, and recipient may not make use of, disseminate, or in any way disclose this document or any of the enclosed information to any person other than employees of addressee to the extent necessary for consultations with authorized personnel of Uber.
  60. Attribution for Images used in this deck • Slide 16

    - http://tfwiki.net/wiki/File:WFC_OmegaSupreme.jpg • Slide 23 - https://media.giphy.com/media/5H7pDI6uF7ZNm/giphy.gif • Slide 24 - https://imgflip.com/memetemplate/One-Does-Not-Simply • Slide 35 - http://replycandy.com/ew/yao-ming-scared-rage-face • Slide 38 - https://upload.wikimedia.org/wikipedia/commons/0/0e/Matanuska_Glacier_mouth.jpg • Slide 41 - https://giphy.com/gifs/wLOk1aJX9Mis0/ • Slide 42 - https://imgflip.com/memegenerator/1489292/Vegeta-over-9000 • Slide 49 - https://az616578.vo.msecnd.net/files/2016/03/24/6359443246595527281092097913_grass_is_greener.jpg