Mono Repo vs Multi Repo: Deep Dive Into The Neverending Debate

Mono Repo vs Multi Repo: Deep Dive Into The Neverending Debate

As an active practitioner of mono repository and multi repository for years, I tried to compare these two important repository strategies and guide the readers which strategy should be used in which conditions. The deck is the one I represent in webinars and in my talks.

Dafc4723e9a1c067796c0fec6f509774?s=128

Lemi Orhan Ergin

April 27, 2020
Tweet

Transcript

  1. Deep Dive Into the NeverEnding Debate lemi orhan ergin Multi

    Repo co-founder, vs Mono Repo
  2. LEMi ORHAN ERGiN co-founder, Craftbase founder, Software Craftsmanship Turkey alumni,

    Sony, eBay, ACM, iyzico programming, since 2001 with love practitioner, multi & mono repo for years speakerdeck.com/lemiorhan @lemiorhan
  3. Microservice world introduces huge amount of dependencies among applications and

    databases
  4. How should we keep the repositories? One for each, or

    one for all?
  5. It is a question of whether the repositories will be

    used by a single team or by many teams
  6. can explain the problem and show the solution It is

    a question of whether the repositories will be used by a single team or by many teams Conway's Law
  7. Any organization that designs a system will inevitably produce a

    design whose structure is a copy of the organization's communication structure. Mel Conway From his original paper called "How Do Committees Invent?" Conway's Law
  8. "A!er some initial estimates of difficulty and time, five people

    were assigned to the COBOL job and three to the ALGOL job. The resulting COBOL compiler ran in five phases, the ALGOL compiler ran in three.” Dr. Melvin Conway performed a famous experiment in 1968 to produce a COBOL and an ALGOL compiler:
  9. If you have four groups working on a compiler, you’ll

    get a 4-pass compiler. Eric S. Raymond From his book "The Cathedral and The Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary"
  10. Communication dictates design Mike Amundsen Author of the article "Conway’s

    Law under Remote Distance — Team Construction in a Distributed World" A good system design addresses issues about communication between human beings. by Manu Cornet, https://en.wikipedia.org/wiki/Manu_Cornet
  11. Your code is shaped by teammates you talk to the

    most Don’t go into the code alone - from the book "Pragmatic Programmer" Microservices is not a technical solution, it’s a solution to team communication - Armağan Amcalar, from the talk "The Human Side of Microservices" Structure your team the way you want your code to look - from the book "Pragmatic Programmer"
  12. Socio-technical alignment is the key for successful teams The structures

    of large systems tend to disintegrate during development. - Mike Amundsen, from article "Conway’s Law under Remote Distance" The system design and communication structure and team structure should align all these areas to the overall business model. - Bronwyn Davies, devops.com article "What Can Conway’s Law Teach Us About DevOps?" Architectural-level alignment wasn’t sufficient to enable the coordination required to avoid issues such as major quality problems. - Marcelo Cataldo, from article "Conway's Law Revisited"
  13. lets us move fast and get the things done enables

    experimenting cross-project prevents generating silos in the team promotes collective ownership We need a repository model that...
  14. Multi Repository Having one individual source code repository for each

    logical code group
  15. Mono Repository Even though our services are still developed and

    deployed independently, the code for all services lives in one repository The repository can contain more than one logical project (e.g. an iOS client and a web-application) These projects are most likely unrelated, loosely connected or can be connected by other means (e.g via dependency management tools)
  16. Change-set detection for building only changed apis (uncommitted, committed changes,

    and changes in dependent modules) Task ordering for running infrastructure tasks and common library builds first Toggling change set detection to run tests for all modules Triggering external contract verifier stub tests according to the task dependency graph Disabling the tasks of removed modules by sparse checkout Custom Tooling for Mono Repository
  17. MULTI mono vs working with ide All codebase can be

    opened in one ide instance Easier to navigate the source code If codebase is too big, that requires long indexing times Lightweight to open one single repo Need to open more than one ide if multiple repos should be opened, that may cause insufficient memory Requires to apply workarounds to open all repos in one ide
  18. MULTI mono vs command line We can pull and build

    any module with one command from one directory Build times may need longer than multi repo that requires custom tooling We have to jump from one directory into other to build more than one repo one after another Needs to write custom script to build or pull all repos once
  19. MULTI mono vs access restrictions clear ownership of each repo

    possible to restrict access for each repo to specific people No way to restrict specific modules to specific people for the mono repo. Collaboration across teams is possible Do you really need that? If you really need, may be you should have put the module into a separate repo.
  20. MULTI mono vs refactoring Cross repo changes and refactorings are

    painful Hard to keep common standards for each repo Atomic commits in entire codebase Cross-module refactoring and implementation is much easier Project gardening is simpler
  21. MULTI mono vs discoverability Lack of understanding the big picture

    You can discover new solutions and standards from other's work
  22. MULTI mono vs managing repository After creating a new repo,

    the CD pipeline for the new repo has to be configured, usually as a duplicated work from others No new configuration is required to enable CD pipeline, just create a new folder and go. Custom tooling is required by using bazel, buck or gradle. Working with non-graph based build tool like "maven" is hard
  23. MULTI mono vs dependency management Each repo can manage its

    own dependencies and versions Same dependency library can have different versions on different repos. That is hard to maintain and upgrade. You can manage dependency versions from one place, that is unified versioning, one source of truth Forcing dependencies to update to latest Easy to discover and refactor overall dependency graph Loss of version information
  24. MULTI mono vs pull requests Cross cutting changes require many

    pull requests to deal with Promotes being silos One pull request can have all cross cutting changes Too many pull requests causes long waiting time at CI builds
  25. MULTI mono vs deployment An artifact for each repo can

    be created after each merge Each repo can be deployed separately Special tooling required (on our side or the host side) to identify the module triggered Each repo can be deployed separately after custom tooling git tags become useless
  26. MULTI mono vs open source Need to extract the module

    to open source to separate repo and need extra development to make it public Manage like open source applications Easy to open source to public
  27. You can not do CI without using mono-repos Ivan Moore

    https://blog.rocketpoweredjetpants.com/2018/01/why-use-monorepo.html
  28. You can not do CI without using mono-repos Ivan Moore

    B C A unit tests integration tests component tests contract tests external verifier contract stub tests functional tests
  29. Spli!ing: A Way of Discovering and Deploying Artifacts Splitting mono-repo

    to read-only mono-repos to only deploy changed microservices
  30. Microservice Microservice Microservice Microservice Microservice Microservice Mono Repository Microservice Microservice

    Microservice Microservice Microservice Microservice
  31. Microservice Microservice Microservice Microservice Microservice Microservice Mono Repository Microservice Microservice

    Microservice Microservice Microservice Microservice Read-only Multi Repositories splitting
  32. Microservice Microservice Microservice Microservice Microservice Microservice Mono Repository Microservice Microservice

    Microservice Microservice Microservice Microservice Read-only Multi Repositories packaging & deployment splitting
  33. Sparse Checkout Sparse checkout simply help you work on the

    folders you really need to work on and dismiss all the others.
  34. None
  35. None
  36. 80 terabyte codebase at Google Google created Piper Facebook improved

    Mercurial Microsoft improved Git Buck, Bazel, Please works well Twitter open sourced Pants Reference: https://en.wikipedia.org/wiki/Monorepo
  37. lets us move fast and get the things done enables

    experimenting cross-project prevents generating silos in the team promotes collective ownership We need a repository model that... LET'S REMEMBER
  38. REPO MONO REPO MONO REPO MONO REPO REPO REPO MONO

    REPO MONO REPO MONO REPO REPO REPO REPO REPO Why don't you start with a mono repo and let it evolve into many mono and multi repos in time t t+1 t+2
  39. @craftbaseio craftbase.io LEMi ORHAN ERGiN @lemiorhan