Mono Repo vs Multi Repo: Deep Dive Into The Neverending Debate

As an active practitioner of mono repository and multi repository for years, I tried to compare these two important repository strategies and guide the readers which strategy should be used in which conditions. The deck is the one I represent in webinars and in my talks.

Transcript

1. Deep Dive Into the NeverEnding Debate
   lemi orhan ergin
   Multi Repo
   co-founder,
   vs
   Mono Repo
   

   View Slide

2. LEMi ORHAN ERGiN
   co-founder, Craftbase
   founder, Software Craftsmanship Turkey
   alumni, Sony, eBay, ACM, iyzico
   programming, since 2001 with love
   practitioner, multi & mono repo for years
   speakerdeck.com/lemiorhan
   @lemiorhan
   

   View Slide

3. Microservice world
   introduces huge amount of
   dependencies among
   applications and databases
   

   View Slide

4. How should we keep the repositories?
   One for each, or one for all?
   

   View Slide

5. It is a question of whether the
   repositories will be used
   by a single team
   or by many teams
   

   View Slide

6. can explain the problem
   and show the solution
   It is a question of whether the
   repositories will be used
   by a single team
   or by many teams
   Conway's Law
   

   View Slide

7. Any organization that designs a system will
   inevitably produce a design whose structure
   is a copy of the organization's
   communication structure.
   Mel Conway
   From his original paper called
   "How Do Committees Invent?"
   Conway's Law
   

   View Slide

8. "A!er some initial estimates of difficulty and
   time, five people were assigned to the COBOL job
   and three to the ALGOL job.
   The resulting COBOL compiler ran in five phases,
   the ALGOL compiler ran in three.”
   Dr. Melvin Conway performed a famous experiment in
   1968 to produce a COBOL and an ALGOL compiler:
   

   View Slide

9. If you have four groups working on a
   compiler, you’ll get a 4-pass compiler.
   Eric S. Raymond
   From his book "The Cathedral and The Bazaar: Musings on
   Linux and Open Source by an Accidental Revolutionary"
   

   View Slide

10. Communication
    dictates design
    Mike Amundsen
    Author of the article "Conway’s Law
    under Remote Distance — Team
    Construction in a Distributed World"
    A good system design addresses
    issues about communication
    between human beings.
    by Manu Cornet, https://en.wikipedia.org/wiki/Manu_Cornet
    

    View Slide

11. Your code is shaped by teammates you talk to the most
    Don’t go into the code alone - from the book "Pragmatic Programmer"
    Microservices is not a technical solution,
    it’s a solution to team communication
    - Armağan Amcalar, from the talk "The
    Human Side of Microservices"
    Structure your team the way you want
    your code to look
    - from the book "Pragmatic Programmer"
    

    View Slide

12. Socio-technical alignment is the key for successful teams
    The structures of large systems tend to
    disintegrate during development.
    - Mike Amundsen, from article
    "Conway’s Law under Remote Distance"
    The system design and communication
    structure and team structure should align all
    these areas to the overall business model.
    - Bronwyn Davies, devops.com
    article "What Can Conway’s Law
    Teach Us About DevOps?"
    Architectural-level alignment wasn’t sufficient
    to enable the coordination required to avoid
    issues such as major quality problems.
    - Marcelo Cataldo, from article
    "Conway's Law Revisited"
    

    View Slide

13. lets us move fast and get the things done
    enables experimenting cross-project
    prevents generating silos in the team
    promotes collective ownership
    We need a repository model that...
    

    View Slide

14. Multi Repository
    Having one individual source code repository
    for each logical code group
    

    View Slide

15. Mono Repository
    Even though our services are still developed and
    deployed independently, the code for all services
    lives in one repository
    The repository can contain more than one logical
    project (e.g. an iOS client and a web-application)
    These projects are most likely unrelated,
    loosely connected or can be connected by
    other means (e.g via dependency
    management tools)
    

    View Slide

16. Change-set detection for building only changed apis (uncommitted,
    committed changes, and changes in dependent modules)
    Task ordering for running infrastructure tasks and common library builds first
    Toggling change set detection to run tests for all modules
    Triggering external contract verifier stub tests according to the task
    dependency graph
    Disabling the tasks of removed modules by sparse checkout
    Custom Tooling
    for Mono Repository
    

    View Slide

17. MULTI mono
    vs
    working with ide
    All codebase can be opened in one
    ide instance
    Easier to navigate the source code
    If codebase is too big, that requires
    long indexing times
    Lightweight to open one single repo
    Need to open more than one ide if
    multiple repos should be opened,
    that may cause insufficient memory
    Requires to apply workarounds to
    open all repos in one ide
    

    View Slide

18. MULTI mono
    vs
    command line
    We can pull and build any module
    with one command from one directory
    Build times may need longer than
    multi repo that requires custom
    tooling
    We have to jump from one directory
    into other to build more than one
    repo one after another
    Needs to write custom script to build
    or pull all repos once
    

    View Slide

19. MULTI mono
    vs
    access restrictions
    clear ownership of each repo
    possible to restrict access for
    each repo to specific people
    No way to restrict specific modules to
    specific people for the mono repo.
    Collaboration across teams is possible
    Do you really need that? If you really
    need, may be you should have put the
    module into a separate repo.
    

    View Slide

20. MULTI mono
    vs
    refactoring
    Cross repo changes and
    refactorings are painful
    Hard to keep common standards
    for each repo
    Atomic commits in entire
    codebase
    Cross-module refactoring and
    implementation is much easier
    Project gardening is simpler
    

    View Slide

21. MULTI mono
    vs
    discoverability
    Lack of understanding the big
    picture
    You can discover new solutions
    and standards from other's work
    

    View Slide

22. MULTI mono
    vs
    managing repository
    After creating a new repo, the CD
    pipeline for the new repo has to
    be configured, usually as a
    duplicated work from others
    No new configuration is
    required to enable CD pipeline,
    just create a new folder and go.
    Custom tooling is required by
    using bazel, buck or gradle.
    Working with non-graph based
    build tool like "maven" is hard
    

    View Slide

23. MULTI mono
    vs
    dependency management
    Each repo can manage its own
    dependencies and versions
    Same dependency library can
    have different versions on
    different repos. That is hard to
    maintain and upgrade.
    You can manage dependency
    versions from one place, that is
    unified versioning, one source
    of truth
    Forcing dependencies to
    update to latest
    Easy to discover and refactor
    overall dependency graph
    Loss of version information
    

    View Slide

24. MULTI mono
    vs
    pull requests
    Cross cutting changes require
    many pull requests to deal with
    Promotes being silos
    One pull request can have all
    cross cutting changes
    Too many pull requests causes
    long waiting time at CI builds
    

    View Slide

25. MULTI mono
    vs
    deployment
    An artifact for each repo can be
    created after each merge
    Each repo can be deployed
    separately
    Special tooling required (on
    our side or the host side) to
    identify the module triggered
    Each repo can be deployed
    separately after custom tooling
    git tags become useless
    

    View Slide

26. MULTI mono
    vs
    open source
    Need to extract the module to
    open source to separate repo and
    need extra development to make
    it public
    Manage like open source
    applications
    Easy to open source to public
    

    View Slide

27. You can not do CI without using mono-repos
    Ivan Moore
    https://blog.rocketpoweredjetpants.com/2018/01/why-use-monorepo.html
    

    View Slide

28. You can not do CI without using mono-repos
    Ivan Moore
    B
    C
    A
    unit tests
    integration tests
    component tests
    contract tests
    external verifier contract stub tests
    functional tests
    

    View Slide

29. Spli!ing:
    A Way of Discovering
    and Deploying Artifacts
    Splitting mono-repo to read-only mono-repos
    to only deploy changed microservices
    

    View Slide

30. Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Mono Repository
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    

    View Slide

31. Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Mono Repository
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Read-only
    Multi Repositories
    splitting
    

    View Slide

32. Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Mono Repository
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Read-only
    Multi Repositories
    packaging & deployment
    splitting
    

    View Slide

33. Sparse Checkout
    Sparse checkout simply help you
    work on the folders you really need
    to work on and dismiss all the others.
    

    View Slide

34. View Slide

35. View Slide

36. 80 terabyte codebase at Google
    Google created Piper
    Facebook improved Mercurial
    Microsoft improved Git
    Buck, Bazel, Please works well
    Twitter open sourced Pants
    Reference: https://en.wikipedia.org/wiki/Monorepo
    

    View Slide

37. lets us move fast and get the things done
    enables experimenting cross-project
    prevents generating silos in the team
    promotes collective ownership
    We need a repository model that...
    LET'S REMEMBER
    

    View Slide

38. REPO
    MONO REPO
    MONO REPO
    MONO REPO
    REPO
    REPO
    MONO REPO
    MONO REPO
    MONO REPO
    REPO
    REPO
    REPO
    REPO
    Why don't you start
    with a mono repo and
    let it evolve into many
    mono and multi repos
    in time
    t t+1 t+2
    

    View Slide

39. @craftbaseio
    craftbase.io
    LEMi ORHAN ERGiN
    @lemiorhan
    

    View Slide