$30 off During Our Annual Pro Sale. View Details »

Mono Repo vs Multi Repo: Deep Dive Into The Neverending Debate

Mono Repo vs Multi Repo: Deep Dive Into The Neverending Debate

As an active practitioner of mono repository and multi repository for years, I tried to compare these two important repository strategies and guide the readers which strategy should be used in which conditions. The deck is the one I represent in webinars and in my talks.

Lemi Orhan Ergin

April 27, 2020
Tweet

More Decks by Lemi Orhan Ergin

Other Decks in Technology

Transcript

  1. Deep Dive Into the NeverEnding Debate
    lemi orhan ergin
    Multi Repo
    co-founder,
    vs
    Mono Repo

    View Slide

  2. LEMi ORHAN ERGiN
    co-founder, Craftbase
    founder, Software Craftsmanship Turkey
    alumni, Sony, eBay, ACM, iyzico
    programming, since 2001 with love
    practitioner, multi & mono repo for years
    speakerdeck.com/lemiorhan
    @lemiorhan

    View Slide

  3. Microservice world
    introduces huge amount of
    dependencies among
    applications and databases

    View Slide

  4. How should we keep the repositories?
    One for each, or one for all?

    View Slide

  5. It is a question of whether the
    repositories will be used
    by a single team
    or by many teams

    View Slide

  6. can explain the problem
    and show the solution
    It is a question of whether the
    repositories will be used
    by a single team
    or by many teams
    Conway's Law

    View Slide

  7. Any organization that designs a system will
    inevitably produce a design whose structure
    is a copy of the organization's
    communication structure.
    Mel Conway
    From his original paper called
    "How Do Committees Invent?"
    Conway's Law

    View Slide

  8. "A!er some initial estimates of difficulty and
    time, five people were assigned to the COBOL job
    and three to the ALGOL job.
    The resulting COBOL compiler ran in five phases,
    the ALGOL compiler ran in three.”
    Dr. Melvin Conway performed a famous experiment in
    1968 to produce a COBOL and an ALGOL compiler:

    View Slide

  9. If you have four groups working on a
    compiler, you’ll get a 4-pass compiler.
    Eric S. Raymond
    From his book "The Cathedral and The Bazaar: Musings on
    Linux and Open Source by an Accidental Revolutionary"

    View Slide

  10. Communication
    dictates design
    Mike Amundsen
    Author of the article "Conway’s Law
    under Remote Distance — Team
    Construction in a Distributed World"
    A good system design addresses
    issues about communication
    between human beings.
    by Manu Cornet, https://en.wikipedia.org/wiki/Manu_Cornet

    View Slide

  11. Your code is shaped by teammates you talk to the most
    Don’t go into the code alone - from the book "Pragmatic Programmer"
    Microservices is not a technical solution,
    it’s a solution to team communication
    - Armağan Amcalar, from the talk "The
    Human Side of Microservices"
    Structure your team the way you want
    your code to look
    - from the book "Pragmatic Programmer"

    View Slide

  12. Socio-technical alignment is the key for successful teams
    The structures of large systems tend to
    disintegrate during development.
    - Mike Amundsen, from article
    "Conway’s Law under Remote Distance"
    The system design and communication
    structure and team structure should align all
    these areas to the overall business model.
    - Bronwyn Davies, devops.com
    article "What Can Conway’s Law
    Teach Us About DevOps?"
    Architectural-level alignment wasn’t sufficient
    to enable the coordination required to avoid
    issues such as major quality problems.
    - Marcelo Cataldo, from article
    "Conway's Law Revisited"

    View Slide

  13. lets us move fast and get the things done
    enables experimenting cross-project
    prevents generating silos in the team
    promotes collective ownership
    We need a repository model that...

    View Slide

  14. Multi Repository
    Having one individual source code repository
    for each logical code group

    View Slide

  15. Mono Repository
    Even though our services are still developed and
    deployed independently, the code for all services
    lives in one repository
    The repository can contain more than one logical
    project (e.g. an iOS client and a web-application)
    These projects are most likely unrelated,
    loosely connected or can be connected by
    other means (e.g via dependency
    management tools)

    View Slide

  16. Change-set detection for building only changed apis (uncommitted,
    committed changes, and changes in dependent modules)
    Task ordering for running infrastructure tasks and common library builds first
    Toggling change set detection to run tests for all modules
    Triggering external contract verifier stub tests according to the task
    dependency graph
    Disabling the tasks of removed modules by sparse checkout
    Custom Tooling
    for Mono Repository

    View Slide

  17. MULTI mono
    vs
    working with ide
    All codebase can be opened in one
    ide instance
    Easier to navigate the source code
    If codebase is too big, that requires
    long indexing times
    Lightweight to open one single repo
    Need to open more than one ide if
    multiple repos should be opened,
    that may cause insufficient memory
    Requires to apply workarounds to
    open all repos in one ide

    View Slide

  18. MULTI mono
    vs
    command line
    We can pull and build any module
    with one command from one directory
    Build times may need longer than
    multi repo that requires custom
    tooling
    We have to jump from one directory
    into other to build more than one
    repo one after another
    Needs to write custom script to build
    or pull all repos once

    View Slide

  19. MULTI mono
    vs
    access restrictions
    clear ownership of each repo
    possible to restrict access for
    each repo to specific people
    No way to restrict specific modules to
    specific people for the mono repo.
    Collaboration across teams is possible
    Do you really need that? If you really
    need, may be you should have put the
    module into a separate repo.

    View Slide

  20. MULTI mono
    vs
    refactoring
    Cross repo changes and
    refactorings are painful
    Hard to keep common standards
    for each repo
    Atomic commits in entire
    codebase
    Cross-module refactoring and
    implementation is much easier
    Project gardening is simpler

    View Slide

  21. MULTI mono
    vs
    discoverability
    Lack of understanding the big
    picture
    You can discover new solutions
    and standards from other's work

    View Slide

  22. MULTI mono
    vs
    managing repository
    After creating a new repo, the CD
    pipeline for the new repo has to
    be configured, usually as a
    duplicated work from others
    No new configuration is
    required to enable CD pipeline,
    just create a new folder and go.
    Custom tooling is required by
    using bazel, buck or gradle.
    Working with non-graph based
    build tool like "maven" is hard

    View Slide

  23. MULTI mono
    vs
    dependency management
    Each repo can manage its own
    dependencies and versions
    Same dependency library can
    have different versions on
    different repos. That is hard to
    maintain and upgrade.
    You can manage dependency
    versions from one place, that is
    unified versioning, one source
    of truth
    Forcing dependencies to
    update to latest
    Easy to discover and refactor
    overall dependency graph
    Loss of version information

    View Slide

  24. MULTI mono
    vs
    pull requests
    Cross cutting changes require
    many pull requests to deal with
    Promotes being silos
    One pull request can have all
    cross cutting changes
    Too many pull requests causes
    long waiting time at CI builds

    View Slide

  25. MULTI mono
    vs
    deployment
    An artifact for each repo can be
    created after each merge
    Each repo can be deployed
    separately
    Special tooling required (on
    our side or the host side) to
    identify the module triggered
    Each repo can be deployed
    separately after custom tooling
    git tags become useless

    View Slide

  26. MULTI mono
    vs
    open source
    Need to extract the module to
    open source to separate repo and
    need extra development to make
    it public
    Manage like open source
    applications
    Easy to open source to public

    View Slide

  27. You can not do CI without using mono-repos
    Ivan Moore
    https://blog.rocketpoweredjetpants.com/2018/01/why-use-monorepo.html

    View Slide

  28. You can not do CI without using mono-repos
    Ivan Moore
    B
    C
    A
    unit tests
    integration tests
    component tests
    contract tests
    external verifier contract stub tests
    functional tests

    View Slide

  29. Spli!ing:
    A Way of Discovering
    and Deploying Artifacts
    Splitting mono-repo to read-only mono-repos
    to only deploy changed microservices

    View Slide

  30. Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Mono Repository
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice

    View Slide

  31. Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Mono Repository
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Read-only
    Multi Repositories
    splitting

    View Slide

  32. Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Mono Repository
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Microservice
    Read-only
    Multi Repositories
    packaging & deployment
    splitting

    View Slide

  33. Sparse Checkout
    Sparse checkout simply help you
    work on the folders you really need
    to work on and dismiss all the others.

    View Slide

  34. View Slide

  35. View Slide

  36. 80 terabyte codebase at Google
    Google created Piper
    Facebook improved Mercurial
    Microsoft improved Git
    Buck, Bazel, Please works well
    Twitter open sourced Pants
    Reference: https://en.wikipedia.org/wiki/Monorepo

    View Slide

  37. lets us move fast and get the things done
    enables experimenting cross-project
    prevents generating silos in the team
    promotes collective ownership
    We need a repository model that...
    LET'S REMEMBER

    View Slide

  38. REPO
    MONO REPO
    MONO REPO
    MONO REPO
    REPO
    REPO
    MONO REPO
    MONO REPO
    MONO REPO
    REPO
    REPO
    REPO
    REPO
    Why don't you start
    with a mono repo and
    let it evolve into many
    mono and multi repos
    in time
    t t+1 t+2

    View Slide

  39. @craftbaseio
    craftbase.io
    LEMi ORHAN ERGiN
    @lemiorhan

    View Slide