Mono Repo vs Multi Repo: Deep Dive Into The Neverending Debate

As an active practitioner of mono repository and multi repository for years, I tried to compare these two important repository strategies and guide the readers which strategy should be used in which conditions. The deck is the one I represent in webinars and in my talks.

Transcript

1. Deep Dive Into the NeverEnding Debate lemi orhan ergin Multi Repo co-founder, vs Mono Repo
2. LEMi ORHAN ERGiN co-founder, Craftbase founder, Software Craftsmanship Turkey alumni, Sony, eBay, ACM, iyzico programming, since 2001 with love practitioner, multi & mono repo for years speakerdeck.com/lemiorhan @lemiorhan
3. Microservice world introduces huge amount of dependencies among applications and databases
4. How should we keep the repositories? One for each, or one for all?
5. It is a question of whether the repositories will be used by a single team or by many teams
6. can explain the problem and show the solution It is a question of whether the repositories will be used by a single team or by many teams Conway's Law
7. Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization's communication structure. Mel Conway From his original paper called "How Do Committees Invent?" Conway's Law
8. "A!er some initial estimates of difficulty and time, five people were assigned to the COBOL job and three to the ALGOL job. The resulting COBOL compiler ran in five phases, the ALGOL compiler ran in three.” Dr. Melvin Conway performed a famous experiment in 1968 to produce a COBOL and an ALGOL compiler:
9. If you have four groups working on a compiler, you’ll get a 4-pass compiler. Eric S. Raymond From his book "The Cathedral and The Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary"
10. Communication dictates design Mike Amundsen Author of the article "Conway’s Law under Remote Distance — Team Construction in a Distributed World" A good system design addresses issues about communication between human beings. by Manu Cornet, https://en.wikipedia.org/wiki/Manu_Cornet
11. Your code is shaped by teammates you talk to the most Don’t go into the code alone - from the book "Pragmatic Programmer" Microservices is not a technical solution, it’s a solution to team communication - Armağan Amcalar, from the talk "The Human Side of Microservices" Structure your team the way you want your code to look - from the book "Pragmatic Programmer"
12. Socio-technical alignment is the key for successful teams The structures of large systems tend to disintegrate during development. - Mike Amundsen, from article "Conway’s Law under Remote Distance" The system design and communication structure and team structure should align all these areas to the overall business model. - Bronwyn Davies, devops.com article "What Can Conway’s Law Teach Us About DevOps?" Architectural-level alignment wasn’t sufficient to enable the coordination required to avoid issues such as major quality problems. - Marcelo Cataldo, from article "Conway's Law Revisited"
13. lets us move fast and get the things done enables experimenting cross-project prevents generating silos in the team promotes collective ownership We need a repository model that...
14. Multi Repository Having one individual source code repository for each logical code group
15. Mono Repository Even though our services are still developed and deployed independently, the code for all services lives in one repository The repository can contain more than one logical project (e.g. an iOS client and a web-application) These projects are most likely unrelated, loosely connected or can be connected by other means (e.g via dependency management tools)
16. Change-set detection for building only changed apis (uncommitted, committed changes, and changes in dependent modules) Task ordering for running infrastructure tasks and common library builds first Toggling change set detection to run tests for all modules Triggering external contract verifier stub tests according to the task dependency graph Disabling the tasks of removed modules by sparse checkout Custom Tooling for Mono Repository
17. MULTI mono vs working with ide All codebase can be opened in one ide instance Easier to navigate the source code If codebase is too big, that requires long indexing times Lightweight to open one single repo Need to open more than one ide if multiple repos should be opened, that may cause insufficient memory Requires to apply workarounds to open all repos in one ide
18. MULTI mono vs command line We can pull and build any module with one command from one directory Build times may need longer than multi repo that requires custom tooling We have to jump from one directory into other to build more than one repo one after another Needs to write custom script to build or pull all repos once
19. MULTI mono vs access restrictions clear ownership of each repo possible to restrict access for each repo to specific people No way to restrict specific modules to specific people for the mono repo. Collaboration across teams is possible Do you really need that? If you really need, may be you should have put the module into a separate repo.
20. MULTI mono vs refactoring Cross repo changes and refactorings are painful Hard to keep common standards for each repo Atomic commits in entire codebase Cross-module refactoring and implementation is much easier Project gardening is simpler
21. MULTI mono vs discoverability Lack of understanding the big picture You can discover new solutions and standards from other's work
22. MULTI mono vs managing repository After creating a new repo, the CD pipeline for the new repo has to be configured, usually as a duplicated work from others No new configuration is required to enable CD pipeline, just create a new folder and go. Custom tooling is required by using bazel, buck or gradle. Working with non-graph based build tool like "maven" is hard
23. MULTI mono vs dependency management Each repo can manage its own dependencies and versions Same dependency library can have different versions on different repos. That is hard to maintain and upgrade. You can manage dependency versions from one place, that is unified versioning, one source of truth Forcing dependencies to update to latest Easy to discover and refactor overall dependency graph Loss of version information
24. MULTI mono vs pull requests Cross cutting changes require many pull requests to deal with Promotes being silos One pull request can have all cross cutting changes Too many pull requests causes long waiting time at CI builds
25. MULTI mono vs deployment An artifact for each repo can be created after each merge Each repo can be deployed separately Special tooling required (on our side or the host side) to identify the module triggered Each repo can be deployed separately after custom tooling git tags become useless
26. MULTI mono vs open source Need to extract the module to open source to separate repo and need extra development to make it public Manage like open source applications Easy to open source to public
27. You can not do CI without using mono-repos Ivan Moore https://blog.rocketpoweredjetpants.com/2018/01/why-use-monorepo.html
28. You can not do CI without using mono-repos Ivan Moore B C A unit tests integration tests component tests contract tests external verifier contract stub tests functional tests
29. Spli!ing: A Way of Discovering and Deploying Artifacts Splitting mono-repo to read-only mono-repos to only deploy changed microservices
30. Microservice Microservice Microservice Microservice Microservice Microservice Mono Repository Microservice Microservice Microservice Microservice Microservice Microservice
31. Microservice Microservice Microservice Microservice Microservice Microservice Mono Repository Microservice Microservice Microservice Microservice Microservice Microservice Read-only Multi Repositories splitting
32. Microservice Microservice Microservice Microservice Microservice Microservice Mono Repository Microservice Microservice Microservice Microservice Microservice Microservice Read-only Multi Repositories packaging & deployment splitting
33. Sparse Checkout Sparse checkout simply help you work on the folders you really need to work on and dismiss all the others.
36. 80 terabyte codebase at Google Google created Piper Facebook improved Mercurial Microsoft improved Git Buck, Bazel, Please works well Twitter open sourced Pants Reference: https://en.wikipedia.org/wiki/Monorepo
37. lets us move fast and get the things done enables experimenting cross-project prevents generating silos in the team promotes collective ownership We need a repository model that... LET'S REMEMBER
38. REPO MONO REPO MONO REPO MONO REPO REPO REPO MONO REPO MONO REPO MONO REPO REPO REPO REPO REPO Why don't you start with a mono repo and let it evolve into many mono and multi repos in time t t+1 t+2
39. @craftbaseio craftbase.io LEMi ORHAN ERGiN @lemiorhan