Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Developing in a Large Monorepo

64abf2cf90cc5f04fd9a471947a3cc09?s=47 Jai Santhosh
September 28, 2019

Developing in a Large Monorepo

In Microsoft, we have a very large TypeScript-based git repository where over 250 developers build and write code for high-value frontend components which is used across all Microsoft365 products like Outlook, Office, Bing, SharePoint, etc.It contains over a 100 npm packages, containing over a million lines of TypeScript code. Co-locating these components encourages collaboration and sharing code across teams very easier.

In this talk, we’ll focus on the tooling used and code organization to make the development easier, fast and reliable and will not discuss in detail unit tests and automation for a better focus on development.

64abf2cf90cc5f04fd9a471947a3cc09?s=128

Jai Santhosh

September 28, 2019
Tweet

Transcript

  1. JSConf Korea 2020 Developing in a Large Monorepo

  2. Jai Santhosh Engineering Manager @jaisanth

  3. Microsoft Search

  4. Re-using Components They are the user’s most important, cross-app and

    cross-platform interactions in the Office Suite. High-Value Experiences These experiences must feel familiar across apps and platforms, but also look and feel consistent within a platform and within an app. Coherence
  5. People Card

  6. People Card

  7. People Card Card Conversations Documents Events Organization Analytics Contact

  8. Write new code Test locally Create a PR Receive Peer

    Review Verify build and test reports Merge PR with main Checkin new code Deploy through rings to production Listen and measure for feedback Propose next steps Write new code Agility Inner Loop Outer Loop
  9. Web Win32 UWP iOS Mac Android src src src src

    src src Agility
  10. Web Win32 UWP iOS Mac Android src Agility

  11. Agility & Coherence

  12. Monorepo

  13. What is a Monorepo? A repository with multiple packages or

    projects • Other repositories can re-use small parts of our code • Better code re-use unit (better tooling) Singularity across multiple packages • Developer agility • Code consistency as conventions are shared • Unified processes, versioned centrally • Code discoverability • ... Why in one repository? • Various projects or packages can co-exist. • Micro platform with an entire application stack in one place. Not necessarily a monolith
  14. Using separate packages Re-using can be made easier with singular

    responsibility packages. Can be published easily. Enables cross-team usage of required packages. Can manage releases easily.
  15. You need reliable tooling! Package Management Linters Build system /

    Infrastructure Testing Tools Cross-package orchestrators and more… Package Management Cross-package orchestrators
  16. Package Management

  17. JavaScript package JavaScript Repo Repo’s external dependencies

  18. package.json NPM Yarn Files on disk NodeJS require() is read

    by which writes is resolved by which finds
  19. require("react") • /User/../git/Monorepo/packages/people-core/node_modules/react • /User/../git/Monorepo/packages/node_modules/react • /User/../git/Monorepo/node_modules/react • /User/../git/node_modules/react •

    /User/../node_modules/react • /node_modules/react Node Module Resolution
  20. package-1 package-2 Repository 1 Repository 2

  21. package-1 package-2 Repository mono monorepo

  22. package-1 package-2 Repository mono monorepo A (1.0) C (1.0) B

    (1.0) B (2.0) B (1.0)
  23. package-1 package-2 monorepo A (1.0) C (1.0) B (1.0) B

    (2.0) node_modules packages package-1 package-2 node_modules symlinks https://yarnpkg.com/blog/2018/02/15/nohoist/ Package Hoisting
  24. rush yarn workspaces pnpm workspaces Lerna Package Management - Tools

    Multiple popular tools for package management, let’s check how yarn workspaces works.
  25. • Set "private": true • Add workspace packages • Optionally,

    you can add a nohoist list of packages as well • Run yarn from the root of the monorepo. • Package installation time for about 150+ packages came down from ~20mins to ~6mins. monorepo/package.json Yarn Workspaces
  26. Cross-package Orchestration

  27. rush Lerna Shell scripts Gulp Cross-package Orchestration - Tools Perform

    operations on files across multiple packages, usually from the root folder
  28. monorepo/lerna.json • Add "packages": [<<paths>>] • Specify npmClient • Set

    useWorkspaces value. • lerna run [script] - Run the script across packages Lerna
  29. Lerna - Example

  30. Lerna - Example

  31. • We use test dev apps for each of the

    experiences for quick dev validation. • lerna run --scope [<packages>] [script] Lerna - Scoping
  32. • lerna publish - create new release versions of packages

    that have been updated. • lerna diff [package?] - Diff package(s) since the last release. • lerna changed – Check which packages have been changed since last release. • lerna version – Bump version of packages changed since the last release. Lerna - Versioning
  33. • Brought down our build times by more than 5x.

    • Good community adoption • babeljs • create-react-app • react-router • jest • and many more... Lerna - Wins
  34. Summing it up… • Pick the right tools for your

    monorepo, based on the complexity of the packages, to improve the developer experience. • For linting, unit/functional/perf testing you can combine it with tools like lerna to easily run these commands across packages. • If you are using TypeScript, you will have to handle TS compilation and for a better developer experience, manage typings for better Intellisense.
  35. References https://medium.com/@vincentbailly/evolving-a-large-typescript-repository-4a32c61e94fd https://rushjs.io/ https://lerna.js.org/ https://yarnpkg.com/lang/en/docs/workspaces/ Image courtesy: @redbadgerteam, Adobe Stock

    images
  36. Thank you! Jai Santhosh Engineering Manager @jaisanth