Managing Your Node.js Monorepos With NX

Slide 1

Slide 1 text

Tamar Twena-Stern Implementing Monorepo in your Node.js project with NX

Slide 2

Slide 2 text

Tamar Twena-Stern twitter @SternTwena linkedin /tamarstern [email protected]

Slide 3

Slide 3 text

Traditional Microservices Architecture

Slide 4

Slide 4 text

Should I Write My Code In The Same Git Repository ? Or Should I Write Every Service In It’s Own Repository ?

Slide 5

Slide 5 text

Monorepo - All micro service in one git repository Multi-repo - Every micro service in a separate git repo

Slide 6

Slide 6 text

Choose Monorepo Or Multi Repo ? There is a real debate on that question

Slide 7

Slide 7 text

Pros And Cons

Slide 8

Slide 8 text

In Multi-repos - All Those Are Easy Maintain version for each component Each service to have it’s own deployment cycle and releases Define access control to different components

Slide 9

Slide 9 text

Disadvantages Of Multi-repos Manage dependencies Multiple copy-paste Refactoring across multiple repositories

Slide 10

Slide 10 text

Since We Will Focus On Mono-repos Let’s Dive To Them

Slide 11

Slide 11 text

Advantages Of Mono-repos Easier for new developers to get familiar with the code and setup environment Less copy-paste, more code reuse between components Easier to refactor code across services / components Separate development teams to share the same culture - libraries, programming methodologies, development tools

Slide 12

Slide 12 text

Disadvantages Of Mono-repos Maintain feature branches for a long period, conflict resolving becomes very complicated Version control slow down - Pulling, cloning, checking out becomes long, it slows down the team’s work Managing access control - Impossible to block read access to all the code

Slide 13

Slide 13 text

The Nightmare - One Build To Rule Them All

Slide 14

Slide 14 text

Long, Complicated build process and CI/CD No ability to test separately only specific service Inability to version and deploy specific service One Build For All Services

Slide 15

Slide 15 text

NX - A Tool To Manage Mono-Repos

Slide 16

Slide 16 text

NX In A Nutshell A tool to manage all your code components in one repo Gives the ability to build, deploy and test each component separately

Slide 17

Slide 17 text

Building An NX Project With Several NPM Packages

Slide 18

Slide 18 text

Creating a new NX project

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

Initial Project Structure After Creation- No Packages Added

Slide 21

Slide 21 text

Project Structure Under Packages - All the packages in the repo will be located. Each package will have each own package.json and it’s own node_modules folder A global package.json to control the main build Typescript is not installed - if you use it , you need to install it globally/ per package

Slide 22

Slide 22 text

Creating A First Package For Generating Random Number

Slide 23

Slide 23 text

Creating A Folder And Module Files Manually

Slide 24

Slide 24 text

Index.ts const getRandomInt = (max) : number => { return Math.floor(Math.random() * max); } export default getRandomInt;

Slide 25

Slide 25 text

Package.json { "name": "rand-number", "version": "0.0.0", "main": "dist/index.js", "devDependencies": {}, "scripts": { "build": "tsc index.ts --outDir dist" } }

Slide 26

Slide 26 text

Building rand-number

Slide 27

Slide 27 text

Creating Manually A Linked Module Called Print Number

Slide 28

Slide 28 text

Creating Manually A Linked Module Called Print Number

Slide 29

Slide 29 text

Print Number - Index.ts import getRandomInt from 'rand-number'; const printRandomInt = (max) : void => { console.log('Printing number' + getRandomInt(max)) } export default printRandomInt;

Slide 30

Slide 30 text

Print Number - Package.json { "name": "print-number", "version": "0.0.0", "main": "dist/index.js", "devDependencies": {}, "scripts": { "build": "tsc index.ts --outDir dist", "test": "jest --coverage", "lint": "eslint src --ext .ts" }, "dependencies": { "rand-number": "*" } }

Slide 31

Slide 31 text

NPM Install On Root Level Identified That Projects Are Linked

Slide 32

Slide 32 text

Building ‘print-number’ - ‘rand-number’ Will Be Built As Well

Slide 33

Slide 33 text

Building All The Project

Slide 34

Slide 34 text

Generate NX Visualization Of Package Dependencies

Slide 35

Slide 35 text

NX Core Features

Slide 36

Slide 36 text

NX Tasks - All Scripts Defined In Package.json "scripts": { "build": "tsc index.ts --outDir dist", "test": "jest --coverage", "lint": "eslint src --ext .ts" } A project can have thousands of tasks !

Slide 37

Slide 37 text

NX Tasks ● Each script in package json defined as a task ● Possibility to run single task, multiple tasks or all of them ● If a specific task is running - nx will run all the task dependencies ● Possibility to control the order of tasks

Slide 38

Slide 38 text

NX Caching System

Slide 39

Slide 39 text

Setup The Cache- nx.json { "extends": "nx/presets/npm.json", "$schema": "./node_modules/nx/schemas/nx-schema.json", "tasksRunnerOptions": { "default": { "runner": "@nrwl/nx-cloud", "options": { "cacheableOperations": [ "build", "lint", "test", "E2e" ], "accessToken": "" } } } }

Slide 40

Slide 40 text

Slide 41

Slide 41 text

How The Caching Hash Is Generated

Slide 42

Slide 42 text

Distributed Caching

Slide 43

Slide 43 text

What The Cache Is Storing ? ● Build Operations - Output of typescript compiler / babel ● Test - test results of Jest/Mocha ● Lint - ESlint results

Slide 44

Slide 44 text

What Nx Cache Adds Above TypeScript Compiler Build Optimizations ?

Slide 45

Slide 45 text

Caching Build Operations - TypeScript And NX

Slide 46

Slide 46 text

The Project Graph

Slide 47

Slide 47 text

The Elements Used To Build The Graph import getRandomInt from 'rand-number'; "dependencies": { "rand-number": "*" } Typescript Imports: Package.json dependencies

Slide 48

Slide 48 text

What You Can Do With The Project Graph ? See How Projects Are dependent Technical Description Of Each Module

Slide 49

Slide 49 text

The Project Graph Makes Detecting Circular Dependencies To A Much Easier Task

Slide 50

Slide 50 text

We talked a lot about NX packaged project . Does NX has other project types ?

Slide 51

Slide 51 text

Node.js NX Project

Slide 52

Slide 52 text

Generating Node.js Project Choose ‘node-standalone’ from the options presented

Slide 53

Slide 53 text

General Information - Node.js NX Project ● A skeleton project is generated with native support of Typescript and Jest. ● A single API file is generated under the root directory containing express endpoints ● Utilities can be generated as a NX modules inside the project.

Slide 54

Slide 54 text

The Project Structure - Main Entry Point For Express Routes

Slide 55

Slide 55 text

Main Server File Under src/ Folder - Node.js Project import express from 'express'; import { doAuth } from '@products-api/auth'; const host = process.env.HOST ?? 'localhost'; const port = process.env.PORT ? Number(process.env.PORT) : 3000; const app = express(); app.get('/', (req, res) => { res.send({ message: 'Hello API' }); }); app.post('/auth', (req, res) => { res.send(doAuth()); }); app.listen(port, host, () => { console.log(`[ ready ] http://${host}:${port}`); });

Slide 56

Slide 56 text

Project Structure For An Internal Module Module is generated with a test file

Slide 57

Slide 57 text

Which NX Project Type Fit To Which Component In Our Architecture ? Packaged Based NX Project

Slide 58

Slide 58 text

Which NX Project Type Fit To Which Component In Our Architecture ? Node Standalone NX Project

Slide 59

Slide 59 text

How NX Helps Us To Solve Problems In The Mono-repo Approach ?

Slide 60

Slide 60 text

Separate Build For every Package Use That Command In Your CI/CD Tool

Slide 61

Slide 61 text

Build The Service Separately Gives Us The Ability To deploy It separately

Slide 62

Slide 62 text

Testing Each Component Separately ● A CI / CD which contains all the tests of all the micro services of the component can be very complex. ● By adding a test script for each microservice package.json we can build a CI/CD that will enable you to test each component by itself.

Slide 63

Slide 63 text

NX Cache - Solving Long Builds Problem NX cache optimizes the build process saves a lot of time - especially when you have hundreds of different services and libraries Critical aspect that will make you consider add it to you projects

Slide 64

Slide 64 text

Strategy To Integrate Existing Code To A Packaged Based Project

Slide 65

Slide 65 text

Start With Making Sure Your Code Dependencies Are Aligned

Slide 66

Slide 66 text

Suggested Code Structure

Slide 67

Slide 67 text

And If They Are Not Aligned - Start Breaking Them

Slide 68

Slide 68 text

Best Project Type To Integrate Existing Code With - Packaged Based NX Project

Slide 69

Slide 69 text

Why ‘Packaged Based’ Is The Best Project Type To Integrate Existing Code With ? ● You generally leave the module/express/nest code’s existing build/test/lint tooling untouched. ● In other NX project types - you need to change your build/test/lint tools to ﬁt to NX built in tools

Slide 70

Slide 70 text

Steps To Integrate Existing Code With Packaged Based NX Project ● Move an existing express/nest/utility into a package-based including package.json ● NX will detect the your code as a package, cache the relevant scripts and build the package dependency graph.