Tweet
Tweet

Slide 1

Slide 1 text

Node.js Scalability Tips Luciano Mammino ( ) @loige 2020-07-09 loige.link/node-scale 1

Slide 2

Slide 2 text

Hello, I am Luciano! Principal Software Engineer at FabFitFun Blog: Twitter: GitHub: loige.co @loige @lmammino nodejsdp.link/3rd 2

Slide 3

Slide 3 text

@loige Get the slides! loige.link/node-scale 3

Slide 4

Slide 4 text

Node.js + Scalability? @loige 4

Slide 5

Slide 5 text

@loige 5

Slide 6

Slide 6 text

@loige 6

Slide 7

Slide 7 text

@loige 6

Slide 8

Slide 8 text

@loige 6

Slide 9

Slide 9 text

@loige 6

Slide 10

Slide 10 text

@loige 7

Slide 11

Slide 11 text

@loige 7

Slide 12

Slide 12 text

@loige 7

Slide 13

Slide 13 text

@loige 7

Slide 14

Slide 14 text

@loige 7

Slide 15

Slide 15 text

"Scalability is the property of a system to handle a growing amount of work by adding resources to the system" — Wikipedia @loige 8

Slide 16

Slide 16 text

"A service is said to be scalable if when we increase the resources in a system, it results in increased performance in a manner proportional to resources added" — Werner Vogels @loige 9

Slide 17

Slide 17 text

Tip 1. Establish a baseline @loige 10

Slide 18

Slide 18 text

/?data=Hello%20Shift https://repl.it/@lmammino/QRGen @loige 11

Slide 19

Slide 19 text

const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) @loige 12

Slide 20

Slide 20 text

autocannon -c 200 --on-port / -- node server.js wrk node server.js& wrk -t8 -c200 -d10s http://localhost:8080/ @loige 13

Slide 21

Slide 21 text

autocannon -c 200 --on-port /?data=Hello%20Shift -- node server.js Running 10s test @ http://localhost:8080/?data=Hello%20Shift 200 connections ┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤ │ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │ └─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘ ┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Req/Sec │ 0 │ 0 │ 30 │ 199 │ 99.5 │ 94.27 │ 30 │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Bytes/Sec │ 0 B │ 0 B │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │ └───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘ Req/Bytes counts sampled once per second. 995 requests in 10.08s, 1.68 MB read @loige 14

Slide 22

Slide 22 text

autocannon -c 200 --on-port /?data=Hello%20Shift -- node server.js Running 10s test @ http://localhost:8080/?data=Hello%20Shift 200 connections ┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤ │ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │ └─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘ ┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Req/Sec │ 0 │ 0 │ 30 │ 199 │ 99.5 │ 94.27 │ 30 │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Bytes/Sec │ 0 B │ 0 B │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │ └───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘ Req/Bytes counts sampled once per second. 995 requests in 10.08s, 1.68 MB read @loige 14

Slide 23

Slide 23 text

⛅ Tip 1-bis Also, find out your ceiling @loige 15

Slide 24

Slide 24 text

const { createServer } = require('http') createServer((req, res) => { if (req.method === 'GET' && req.url === '/') { res.writeHead(200, { 'Content-Type': 'text/plain' }) res.end('Hello World\n') } else { res.statusCode = 404 res.end() } }) .listen(8080) @loige 16

Slide 25

Slide 25 text

autocannon -c 200 --on-port / -- node server.js Running 10s test @ http://localhost:8080/ 200 connections ┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤ │ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │ └─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘ ┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Req/Sec │ 21087 │ 21087 │ 34623 │ 35487 │ 33258.4 │ 4107.01 │ 21077 │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB │ 3.29 MB │ └───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘ Req/Bytes counts sampled once per second. 333k requests in 10.1s, 51.9 MB read @loige 17

Slide 26

Slide 26 text

autocannon -c 200 --on-port / -- node server.js Running 10s test @ http://localhost:8080/ 200 connections ┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤ │ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │ └─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘ ┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Req/Sec │ 21087 │ 21087 │ 34623 │ 35487 │ 33258.4 │ 4107.01 │ 21077 │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB │ 3.29 MB │ └───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘ Req/Bytes counts sampled once per second. 333k requests in 10.1s, 51.9 MB read @loige 17

Slide 27

Slide 27 text

Tip 2. Find your bottleneck @loige 18

Slide 28

Slide 28 text

Clinic.js clinicjs.org @loige 19

Slide 29

Slide 29 text

clinic doctor --autocannon [ -c 200 '/?data=Hello%20Shift' ] -- node server.js @loige 20

Slide 30

Slide 30 text

clinic flame --autocannon [ -c 200 '/?data=Hello%20Shift' ] -- node server.js @loige 21

Slide 31

Slide 31 text

clinic bubble --autocannon [ -c 200 '/?data=Hello%20Shift' ] -- node server.js @loige 22

Slide 32

Slide 32 text

Tip 3. Understand your goals @loige 23

Slide 33

Slide 33 text

What do we optimize for? Throughput? Memory? Latency? @loige 24

Slide 34

Slide 34 text

Tip 4. Always "observe" @loige 25

Slide 35

Slide 35 text

I mean, in production! Logs - Metrics - Traces @loige 26

Slide 36

Slide 36 text

Tip 5. Scale your architecture @loige 27

Slide 37

Slide 37 text

Performance != Scalability @loige 28

Slide 38

Slide 38 text

How can we scale a system by adding resources? @loige 29

Slide 39

Slide 39 text

The " " Scale Cube x-axis cloning z-axis partitioning y-axis functional decomposition @loige 30

Slide 40

Slide 40 text

The " " Scale Cube x-axis cloning z-axis partitioning y-axis functional decomposition @loige 30

Slide 41

Slide 41 text

Cloning Reverse proxy 31 Inside the same server Load Balancer Using multiple server @loige

Slide 42

Slide 42 text

The module cluster Master process Worker process Worker process Worker process 32 @loige

Slide 43

Slide 43 text

The module cluster Master process Worker process Worker process Worker process 32 @loige

Slide 44

Slide 44 text

The module cluster Master process Worker process Worker process Worker process 32 @loige

Slide 45

Slide 45 text

The module cluster Master process Worker process Worker process Worker process 32 @loige

Slide 46

Slide 46 text

The module cluster Master process Worker process Worker process Worker process 32 @loige

Slide 47

Slide 47 text

The module cluster Master process Worker process Worker process Worker process 32 @loige

Slide 48

Slide 48 text

The module cluster Master process Worker process Worker process Worker process 32 @loige

Slide 49

Slide 49 text

const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } @loige 33

Slide 50

Slide 50 text

const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 3-4x req/sec (8 core) @loige 33

Slide 51

Slide 51 text

You could also use Check out ! Worker Threads piscina @loige 34

Slide 52

Slide 52 text

Cloning is the easiest strategy to scale a service... ... as long as your application is Stateless @loige 35

Slide 53

Slide 53 text

API Gateway Functional decomposition a.k.a. "Micro-services" 36 /products /cart cart DB products DB @loige

Slide 54

Slide 54 text

API Gateway Functional decomposition a.k.a. "Micro-services" 37 /products /cart Functional decomposition can also be combined with cloning! cart DB products DB @loige

Slide 55

Slide 55 text

Node.js is great for microservices @loige 38

Slide 56

Slide 56 text

Microservices can also help with scaling the organisation! @loige 39

Slide 57

Slide 57 text

Microservices add complexity Observability Deployments Versioning Integration @loige 40

Slide 58

Slide 58 text

Partitioning Service and Data Partitioning along Customer Boundaries Shard partitioning /products/[A-L]/ /products/[M-Z]/ DB 2 41 DB 1 @loige

Slide 59

Slide 59 text

Partitioning is generally used to scale databases and SaaS software geographically @loige 42

Slide 60

Slide 60 text

Summary @loige 43

Slide 61

Slide 61 text

Summary Establish a baseline @loige 43

Slide 62

Slide 62 text

Summary Establish a baseline Find your bottleneck @loige 43

Slide 63

Slide 63 text

Summary Establish a baseline Find your bottleneck Understand your goals @loige 43

Slide 64

Slide 64 text

Summary Establish a baseline Find your bottleneck Understand your goals Always "observe" @loige 43

Slide 65

Slide 65 text

Summary Establish a baseline Find your bottleneck Understand your goals Always "observe" Scale your architecture (cloning, decomposition & partitioning) @loige 43

Slide 66

Slide 66 text

Thank you! Special thanks to , , , , , , , , , , , , , , , , , , , , , Icons and SVGs by @StefanoAbalsamo @matteocollina @dagonzago @NullishCoalesce @DublinSvelte @KViglucci @gjohnson391 @lucamaraschi @laurekamalandua @giltayar @mrm8488 @adrirai @harafise @EugeneWare @Jauny @tlivings @michaelcfine @leojino @shahidontech @Lordoomer @zsadigov @dottorblaster freepik.com loige.link/node-scale @loige 44