Node.js: scalability tips - Azure Dev Community Vijayawada

NODE.JS SCALABILITY TIPS
NODE.JS SCALABILITY TIPS
Luciano Mammino ( )
@loige
loige.link/node-adc
Vijayawada - 2021-12-13
1

View Slide

@loige
Get the slides!
👇
loige.link/node-adc
2

View Slide

HELLO!
3

View Slide

HELLO!
👋 I'm Luciano (
🍕🍝)
3

View Slide

HELLO!
👋 I'm Luciano (
🍕🍝)
Senior Architect @ fourTheorem (Dublin )
3

View Slide

HELLO!
👋 I'm Luciano (
🍕🍝)
nodejsdp.link
📔 Co-Author of Node.js Design Patterns
👉
3

View Slide

HELLO!
👋 I'm Luciano (
🍕🍝)
nodejsdp.link
📔 Co-Author of Node.js Design Patterns
👉
Let's connect!
(blog)
(twitter)
(twitch)
(github)
loige.co
@loige
loige
lmammino
3

View Slide

WE ARE BUSINESS FOCUSED TECHNOLOGISTS THAT DELIVER.
| |
Accelerated Serverless AI as a Service Platform Modernisation
We are hiring: do you want to ?
work with us
loige 4

View Slide

"A service is said to be scalable if when we
increase the resources in a system, it
results in increased performance in a
manner proportional to resources added"
— Werner Vogels
@loige
5

View Slide

🛫
TIP 1.
ESTABLISH A BASELINE
@loige
6

View Slide

/?data=ciao
👋 @loige
Site: - Code:
qrgen.lmammino.repl.co replit.com/@lmammino/QRGen
7

View Slide

import { createServer } from 'http'
import QRCode from 'qrcode'
createServer(function handler (req, res) {
const url = new URL(req.url, 'http://localhost:8080')
const data = url.searchParams.get('data')
if (!data) {
res.writeHead(200, {'Content-Type': 'text/html'})
return res.end(`

To use this app you need to set the data querystrin
Try for example /?data=hello

`)
}
res.writeHead(200, { 'Content-Type': 'image/png' })
QRCode.toFileStream(res, data, { width: 300 })
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
@loige
8

View Slide

if (!data) {
return res.end(`


`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
}).listen(8080)
18
@loige
8

View Slide

if (!data) {
return res.end(`


`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
}).listen(8080)
18
}).listen(8080)
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
18
@loige
8

View Slide

if (!data) {
return res.end(`


`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
}).listen(8080)
18
}).listen(8080)
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
}).listen(8080)
18
@loige
8

View Slide

if (!data) {
return res.end(`


`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
}).listen(8080)
18
}).listen(8080)
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
}).listen(8080)
18
if (!data) {
return res.end(`


`)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
}).listen(8080)
18
@loige
8

View Slide

if (!data) {
return res.end(`


`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
}).listen(8080)
18
}).listen(8080)
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
}).listen(8080)
18
if (!data) {
return res.end(`


`)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
}).listen(8080)
18
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
}).listen(8080)
18
@loige
8

View Slide

if (!data) {
return res.end(`


`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
}).listen(8080)
18
}).listen(8080)
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
18
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
}).listen(8080)
18
if (!data) {
return res.end(`


`)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
}).listen(8080)
18
1
2
3
4
5
6
if (!data) {
7
8
return res.end(`
9

10
11
12

13
`)
14
}
15
16
17
}).listen(8080)
18
if (!data) {
return res.end(`


`)
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
@loige
8

View Slide

autocannon -c 200 --on-port / -- node server.js
WRK
node server.js&
wrk -t8 -c200 -d10s http://localhost:8080/
@loige
9

View Slide

autocannon -c 200 --on-port /?data=ciao
👋
-- node server.js
Running 10s test @ http://localhost:8080/?data=ciao
👋
200 connections
┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤
│ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │
└─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘
┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤
│ Req/Sec │ 0 │ 0 │ 30 │ 199 │ 99.5 │ 94.27 │ 30 │
├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤
│ Bytes/Sec │ 0 B │ 0 B │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │
└───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘
Req/Bytes counts sampled once per second.
995 requests in 10.08s, 1.68 MB read
@loige
10

View Slide

⛅
TIP 1-BIS
ALSO, FIND OUT YOUR CEILING
@loige
11

View Slide

createServer((req, res) => {
if (req.method === 'GET' && req.url === '/') {
res.writeHead(200, { 'Content-Type': 'text/plain' })
res.end('Hello World\n')
} else {
res.statusCode = 404
res.end()
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
@loige
12

View Slide

} else {
res.end()
}
}).listen(8080)
1
2
3
4
5
6
7
8
9
10
11
1
2
3
4
5
6
} else {
7
8
res.end()
9
}
10
}).listen(8080)
11
@loige
12

View Slide

autocannon -c 200 --on-port / -- node server.js
Running 10s test @ http://localhost:8080/
200 connections
┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤
│ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │
└─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘
┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec │ 21087 │ 21087 │ 34623 │ 35487 │ 33258.4 │ 4107.01 │ 21077 │
├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB │ 3.29 MB │
└───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘
Req/Bytes counts sampled once per second.
333k requests in 10.1s, 51.9 MB read
@loige
13

View Slide

🍾
TIP 2.
FIND YOUR BOTTLENECK
@loige
14

View Slide

Clinic.js
clinicjs.org
@loige
15

View Slide

clinic doctor --autocannon [ -c 200 '/?data=ciao
👋
' ] -- node server.js
@loige
16

View Slide

clinic flame --autocannon [ -c 200 '/?data=ciao
👋
@loige
17

View Slide

clinic bubble --autocannon [ -c 200 '/?data=ciao
👋
@loige
18

View Slide

🎳
TIP 3.
UNDERSTAND YOUR GOALS
@loige
19

View Slide

WHAT DO WE OPTIMIZE FOR?
THROUGHPUT?
MEMORY?
LATENCY?
@loige
20

View Slide

👁
TIP 4.
ALWAYS "OBSERVE"
@loige
21

View Slide

I MEAN, IN PRODUCTION!
Logs - Metrics - Traces
@loige
22

View Slide

🚀
TIP 5.
SCALE YOUR ARCHITECTURE
@loige
23

View Slide

PERFORMANCE != SCALABILITY
@loige
24

View Slide

HOW CAN WE SCALE A SYSTEM
BY ADDING RESOURCES?
@loige
25

View Slide

THE " "
SCALE CUBE
x-axis
cloning
z-axis
partitioning
y-axis
functional
decomposition
@loige
26

View Slide

27
Reverse proxy
CLONING
Inside the same server
Load Balancer
Using multiple server
@loige

View Slide

THE MODULE
CLUSTER
Master process
Worker
process
Worker
process
Worker
process
28
@loige

View Slide

import cluster from 'cluster'
import { cpus } from 'os'
const numCPUs = cpus().length
if (cluster.isMaster) {
// Fork workers
for (let i = 0; i < numCPUs; i++) {
cluster.fork()
}
} else {
// Worker code here...
}
1
2
3
4
5
6
7
8
9
10
11
12
13
@loige
29

View Slide

// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
3-4x req/sec
(8 core)
@loige
29

View Slide

// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
3-4x req/sec
(8 core)
@loige
29

View Slide

// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
} else {
}
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
11
12
13
3-4x req/sec
(8 core)
@loige
29

View Slide

// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
} else {
}
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
11
12
13
// Fork workers
cluster.fork()
}
1
2
3
4
5
6
7
8
9
10
} else {
11
12
}
13
3-4x req/sec
(8 core)
@loige
29

View Slide

// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
} else {
}
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
11
12
13
// Fork workers
cluster.fork()
}
1
2
3
4
5
6
7
8
9
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
3-4x req/sec
(8 core)
@loige
29

View Slide

// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
} else {
}
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
11
12
13
// Fork workers
cluster.fork()
}
1
2
3
4
5
6
7
8
9
10
} else {
11
12
}
13
1
2
3
4
5
6
// Fork workers
7
8
cluster.fork()
9
}
10
} else {
11
12
}
13
// Fork workers
cluster.fork()
}
} else {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
3-4x req/sec
(8 core)
@loige
29

View Slide

YOU COULD ALSO USE
CHECK OUT !
WORKER
THREADS
PISCINA
@loige
30

View Slide

CLONING IS THE EASIEST STRATEGY TO SCALE A SERVICE...
... AS LONG AS YOUR APPLICATION IS "STATELESS"
@loige
31

View Slide

API Gateway
FUNCTIONAL DECOMPOSITION
a.k.a. "Micro-services"
32
/products
/cart
cart DB
products DB
@loige

View Slide

API Gateway
FUNCTIONAL DECOMPOSITION
a.k.a. "Micro-services"
33
/products
/cart
Functional decomposition can
also be combined with cloning!
cart DB
products DB
@loige

View Slide

NODE.JS IS GREAT FOR MICROSERVICES
@loige
34

View Slide

MICROSERVICES CAN ALSO HELP WITH
SCALING THE ORGANISATION!
@loige
35

View Slide

MICROSERVICES ADD COMPLEXITY
Observability
Deployments
Versioning
Integration
@loige
36

View Slide

PARTITIONING
Service and Data Partitioning along Customer Boundaries
Shard partitioning
/products/[A-L]/
/products/[M-Z]/
DB 2
37
DB 1
@loige

View Slide

PARTITIONING IS GENERALLY USED
TO SCALE DATABASES
AND
SAAS SOFTWARE GEOGRAPHICALLY
@loige
38

View Slide

SUMMARY @loige
39

View Slide

SUMMARY
🛫 Establish a baseline
@loige
39

View Slide

SUMMARY
🍾 Find your bottleneck
@loige
39

View Slide

SUMMARY
🎳 Understand your goals
@loige
39

View Slide

SUMMARY
👁 Always "observe"
@loige
39

View Slide

SUMMARY
👁 Always "observe"
🚀 Scale your architecture
(cloning, decomposition & partitioning)
@loige
39

View Slide

☝ nodejsdp.link
THANK YOU!
Special thanks to , ,
, , ,
, , ,
, , , ,
, , , ,
, , , ,
,
Icons and SVGs by
Photo by on
@StefanoAbalsamo @matteocollina
@dagonzago @NullishCoalesce @DublinSvelte
@KViglucci @gjohnson391 @lucamaraschi
@laurekamalandua @giltayar @mrm8488 @adrirai
@haraﬁse @EugeneWare @Jauny @tlivings
@michaelcﬁne @leojino @shahidontech @Lordoomer
@zsadigov @dottorblaster
freepik.com
Jonas Kaiser Unsplash
@loige
loige.link/node-adc
40

View Slide

Node.js: scalability tips - Azure Dev Community Vijayawada

Node.js: scalability tips - Azure Dev Community Vijayawada

Luciano Mammino

More Decks by Luciano Mammino

Other Decks in Technology

Featured

Transcript