Node.js Scalability Tips

You finally built that amazing start-up idea you had in mind for years and you did it using Node.js! That's Great! You just launched it on Hacker News and you are very happy and proud... but now more and more people are using it and you start to have a nasty fear that Node.js won't scale because you now... it's single-threaded! Is your project doomed now? Do you have to invest your time on rewriting it in something like C++ or maybe Rust or even Go? You'd rather invest your time on adding valuable features for your users rather than learning a new language and rewriting everything from scratch, but what if nothing works anymore? And... by the way, what the heck "single-threaded" really means?! Fear no more, dear fellow developer! In this talk, we will discuss the architecture of Node.js going through its strengths and its weaknesses. We will then talk about scalability and I will share some valuable tips and tricks to make your Node.js app scale! Spoiler alert: you probably won't need Go or Rust :)

Transcript

1. NODE.JS SCALABILITY TIPS Luciano Mammino ( ) @loige loige.link/nodescale 1
2. HELLO, I AM LUCIANO! Cloud Architect and Fullstack developer Blog: Twitter: GitHub: loige.co @loige @lmammino nodejsdp.link 2
3. @loige Get the slides! loige.link/nodescale 3
4. "A service is said to be scalable if when we increase the resources in a system, it results in increased performance in a manner proportional to resources added" — Werner Vogels @loige 4
5. TIP 1. ESTABLISH A BASELINE @loige 5
6. /?data=ciao @loige https://repl.it/@lmammino/QRGen 6
7. const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 @loige 7
8. const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') 1 2 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 @loige 7
9. const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') 1 2 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 createServer(function handler (req, res) { }) .listen(8080) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 14 15 @loige 7
10. const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') 1 2 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 createServer(function handler (req, res) { }) .listen(8080) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 14 15 const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 6 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 @loige 7
11. const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') 1 2 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 createServer(function handler (req, res) { }) .listen(8080) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 14 15 const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 6 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 if (!data) { res.writeHead(400) // bad request return res.end() } const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 8 9 10 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 @loige 7
12. const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') 1 2 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 createServer(function handler (req, res) { }) .listen(8080) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 14 15 const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 6 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 if (!data) { res.writeHead(400) // bad request return res.end() } const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 8 9 10 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 12 13 }) 14 .listen(8080) 15 @loige 7
13. const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') 1 2 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 createServer(function handler (req, res) { }) .listen(8080) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 14 15 const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 6 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 if (!data) { res.writeHead(400) // bad request return res.end() } const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 8 9 10 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 12 13 }) 14 .listen(8080) 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 @loige 7
14. autocannon -c 200 --on-port / -- node server.js WRK node server.js& wrk -t8 -c200 -d10s http://localhost:8080/ @loige 8
15. autocannon -c 200 --on-port /?data=hello%20cityjs -- node server.js Running 10s test @ http://localhost:8080/?data=ciao 200 connections ┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤ │ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │ └─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘ ┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Req/Sec │ 0 │ 0 │ 30 │ 199 │ 99.5 │ 94.27 │ 30 │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Bytes/Sec │ 0 B │ 0 B │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │ └───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘ Req/Bytes counts sampled once per second. 995 requests in 10.08s, 1.68 MB read @loige 9
16. autocannon -c 200 --on-port /?data=hello%20cityjs -- node server.js Running 10s test @ http://localhost:8080/?data=ciao 200 connections ┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤ │ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │ └─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘ ┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Req/Sec │ 0 │ 0 │ 30 │ 199 │ 99.5 │ 94.27 │ 30 │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Bytes/Sec │ 0 B │ 0 B │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │ └───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘ Req/Bytes counts sampled once per second. 995 requests in 10.08s, 1.68 MB read @loige 9
17. ⛅ TIP 1-BIS ALSO, FIND OUT YOUR CEILING @loige 10
18. const { createServer } = require('http') createServer((req, res) => { if (req.method === 'GET' && req.url === '/') { res.writeHead(200, { 'Content-Type': 'text/plain' }) res.end('Hello World\n') } else { res.statusCode = 404 res.end() } }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 @loige 11
19. const { createServer } = require('http') createServer((req, res) => { if (req.method === 'GET' && req.url === '/') { res.writeHead(200, { 'Content-Type': 'text/plain' }) res.end('Hello World\n') } else { res.statusCode = 404 res.end() } }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 res.writeHead(200, { 'Content-Type': 'text/plain' }) res.end('Hello World\n') const { createServer } = require('http') 1 2 createServer((req, res) => { 3 if (req.method === 'GET' && req.url === '/') { 4 5 6 } else { 7 res.statusCode = 404 8 res.end() 9 } 10 }) 11 .listen(8080) 12 @loige 11
20. autocannon -c 200 --on-port / -- node server.js Running 10s test @ http://localhost:8080/ 200 connections ┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤ │ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │ └─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘ ┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Req/Sec │ 21087 │ 21087 │ 34623 │ 35487 │ 33258.4 │ 4107.01 │ 21077 │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB │ 3.29 MB │ └───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘ Req/Bytes counts sampled once per second. 333k requests in 10.1s, 51.9 MB read @loige 12
21. autocannon -c 200 --on-port / -- node server.js Running 10s test @ http://localhost:8080/ 200 connections ┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤ │ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │ └─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘ ┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Req/Sec │ 21087 │ 21087 │ 34623 │ 35487 │ 33258.4 │ 4107.01 │ 21077 │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB │ 3.29 MB │ └───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘ Req/Bytes counts sampled once per second. 333k requests in 10.1s, 51.9 MB read @loige 12
22. TIP 2. FIND YOUR BOTTLENECK @loige 13
23. Clinic.js clinicjs.org @loige 14
24. clinic doctor --autocannon [ -c 200 '/?data=ciao ' ] -- node server.js @loige 15
25. clinic flame --autocannon [ -c 200 '/?data=ciao ' ] -- node server.js @loige 16
26. clinic bubble --autocannon [ -c 200 '/?data=ciao ' ] -- node server.js @loige 17
27. TIP 3. UNDERSTAND YOUR GOALS @loige 18
28. WHAT DO WE OPTIMIZE FOR? THROUGHPUT? MEMORY? LATENCY? @loige 19
29. TIP 4. ALWAYS "OBSERVE" @loige 20
30. I MEAN, IN PRODUCTION! Logs - Metrics - Traces @loige 21
31. TIP 5. SCALE YOUR ARCHITECTURE @loige 22
32. PERFORMANCE != SCALABILITY @loige 23
33. HOW CAN WE SCALE A SYSTEM BY ADDING RESOURCES? @loige 24
34. THE " " SCALE CUBE x-axis cloning z-axis partitioning y-axis functional decomposition @loige 25
35. THE " " SCALE CUBE x-axis cloning z-axis partitioning y-axis functional decomposition @loige 25
36. 26 Reverse proxy CLONING Inside the same server Load Balancer Using multiple server @loige
37. THE MODULE CLUSTER Master process Worker process Worker process Worker process 27 @loige
38. THE MODULE CLUSTER Master process Worker process Worker process Worker process 27 @loige
39. THE MODULE CLUSTER Master process Worker process Worker process Worker process 27 @loige
40. THE MODULE CLUSTER Master process Worker process Worker process Worker process 27 @loige
41. THE MODULE CLUSTER Master process Worker process Worker process Worker process 27 @loige
42. THE MODULE CLUSTER Master process Worker process Worker process Worker process 27 @loige
43. THE MODULE CLUSTER Master process Worker process Worker process Worker process 27 @loige
44. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 @loige 28
45. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 @loige 28
46. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 const numCPUs = require('os').cpus().length const cluster = require('cluster') 1 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 @loige 28
47. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 const numCPUs = require('os').cpus().length const cluster = require('cluster') 1 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 if (cluster.isMaster) { } else { } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 9 // Worker code 10 require('./server.js') 11 12 @loige 28
48. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 const numCPUs = require('os').cpus().length const cluster = require('cluster') 1 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 if (cluster.isMaster) { } else { } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 9 // Worker code 10 require('./server.js') 11 12 // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 5 6 7 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 @loige 28
49. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 const numCPUs = require('os').cpus().length const cluster = require('cluster') 1 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 if (cluster.isMaster) { } else { } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 9 // Worker code 10 require('./server.js') 11 12 // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 5 6 7 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 // Worker code require('./server.js') const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 10 11 } 12 @loige 28
50. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 const numCPUs = require('os').cpus().length const cluster = require('cluster') 1 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 if (cluster.isMaster) { } else { } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 9 // Worker code 10 require('./server.js') 11 12 // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 5 6 7 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 // Worker code require('./server.js') const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 10 11 } 12 const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 @loige 28
51. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 const numCPUs = require('os').cpus().length const cluster = require('cluster') 1 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 if (cluster.isMaster) { } else { } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 9 // Worker code 10 require('./server.js') 11 12 // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 5 6 7 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 // Worker code require('./server.js') const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 10 11 } 12 const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 3-4x req/sec (8 core) @loige 28
52. YOU COULD ALSO USE CHECK OUT ! WORKER THREADS PISCINA @loige 29
53. CLONING IS THE EASIEST STRATEGY TO SCALE A SERVICE... ... AS LONG AS YOUR APPLICATION IS "STATELESS" @loige 30
54. API Gateway FUNCTIONAL DECOMPOSITION a.k.a. "Micro-services" 31 /products /cart cart DB products DB @loige
55. API Gateway FUNCTIONAL DECOMPOSITION a.k.a. "Micro-services" 32 /products /cart Functional decomposition can also be combined with cloning! cart DB products DB @loige
56. NODE.JS IS GREAT FOR MICROSERVICES @loige 33
57. MICROSERVICES CAN ALSO HELP WITH SCALING THE ORGANISATION! @loige 34
58. MICROSERVICES ADD COMPLEXITY Observability Deployments Versioning Integration @loige 35
59. PARTITIONING Service and Data Partitioning along Customer Boundaries Shard partitioning /products/[A-L]/ /products/[M-Z]/ DB 2 36 DB 1 @loige
60. PARTITIONING IS GENERALLY USED TO SCALE DATABASES AND SAAS SOFTWARE GEOGRAPHICALLY @loige 37
61. SUMMARY @loige 38
62. SUMMARY Establish a baseline @loige 38
63. SUMMARY Establish a baseline Find your bottleneck @loige 38
64. SUMMARY Establish a baseline Find your bottleneck Understand your goals @loige 38
65. SUMMARY Establish a baseline Find your bottleneck Understand your goals Always "observe" @loige 38
66. SUMMARY Establish a baseline Find your bottleneck Understand your goals Always "observe" Scale your architecture (cloning, decomposition & partitioning) @loige 38
67. THANK YOU! Special thanks to , , , , , , , , , , , , , , , , , , , , , Icons and SVGs by Cover picture by on @StefanoAbalsamo @matteocollina @dagonzago @NullishCoalesce @DublinSvelte @KViglucci @gjohnson391 @lucamaraschi @laurekamalandua @giltayar @mrm8488 @adrirai @harafise @EugeneWare @Jauny @tlivings @michaelcfine @leojino @shahidontech @Lordoomer @zsadigov @dottorblaster freepik.com Markus Spiske Unsplash @loige loige.link/nodescale 39