Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Node.js: scalability tips - CityJS 2020

Node.js: scalability tips - CityJS 2020

You finally built that amazing start-up idea you had in mind for years and you did it using Node.js! That's Great! You just launched it on Hacker News and you are very happy and proud... but now more and more people are using it and you start to have a nasty fear that Node.js won't scale because you now... it's single-threaded! Is your project doomed now? Do you have to invest your time on rewriting it in something like C++ or maybe Rust or even Go? You'd rather invest your time on adding valuable features for your users rather than learning a new language and rewriting everything from scratch, but what if nothing works anymore? And... by the way, what the heck "single-threaded" really means?! Fear no more, dear fellow developer! In this talk, we will discuss the architecture of Node.js going through its strengths and its weaknesses. We will then talk about scalability and I will share some valuable tips and tricks to make your Node.js app scale! Spoiler alert: you probably won't need Go or Rust :)

F3a6662b3cd161c3c2f13604965ed0f2?s=128

Luciano Mammino

September 15, 2020
Tweet

Transcript

  1. Node.js Scalability Tips Luciano Mammino ( ) @loige loige.link/node-scale-city 2020-09-15

    1
  2. Hello, I am Luciano! Principal Software Engineer at FabFitFun Blog:

    Twitter: GitHub: loige.co @loige @lmammino nodejsdp.link/buy 2
  3. Hello, I am Luciano! Principal Software Engineer at FabFitFun Blog:

    Twitter: GitHub: loige.co @loige @lmammino nodejsdp.link/buy CITYJS SPECIAL DISCOUNT: 20% OFF on Amazon.com - www.amazon.com/gp/mpc/A2L4T1QGT3DXYX 2
  4. @loige Get the slides! loige.link/node-scale-city 3

  5. Node.js + Scalability? @loige 4

  6. @loige 5

  7. @loige 6

  8. @loige 6

  9. @loige 6

  10. @loige 6

  11. @loige 7

  12. @loige 7

  13. @loige 7

  14. @loige 7

  15. @loige 7

  16. "Scalability is the property of a system to handle a

    growing amount of work by adding resources to the system" — Wikipedia @loige 8
  17. "A service is said to be scalable if when we

    increase the resources in a system, it results in increased performance in a manner proportional to resources added" — Werner Vogels @loige 9
  18. Tip 1. Establish a baseline @loige 10

  19. /?data=hello%20cityjs https://repl.it/@lmammino/QRGen @loige 11

  20. const { createServer } = require('http') const { URL }

    = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 @loige 12
  21. const { createServer } = require('http') const { URL }

    = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') 1 2 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 @loige 12
  22. const { createServer } = require('http') const { URL }

    = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') 1 2 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 createServer(function handler (req, res) { }) .listen(8080) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 14 15 @loige 12
  23. const { createServer } = require('http') const { URL }

    = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') 1 2 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 createServer(function handler (req, res) { }) .listen(8080) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 14 15 const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 6 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 @loige 12
  24. const { createServer } = require('http') const { URL }

    = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') 1 2 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 createServer(function handler (req, res) { }) .listen(8080) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 14 15 const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 6 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 if (!data) { res.writeHead(400) // bad request return res.end() } const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 8 9 10 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 @loige 12
  25. const { createServer } = require('http') const { URL }

    = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') 1 2 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 createServer(function handler (req, res) { }) .listen(8080) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 14 15 const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 6 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 if (!data) { res.writeHead(400) // bad request return res.end() } const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 8 9 10 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 12 13 }) 14 .listen(8080) 15 @loige 12
  26. const { createServer } = require('http') const { URL }

    = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') 1 2 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 createServer(function handler (req, res) { }) .listen(8080) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 14 15 const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 6 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 if (!data) { res.writeHead(400) // bad request return res.end() } const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 8 9 10 11 res.writeHead(200, { 'Content-Type': 'image/png' }) 12 QRCode.toFileStream(res, data, { width: 300 }) 13 }) 14 .listen(8080) 15 res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) const { createServer } = require('http') 1 const { URL } = require('url') 2 const QRCode = require('qrcode') 3 4 createServer(function handler (req, res) { 5 const url = new URL(req.url, 'http://localhost:8080') 6 const data = url.searchParams.get('data') 7 if (!data) { 8 res.writeHead(400) // bad request 9 return res.end() 10 } 11 12 13 }) 14 .listen(8080) 15 const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 @loige 12
  27. autocannon -c 200 --on-port / -- node server.js wrk node

    server.js& wrk -t8 -c200 -d10s http://localhost:8080/ @loige 13
  28. autocannon -c 200 --on-port /?data=hello%20cityjs -- node server.js Running 10s

    test @ http://localhost:8080/?data=hello%20cityjs 200 connections ┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤ │ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │ └─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘ ┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Req/Sec │ 0 │ 0 │ 30 │ 199 │ 99.5 │ 94.27 │ 30 │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Bytes/Sec │ 0 B │ 0 B │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │ └───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘ Req/Bytes counts sampled once per second. 995 requests in 10.08s, 1.68 MB read @loige 14
  29. autocannon -c 200 --on-port /?data=hello%20cityjs -- node server.js Running 10s

    test @ http://localhost:8080/?data=hello%20cityjs 200 connections ┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤ │ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │ └─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘ ┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Req/Sec │ 0 │ 0 │ 30 │ 199 │ 99.5 │ 94.27 │ 30 │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Bytes/Sec │ 0 B │ 0 B │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │ └───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘ Req/Bytes counts sampled once per second. 995 requests in 10.08s, 1.68 MB read @loige 14
  30. ⛅ Tip 1-bis Also, find out your ceiling @loige 15

  31. const { createServer } = require('http') createServer((req, res) => {

    if (req.method === 'GET' && req.url === '/') { res.writeHead(200, { 'Content-Type': 'text/plain' }) res.end('Hello World\n') } else { res.statusCode = 404 res.end() } }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 @loige 16
  32. const { createServer } = require('http') createServer((req, res) => {

    if (req.method === 'GET' && req.url === '/') { res.writeHead(200, { 'Content-Type': 'text/plain' }) res.end('Hello World\n') } else { res.statusCode = 404 res.end() } }) .listen(8080) 1 2 3 4 5 6 7 8 9 10 11 12 res.writeHead(200, { 'Content-Type': 'text/plain' }) res.end('Hello World\n') const { createServer } = require('http') 1 2 createServer((req, res) => { 3 if (req.method === 'GET' && req.url === '/') { 4 5 6 } else { 7 res.statusCode = 404 8 res.end() 9 } 10 }) 11 .listen(8080) 12 @loige 16
  33. autocannon -c 200 --on-port / -- node server.js Running 10s

    test @ http://localhost:8080/ 200 connections ┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤ │ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │ └─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘ ┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Req/Sec │ 21087 │ 21087 │ 34623 │ 35487 │ 33258.4 │ 4107.01 │ 21077 │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB │ 3.29 MB │ └───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘ Req/Bytes counts sampled once per second. 333k requests in 10.1s, 51.9 MB read @loige 17
  34. autocannon -c 200 --on-port / -- node server.js Running 10s

    test @ http://localhost:8080/ 200 connections ┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤ │ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │ └─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘ ┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Req/Sec │ 21087 │ 21087 │ 34623 │ 35487 │ 33258.4 │ 4107.01 │ 21077 │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB │ 3.29 MB │ └───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘ Req/Bytes counts sampled once per second. 333k requests in 10.1s, 51.9 MB read @loige 17
  35. Tip 2. Find your bottleneck @loige 18

  36. Clinic.js clinicjs.org @loige 19

  37. clinic doctor --autocannon [ -c 200 '/?data=hello%20cityjs' ] -- node

    server.js @loige 20
  38. clinic flame --autocannon [ -c 200 '/?data=hello%20cityjs' ] -- node

    server.js @loige 21
  39. clinic bubble --autocannon [ -c 200 '/?data=hello%20cityjs' ] -- node

    server.js @loige 22
  40. Tip 3. Understand your goals @loige 23

  41. What do we optimize for? Throughput? Memory? Latency? @loige 24

  42. Tip 4. Always "observe" @loige 25

  43. I mean, in production! Logs - Metrics - Traces @loige

    26
  44. Tip 5. Scale your architecture @loige 27

  45. Performance != Scalability @loige 28

  46. How can we scale a system by adding resources? @loige

    29
  47. The " " Scale Cube x-axis cloning z-axis partitioning y-axis

    functional decomposition @loige 30
  48. The " " Scale Cube x-axis cloning z-axis partitioning y-axis

    functional decomposition @loige 30
  49. Cloning Reverse proxy 31 Inside the same server Load Balancer

    Using multiple server @loige
  50. The module cluster Master process Worker process Worker process Worker

    process 32 @loige
  51. The module cluster Master process Worker process Worker process Worker

    process 32 @loige
  52. The module cluster Master process Worker process Worker process Worker

    process 32 @loige
  53. The module cluster Master process Worker process Worker process Worker

    process 32 @loige
  54. The module cluster Master process Worker process Worker process Worker

    process 32 @loige
  55. The module cluster Master process Worker process Worker process Worker

    process 32 @loige
  56. The module cluster Master process Worker process Worker process Worker

    process 32 @loige
  57. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster)

    { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 @loige 33
  58. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster)

    { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 3-4x req/sec (8 core) @loige 33
  59. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster)

    { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 3-4x req/sec (8 core) @loige 33
  60. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster)

    { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 const numCPUs = require('os').cpus().length const cluster = require('cluster') 1 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 3-4x req/sec (8 core) @loige 33
  61. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster)

    { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 const numCPUs = require('os').cpus().length const cluster = require('cluster') 1 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 if (cluster.isMaster) { } else { } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 9 // Worker code 10 require('./server.js') 11 12 3-4x req/sec (8 core) @loige 33
  62. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster)

    { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 const numCPUs = require('os').cpus().length const cluster = require('cluster') 1 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 if (cluster.isMaster) { } else { } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 9 // Worker code 10 require('./server.js') 11 12 // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 5 6 7 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 3-4x req/sec (8 core) @loige 33
  63. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster)

    { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 const numCPUs = require('os').cpus().length const cluster = require('cluster') 1 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 if (cluster.isMaster) { } else { } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 9 // Worker code 10 require('./server.js') 11 12 // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 5 6 7 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 // Worker code require('./server.js') const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 10 11 } 12 3-4x req/sec (8 core) @loige 33
  64. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster)

    { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 const numCPUs = require('os').cpus().length const cluster = require('cluster') 1 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 if (cluster.isMaster) { } else { } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 9 // Worker code 10 require('./server.js') 11 12 // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 5 6 7 8 } else { 9 // Worker code 10 require('./server.js') 11 } 12 // Worker code require('./server.js') const cluster = require('cluster') 1 const numCPUs = require('os').cpus().length 2 3 if (cluster.isMaster) { 4 // Fork workers 5 for (let i = 0; i < numCPUs; i++) { 6 cluster.fork() 7 } 8 } else { 9 10 11 } 12 const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 1 2 3 4 5 6 7 8 9 10 11 12 3-4x req/sec (8 core) @loige 33
  65. You could also use Check out ! Worker Threads piscina

    @loige 34
  66. Cloning is the easiest strategy to scale a service... ...

    as long as your application is "Stateless" @loige 35
  67. API Gateway Functional decomposition a.k.a. "Micro-services" 36 /products /cart cart

    DB products DB @loige
  68. API Gateway Functional decomposition a.k.a. "Micro-services" 37 /products /cart Functional

    decomposition can also be combined with cloning! cart DB products DB @loige
  69. Node.js is great for microservices @loige 38

  70. Microservices can also help with scaling the organisation! @loige 39

  71. Microservices add complexity Observability Deployments Versioning Integration @loige 40

  72. Partitioning Service and Data Partitioning along Customer Boundaries Shard partitioning

    /products/[A-L]/ /products/[M-Z]/ DB 2 41 DB 1 @loige
  73. Partitioning is generally used to scale databases and SaaS software

    geographically @loige 42
  74. Summary @loige 43

  75. Summary Establish a baseline @loige 43

  76. Summary Establish a baseline Find your bottleneck @loige 43

  77. Summary Establish a baseline Find your bottleneck Understand your goals

    @loige 43
  78. Summary Establish a baseline Find your bottleneck Understand your goals

    Always "observe" @loige 43
  79. Summary Establish a baseline Find your bottleneck Understand your goals

    Always "observe" Scale your architecture (cloning, decomposition & partitioning) @loige 43
  80. Thank you! Special thanks to , , , , ,

    , , , , , , , , , , , , , , , , Icons and SVGs by @StefanoAbalsamo @matteocollina @dagonzago @NullishCoalesce @DublinSvelte @KViglucci @gjohnson391 @lucamaraschi @laurekamalandua @giltayar @mrm8488 @adrirai @harafise @EugeneWare @Jauny @tlivings @michaelcfine @leojino @shahidontech @Lordoomer @zsadigov @dottorblaster freepik.com loige.link/node-scale-city @loige 44