Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scaling NodeJS beyond the ordinary

Scaling NodeJS beyond the ordinary

These are the slides to accompany the talk I gave at JSConf Iceland 2018.

Abhinav Rastogi

March 02, 2018
Tweet

More Decks by Abhinav Rastogi

Other Decks in Technology

Transcript

  1. Scaling NodeJS
    Beyond the ordinary
    Abhinav Rastogi
    Lead UI Engineer
    _abhinavrastogi

    View Slide

  2. View Slide

  3. 150M Registered
    Users
    500K Phones sold in
    15 seconds
    200M Visits over 5 days
    during annual sale event

    View Slide

  4. – Andre Bondi
    “Scalability is the capability of a system to handle a
    growing amount of work, or its potential to be enlarged to
    accommodate that growth.”

    View Slide

  5. Why should it concern the JS
    community?

    View Slide

  6. Why should it concern the JS
    community?
    • JS is running on servers now
    • Failure is shifting to application code
    • Failure is better handled by devs

    View Slide

  7. Types of Scalability
    x
    y
    z
    Add more machines
    Add more resources
    Application layer

    View Slide

  8. Horizontal Scaling

    View Slide

  9. Horizontal Scaling
    Machine
    Node
    API

    View Slide

  10. Horizontal Scaling
    Machine
    Node
    API
    Machine
    Node
    ELB

    View Slide

  11. Vertical Scaling

    View Slide

  12. Vertical Scaling
    Machine
    Node
    API

    View Slide

  13. Vertical Scaling
    Machine
    Node
    API
    Node
    ??

    View Slide

  14. Application Layer
    Optimisations

    View Slide

  15. The Optimisation Cycle
    Load Test
    Find Bottleneck
    Fix Issues

    View Slide

  16. network, cpu, memory, disk

    View Slide

  17. network bandwidth

    View Slide

  18. network bandwidth

    View Slide

  19. 1000 kb per page
    100 rps per machine
    100 machines
    = 10gbps

    View Slide

  20. compression

    View Slide

  21. const app = express()
    app.use(compression)

    View Slide

  22. co-hosted nginx

    View Slide

  23. co-hosted nginx
    Machine
    Node
    Node
    PM2
    nginx
    API
    ELB

    View Slide

  24. network profiling

    View Slide

  25. netstat / ss
    lsof
    watch

    View Slide

  26. everything is a file in unix!

    View Slide

  27. ulimit

    View Slide

  28. ulimit
    core file size (blocks, -c) 0
    file size (blocks, -f) unlimited
    pending signals (-i) 32767
    max locked memory (kbytes, -l) 32
    max memory size (kbytes, -m) unlimited
    open files (-n) 1024
    max user processes (-u) 50

    View Slide

  29. increase the limits
    /
    reduce the usage

    View Slide

  30. keep-alive header

    View Slide

  31. keep-alive header
    Object.assign(headers, {
    'Connection': 'keep-alive',
    'Keep-Alive': 'timeout=200'
    });

    View Slide

  32. connection pooling

    View Slide

  33. connection pooling
    const http = require('http');
    fetch(url, {
    agent: new http.Agent({
    keepAlive: true,
    maxSockets: 24
    })
    });

    View Slide

  34. ephemeral ports

    View Slide

  35. tcp connection states

    View Slide

  36. tcp connection states
    source: http://cnp3book.info.ucl.ac.be/1st/html/transport/transport.html

    View Slide

  37. tcp connection states

    View Slide

  38. cpu

    View Slide

  39. cpu profiling

    View Slide

  40. node --prof app.js

    View Slide

  41. import crypto from 'crypto';
    app.get('/auth', (req, res) => {
    const hash = crypto.pbkdf2Sync(password, users[username].salt, 100, 512);
    if (users[username].hash.toString() === hash.toString()) {
    res.sendStatus(200);
    } else {
    res.sendStatus(401);
    }
    });

    View Slide

  42. [Summary]:
    ticks total nonlib name
    79 0.2% 0.2% JavaScript
    36703 97.2% 99.2% C++
    7 0.0% 0.0% GC
    767 2.0% Shared libraries
    215 0.6% Unaccounted

    View Slide

  43. [C++]:
    ticks total nonlib name
    19557 51.8% 52.9% node::crypto::PBKDF2(v8)
    4510 11.9% 12.2% _sha1_block_data_order
    3165 8.4% 8.6% _malloc_zone_malloc

    View Slide

  44. View Slide

  45. $ npm install -g 0x
    $ 0x app.js

    View Slide

  46. View Slide

  47. disk

    View Slide

  48. memory

    View Slide

  49. real-time monitoring

    View Slide

  50. load, profile, fix, repeat!

    View Slide

  51. thank you
    @_abhinavrastogi
    http://tiny.cc/scalingnodejs

    View Slide