Scaling NodeJS beyond the ordinary

These are the slides to accompany the talk I gave at JSConf Iceland 2018.

Transcript

1. Scaling NodeJS
   Beyond the ordinary
   Abhinav Rastogi
   Lead UI Engineer
   _abhinavrastogi
   

   View Slide

2. View Slide

3. 150M Registered
   Users
   500K Phones sold in
   15 seconds
   200M Visits over 5 days
   during annual sale event
   

   View Slide

4. – Andre Bondi
   “Scalability is the capability of a system to handle a
   growing amount of work, or its potential to be enlarged to
   accommodate that growth.”
   

   View Slide

5. Why should it concern the JS
   community?
   

   View Slide

6. Why should it concern the JS
   community?
   • JS is running on servers now
   • Failure is shifting to application code
   • Failure is better handled by devs
   

   View Slide

7. Types of Scalability
   x
   y
   z
   Add more machines
   Add more resources
   Application layer
   

   View Slide

8. Horizontal Scaling
   

   View Slide

9. Horizontal Scaling
   Machine
   Node
   API
   

   View Slide

10. Horizontal Scaling
    Machine
    Node
    API
    Machine
    Node
    ELB
    

    View Slide

11. Vertical Scaling
    

    View Slide

12. Vertical Scaling
    Machine
    Node
    API
    

    View Slide

13. Vertical Scaling
    Machine
    Node
    API
    Node
    ??
    

    View Slide

14. Application Layer
    Optimisations
    

    View Slide

15. The Optimisation Cycle
    Load Test
    Find Bottleneck
    Fix Issues
    

    View Slide

16. network, cpu, memory, disk
    

    View Slide

17. network bandwidth
    

    View Slide

18. network bandwidth
    

    View Slide

19. 1000 kb per page
    100 rps per machine
    100 machines
    = 10gbps
    

    View Slide

20. compression
    

    View Slide

21. const app = express()
    app.use(compression)
    

    View Slide

22. co-hosted nginx
    

    View Slide

23. co-hosted nginx
    Machine
    Node
    Node
    PM2
    nginx
    API
    ELB
    

    View Slide

24. network profiling
    

    View Slide

25. netstat / ss
    lsof
    watch
    

    View Slide

26. everything is a file in unix!
    

    View Slide

27. ulimit
    

    View Slide

28. ulimit
    core file size (blocks, -c) 0
    file size (blocks, -f) unlimited
    pending signals (-i) 32767
    max locked memory (kbytes, -l) 32
    max memory size (kbytes, -m) unlimited
    open files (-n) 1024
    max user processes (-u) 50
    

    View Slide

29. increase the limits
    /
    reduce the usage
    

    View Slide

30. keep-alive header
    

    View Slide

31. keep-alive header
    Object.assign(headers, {
    'Connection': 'keep-alive',
    'Keep-Alive': 'timeout=200'
    });
    

    View Slide

32. connection pooling
    

    View Slide

33. connection pooling
    const http = require('http');
    fetch(url, {
    agent: new http.Agent({
    keepAlive: true,
    maxSockets: 24
    })
    });
    

    View Slide

34. ephemeral ports
    

    View Slide

35. tcp connection states
    

    View Slide

36. tcp connection states
    source: http://cnp3book.info.ucl.ac.be/1st/html/transport/transport.html
    

    View Slide

37. tcp connection states
    

    View Slide

38. cpu
    

    View Slide

39. cpu profiling
    

    View Slide

40. node --prof app.js
    

    View Slide

41. import crypto from 'crypto';
    app.get('/auth', (req, res) => {
    const hash = crypto.pbkdf2Sync(password, users[username].salt, 100, 512);
    if (users[username].hash.toString() === hash.toString()) {
    res.sendStatus(200);
    } else {
    res.sendStatus(401);
    }
    });
    

    View Slide

42. [Summary]:
    ticks total nonlib name
    79 0.2% 0.2% JavaScript
    36703 97.2% 99.2% C++
    7 0.0% 0.0% GC
    767 2.0% Shared libraries
    215 0.6% Unaccounted
    

    View Slide

43. [C++]:
    ticks total nonlib name
    19557 51.8% 52.9% node::crypto::PBKDF2(v8)
    4510 11.9% 12.2% _sha1_block_data_order
    3165 8.4% 8.6% _malloc_zone_malloc
    

    View Slide

44. View Slide

45. $ npm install -g 0x
    $ 0x app.js
    

    View Slide

46. View Slide

47. disk
    

    View Slide

48. memory
    

    View Slide

49. real-time monitoring
    

    View Slide

50. load, profile, fix, repeat!
    

    View Slide

51. thank you
    @_abhinavrastogi
    http://tiny.cc/scalingnodejs
    

    View Slide