Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Troubleshooting Node.js

Troubleshooting Node.js

what to do when things go wrong

C29ecd0460bf9361001f47bbb8455b9a?s=128

Igor Soarez

July 23, 2015
Tweet

Transcript

  1. TROUBLESHOOTING NODE.JS what to do when things go wrong Igor

    Soarez igor@yld.io @igorsoarez Hassy Veldstra h@yld.io @hveldstra WDCNZ, July 2015
  2. HELLO HELLO • YLD! • Node.js Consultancy & Training •

    Digital transformation for enterprise clients
  3. FIRES Someone has to put them out

  4. BE PREPARED

  5. LOG EVERYTHING Make it easy to grep & visualise

  6. TEST TEST TEST • Functional tests first • Good coverage

    for unit tests • Randomised testing & fuzz testing
  7. MONITORING & ALERTING • Resource monitoring • CPU • Memory

    • I/O • Application monitoring • Errors • Real-user monitoring • Latency • Availability
  8. AUTOMATED DEPLOYMENTS

  9. PAIR PROGRAMMING

  10. TAKE YOUR TIME WHEN BUILDING YOUR APPLICATION Tight deadlines are

    the root of all evil
  11. LOAD TESTING

  12. LOAD TESTING • You should do it

  13. LOAD TESTING • You should do it • Find the

    bottlenecks
  14. LOAD TESTING • You should do it • Find the

    bottlenecks • Understand your system
  15. LOAD TESTING • You should do it • Find the

    bottlenecks • Understand your system
  16. LOAD TESTING

  17. LOAD TESTING

  18. YOU SHOULD DO IT IN PRODUCTION

  19. IN PRODUCTION??? I must have heard you wrong

  20. WHY NOT? Why — Reason 1

  21. STAGING IS NEVER THE SAME Why — Reason 1I

  22. GAIN EXTRA CONFIDENCE Why — Reason 1II

  23. IF YOU DO THIS • Analytics

  24. IF YOU DO THIS • Analytics • Custom header •

    Cookie
  25. IF YOU DO THIS • Analytics • Custom header •

    Cookie • Let Marketing know
  26. IF YOU DO THIS • Analytics • Custom header •

    Cookie • Let Marketing know • Start outside of peak times
  27. IF YOU DO THIS • Analytics • Custom header •

    Cookie • Let Marketing know • Start outside of peak times • Start slow & ramp up
  28. LOAD TESTING TOOLS • JMeter

  29. LOAD TESTING TOOLS • JMeter • Tsung

  30. LOAD TESTING TOOLS • JMeter • Tsung • Minigun -

    https://artillery.io/minigun
  31. AFTER THE FIRE STARTS how to approach it

  32. PERFORMANCE PROBLEMS

  33. ATTACK THE PROBLEM 1. Isolated test case * 2. Measure

    3. Change / Tweak / Improve 4. Goto 2 * If possible
  34. TYPES OF ISSUES • CPU-bound code • Memory leaks •

    GC pressure • I/O performance • Event loop lag (Node.js)
  35. KNOW THE TOOLS AVAILABLE • OS-level • perf • sysstat,

    netstat, netmon, ps, top • sysdig
  36. KNOW THE TOOLS AVAILABLE • Platform-specific (Node.js)

  37. KNOW THE TOOLS AVAILABLE • Platform-specific (Node.js) • V8 profiler

    • find hot-path functions that cannot be optimized • trace GC activity
  38. KNOW THE TOOLS AVAILABLE • Platform-specific (Node.js) • V8 profiler

    • find hot-path functions that cannot be optimized • trace GC activity • heapdump • find memory leaks
  39. MEMORY • ps • top / htop • Heapdumps

  40. KNOW THE TOOLS AVAILABLE • Platform-specific (Node.js) • V8 profiler

    • find hot-path functions that cannot be optimized • trace GC activity • heapdump • find memory leaks • flamegraphs • see what's hot on the CPU
  41. CPU • ps • top / htop • Flamegraphs

  42. KNOW THE TOOLS AVAILABLE • Platform-specific (Node.js) • V8 profiler

    • find hot-path functions that cannot be optimized • trace GC activity • heapdump • find memory leaks • flamegraphs • see what's hot on the CPU • node-toobusy-js • track the event loop lag
  43. FLAMEGRAPHS

  44. PERF aka Performance Counters

  45. Collected stack frames h() c() h() c() c() z() c()

    b() f() b() b() f() b() y() b() b() a() a() a() m() a() m() x() m() a() main() main() main() main() main() main() main() main() main()
  46. Collapsed stack frames h() c() h() c() z() b() f()

    b() y() a() m() x() main()
  47. DEMO!

  48. THANK YOU! @hveldstra @igorsoarez