$30 off During Our Annual Pro Sale. View Details »

A Journey Into Node.js Internals

A Journey Into Node.js Internals

An overview on Node.js Architecture, event loop mechanism, and v8 interpreter and memory management

Tamar Twena-Stern

May 14, 2019

More Decks by Tamar Twena-Stern

Other Decks in Technology


  1. A Journey Into Node.js Internals Tamar Twena-Stern

  2. Tamar Twena-Stern • Software Engineer - manager and architect •

    Architect @PaloAltoNetworks • Was a CTO of my own startup • Passionate about Node.js ! • Twitter: @SternTwena
  3. Tamar Twena-Stern • On Maternity Leave • Have 3 kids

    • Loves to play my violin • Javascript Israel community leader
  4. Introduction

  5. Traditional Approach - Blocking IO client thread request request request

    thread thread Server
  6. Problems •The system allocates CPU and memory resources for every

    new thread •When the system is stressed – overhead of thread scheduling and context switching •The system waste resources for allocating threads instead of doing actual work
  7. Node.js Architecture

  8. Node.js Architecture - High Level

  9. Now, Lets Get Into The Details

  10. Single Threaded ? • Not really single threaded • Several

    threaded : • Event Loop • The workers thread pool
  11. Event Loop Thread • Every request registers a callback which

    executes immediately • The event loop execute JavaScript callbacks • Offloads I/O operations to worker thread pool. • Handle callbacks for asynchronous I/O operations from multiple requests.
  12. • Thread pool to perform heavy operations • I/O •

    CPU intensive operations • Bounded by fixed capacity • Every application written in node can submit a task to worker thread low level API (libUV) Worker Thread Pool
  13. Submitting A Request To The Worker Pool • Use a

    set of ‘basic’ modules that work with the event loop • Examples: • Fs • Dns • Crypto • And more • Submit a task to libUV using c++ add-on
  14. Is Worker Thread Pool Implemented With A Queue ?

  15. Event Loop Implemented With A Queue ?

  16. The Event Loop Thread Deep Dive

  17. Order In Chaos - Event Loop Phases

  18. While (True)

  19. Timers Phase • setTimeout, setInterval • Timer’s callback will run

    as soon as they can be scheduled after the threshold • Timer’s callback scheduling controlled by the “Poll” phase
  20. I/O Callback Phase (Error) • Executes system error callbacks •

    Example : TCP socket connection error. • Normal I/O operation callbacks are executed in the poll phase.
  21. Poll Phase - Most Importent • Scheduling • Callback Execution

  22. Check Phase And Close Phase • Check Phase - Execute

    callbacks for setImmediate timers • Close Phase – Handles an abruptly close of a socket or a handle
  23. Lets profile some code

  24. V8 Engine

  25. Chrome • Open source JavaScript engine • Developed originally for

    Google Chrome and chromium • Also used for • Couchbase • MongoDB • Node.js
  26. V8 Architecture

  27. What Is Just-In-Time Compilation ? • Compilation during run time

    • Combines two approaches : • Compilation ahead of runtime • Interpreter
  28. JIT Compiler In Java

  29. The JIT Compilation In V8 Source Code Tracing Interpreter Function

    Repeats - Hot Code JIT Compiler Optimised Code
  30. Ignition Interpreter

  31. Ignition Interpreter • Interpreter for V8 • Translates to low

    level Bytecode • Enabling ‘cold’ code to be stored more compactly in Bytecode • Run once code • Non hot code
  32. Turbofan Compiler - JIT

  33. How To Help Turbofan Optimise Hot Code ? •The fewer

    function input type variations lead to smaller and faster resulting code. •Keeping your functions monomorphic or at least polymorphic •Monomorphic: one input type •Polymorphic: two to four input types •Megamorphic: five or more input types
  34. Optimisation And De- optimisation Demo

  35. Optimisation And De-optimisation • Optimisation - All assumptions fulfilled –

    Compiled code runs. • Deoptimisation – Not all assumptions fulfilled – Compiled code erased Assumptions Fulfilled Optimisation Assumptions Break De- Optimisation
  36. Avoid De-optimisation • When a code is optimised and de-optimised

    – it ended up being slower then just use the baseline compiled version • Most browsers and engines will stop trying after several iterations of optimising and de-optimizing
  37. V8 Memory Management

  38. Why Do I Need It ?

  39. V8 Memory Structure

  40. The Stack And The Heap • Stack - data structure

    to store current function variables • From the stack starts the garbage collection • Heap - Data structure to store complex objects
  41. Whats Allocated Where ?

  42. The Heap Structure

  43. Object Life On The Heap Object Allocated On New Space

    ‘Scavenge’ Scan To Detect Live Objects On New Space Object survives 2 ‘Scavenge’ scans and still alive Moved To ‘Old Space’ More Space Needed On Old Space Mark Sweep Scan Starts to Free Space On Heap
  44. Heap New Space - Scavenge Scan

  45. Heap Old Space - Mark Sweep Demonstration

  46. Mark Sweep - DFS

  47. Mark Sweep - DFS

  48. Lets Find Some Memory Leaks !

  49. • Twitter: @SternTwena