A Journey Into Node.js Internals

An overview on Node.js Architecture, event loop mechanism, and v8 interpreter and memory management

Transcript

1. A Journey Into Node.js Internals Tamar Twena-Stern
2. Tamar Twena-Stern • Software Engineer - manager and architect • Architect @PaloAltoNetworks • Was a CTO of my own startup • Passionate about Node.js ! • Twitter: @SternTwena
3. Tamar Twena-Stern • On Maternity Leave • Have 3 kids • Loves to play my violin • Javascript Israel community leader
4. Introduction
5. Traditional Approach - Blocking IO client thread request request request thread thread Server
6. Problems •The system allocates CPU and memory resources for every new thread •When the system is stressed – overhead of thread scheduling and context switching •The system waste resources for allocating threads instead of doing actual work
7. Node.js Architecture
8. Node.js Architecture - High Level
9. Now, Lets Get Into The Details
10. Single Threaded ? • Not really single threaded • Several threaded : • Event Loop • The workers thread pool
11. Event Loop Thread • Every request registers a callback which executes immediately • The event loop execute JavaScript callbacks • Offloads I/O operations to worker thread pool. • Handle callbacks for asynchronous I/O operations from multiple requests.
12. • Thread pool to perform heavy operations • I/O • CPU intensive operations • Bounded by fixed capacity • Every application written in node can submit a task to worker thread low level API (libUV) Worker Thread Pool
13. Submitting A Request To The Worker Pool • Use a set of ‘basic’ modules that work with the event loop • Examples: • Fs • Dns • Crypto • And more • Submit a task to libUV using c++ add-on
14. Is Worker Thread Pool Implemented With A Queue ?
15. Event Loop Implemented With A Queue ?
16. The Event Loop Thread Deep Dive
17. Order In Chaos - Event Loop Phases
18. While (True)
19. Timers Phase • setTimeout, setInterval • Timer’s callback will run as soon as they can be scheduled after the threshold • Timer’s callback scheduling controlled by the “Poll” phase
20. I/O Callback Phase (Error) • Executes system error callbacks • Example : TCP socket connection error. • Normal I/O operation callbacks are executed in the poll phase.
21. Poll Phase - Most Importent • Scheduling • Callback Execution
22. Check Phase And Close Phase • Check Phase - Execute callbacks for setImmediate timers • Close Phase – Handles an abruptly close of a socket or a handle
23. Lets profile some code
24. V8 Engine
25. Chrome • Open source JavaScript engine • Developed originally for Google Chrome and chromium • Also used for • Couchbase • MongoDB • Node.js
26. V8 Architecture
27. What Is Just-In-Time Compilation ? • Compilation during run time • Combines two approaches : • Compilation ahead of runtime • Interpreter
28. JIT Compiler In Java
29. The JIT Compilation In V8 Source Code Tracing Interpreter Function Repeats - Hot Code JIT Compiler Optimised Code
30. Ignition Interpreter
31. Ignition Interpreter • Interpreter for V8 • Translates to low level Bytecode • Enabling ‘cold’ code to be stored more compactly in Bytecode • Run once code • Non hot code
32. Turbofan Compiler - JIT
33. How To Help Turbofan Optimise Hot Code ? •The fewer function input type variations lead to smaller and faster resulting code. •Keeping your functions monomorphic or at least polymorphic •Monomorphic: one input type •Polymorphic: two to four input types •Megamorphic: five or more input types
34. Optimisation And De- optimisation Demo
35. Optimisation And De-optimisation • Optimisation - All assumptions fulfilled – Compiled code runs. • Deoptimisation – Not all assumptions fulfilled – Compiled code erased Assumptions Fulfilled Optimisation Assumptions Break De- Optimisation
36. Avoid De-optimisation • When a code is optimised and de-optimised – it ended up being slower then just use the baseline compiled version • Most browsers and engines will stop trying after several iterations of optimising and de-optimizing
37. V8 Memory Management
38. Why Do I Need It ?
39. V8 Memory Structure
40. The Stack And The Heap • Stack - data structure to store current function variables • From the stack starts the garbage collection • Heap - Data structure to store complex objects
41. Whats Allocated Where ?
42. The Heap Structure
43. Object Life On The Heap Object Allocated On New Space ‘Scavenge’ Scan To Detect Live Objects On New Space Object survives 2 ‘Scavenge’ scans and still alive Moved To ‘Old Space’ More Space Needed On Old Space Mark Sweep Scan Starts to Free Space On Heap
44. Heap New Space - Scavenge Scan
45. Heap Old Space - Mark Sweep Demonstration
46. Mark Sweep - DFS
47. Mark Sweep - DFS
48. Lets Find Some Memory Leaks !
49. • Twitter: @SternTwena