A Journey Into Node.js Internals

Slide 1

Slide 1 text

A Journey Into Node.js Internals Tamar Twena-Stern

Slide 2

Slide 2 text

Tamar Twena-Stern • Software Engineer - manager and architect • Architect @PaloAltoNetworks • Was a CTO of my own startup • Passionate about Node.js ! • Twitter: @SternTwena

Slide 3

Slide 3 text

Tamar Twena-Stern • On Maternity Leave • Have 3 kids • Loves to play my violin • Javascript Israel community leader

Slide 4

Slide 4 text

Introduction

Slide 5

Slide 5 text

Traditional Approach - Blocking IO client thread request request request thread thread Server

Slide 6

Slide 6 text

Problems •The system allocates CPU and memory resources for every new thread •When the system is stressed – overhead of thread scheduling and context switching •The system waste resources for allocating threads instead of doing actual work

Slide 7

Slide 7 text

Node.js Architecture

Slide 8

Slide 8 text

Node.js Architecture - High Level

Slide 9

Slide 9 text

Now, Lets Get Into The Details

Slide 10

Slide 10 text

Single Threaded ? • Not really single threaded • Several threaded : • Event Loop • The workers thread pool

Slide 11

Slide 11 text

Event Loop Thread • Every request registers a callback which executes immediately • The event loop execute JavaScript callbacks • Offloads I/O operations to worker thread pool. • Handle callbacks for asynchronous I/O operations from multiple requests.

Slide 12

Slide 12 text

• Thread pool to perform heavy operations • I/O • CPU intensive operations • Bounded by fixed capacity • Every application written in node can submit a task to worker thread low level API (libUV) Worker Thread Pool

Slide 13

Slide 13 text

Submitting A Request To The Worker Pool • Use a set of ‘basic’ modules that work with the event loop • Examples: • Fs • Dns • Crypto • And more • Submit a task to libUV using c++ add-on

Slide 14

Slide 14 text

Is Worker Thread Pool Implemented With A Queue ?

Slide 15

Slide 15 text

Event Loop Implemented With A Queue ?

Slide 16

Slide 16 text

The Event Loop Thread Deep Dive

Slide 17

Slide 17 text

Order In Chaos - Event Loop Phases

Slide 18

Slide 18 text

While (True)

Slide 19

Slide 19 text

Timers Phase • setTimeout, setInterval • Timer’s callback will run as soon as they can be scheduled after the threshold • Timer’s callback scheduling controlled by the “Poll” phase

Slide 20

Slide 20 text

I/O Callback Phase (Error) • Executes system error callbacks • Example : TCP socket connection error. • Normal I/O operation callbacks are executed in the poll phase.

Slide 21

Slide 21 text

Poll Phase - Most Importent • Scheduling • Callback Execution

Slide 22

Slide 22 text

Check Phase And Close Phase • Check Phase - Execute callbacks for setImmediate timers • Close Phase – Handles an abruptly close of a socket or a handle

Slide 23

Slide 23 text

Lets proﬁle some code

Slide 24

Slide 24 text

V8 Engine

Slide 25

Slide 25 text

Chrome • Open source JavaScript engine • Developed originally for Google Chrome and chromium • Also used for • Couchbase • MongoDB • Node.js

Slide 26

Slide 26 text

V8 Architecture

Slide 27

Slide 27 text

What Is Just-In-Time Compilation ? • Compilation during run time • Combines two approaches : • Compilation ahead of runtime • Interpreter

Slide 28

Slide 28 text

JIT Compiler In Java

Slide 29

Slide 29 text

The JIT Compilation In V8 Source Code Tracing Interpreter Function Repeats - Hot Code JIT Compiler Optimised Code

Slide 30

Slide 30 text

Ignition Interpreter

Slide 31

Slide 31 text

Ignition Interpreter • Interpreter for V8 • Translates to low level Bytecode • Enabling ‘cold’ code to be stored more compactly in Bytecode • Run once code • Non hot code

Slide 32

Slide 32 text

Turbofan Compiler - JIT

Slide 33

Slide 33 text

How To Help Turbofan Optimise Hot Code ? •The fewer function input type variations lead to smaller and faster resulting code. •Keeping your functions monomorphic or at least polymorphic •Monomorphic: one input type •Polymorphic: two to four input types •Megamorphic: five or more input types

Slide 34

Slide 34 text

Optimisation And De- optimisation Demo

Slide 35

Slide 35 text

Optimisation And De-optimisation • Optimisation - All assumptions fulfilled – Compiled code runs. • Deoptimisation – Not all assumptions fulfilled – Compiled code erased Assumptions Fulﬁlled Optimisation Assumptions Break De- Optimisation

Slide 36

Slide 36 text

Avoid De-optimisation • When a code is optimised and de-optimised – it ended up being slower then just use the baseline compiled version • Most browsers and engines will stop trying after several iterations of optimising and de-optimizing

Slide 37

Slide 37 text

V8 Memory Management

Slide 38

Slide 38 text

Why Do I Need It ?

Slide 39

Slide 39 text

V8 Memory Structure

Slide 40

Slide 40 text

The Stack And The Heap • Stack - data structure to store current function variables • From the stack starts the garbage collection • Heap - Data structure to store complex objects

Slide 41

Slide 41 text

Whats Allocated Where ?

Slide 42

Slide 42 text

The Heap Structure

Slide 43

Slide 43 text

Object Life On The Heap Object Allocated On New Space ‘Scavenge’ Scan To Detect Live Objects On New Space Object survives 2 ‘Scavenge’ scans and still alive Moved To ‘Old Space’ More Space Needed On Old Space Mark Sweep Scan Starts to Free Space On Heap