Architectural Changes In V8 And How They Will Improve Server Performance

Slide 1

Slide 1 text

Architectural Changes In V8 And How They Improve Your Node.js Performance Tamar Twena-Stern

Slide 2

Slide 2 text

Tamar Twena-Stern twitter @SternTwena linkedin /tamarstern [email protected]

Slide 3

Slide 3 text

What Is V8 ?

Slide 4

Slide 4 text

Few V8 Facts

Slide 5

Slide 5 text

A Crucial Component In Node.js Architecture

Slide 6

Slide 6 text

V8 In Node.js Has Two Roles Code Parsing And Execution Memory Management

Slide 7

Slide 7 text

Since JavaScript And Node.js Are Very Popular , V8 Have A Hard Task

Slide 8

Slide 8 text

To Be A Super Fast Interpreter For A Dynamic Type Language

Slide 9

Slide 9 text

Let’s talk about compilers, interpreters, dynamic and static type languages

Slide 10

Slide 10 text

Compiler - Translates High level code to machine code in compilation process Interpreter - directly executes source code written in a high-level programming, translate to machine code line by line Dynamic Typed Languages - checks variable types at runtime Static Typed Languages - checks variable types at compile time

Slide 11

Slide 11 text

Compilers Interpreters Execution Overhead Efficient Memory Usage Platform Specific Optimizations Why Compilers Perform Better than interpreters

Slide 12

Slide 12 text

Dynamic Type Languages Have To Do Those Techniques At Runtime ● Runtime Type Checking ● Dynamic Dispatch

Slide 13

Slide 13 text

Compiler, static typed language == Interpreter, dynamic type language == == Interpreter, dynamic type language So In Most Cases

Slide 14

Slide 14 text

To Make V8 Super Fast, 2 Compilers And One Interpreters To The Rescue

Slide 15

Slide 15 text

● Ignition Interpreter ● Turbofan Compiler ● Sparkplug Compiler

Slide 16

Slide 16 text

Code Parsing Pipeline - Evolution

Slide 17

Slide 17 text

What’s Happening When Your Application Code Is Running ?

Slide 18

Slide 18 text

Parser And AST

Slide 19

Slide 19 text

function add(a,b) { return a+b; } Simple AST Example

Slide 20

Slide 20 text

Ignition Interpreter

Slide 21

Slide 21 text

Turbofan Compiler - JIT Compiler

Slide 22

Slide 22 text

Help turbofan work fast - Use fewer function input type variations

Slide 23

Slide 23 text

monomorphic(myNumber : number) : void;

Slide 24

Slide 24 text

polymorphic(myNumber : number) : void; polymorphic(myString: String) : void;

Slide 25

Slide 25 text

megamorohic(myNumber : number) : void; megamorohic(myString: String) : void; megamorohic(myPerson : Person) : void; megamorohic(myFirstAray: Array) : void; megamorohic(mySecondArray: Array) : void;

Slide 26

Slide 26 text

Optimization And De-Optimization

Slide 27

Slide 27 text

De-Optimization - Example class Person { constructor(name, age) { this.name = name; this.age = age; } } function getAge(person) { return person.age; }

Slide 28

Slide 28 text

De-Optimization - Example for (let i = 0 ; i < 1000000 ; i++ ) { // After X iterations the Turbofan binary will be created const p = new Person("Tamar", 30); getAge(p); }

Slide 29

Slide 29 text

De-Optimization - Example const differentPerson = new Person("Tamar", 30);; // Adding dynamic property, hidden class changed differentPerson.salary = 3000; //binary released getAge(differentPerson);

Slide 30

Slide 30 text

● Assumption for turbofan optimizations - checking the Shape as a prerequisite ● The “maps check” checks what hidden class the object is pointing to and makes sure it is the same as the compiler expects. ● If it matches, then the check succeeds; if not, then it fails and the code is deoptimized due to “wrong map.” De-Optimization - Wrong “Hidden Class”

Slide 31

Slide 31 text

Deoptimization Benchmark

Slide 32

Slide 32 text

Optimized Code function concat_str(arg) { return arg + arg; } // Start for (let i = 0 ; i< 100000; i++) { concat_str("a"); } // End

Slide 33

Slide 33 text

Code With DeOptimization function concat_str(arg) { return arg + arg; } // Start for (let i = 0 ; i< 100000; i++) { if (i == 10231) { concat_str(1); } else { concat_str("a"); } } // End

Slide 34

Slide 34 text

Test Results - 100,000 Loop

Slide 35

Slide 35 text

Lets See The Performance Effect On 10,000,000 Loop Iterations

Slide 36

Slide 36 text

Test Results - 10,000,000 Loop

Slide 37

Slide 37 text

Comparing Runs

Slide 38

Slide 38 text

node --trace-opt --trace-deopt myProgram.js

Slide 39

Slide 39 text

What I Got When Running With Those Flags ?

Slide 40

Slide 40 text

First Phase - De-optimizing After Transfering Different Type [marking 0x070b4aa60601 for optimization to TURBOFAN, ConcurrencyMode::kConcurrent, reason: hot and stable] [compiling method 0x070b4aa60601 (target TURBOFAN) using Turbofan OSR] [optimizing 0x070b4aa60601 (target TURBOFAN) - took 0.000, 0.625, 0.000 ms] [bailout (kind: deopt-eager, reason: Insufficient type feedback for call): begin. deoptimizing 0x070b4aa60601 , opt id 0, bytecode offset 30, deopt exit 4, FP to SP delta 96, caller SP 0x00016bc4a310, pc 0x000128853144]

Slide 41

Slide 41 text

Phase 2 - Optimizing Again [compiling method 0x070b4aa61039 (target TURBOFAN) using Turbofan] [marking 0x070b4aa60601 for optimization to TURBOFAN, ConcurrencyMode::kConcurrent, reason: hot and stable] [optimizing 0x151588782981 (target TURBOFAN) - took 0.000, 0.458, 0.000 ms] [completed optimizing 0x151588782981 (target TURBOFAN)] [compiling method 0x1515887829c1 (target TURBOFAN) using Turbofan OSR] [optimizing 0x1515887829c1 (target TURBOFAN) - took 0.000, 0.584, 0.000 ms] [bailout (kind: deopt-eager, reason: Insufficient type feedback for generic named access): begin. deoptimizing 0x1515887829c1 , opt id 2, bytecode offset 74, deopt exit 7, FP to SP delta 112, caller SP 0x00016bc4a310, pc 0x000128853888]

Slide 42

Slide 42 text

Let’s Go Back To V8 Code Parsing Pipeline

Slide 43

Slide 43 text

What Needs Improvement ?

Slide 44

Slide 44 text

● Application start ● Command line application ● Dynamic environment with de-optimizations Week Scenarios

Slide 45

Slide 45 text

Turbofan Helps When Your Application Is Already “Warm”, But What if it is Not The Case Yet ?

Slide 46

Slide 46 text

So This Pipeline Turned Into …

Slide 47

Slide 47 text

V8 New Architecture - Mid Tier Compiler

Slide 48

Slide 48 text

const a = 1; const b = doStuff1(a); const c = doStuff2(a); for (let i=0; i<100000; i++) { doStuff1(a);//After X runs,run Machine Code (TurboFan) } Before Adding SparkPlug

Slide 49

Slide 49 text

const a = 1; const b = doStuff1(a); // run Machine Code (SparkPlug) const c = doStuff2(a); // run Machine Code (SparkPlug) for (let i=0; i<100000; i++) { doStuff1(a); // After X runs, run Machine Code (TurboFan) } After Adding SparkPlug

Slide 50

Slide 50 text

Ignition Does The Hard Work ● variable resolution ● parentheses or arrow functions ● desugaring destructuring statements

Slide 51

Slide 51 text

// The Sparkplug compiler for (; !iterator.done(); iterator.Advance()) { VisitSingleBytecode(); } Transfer Over The Code In A For Loop, No IR

Slide 52

Slide 52 text

SparkPlug Real World Performance Improvement

Slide 53

Slide 53 text

Improve Class Initializations

Slide 54

Slide 54 text

So , How JavaScript Objects Are Built Behind The Scenes ?

Slide 55

Slide 55 text

JavaScript Objects

Slide 56

Slide 56 text

JavaScript Arrays

Slide 57

Slide 57 text

Shapes To Represent Object Types

Slide 58

Slide 58 text

The Offset Lookup Was Very Expensive

Slide 59

Slide 59 text

The Solution - The Inline Cache

Slide 60

Slide 60 text

Monomorphic, Polymorphic, Megamorphic - And the effect of the inline cache

Slide 61

Slide 61 text

getX({ x: 3 }) // Cache size = 1 getX({ x: 3, a: 1 }) // polymorphic getX({ x: 3, b: 1 }) // polymorphic getX({ x: 3, c: 1 }) // polymorphic, several cache entries getX({ x: 3, d: 1 }) // megamorphic - Entries are overwritten on collisions

Slide 62

Slide 62 text

Until recent V8 versions - Inline Cache was not implemented for class ﬁelds and private methods

Slide 63

Slide 63 text

const person = {}; person.age = 42; class Person { age; constructor(age) { this.age = age; } getAge() { return this.age; } }

Slide 64

Slide 64 text

Benchmark - Class Initialization In Previous And Latest Node.js Versions

Slide 65

Slide 65 text

Benchmark Setup class Person { age; constructor(age) { this.age = age; } function getAge(person) { return person.age; } }

Slide 66

Slide 66 text

Benchmark Phase 1 arr = []; // Start for (let i = 0 ; i < 1000000 ; i++ ) { const p = new Person(30); arr[i] = p; } // End

Slide 67

Slide 67 text

Benchmark Phase 2 // Start for (let i = 0 ; i < 1000000 ; i++ ) { arr[i].age; } // End

Slide 68

Slide 68 text

Test Results

Slide 69

Slide 69 text

Class Field Initialization Improvement

Slide 70

Slide 70 text

What’s In It For Me ? class Person { age; constructor(age) { this.age = age; } getAge() { return this.age; } }