Valhalla va-t-il corriger l'erreur à 1 milliards de dollars ?

Slide 1

Slide 1 text

Will Valhalla Fix the 1 Billion $ Mistake? Exploring New Ways to Manage Your In Memory Data José Paumard Java Developer Advocate Java Platform Group Rémi Forax Maître de conferences Université Gustave Eiffel

Slide 2

Slide 2 text

https://twitter.com/JosePaumard https://github.com/JosePaumard https://www.youtube.com/c/JosePaumard01 https://www.youtube.com/user/java https://www.youtube.com/hashtag/jepcafe https://fr.slideshare.net/jpaumard https://www.pluralsight.com/authors/josepaumard https://dev.java

Slide 3

Slide 3 text

https://twitter.com/Nope! https://github.com/forax https://speakerdeck.com/forax OpenJDK, ASM, Tatoo, Pro, etc… One of the Father of invokedynamic (Java 7) Lambda (Java 8), Module (Java 9) Constant dynamic (Java 11) Record, Pattern Matching (Java 17 / 21) Valhalla (Java ??+)

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 9 Bring the benefits of primitives to objects and classes Enable flatter and denser memory layouts Enable adding new primitive-like classes Healing the reference-primitive divide Compatible migration of existing value-based classes Enable runtime specialization of generics Valhalla

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 13 Population as a primitive, City is modifiable Cost of Abstraction int[] populations; class City { final String name; int population; } int populations int int int int[]

Slide 14

Slide 14 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 14 Population and City non-modifiable Cost of Abstraction record Population(int population) {} record City(String name, Population population) {} population int Population Population[]

Slide 15

Slide 15 text

Slide 16

Slide 16 text

total 1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 16 Summing populations Optimization: Stack Scalarization Population sumPopulations( Population[] populations) { Population total = new Population(0); for (Population p: populations) { total = total.add(p); } return total; } Population 0 record Population(int population) {}

Slide 17

Slide 17 text

total 1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 17 Summing populations Optimization: Stack Scalarization Population 1_243 Population sumPopulations( Population[] populations) { Population total = new Population(0); for (Population p: populations) { total = total.add(p); } return total; } record Population(int population) {}

Slide 18

Slide 18 text

total 1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 18 Summing populations Optimization: Stack Scalarization Population 3_851 Population sumPopulations( Population[] populations) { Population total = new Population(0); for (Population p: populations) { total = total.add(p); } return total; } record Population(int population) {}

Slide 19

Slide 19 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 19 Summing populations Optimization: Stack Scalarization total 21_529 Population sumPopulations( Population[] populations) { Population total = new Population(0); for (Population p: populations) { total = total.add(p); } return total; } record Population(int population) {}

Slide 20

Slide 20 text

Slide 21

Slide 21 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 21 Summing populations Optimization: Heap Flattening record Population(int population) {} record City(String name, Population population) {} City City1 population City City2 population City City3 population Population 20_000 Population 10_000 Population total = new Population(0); for (City city: cities) { total = total.add(city.population()); }

Slide 22

Slide 22 text

Slide 23

Slide 23 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 23 Object header + padding Optimization: Heap Flattening City population Header city Population 10_000 Header header =32 + 64 bits (heap < 32GB) 64 + 64 bits String Header N i c e Header length

Slide 24

Slide 24 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 24 Summing populations Optimization: Heap Flattening record Population(int population) {} record City(String name, Population population) {} City City1 population 10_000 City City2 population 20_000 City City3 population 10_000 Population total = new Population(0); for (City city: cities) { total = total.add(city.population()); }

Slide 25

Slide 25 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 25 - No allocation overhead - No extra pointer dereferencing - Denser storage (no object header) - Localized storage (no pointer chasing) Benefits

Slide 26

Slide 26 text

Abstraction for free, indirection for free You should not have to choose between readable code and performance 1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 26 Valhalla: Design Principle Codes like a class, Works like an int Valhalla

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

The Primitive World…

Slide 30

Slide 30 text

1/13/2025 The Object World!

Slide 31

Slide 31 text

Slide 32

Slide 32 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 32 Classes have: - fields, methods, interfaces - identity (thus mutability) Class instances are nullable Classes have encapsulation From Class vs. Primitive to Value Class Primitives have - default value - no integrity (long, double) Primitives are scalarized on stack and flattened on heap

Slide 33

Slide 33 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 33 Classes have: - fields, methods, interfaces - identity (thus mutability) Class instances are nullable Classes have encapsulation Value Class : Giving up Semantic Constraints Primitives have - default value - no integrity (long, double) Primitives are scalarized on stack and flattened on heap

Slide 34

Slide 34 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 34 Value classes: - have fields, methods & interfaces - have no identity (thus no mutability) - instances may be nullable - have encapsulation or default value - may have no integrity - are scalarized on stack / maybe flattened on heap Value Class

Slide 35

Slide 35 text

Step by step approach: 1) Value keyword 2) Nullness emotion 3) Value class like primitive 4) Primitive as value class 5) Operator overloading 6) Specialized generics Rocket Model

Slide 36

Slide 36 text

36 JEP 401 : Value Classes and Object https://openjdk.org/jeps/401 Stage 1: Value Keyword

Slide 37

Slide 37 text

Slide 38

Slide 38 text

No Identity, but Value

Slide 39

Slide 39 text

Demo

Slide 40

Slide 40 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 40 Using the value keyword (on a class or on a record) Value classes are regular classes Declaring a Value Class value record Population(int population) {} value class Population { private final int population; ... }

Slide 41

Slide 41 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 41 Value classes are final, no subtype A value class can implement interfaces A value class can inherit a « value enabled » class j.l.Object, j.l.Number, j.l.Record , … Declaring a Value Class value record Population(int population) {}

Slide 42

Slide 42 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 42 The fields of a value class must be initialized before the call to super Strict Field Initialization value class Population implements Comparable { private final int population; Population(int population) { this.population = population; super(); } ... }

Slide 43

Slide 43 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 43 Value class instances are regular instances They are nullable No identity at runtime The VM may decide not to use a pointer at runtime Transparent to the user Value Class Execution Model value record Population(int population) {}

Slide 44

Slide 44 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 44 No identity at runtime: - what about value1 == object ? - component wise equality (even for double) - System.identityHashCode(value) - computed from components - synchronized(value) throws IdentityException - new WeakRef(value) throws IdentityException - value.getClass().isValue() // true Runtime Semantics

Slide 45

Slide 45 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 45 Wrapper types are value types: Byte, Integer, Long, Double, etc Also more value types: java.util.Optional java.time: Month, Year, LocalDateTime, etc Retrofit Value Based Classes

Slide 46

Slide 46 text

Under the Hood

Slide 47

Slide 47 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 47 Benchmark Scalarization int loopInt() { var total = 0; for (var i = 0; i < 1_000; i++) { total = total + i; } return total; } Population loopPopulation() { var total = new Population(0); for (var i = 0; i < 1_000; i++) { total = total.add(new Population(i)); } return total; } Value Class Score Error Units loopInt 283.386 ± 0.620 ns/op loopPopulation 283.414 ± 0.302 ns/op Identity Class Score Error Units loopInt 283.340 ± 0.434 ns/op loopPopulation 1229.171 ± 4.104 ns/op

Slide 48

Slide 48 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 49 Scalarization but few flattening On stack: - optimized calling convention (+ fallback) - scalar replacement (in registers) optimization only by C2 On heap: - use pointers unless size < 64 bits From Hotspot Point of View

Slide 49

Slide 49 text

New value keyword on class and record No identity at runtime Scalarization on stack Flattening for very small object on heap Wrapping Up Stage 1

Slide 50

Slide 50 text

Coffee Break!

Slide 51

Slide 51 text

52 JEP draft : Null-Restricted and Nullable type https://openjdk.org/jeps/8303099 Stage 2: Nullness Emotion

Slide 52

Slide 52 text

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Slide 56

Slide 56 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 57 Flattening needs to know that a reference cannot be null Flattening on Heap? cities String int String int String int String int array of not-null City / not-null Population

Slide 57

Slide 57 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 58 Declaring Nullness Emotion? value class City { private final Population population; // initialized with null City(Population population) { this.population = population; // can store null super(); } // arguments can be null public Population[] toArray(Population p1) { var array = new Population[1]; // initialized with null array[0] = p1; // can store null return array; } }

Slide 58

Slide 58 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 59 Declaring Nullness Emotion? value class City { private final Population! population; // not initialized with null City(Population! population) { this.population = population; // cannot store null super(); } // arguments cannot be null public Population![] toArray(Population! p1 /* cannot be null */) { var array = new Population![] {p1}; // not initialized with null array[0] = p1; // cannot store null return array; } }

Slide 59

Slide 59 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 60 Produce a warning when dereferencing null is possible 3 states - not-null - maybe-null - unspecified (maybe-null, produces no warning if null) Null State Analysis

Slide 60

Slide 60 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 61 Null state analysis - enhance existing Definite Assignment analysis - not-null if - the expression is known to be not null - after a null check (explicit or implicit) Otherwise maybe-null (or unspecified) Null State Analysis

Slide 61

Slide 61 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 62 Kotlin, C# (via opt-in) semantics are non-null by default The goal is to avoid NPE at runtime In Java, we propose type references to be nullable by default Nullness By Default?

Slide 62

Slide 62 text

Are your craaaazy?

Slide 63

Slide 63 text

Demo Java + Jspecify vs. Kotlin

Slide 64

Slide 64 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 65 Fields are first initialized to null at runtime Field Initialization Problems (1) class Village { final Population! population; Village(Population! population) { super(); println(this); // Village(null)  this.population = population; } public String toString() { return "Village(" + population + ")"; } }

Slide 65

Slide 65 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 66 Super constructors can call instance methods Field Initialization Problems (2) class City { City() { init(); // leaks this  } void init() { // nothing } } class Capital extends City { final Population! population; Capital(Population! population) { this.population = population; } void init() { println(this.population); // null  } }

Slide 66

Slide 66 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 67 The null state analysis is local to a method - requires checks at boundaries - problem with overriding Should not be strongly enforced by the VM - migration issues ⇒ the compiler inserts Objects.requireNonNull() Parameter and Return Value

Slide 67

Slide 67 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 68 We need the null state on field/array element to be enforced by the VM at runtime So value types can be flattened This runtime semantics is different from existing ones - Jspecify erases null state annotations everywhere - Kotlin erases null state info everywhere but on method parameters Not Your Typical Implementation

Slide 68

Slide 68 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 69 Not-null has a stronger semantics than Kotlin/C# Not-null should be opt-in - It is enforced by the VM or the compiler - Mark where a FieldStoreException, ArrayStoreException, NullPointerException can occur at runtime - Better for migration ! Not-null Field / Array / Parameter

Slide 69

Slide 69 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 70 Java reference type are nullable by default Two ideas to make it sufferable - two states : the user can not denote “unspecified” if you opt-in to nullable analysis, everything not not-null is nullable - inference of local variable nullness only methods and fields needs nullness emotion Taming the Dragon

Slide 70

Slide 70 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 71 Not-null fields must be initialized It must happen before super() Not-null Fields Initialization class City { City() { init(); // leaks this  } void init() { // nothing } } value class Capital extends City { Population! population; Capital(Population! population) { this.population = population; super(); } void init() { println(this.population); } }

Slide 71

Slide 71 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 72 Array of references are initialized with nulls We need a new syntax for arrays of not-null references Proposed syntax: Strict Array Initialization var populations = new Population![4](new Population(0)); var populations = new Population![4](index -> new Population(index));

Slide 72

Slide 72 text

Under the Hood

Slide 73

Slide 73 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 74 Benchmark Flattening via Nullness var ints = new int[] { /* 36k ints */ }; record Population(int value) {} var populations = new Population[] {/* 36k populations */ } var populationBangs = new Population![] {/* 36k populations */ }

Slide 74

Slide 74 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 75 Benchmark Score Error Units totalInt 10.454 ± 0.007 us/op totalPopulation 19.271 ± 0.216 us/op Benchmark Flattening via Nullness int totalInt(int[] ints) { var total = 0; for (var population: ints) { total = total + population; } return total; } Population totalPopulation(Population[] populations) { var total = new Population(0); for (var population: populations) { total = total.add(population); } return total; }

Slide 75

Slide 75 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 76 Benchmark Score Error Units totalInt 10.454 ± 0.007 us/op totalPopulation 19.271 ± 0.216 us/op totalPopulationBang 10.456 ± 0.013 us/op Benchmark Flattening via Nullness int totalInt(int[] ints) { var total = 0; for (var population: ints) { total = total + population; } return total; } Population totalPopulation(Population[] populationBangs) { var total = new Population(0); for (var population: populationBangs) { total = total.add(population); } return total; }

Slide 76

Slide 76 text

Slide 77

Slide 77 text

Nullness analysis as an enhanced of the DA Reference type are nullable by default Nullness emotion are not erased* The ! is enforced at runtime Maybe : Two states user model + inference for local variables Wrapping Up Stage 2 * still erased for generics (type variable or parametrized type)

Slide 78

Slide 78 text

80 JEP Draft : Null-Restricted Value Class type https://openjdk.org/jeps/8316779 Stage 3: Value Classes As Primitives

Slide 79

Slide 79 text

Slide 80

Slide 80 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 82 Some value classes act more like primitive types - they may have a default value Float16, Population - they may allow tearing of components Complex Value Classes As Primitives

Slide 81

Slide 81 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 83 Having a default value (all fields are zeroes) bypass encapsulation Users should opt-in to an implicit constructor Value With Implicit Constructor value class Population { private final int value; implicit Population(); // no support for user-defined default value Population(int value) { this.value = value; } }

Slide 82

Slide 82 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 84 Not-null field / array can be left uninitialized Value Class With a Default Value class City { private Population! population; // equivalent to Population(0) City() { // can be empty } Population![] toArray(Population! population) { var array = new Population![4]; // initialized with Population(0) array[0] = population; return array; } }

Slide 83

Slide 83 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 85 User can choose to relax integrity of value classes Some value classes allow tearing Complex(re, im) // OK Range(from, to) // not OK Tearing and Integrity

Slide 84

Slide 84 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 86 To give up integrity: annotate with @LooselyConsistentValue Value Classes With no Integrity @LooselyConsistentValue value record Complex(double re, double im) { }

Slide 85

Slide 85 text

Under the Hood

Slide 86

Slide 86 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 88 Benchmark Flattening via Nullness value record Population(int value) { implicit Population(); ... } value class City { final String name; final Population! population; implicit City(); ... } City[] cities = ...; City![] citiesBang = ...;

Slide 87

Slide 87 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 89 Benchmark Flattening via Nullness City[] cities null name population 12_000 City City![] citiesBang name population 12_000 City name population 8_000 City name population 1_000 City

Slide 88

Slide 88 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 90 Benchmark Score Error Units totalInt 42.294 ± 0.255 us/op totalCity 41.791 ± 0.587 us/op Benchmark Flattening via Nullness int totalInt() { var total = 0; for (var city : cities) { total += city.population().value(); } return total; } Population totalCity() { var total = new Population(0); for (var city : cities) { total = total.add(city.population()); } return total; }

Slide 89

Slide 89 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 91 Benchmark Score Error Units totalInt 42.294 ± 0.255 us/op totalCity 41.791 ± 0.587 us/op totalCityBang 20.822 ± 0.032 us/op Benchmark Flattening via Nullness int totalInt() { var total = 0; for (var city : cities) { total += city.population().value(); } return total; } Population totalCityBang() { var total = new Population(0); for (var city : citiesBang) { total = total.add(city.population()); } return total; }

Slide 90

Slide 90 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 92 Benchmark Comparison 18 25 8 84 int[] Population![] 10.454 ± 0.007 us/op 10.456 ± 0.013 us/op City!{Population!}[] 20.822 ± 0.032 us/op 18 25 8 84 18 25 8 84

Slide 91

Slide 91 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 94 Flattening requested if ! If Integrity yes if size ≤ 64 (still waiting for Intel) no if size ≤ 512 ? (or 1024 ?) From Hotspot Point of View

Slide 92

Slide 92 text

95 JEP 402 : Enhance primitive boxing https://openjdk.org/jeps/402 Stage 4: Primitives as Value Classes

Slide 93

Slide 93 text

Slide 94

Slide 94 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 97 Primitive types should behave like value classes One model to rule them all! We can not fully have int == Integer! Because of the bytecode format Primitives as Value Classes

Slide 95

Slide 95 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 98 Method calls on primitive int i = 3; i.toString() Primitive as type argument new ArrayList() Array reinterpretation at runtime int[] <=> Integer![] Overriding with primitive int m() can override Integer m() Fix Inconsistencies

Slide 96

Slide 96 text

Stage 5: Operator Overloading? Brian Goetz has some ideas: type class from Haskell? Stage 6: Generic Specialization? John Rose has some ideas: parametric constant pool? More Stages!

Slide 97

Slide 97 text

1/13/2025 Copyright © 2021, Oracle and/or its affiliates | 103 Executive Summary Class No Identity Field / array Element not-null Strict initialization No encapsulation: implicit constructor Flattening if size ≤ 64 bits No integrity Flattening if size ≤ 512 bits Scalarization on stack