Code Duplication in the Graal Compiler

Slide 1

Slide 1 text

CODE DUPLICATION IN THE GRAAL COMPILER Virtual Machine Meetup (VMM), Prague 2017 David Leopoldseder, JKU Linz, Austria Lukas Stadler, Oracle Labs, Linz - Austria Thomas Würthinger, Oracle Labs, Zurich - CH 1

Slide 2

Slide 2 text

2 DUPLICATION IN GRAAL Duplication

Slide 3

Slide 3 text

3 DUPLICATION IN GRAAL Inlining Duplication

Slide 4

Slide 4 text

4 DUPLICATION IN GRAAL Inlining Duplication Loop Optimizations Loop Unrolling Loop Unswitching Loop Peeling Loop Inversion

Slide 5

Slide 5 text

5 DUPLICATION IN GRAAL Duplication Simulation Duplication Inlining Loop Optimizations Loop Unrolling Loop Unswitching Loop Peeling Loop Inversion

Slide 6

Slide 6 text

6 DUPLICATION IN GRAAL Duplication Inlining Loop Optimizations Loop Unrolling Loop Unswitching Loop Peeling Loop Inversion Duplication Simulation

Slide 7

Slide 7 text

7 MOTIVATING EXAMPLE int f(int a, int b, int[] x) { int p; if (a > b) { p = a; } else { p = 2; } return x.length / p; }

Slide 8

Slide 8 text

int f(int a, int b, int[] x) { int p; if (a > b) { p = a; } else { p = 2; } return x.length / p; } 8 MOTIVATING EXAMPLE Fastpath Slowpath

Slide 9

Slide 9 text

int f(int a, int b, int[] x) { int p; if (a > b) { p = a; } else { p = 2; } return x.length / p; } 9 MOTIVATING EXAMPLE Fastpath Slowpath Expensive operation

Slide 10

Slide 10 text

int f(int a, int b, int[] x) { int p; if (a > b) { p = a; } else { p = 2; } return x.length / p; } 10 MOTIVATING EXAMPLE Fastpath Slowpath Expensive operation p=2 in the false branch

Slide 11

Slide 11 text

int f(int a, int b, int[] x) { int p; if (a > b) { p = a; } else { p = 2; } return x.length / p; } 11 MOTIVATING EXAMPLE Fastpath Slowpath Expensive operation p=2 in the false branch Strength Reduction if we know p == 2 and x.length >= 0 x.length / 2  x.length >> 1

Slide 12

Slide 12 text

int f(int a, int b, int[] x) { int p; if (a > b) { p = a; } else { p = 2; } return x.length / p; } 12 MOTIVATING EXAMPLE Fastpath Slowpath Expensive operation p=2 in the false branch Merge is an optimization boundary Strength Reduction if we know p == 2 and x.length >= 0 x.length / 2  x.length >> 1

Slide 13

Slide 13 text

int f(int a, int b, int[] x) { int p; if (a > b) { p = a; } else { p = 2; } return x.length / p; } 13 MOTIVATING EXAMPLE Fastpath Slowpath Expensive operation p=2 in the false branch Merge is an optimization boundary Only allowed if isPowerOf2(a) == true Strength Reduction if we know p == 2 and x.length >= 0 x.length / 2  x.length >> 1

Slide 14

Slide 14 text

int f(int a, int b, int[] x) { int p; if (a > b) { p = a; } else { p = 2; } return x.length / p; } 14 MOTIVATING EXAMPLE Fastpath Slowpath Expensive operation p=2 in the false branch Merge is an optimization boundary Solution  Duplication Strength Reduction if we know p == 2 and x.length >= 0 x.length / 2  x.length >> 1

Slide 15

Slide 15 text

15 MOTIVATING EXAMPLE – WHAT WE WANT IN THE END int f(int a, int b, int[] x) { if (a > b) { return x.length / a; } else { return x.length >> 1; } }

Slide 16

Slide 16 text

16 MOTIVATING EXAMPLE – WHAT WE WANT IN THE END int f(int a, int b, int[] x) { if (a > b) { return x.length / a; } else { return x.length >> 1; } } Slowpath Fastpath Approx. 15-90x faster on Nehalem

Slide 17

Slide 17 text

17 ENABLED OPTIMIZATIONS Read Eliminations Escape Analysis Opportunities Canonicalizations Conditional Eliminations Check Eliminations (Devirtualization) (Partial Redundancy Eliminations)

Slide 18

Slide 18 text

18 PROBLEMS TO SOLVE

Slide 19

Slide 19 text

19 PROBLEMS TO SOLVE P1 We need to determine which duplications will increase peak performance

Slide 20

Slide 20 text

20 PROBLEMS TO SOLVE P1 We need to determine which duplications will increase peak performance P2 We need to know which optimizations are enabled by a certain duplication

Slide 21

Slide 21 text

21 PROBLEMS TO SOLVE P1 We need to determine which duplications will increase peak performance P2 We need to know which optimizations are enabled by a certain duplication P3 Finding those optimization opportunities after duplication is compile time intensive, therefore, we need to find a way to perform this kind of analysis in acceptable time in a JIT compiler

Slide 22

Slide 22 text

22 APPROACH Simulate a duplication per merge Constant Folding Trade-off every possible duplication Sort by • Benefit • Cost • Probability Duplicate & Optimize Conditional Elimination PEA & Scalar Replacement Strength Reduction Duplicate Optimize Initial IR Optimization Potential For each Duplication Beneficial Duplications Optimized IR Decide if duplication is beneficial

Slide 23

Slide 23 text

23 APPROACH Simulate a duplication per merge Constant Folding Trade-off every possible duplication Sort by • Benefit • Cost • Probability Duplicate & Optimize Conditional Elimination PEA & Scalar Replacement Strength Reduction Duplicate Optimize Initial IR Optimization Potential For each Duplication Beneficial Duplications Optimized IR Decide if duplication is beneficial

Slide 24

Slide 24 text

24 APPROACH Simulate a duplication per merge Constant Folding Trade-off every possible duplication Sort by • Benefit • Cost • Probability Duplicate & Optimize Conditional Elimination PEA & Scalar Replacement Strength Reduction Duplicate Optimize Initial IR Optimization Potential For each Duplication Beneficial Duplications Optimized IR Decide if duplication is beneficial

Slide 25

Slide 25 text

25 APPROACH Simulate a duplication per merge Constant Folding Trade-off every possible duplication Sort by • Benefit • Cost • Probability Duplicate & Optimize Conditional Elimination PEA & Scalar Replacement Strength Reduction Duplicate Optimize Initial IR Optimization Potential For each Duplication Beneficial Duplications Optimized IR Decide if duplication is beneficial

Slide 26

Slide 26 text

26 APPROACH Simulate a duplication per merge Constant Folding Trade-off every possible duplication Sort by • Benefit • Cost • Probability Duplicate & Optimize Simulation Tier Trade-off Tier Optimization Tier Conditional Elimination PEA & Scalar Replacement Strength Reduction Duplicate Optimize Initial IR Optimization Potential For each Duplication Beneficial Duplications Optimized IR Decide if duplication is beneficial

Slide 27

Slide 27 text

27 RECAP MOTIVATING EXAMPLE int f(int a, int b, int[] x) { int p; if (a > b) { p = a; } else { p = 2; } return x.length / p; }

Slide 28

Slide 28 text

28 APPROACH int p; if (a > b) p = a; p = 2; return x.length / p;

Slide 29

Slide 29 text

29 DOMINATOR TREE int p; if (a > b) p = a; p = 2; return x.length / p; Merge Block

Slide 30

Slide 30 text

30 DUPLICATION SIMULATION int p; if (a > b) p = a; p = 2; return x.length / p; Copy Merge Block return x.length / p; return x.length / p;

Slide 31

Slide 31 text

31 DUPLICATION SIMULATION int p; if (a > b) p = a; p = 2; return x.length / p; Simulate Dominance Relation return x.length / p; return x.length / p;

Slide 32

Slide 32 text

int p; if (a > b) p = a; p = 2; return x.length / p; Copy Propagation return x.length / p; return x.length / p; a 2 32 DUPLICATION SIMULATION

Slide 33

Slide 33 text

33 DUPLICATION SIMULATION int p; if (a > b) return x.length / p; return x.length / 2; return x.length / a; Apply Optimizations

Slide 34

Slide 34 text

34 DUPLICATION SIMULATION int p; if (a > b) return x.length / p; return x.length / 2; return x.length / a; Strength Reduction x.length >> 1

Slide 35

Slide 35 text

35 BENEFIT / COST -> TRADE OFF int p; if (a > b) return x.length / p; return x.length >> 1; return x.length / a;

Slide 36

Slide 36 text

36 BENEFIT / COST -> TRADE OFF int p; if (a > b) return x.length / p; return x.length >> 1; return x.length / a; Benefit  (Latency(Div) - Latency(Shift)) * Probability = 31 * 0.9 = 27.9

Slide 37

Slide 37 text

int p; if (a > b) return x.length / p; return x.length >> 1; return x.length / a; 37 BENEFIT / COST -> TRADE OFF Benefit  (Latency(Div) - Latency(Shift)) * Probability = 31 * 0.9 = 27.9 Cost  1 Additional Return (+ 4 Instructions) + 1 Additional Shift + 1 Additional Read = 6 6 – 1 Jump from branch = 5 Instructions