• Yusuke Izawa , Hidehiko Masuhara, Tomoyuki Aotani • Tokyo Institute of Technology, Dept. of Math. And Comp. Science • MoreVMs’19 Workshop on Modern Language Runtimes, Ecosystems and VMs Extending a Meta-Tracing Compiler to Mix Method and Tracing Compilation 2019/4/2 2019 1 @yusuke_izawa 

•A performance penalty in tracing compilation. • Tracing JIT works well in many programs. • In some recursive functions, tracing JIT performs worse. 2019/4/2 2019 2 What is the problem? 0 0.2 0.4 0.6 0.8 tak fib geo_mean Pyrlang HiPE 0 0.2 0.4 0.6 0.8 1 tak fib geo_mean Pycket Larceny Pyrlang[R. Huang, el. al., ‘16] vs. HiPE Pycket[S. Bauman, el. al. ,‘15] vs. Larceny Tracing JIT (RPython) vs. AOT compiler better better 

•Mismatch between tracing and running. 2019/4/2 2019 3 Problem Characterization: Path Divergence Problem Getting traces of a varying control-flow. Resulting in not covering all paths. Resulting trace A control flow of the Fibonacci Most executions are run in an interpreter. 

•Apply different compilation strategies for different program parts. 2019/4/2 2019 4 Our Solution to Path Divergence Problem Branching possibilities Varying control flow Apply Tracing Compilation Apply Method Compilation 

Goals •Cooperate method- and trace-based compilation. •Requires only a single interpreter definition. Proof-of-concept implementation •BacCaml, based on MinCaml[E.Sumii, ‘05] compiler. 2019/4/2 2019 5 Proposal of Meta-Hybrid Compilation Framework 

Principle •Re-using a meta-tracing compilation in method compilation. • Creating both strategies in a common architecture. Implementation • From scratch • RPython: too large to comprehend (2,000,000 lines in PyPy repo.). • MinCaml: simple yet efficient (2,000 lines, as fast as GCC). 2019/4/2 2019 6 Designing Meta-Hybrid Compilation Framework 

2019/4/2 7 2019 Architecture of Meta-Hybrid Compilation Framework Tracing Cmpl. Method Cmpl. Trace Specializer Profiler Dispatcher Base Language Interpreter Base Language Program Write with BacCaml Optimizing + Generating native BacCaml Following exec. and recording instructions of the base lang. interp. Implemented as a BacCaml interpreter Tracer 

Key idea •Trace all possible paths in basic blocks in a method. • Followings are customized for method compilation. • Conditional branches • Function calls • Loops 2019/4/2 2019 8 Compiling a Method with a Meta- Tracing Compiler 

•Tracing both branches. •Rolling back values in registers and heaps. 2019/4/2 2019 9 Compiling a Method with a Meta- Tracing Compiler: Conditional Branches 1/3 Resulting trace Base program Stores values Rolls back stored values 

2019/4/2 10 2019 Compiling a Method with a Meta- Tracing Compiler: Conditional Branches 2/3 @method_entry int f(int x) { @condition if (x < 0) { return x + 10; } else { return x - 10; }} void interp(...) { switch(instruction) { ... case COND_BRANCH: cond_branch(); break; } } Source code of base program Source code of base lang. interp. Specifying the annotation Telling the tracer the position of a conditional branch 

•Why can the tracer roll back values? • Registers and heaps are implemented as “array” of OCaml’s data structures. • The tracer is implemented as an interpreter for MinCaml IR. •Any problem with tracing an unexecuted branch? • No problem: the tracer follows only variables related to a program counter or a bytecode. 2019/4/2 2019 11 Compiling a Method with a Meta- Tracing Compiler: Conditional Branches 3/3 

•Leaving a call instruction. •Continuing to trace successors. 2019/4/2 2019 12 Compiling a Method with a Meta- Tracing Compiler: Function Calls 1/2 call Leaves the call inst. Continues to trace Base program call Resulting trace 

2019/4/2 13 2019 Compiling a Method with a Meta- Tracing Compiler: Function Calls 2/2 @method_entry int g(int x) { return x - 1; } @method_entry int f(int x) { int r = g(x); return r + 10; } void interp(...) { switch(instruction) { case CALL: fun_call(); a = interp(...); stack.push(a); break; }} Source code of base program Source code of base lang. interp. Telling the tracer the position of a function call 

•Trace-splitting and not following a back edge. 2019/4/2 2019 14 Compiling a Method with a Meta- Tracing Compiler: Loops 1/3 outer loop inner loop Doesn’t follow a backward jump Splits and combines with a jump inst. Generates a jump instruction to B Traces it as in C Base program Resulting trace jump jump jump 

2019/4/2 15 2019 Compiling a Method with a Meta- Tracing Compiler: Loops 2/3 @method_entry int f(int x) { int i, sum = 0; @loop while(i < 100) { i++; sum += i; } return sum; } void interp(int pc) { instrucetion = bytecodoe[pc]; switch(instruction) { ... case LOOP_ENTRY: loop_entry(); break; case BACK_EDGE: back_edge(); break; }} Source code of base program Source code of base lang. interp. Specifying the annotation Telling the tracer the position of a loop 

•Why need trace-splitting? • To trace a back-edge as a tail-recursive call. • In MinCaml IR, a loop is a “tail-recursive” function call. 2019/4/2 2019 16 Compiling a Method with a Meta- Tracing Compiler: Loops 3/3 

•A suitable definition is different in function calls. 2019/4/2 2019 17 Enabling a Hybrid Compilation with a single interpreter 1/2 User Stack Host Language Stack case CALL: addr = bytecode[pc]; stack.push(pc + 1); pc = addr; break; case CALL: addr = bytecode[pc] return_value = interp(addr); stack.push(return_value); break; case RETURN: return_value = stack.pop(); return_address = stack.pop(); stack.push(return_value); pc = return_address; break; case RETURN; return stask.pop(); Sutable for Tracing Compilation Sutable for Method Compilation 

•Prepared a pseudo function (is_mj()) • Enabling to embed two definitions in one interpreter. 2019/4/2 2019 18 Enabling a Hybrid Compilation with a single interpreter 2/2 case CALL: if (is_mj()) { // using a host language’s stack } else { // using a user’s stack } break; Only this part are traced in method compilation case RETURN; if (is_mj()) { // using a host language’s stack } else { // using a user’s stack } break; 

What we have done •Running method- and trace- based compilations separately. •Running the tracer offline by manually specifying: • (Tracing) the entry / exit points of the traces. • (Method) the entry points of the method. What we have not done • Profiling and dynamic linking. • Mixing both compilations. • and more… 2019/4/2 2019 19 Preliminary Benchmarking Test 1/3 

Preliminary Benchmarking Test 2/3 • sum: calculating a summation from 1 to 1000, 10000 times. • fib: calculating a Fibonacci number 28, 10000 times 2019/4/2 2019 20 RET Control-flow of sum Control-flow of sum Took two microbenchmarks: Suitable for Tracing Suitable for Method 

•Our compiler works well. • In average, method and tracing compilations are 4 times faster than the interpreter. •Our method compilation performs well in fib causing Path Divergence Problem. 2019/4/2 2019 21 Preliminary Benchmarking Test 3/3 4.04 15 4.53 3.47 19.8 25.8 0 10 20 30 SUM FIB Tracing compilation Method compilation Interpreter only better Environment Mac Pro (Late 2013) with CPU: 3.5 GHz 6-Core Intel Xeon E5, Mem: 16 GB 1866 MHz DDR3, running macOS Mojave version 10.14.2 Execution times relative to MinCaml 

•Proposed a meta-hybrid compilation framework. • Applied tracing compilation for all possible paths in a method. • Enabled by customizing following points (conditional branches, function calls and loops). • Discovered a way to commonalize an user interpreter in our hybrid compilation. •Proof-of-concept implementation BacCaml. 2019/4/2 2019 22 Conclusion 

•Implement many runtime optimizations. •Investigate: • a strategy of switching compilations. • good programming interface. •Apply this meta-hybrid compilation approach to RPython. 2019/4/2 2019 23 Future work 

2019/4/2 24 2019 Which path the tracer follows in method-based compilation? The control-flow of Fibonacci (w/o return) The paths the tracer follows 

2019/4/2 25 2019 How to switch both compilations? void f(int x) { ... ... ... ... } Apply Tracing Compilation Many guard failures Becomes hot void f(int x) { ... ... ... ... } void f(int x) { ... ... ... ... } void f(int x) { ... ... ... ... } Apply Method Compilation