Two-level Just-in-Time Compilation with One Interpreter and One Engine

Presented at PEPM 2022.

Transcript

1. Two-level Just-in-Time Compilation with One Interpreter and One Engine Yusuke Izawa 1 Hidehiko Masuhara 1 Carl Friedrich Bolz-Tereick 2 1Tokyo Institute of Technology 2Heinrich-Heine-Universität Düsseldorf PEPM 2022 January 18, 2022 Two-level JIT Compilation with .. PEPM 2022 1 / 15
2. Outline 1. Introduction: the folklore in a JIT community and our findings 2. Proposal: Adaptive RPython that performs multitier compilation with “one interpreter” and “one engine” 3. Observation: to confirm that Adaptive RPython “actually” works Two-level JIT Compilation with .. PEPM 2022 2 / 15
3. Folklore: A Meta-JIT Compiler Performs a Fixed Kind of JIT Compilation • Build an interpreter from scratch for realizing different kinds of JIT compilers Interptracing Interpmethod Interpthreaded Meta-JIT compiler Meta-JIT compiler Meta-JIT compiler Tracing JIT Method JIT Threaded Code Gen.[ICOOOLPS 2021] Two-level JIT Compilation with .. PEPM 2022 3 / 15
4. Folklore: A Meta-JIT Compiler Performs a Fixed Kind of JIT Compilation • Build an interpreter from scratch for realizing different kinds of JIT compilers Interptracing Interpmethod Interpthreaded Meta-JIT compiler Meta-JIT compiler Meta-JIT compiler Tracing JIT Method JIT Threaded Code Gen.[ICOOOLPS 2021] consisting CALL insts in bytecode: removing indirect-branching Two-level JIT Compilation with .. PEPM 2022 3 / 15
5. Folklore: A Meta-JIT Compiler Performs a Fixed Kind of JIT Compilation • Build an interpreter from scratch for realizing different kinds of JIT compilers Interptracing Interpmethod Interpthreaded RPython Truffle/Graal Meta-JIT compiler PyPy TruffleRuby Threaded Code Gen.[ICOOOLPS 2021] Two-level JIT Compilation with .. PEPM 2022 3 / 15
6. Our Findings Will Affect JIT Community’s Assumption JIT community assumes that .. • Meta-tracing JIT compiler can only do tracing compilation RPython[interp, source] = ptracing Two-level JIT Compilation with .. PEPM 2022 4 / 15
7. Our Findings Will Affect JIT Community’s Assumption JIT community assumes that .. • Meta-tracing JIT compiler can only do tracing compilation But, with our findings .. • Let meta-tracing JIT do several compilations, like − method compilation − threaded code compilation − etc. RPython[interp, source] = ptracing RPython[interp, source] = p α RPython[interp, source] = p β RPython[interp, source] = p γ · · · Two-level JIT Compilation with .. PEPM 2022 4 / 15
8. Our Findings Two-level JIT Compilation with .. PEPM 2022 5 / 15
9. Our Findings By providing different interp definitions to RPython, can derive different kinds of outputs E.g. when passes .. Two-level JIT Compilation with .. PEPM 2022 5 / 15
10. Our Findings By providing different interp definitions to RPython, can derive different kinds of outputs E.g. when passes .. • interptracing to RPython → tracing compilation RPython[interptracing, source] = ptracing Two-level JIT Compilation with .. PEPM 2022 5 / 15
11. Our Findings By providing different interp definitions to RPython, can derive different kinds of outputs E.g. when passes .. • interptracing to RPython → tracing compilation • interpmethod to RPython → method compilation RPython[interptracing, source] = ptracing RPython[interpmethod, source] = pmethod Two-level JIT Compilation with .. PEPM 2022 5 / 15
12. Our Findings By providing different interp definitions to RPython, can derive different kinds of outputs E.g. when passes .. • interptracing to RPython → tracing compilation • interpmethod to RPython → method compilation • interpthreaded to RPython → threaded compilation RPython[interptracing, source] = ptracing RPython[interpmethod, source] = pmethod RPython[interpthreaded, source] = pthreaded Two-level JIT Compilation with .. PEPM 2022 5 / 15
13. Our Findings By providing different interp definitions to RPython, can derive different kinds of outputs E.g. when passes .. • interptracing to RPython → tracing compilation • interpmethod to RPython → method compilation • interpthreaded to RPython → threaded compilation RPython[interptracing, source] = ptracing RPython[interpmethod, source] = pmethod RPython[interpthreaded, source] = pthreaded  In other words .. By changing an interpreter, we can get different kinds of compilers Two-level JIT Compilation with .. PEPM 2022 5 / 15
14. Proposal: Multitier Compilation on Adaptive RPython Adaptive RPython performs multitier compilation with “one interpreter” and “one engine” optimization level threaded code baseline 2 · · · tracing method tracing + method Two-level JIT Compilation with .. PEPM 2022 6 / 15
15. Proposal: Multitier Compilation on Adaptive RPython Adaptive RPython performs multitier compilation with “one interpreter” and “one engine” optimization level threaded code baseline 2 · · · tracing method tracing + method With Adaptive RPython .. one generic interp. → common interp + a bit different definitions Two-level JIT Compilation with .. PEPM 2022 6 / 15
16. Proposal: Multitier Compilation on Adaptive RPython Adaptive RPython performs multitier compilation with “one interpreter” and “one engine” optimization level threaded code baseline 2 · · · tracing method tracing + method With Adaptive RPython .. one generic interp. → common interp + a bit different definitions perform on one engine = RPython Two-level JIT Compilation with .. PEPM 2022 6 / 15
17. Overview: Adaptive RPython Performs Multitier Compilation (1) A developer writes a generic interp Generic interp Adaptive RPython Pre-processor Adaptive RPython P.E. System Two-level JIT Compilation with .. PEPM 2022 7 / 15
18. Overview: Adaptive RPython Performs Multitier Compilation (2) Pass information to the pre-processor Generic interp Adaptive RPython Pre-processor Which instruc- tions will be transformed? Adaptive RPython P.E. System Two-level JIT Compilation with .. PEPM 2022 7 / 15
19. Overview: Adaptive RPython Performs Multitier Compilation (3) Generate interps from generic interp Generic interp Adaptive RPython Pre-processor Which instruc- tions will be transformed? Icommon Itracing Ithreaded Imethod Adaptive RPython P.E. System Two-level JIT Compilation with .. PEPM 2022 7 / 15
20. Overview: Adaptive RPython Performs Multitier Compilation (4) Pass information to the offline P.E. Generic interp Adaptive RPython Pre-processor Which instruc- tions will be transformed? Icommon Itracing Ithreaded Imethod Adaptive RPython P.E. System Source program and info about static and dy- namic inputs Two-level JIT Compilation with .. PEPM 2022 7 / 15
21. Overview: Adaptive RPython Performs Multitier Compilation (5) Tracing compilation: choose Icommon and I tracing RPython[Icommon + Itracing, P, V] = P′ tracing Generic interp Adaptive RPython Pre-processor Which instruc- tions will be transformed? Icommon Itracing Ithreaded Imethod Adaptive RPython P.E. System Source program and info about static and dy- namic inputs Ptracing Two-level JIT Compilation with .. PEPM 2022 7 / 15
22. Overview: Adaptive RPython Performs Multitier Compilation (6) Threaded code gen. [Izawa et al., 2021]: choose Icommon and Ithreaded RPython[Icommon + Ithreaded, P, V] = P′ threaded Generic interp Adaptive RPython Pre-processor Which instruc- tions will be transformed? Icommon Itracing Ithreaded Imethod Adaptive RPython P.E. System Source program and info about static and dy- namic inputs Ptracing Pthreaded Two-level JIT Compilation with .. PEPM 2022 7 / 15
23. Overview: Adaptive RPython Performs Multitier Compilation (7) Method compilation: choose Icommon and Imethod RPython[Icommon + Imethod, P, V] = P′ method Generic interp Adaptive RPython Pre-processor Which instruc- tions will be transformed? Icommon Itracing Ithreaded Imethod Adaptive RPython P.E. System Source program and info about static and dy- namic inputs Ptracing Pthreaded Pmethod Two-level JIT Compilation with .. PEPM 2022 7 / 15
24. Overview: How to Drive the RPython Engine [ICOOOLPS 2021] Meta-tracing JIT • Trace the execution of an interp. Threaded code generation A B C D JUMP E F RET call [p0] i1 = load(..) i1 = int_add(..) i2 = int_lt(..) guard_true(i2) [p0] .. .. jump(p0) Two-level JIT Compilation with .. PEPM 2022 8 / 15
25. Overview: How to Drive the RPython Engine [ICOOOLPS 2021] Meta-tracing JIT • Trace the execution of an interp Threaded code generation Two-level JIT Compilation with .. PEPM 2022 9 / 15
26. Overview: How to Drive the RPython Engine [ICOOOLPS 2021] Meta-tracing JIT • Trace the execution of an interp Threaded code generation • Traverse the entire method body • Not trace inside the handlers Two-level JIT Compilation with .. PEPM 2022 9 / 15
27. Overview: How to Drive the RPython Engine [ICOOOLPS 2021] Meta-tracing JIT • Trace the execution of an interp Threaded code generation • Traverse the entire method body • Not trace inside the handlers A B C D JUMP E F RET call call Traverse the en- tire method body Not trace the inside but leave CALL to the handler Cut/stitch the temporal trace [p0] i7 = call_i(ConstClass(DUP, ..)) i12 = call_i(ConstClass(CONST_I ..)) i16 = call_i(ConstClass(LT, ..)) guard_true(i16) [p0] ... jump(p0) [p0] ... i28 = call_i(ConstClass(CALL, ..)) ... i32 = call_i(ConstClass(RET, ..2)) leave_portal_frame(0) finish(i32) bridge Two-level JIT Compilation with .. PEPM 2022 9 / 15
28. Overview: How to Drive the RPython Engine [ICOOOLPS 2021] Meta-tracing JIT • Trace the execution of an interp Threaded code generation • Traverse the entire method body • Not trace inside the handlers tweaking an interp A B C D JUMP E F RET call call Traverse the en- tire method body Not trace the inside but leave CALL to the handler Cut/stitch the temporal trace [p0] i7 = call_i(ConstClass(DUP, ..)) i12 = call_i(ConstClass(CONST_I ..)) i16 = call_i(ConstClass(LT, ..)) guard_true(i16) [p0] ... jump(p0) [p0] ... i28 = call_i(ConstClass(CALL, ..)) ... i32 = call_i(ConstClass(RET, ..2)) leave_portal_frame(0) finish(i32) bridge Two-level JIT Compilation with .. PEPM 2022 9 / 15
29. Method-traversal Interpreter: How to Drive the RPython Engine [ICOOOLPS 2021]  traverse depth-firstly the entire method body w/ traverse_stack @dont_look_insdie def ADD(): .. while True: if opcde == JUMP_IF: top = pop() target = bytecode[pc++] if top.is_true(): traverse_stack.push(pc++) pc = target else: traverse_stack.push(target) pc++ elif opcode == JUMP: target = bytecode[pc++] if not traverse_stack.is_empty(): pc = traverse_stack.pop() else: finish() elif opcode == RET: if not traverse_stack.is_empty(): pc = traverse_stack.pop() else: return pop()
30. Method-traversal Interpreter: How to Drive the RPython Engine [ICOOOLPS 2021]  traverse depth-firstly the entire method body w/ traverse_stack @dont_look_insdie def ADD(): .. while True: if opcde == JUMP_IF: top = pop() target = bytecode[pc++] if top.is_true(): traverse_stack.push(pc++) pc = target else: traverse_stack.push(target) pc++ elif opcode == JUMP: target = bytecode[pc++] if not traverse_stack.is_empty(): pc = traverse_stack.pop() else: finish() elif opcode == RET: if not traverse_stack.is_empty(): pc = traverse_stack.pop() else: return pop() Suppress inlining
31. Method-traversal Interpreter: How to Drive the RPython Engine [ICOOOLPS 2021]  traverse depth-firstly the entire method body w/ traverse_stack @dont_look_insdie def ADD(): .. while True: if opcde == JUMP_IF: top = pop() target = bytecode[pc++] if top.is_true(): traverse_stack.push(pc++) pc = target else: traverse_stack.push(target) pc++ elif opcode == JUMP: target = bytecode[pc++] if not traverse_stack.is_empty(): pc = traverse_stack.pop() else: finish() elif opcode == RET: if not traverse_stack.is_empty(): pc = traverse_stack.pop() else: return pop() Suppress inlining Save another side to traverse later
32. Method-traversal Interpreter: How to Drive the RPython Engine [ICOOOLPS 2021]  traverse depth-firstly the entire method body w/ traverse_stack @dont_look_insdie def ADD(): .. while True: if opcde == JUMP_IF: top = pop() target = bytecode[pc++] if top.is_true(): traverse_stack.push(pc++) pc = target else: traverse_stack.push(target) pc++ elif opcode == JUMP: target = bytecode[pc++] if not traverse_stack.is_empty(): pc = traverse_stack.pop() else: finish() elif opcode == RET: if not traverse_stack.is_empty(): pc = traverse_stack.pop() else: return pop() Suppress inlining Save another side to traverse later Jump to another side Two-level JIT Compilation with .. PEPM 2022 10 / 15
33. The Design of Generic Interpreter • From a generic interp, Adaptive RPython generates interps including MTI  Embed tier-specific definitions in a meta-tracing-based interpreter 1. Declare JitTierDriver jittierdriver = JitTierDriver(pc='pc') def interp(self); .. if opcode == JUMP_IF: target = bytecode[pc] elif opcode == JUMP: target = bytecode[pc] elif opcode == RET: w_x = self.pop() elif .. Two-level JIT Compilation with .. PEPM 2022 11 / 15
34. The Design of Generic Interpreter • From a generic interp, Adaptive RPython generates interps including MTI  Embed tier-specific definitions in a meta-tracing-based interpreter 1. Declare JitTierDriver 2. Define can_enter_tier1_XX at JUMP_IF, JUMP, RET for threaded code gen. and method comp. jittierdriver = JitTierDriver(pc='pc') def interp(self); .. if opcode == JUMP_IF: target = bytecode[pc] jittierdriver.can_enter_tier1_branch( true_path=target,false_path=pc+1, cond=self.is_true) if we_are_in_tier2(): elif opcode == JUMP: target = bytecode[pc] jittierdriver.can_enter_tier1_jump(target=target) elif opcode == RET: w_x = self.pop() jittierdriver.can_enter_tier1_ret(ret_value=w_x) elif .. Two-level JIT Compilation with .. PEPM 2022 11 / 15
35. The Design of Generic Interpreter • From a generic interp, Adaptive RPython generates interps including MTI  Embed tier-specific definitions in a meta-tracing-based interpreter 1. Declare JitTierDriver 2. Define can_enter_tier1_XX at JUMP_IF, JUMP, RET for threaded code gen. and method comp. 3. Define interp. for tracing JIT inside we_are_in_tier2 jittierdriver = JitTierDriver(pc='pc') def interp(self); .. if opcode == JUMP_IF: target = bytecode[pc] jittierdriver.can_enter_tier1_branch( true_path=target,false_path=pc+1, cond=self.is_true) if we_are_in_tier2(): do stuff for tracing JIT elif opcode == JUMP: target = bytecode[pc] jittierdriver.can_enter_tier1_jump(target=target) if we_are_in_tier2(): do stuff for tracing JIT elif opcode == RET: w_x = self.pop() jittierdriver.can_enter_tier1_ret(ret_value=w_x) if we_are_in_tier2(): do stuff for tracing JIT elif .. Two-level JIT Compilation with .. PEPM 2022 11 / 15
36. The Design of Generic Interpreter • From a generic interp, Adaptive RPython generates interps including MTI  Embed tier-specific definitions in a meta-tracing-based interpreter 1. Declare JitTierDriver 2. Define can_enter_tier1_XX at JUMP_IF, JUMP, RET for threaded code gen. and method comp. 3. Define interp. for tracing JIT inside we_are_in_tier2 4. The pre-processor generates method-traversal interp and tracing interp from this jittierdriver = JitTierDriver(pc='pc') def interp(self); .. if opcode == JUMP_IF: target = bytecode[pc] jittierdriver.can_enter_tier1_branch( true_path=target,false_path=pc+1, cond=self.is_true) if we_are_in_tier2(): do stuff for tracing JIT elif opcode == JUMP: target = bytecode[pc] jittierdriver.can_enter_tier1_jump(target=target) if we_are_in_tier2(): do stuff for tracing JIT elif opcode == RET: w_x = self.pop() jittierdriver.can_enter_tier1_ret(ret_value=w_x) if we_are_in_tier2(): do stuff for tracing JIT elif .. Two-level JIT Compilation with .. PEPM 2022 11 / 15
37. Observation: Can Adaptive RPython “Actually” Work? (1) Setup • Write TLA lang. interpreter in Adaptive RPython • Run TLA interpreter on small examples − loopabit: nested loop − callabit: two functions – one is suitable for tracing, the other is for thraeded code gen. (method) NOTE • Current multitier is the combination of threaded code gen. and tracing (two-level) Two-level JIT Compilation with .. PEPM 2022 12 / 15
38. Observation: Can Adaptive RPython “Actually” Work? (2) Situation in callabit: increasing an optimization level program JIT applied to f JIT applied to g callabit_baseline_interp threaded (interpreted) function f (for threaded code gen.) function g (for trac- ing comp.) call ret threaded code generation interpreted Two-level JIT Compilation with .. PEPM 2022 13 / 15
39. Observation: Can Adaptive RPython “Actually” Work? (2) Situation in callabit: increasing an optimization level program JIT applied to f JIT applied to g callabit_baseline_interp threaded (interpreted) callabit_baseline_only threaded threaded function f (for threaded code gen.) function g (for trac- ing comp.) call ret threaded code generation threaded code generation Two-level JIT Compilation with .. PEPM 2022 13 / 15
40. Observation: Can Adaptive RPython “Actually” Work? (2) Situation in callabit: increasing an optimization level program JIT applied to f JIT applied to g callabit_baseline_interp threaded (interpreted) callabit_baseline_only threaded threaded callabit_tracing_baseline tracing baseline function f (for threaded code gen.) function g (for trac- ing comp.) call ret tracing compi- lation threaded code generation Two-level JIT Compilation with .. PEPM 2022 13 / 15
41. Observation: Can Adaptive RPython “Actually” Work? (2) Situation in callabit: increasing an optimization level program JIT applied to f JIT applied to g callabit_baseline_interp threaded (interpreted) callabit_baseline_only threaded threaded callabit_tracing_baseline tracing baseline callabit_baseline_tracing threaded tracing function f (for threaded code gen.) function g (for trac- ing comp.) call ret threaded code generation tracing compi- lation Two-level JIT Compilation with .. PEPM 2022 13 / 15
42. Observation: Can Adaptive RPython “Actually” Work? (2) Situation in callabit: increasing an optimization level program JIT applied to f JIT applied to g callabit_baseline_interp threaded (interpreted) callabit_baseline_only threaded threaded callabit_tracing_baseline tracing baseline callabit_baseline_tracing threaded tracing callabit_tracing_only tracing tracing function f (for threaded code gen.) function g (for trac- ing comp.) call ret tracing compi- lation tracing compi- lation Two-level JIT Compilation with .. PEPM 2022 13 / 15
43. Observation: Running Speeds and Trace Sizes • Actually worked: compilation speed is reaching to tracing compilation callabit_baseline_interp callabit_baseline_only callabit_baseline_tracing callabit_tracing_baseline callabit_tracing_only 0.0 0.5 1.0 1.5 2.0 2.5 3.0 The speed up ratio (interp = 1) TLA w/ Adaptive RPython (Stable speed) # Traces 0 50 100 150 200 250 300 350 400 callabit_baseline_interp callabit_baseline_only callabit_baseline_tracing callabit_tracing_baseline callabit_tracing_only Increasing better smaller Two-level JIT Compilation with .. PEPM 2022 14 / 15
44. Observation: Running Speeds and Trace Sizes • Actually worked: compilation speed is reaching to tracing compilation • Promising signs: multitier is same speed but smaller code size compared to single tier → might get good performance in the future callabit_baseline_interp callabit_baseline_only callabit_baseline_tracing callabit_tracing_baseline callabit_tracing_only 0.0 0.5 1.0 1.5 2.0 2.5 3.0 The speed up ratio (interp = 1) TLA w/ Adaptive RPython (Stable speed) # Traces 0 50 100 150 200 250 300 350 400 callabit_baseline_interp callabit_baseline_only callabit_baseline_tracing callabit_tracing_baseline callabit_tracing_only better smaller Two-level JIT Compilation with .. PEPM 2022 14 / 15
45. Conclusion and Future Work Conclusion • Adaptive RPython actually worked on a small lang. RPython [I, P, V] = P′ RPython [ Icommon + Itracing, P, V ] = P′ tracing RPython [ Icommon + Ithreaded, P, V ] = P′ threaded RPython [ Icommon + Imethod, P, V ] = P′ method One Engine One Interpreter Multitier Outputs Derive from Generic Interp. Common Interp. Tweaked Defs. Future Work Two-level JIT Compilation with .. PEPM 2022 15 / 15
46. Conclusion and Future Work Conclusion • Adaptive RPython actually worked on a small lang. RPython [I, P, V] = P′ RPython [ Icommon + Itracing, P, V ] = P′ tracing RPython [ Icommon + Ithreaded, P, V ] = P′ threaded RPython [ Icommon + Imethod, P, V ] = P′ method One Engine One Interpreter Multitier Outputs Derive from Generic Interp. Common Interp. Tweaked Defs. Future Work • Decide multitier compilation strategy − How to shift between levels? − How to decide an appropriate level? • Implement our ideas on PyPy Two-level JIT Compilation with .. PEPM 2022 15 / 15
47. Conclusion and Future Work Conclusion • Adaptive RPython actually worked on a small lang. RPython [I, P, V] = P′ RPython [ Icommon + Itracing, P, V ] = P′ tracing RPython [ Icommon + Ithreaded, P, V ] = P′ threaded RPython [ Icommon + Imethod, P, V ] = P′ method One Engine One Interpreter Multitier Outputs Derive from Generic Interp. Common Interp. Tweaked Defs. Future Work • Decide multitier compilation strategy − How to shift between levels? − How to decide an appropriate level? • Implement our ideas on PyPy Two-level JIT Compilation with .. PEPM 2022 15 / 15
48. References I Izawa, Y., Masuhara, H., Bolz-Tereick, C. F., and Cong, Y. (2021). Threaded code generation with a meta-tracing JIT compiler. The Journal of Object Technology Special Issue for ICOOOLPS 2021, pages 1–11. Accepted. Two-level JIT Compilation with .. PEPM 2022 1 / 1