Why Ruby's JIT was slow / RubyKaigi Takeout 2021

RubyKaigi Takeout 2021

Transcript

1. Why Ruby's JIT was slow RubyKaigi Takeout 2021 @k0kubun / Takashi Kokubun
2. Self introduction • GitHub, Twitter: @k0kubun • Ruby committer ◦ JIT ◦ IRB: Color, ls, show_source ◦ Struct keyword_init • Treasure Data
3. My first RubyKaigi: 2015
4. Hamlit will be Haml 6 (?) (Manually merged)
5. Why Ruby's JIT was slow
8. https://gist.github.com/k0kubun/cbc5251be1c19e36b7b7f786db302465
9. https://gist.github.com/k0kubun/cbc5251be1c19e36b7b7f786db302465
10. Why was Ruby's JIT slow? • The "MJIT" architecture was making Rails slow ◦ MJIT: C compiler + dlopen ◦ A lot of duplications in generated codes ▪ Ruby 3.0 fixed it ▪ Ruby 3.1 will have a better default config for it
11. Other drawbacks of MJIT • Too slow compilation ◦ 5 min to fully warm up 1,000 methods on Railsbench • Too large compilation overhead • PIC is slower by several cycles
12. JIT's architecture matters! • It took 3 years to fix MJIT's bottleneck ◦ and still other drawbacks remain • It impacts your daily Ruby usage ◦ MJIT will make your Rails app slower during long warm-up ◦ MJIT requires a C compiler on runtime
13. MJIT for competitive programming? How about supporting JIT in AtCoder? (competitive programming website)
14. MJIT for competitive programming? https://docs.google.com/spreadsheets/d/1PmsqufkF3wjKN6g1L0STS80yP4a6u-VdGiEv5uOHe0M/edit Removed --jit because it actually makes 2s of use slower.
15. vs Golang https://youtu.be/mMwC0QenvcA?t=5188
16. vs Python https://youtu.be/vucLAqv7qpc
17. The goal of this talk • Discuss the JIT architecture of Ruby ◦ It will impact your future use of Ruby ◦ JIT authors could reduce development effort
18. Layers of concerns VM JIT compiler Codegen RTL YARV RTL-MJIT MJIT (C compiler) YARV-MJIT (mjit_compile.c) MIR YJIT yjit_codegen.c MIR-based JIT Ruby 2.6~3.0 Feature #12589 MIR YJIT
19. Discussion points • Maintainability • Internal Representation • How to compile and generate code • What optimization is feasible
20. Maintainability
21. Why did we choose MJIT? • One reason: maintainability ◦ You can use gdb and see C code while debugging JIT-ed methods
22. Automatic support of new instructions • Support new instructions automatically ◦ Koichi's idea ◦ This may be helpful for any JIT • People think MJIT pastes C code and lets GCC do everything, but it’s wrong ◦ GCC alone can’t perform most of Ruby-specific optimizations
23. Language to implement JIT: C vs Ruby • Use Ruby to write the JIT compiler? ◦ Ractor: always multi-ractor or create and stop every time? ◦ Inter-process communication with a JIT process?
24. Internal Representation
25. YARV vs RTL • YARV-MJIT is no longer slower than RTL-MJIT ◦ We didn’t need to rewrite the VM for JIT’s performance.
26. What RTL had • Register-based instructions • Speculative instructions
27. Speculative instructions • These work like a profiler of runtime information • Alternatively, we could let JIT generate code for profiling ◦ The current MJIT generates the most speculative code first, and then recompile code with some optimizations disabled when cancelled ◦ YJIT's basic block versioning also profiles type information, etc.
28. C method inlining • LLVM, MIR • Rewrite everything in Ruby: YARV • TruffleRuby is both
29. Compilation and Code Generation
30. Compilation: Sync vs Async • MJIT: Concurrently JIT-compile methods in an MJIT worker thread • YJIT: Ruby threads JIT-compile methods during execution
31. Compilation speed • MJIT: 50~200ms for a single compile, and minutes for compaction • YJIT, MIR: < 1ms
32. Code generation • C compiler ◦ Slow startup • LLVM ◦ Binary size, build complexity • MIR • YJIT's assembler
33. Feasible optimizations
34. JIT code dispatch • call vs jmp (direct threading) ◦ jmp is hard for MJIT ◦ But fortunately call seems faster, even in YJIT
35. Deoptimization • On-stack replacement ◦ mprotect + SEGV handler ◦ Code patching • It’s hard for MJIT to manipulate low-level information
36. Method frame skip • We already have one since Ruby 2.7 ◦ We also supported frame skip of more methods in Ruby 3.0 • Next: Lazy method frame push ◦ This is probably feasible in MJIT as well
37. Conclusion • JIT's architecture may impact: ◦ When you can use it ◦ Warmup speed ◦ Performance of VM and JIT ◦ Build and runtime dependencies