Why Ruby's JIT was slow / RubyKaigi Takeout 2021

Transcript

1. Why Ruby's JIT was slow RubyKaigi Takeout 2021 @k0kubun /

   Takashi Kokubun

2. Self introduction • GitHub, Twitter: @k0kubun • Ruby committer ◦

   JIT ◦ IRB: Color, ls, show_source ◦ Struct keyword_init • Treasure Data

3. My first RubyKaigi: 2015

4. Hamlit will be Haml 6 (?) (Manually merged)

5. Why Ruby's JIT was slow

6. None
7. None

8. https://gist.github.com/k0kubun/cbc5251be1c19e36b7b7f786db302465

9. https://gist.github.com/k0kubun/cbc5251be1c19e36b7b7f786db302465

10. Why was Ruby's JIT slow? • The "MJIT" architecture was

    making Rails slow ◦ MJIT: C compiler + dlopen ◦ A lot of duplications in generated codes ▪ Ruby 3.0 fixed it ▪ Ruby 3.1 will have a better default config for it

11. Other drawbacks of MJIT • Too slow compilation ◦ 5

    min to fully warm up 1,000 methods on Railsbench • Too large compilation overhead • PIC is slower by several cycles

12. JIT's architecture matters! • It took 3 years to fix

    MJIT's bottleneck ◦ and still other drawbacks remain • It impacts your daily Ruby usage ◦ MJIT will make your Rails app slower during long warm-up ◦ MJIT requires a C compiler on runtime

13. MJIT for competitive programming? How about supporting JIT in AtCoder?

    (competitive programming website)

14. MJIT for competitive programming? https://docs.google.com/spreadsheets/d/1PmsqufkF3wjKN6g1L0STS80yP4a6u-VdGiEv5uOHe0M/edit Removed --jit because it actually

    makes 2s of use slower.

15. vs Golang https://youtu.be/mMwC0QenvcA?t=5188

16. vs Python https://youtu.be/vucLAqv7qpc

17. The goal of this talk • Discuss the JIT architecture

    of Ruby ◦ It will impact your future use of Ruby ◦ JIT authors could reduce development effort

18. Layers of concerns VM JIT compiler Codegen RTL YARV RTL-MJIT

    MJIT (C compiler) YARV-MJIT (mjit_compile.c) MIR YJIT yjit_codegen.c MIR-based JIT Ruby 2.6~3.0 Feature #12589 MIR YJIT

19. Discussion points • Maintainability • Internal Representation • How to

    compile and generate code • What optimization is feasible

20. Maintainability

21. Why did we choose MJIT? • One reason: maintainability ◦

    You can use gdb and see C code while debugging JIT-ed methods

22. Automatic support of new instructions • Support new instructions automatically

    ◦ Koichi's idea ◦ This may be helpful for any JIT • People think MJIT pastes C code and lets GCC do everything, but it’s wrong ◦ GCC alone can’t perform most of Ruby-specific optimizations

23. Language to implement JIT: C vs Ruby • Use Ruby

    to write the JIT compiler? ◦ Ractor: always multi-ractor or create and stop every time? ◦ Inter-process communication with a JIT process?

24. Internal Representation

25. YARV vs RTL • YARV-MJIT is no longer slower than

    RTL-MJIT ◦ We didn’t need to rewrite the VM for JIT’s performance.

26. What RTL had • Register-based instructions • Speculative instructions

27. Speculative instructions • These work like a profiler of runtime

    information • Alternatively, we could let JIT generate code for profiling ◦ The current MJIT generates the most speculative code first, and then recompile code with some optimizations disabled when cancelled ◦ YJIT's basic block versioning also profiles type information, etc.

28. C method inlining • LLVM, MIR • Rewrite everything in

    Ruby: YARV • TruffleRuby is both

29. Compilation and Code Generation

30. Compilation: Sync vs Async • MJIT: Concurrently JIT-compile methods in

    an MJIT worker thread • YJIT: Ruby threads JIT-compile methods during execution

31. Compilation speed • MJIT: 50~200ms for a single compile, and

    minutes for compaction • YJIT, MIR: < 1ms

32. Code generation • C compiler ◦ Slow startup • LLVM

    ◦ Binary size, build complexity • MIR • YJIT's assembler

33. Feasible optimizations

34. JIT code dispatch • call vs jmp (direct threading) ◦

    jmp is hard for MJIT ◦ But fortunately call seems faster, even in YJIT

35. Deoptimization • On-stack replacement ◦ mprotect + SEGV handler ◦

    Code patching • It’s hard for MJIT to manipulate low-level information

36. Method frame skip • We already have one since Ruby

    2.7 ◦ We also supported frame skip of more methods in Ruby 3.0 • Next: Lazy method frame push ◦ This is probably feasible in MJIT as well

37. Conclusion • JIT's architecture may impact: ◦ When you can

    use it ◦ Warmup speed ◦ Performance of VM and JIT ◦ Build and runtime dependencies