Why Ruby's JIT was slow / RubyKaigi Takeout 2021

RubyKaigi Takeout 2021

Transcript

1. Why Ruby's JIT was slow
   RubyKaigi Takeout 2021
   @k0kubun / Takashi Kokubun
   

   View Slide

2. Self introduction
   ● GitHub, Twitter: @k0kubun
   ● Ruby committer
   ○ JIT
   ○ IRB: Color, ls, show_source
   ○ Struct keyword_init
   ● Treasure Data
   

   View Slide

3. My first RubyKaigi: 2015
   

   View Slide

4. Hamlit will be Haml 6 (?)
   (Manually merged)
   

   View Slide

5. Why Ruby's JIT was slow
   

   View Slide

6. View Slide

7. View Slide

8. https://gist.github.com/k0kubun/cbc5251be1c19e36b7b7f786db302465
   

   View Slide

9. https://gist.github.com/k0kubun/cbc5251be1c19e36b7b7f786db302465
   

   View Slide

10. Why was Ruby's JIT slow?
    ● The "MJIT" architecture was making Rails slow
    ○ MJIT: C compiler + dlopen
    ○ A lot of duplications in generated codes
    ■ Ruby 3.0 fixed it
    ■ Ruby 3.1 will have a better default config for it
    

    View Slide

11. Other drawbacks of MJIT
    ● Too slow compilation
    ○ 5 min to fully warm up 1,000 methods on Railsbench
    ● Too large compilation overhead
    ● PIC is slower by several cycles
    

    View Slide

12. JIT's architecture matters!
    ● It took 3 years to fix MJIT's bottleneck
    ○ and still other drawbacks remain
    ● It impacts your daily Ruby usage
    ○ MJIT will make your Rails app slower during long warm-up
    ○ MJIT requires a C compiler on runtime
    

    View Slide

13. MJIT for competitive programming?
    How about supporting JIT in AtCoder?
    (competitive programming website)
    

    View Slide

14. MJIT for competitive programming?
    https://docs.google.com/spreadsheets/d/1PmsqufkF3wjKN6g1L0STS80yP4a6u-VdGiEv5uOHe0M/edit
    Removed --jit because it actually makes 2s of use slower.
    

    View Slide

15. vs Golang
    https://youtu.be/mMwC0QenvcA?t=5188
    

    View Slide

16. vs Python
    https://youtu.be/vucLAqv7qpc
    

    View Slide

17. The goal of this talk
    ● Discuss the JIT architecture of Ruby
    ○ It will impact your future use of Ruby
    ○ JIT authors could reduce development effort
    

    View Slide

18. Layers of concerns
    VM
    JIT
    compiler
    Codegen
    RTL
    YARV
    RTL-MJIT
    MJIT (C compiler)
    YARV-MJIT (mjit_compile.c)
    MIR
    YJIT
    yjit_codegen.c
    MIR-based
    JIT
    Ruby 2.6~3.0 Feature #12589
    MIR
    YJIT
    

    View Slide

19. Discussion points
    ● Maintainability
    ● Internal Representation
    ● How to compile and generate code
    ● What optimization is feasible
    

    View Slide

20. Maintainability
    

    View Slide

21. Why did we choose MJIT?
    ● One reason: maintainability
    ○ You can use gdb and see C code while debugging JIT-ed methods
    

    View Slide

22. Automatic support of new instructions
    ● Support new instructions automatically
    ○ Koichi's idea
    ○ This may be helpful for any JIT
    ● People think MJIT pastes C code and lets GCC do everything,
    but it’s wrong
    ○ GCC alone can’t perform most of Ruby-specific optimizations
    

    View Slide

23. Language to implement JIT: C vs Ruby
    ● Use Ruby to write the JIT compiler?
    ○ Ractor: always multi-ractor or create and stop every time?
    ○ Inter-process communication with a JIT process?
    

    View Slide

24. Internal Representation
    

    View Slide

25. YARV vs RTL
    ● YARV-MJIT is no longer slower than RTL-MJIT
    ○ We didn’t need to rewrite the VM for JIT’s performance.
    

    View Slide

26. What RTL had
    ● Register-based instructions
    ● Speculative instructions
    

    View Slide

27. Speculative instructions
    ● These work like a profiler of runtime information
    ● Alternatively, we could let JIT generate code for profiling
    ○ The current MJIT generates the most speculative code first, and then
    recompile code with some optimizations disabled when cancelled
    ○ YJIT's basic block versioning also profiles type information, etc.
    

    View Slide

28. C method inlining
    ● LLVM, MIR
    ● Rewrite everything in Ruby: YARV
    ● TruffleRuby is both
    

    View Slide

29. Compilation and Code Generation
    

    View Slide

30. Compilation: Sync vs Async
    ● MJIT: Concurrently JIT-compile methods in an MJIT worker thread
    ● YJIT: Ruby threads JIT-compile methods during execution
    

    View Slide

31. Compilation speed
    ● MJIT: 50~200ms for a single compile, and minutes for compaction
    ● YJIT, MIR: < 1ms
    

    View Slide

32. Code generation
    ● C compiler
    ○ Slow startup
    ● LLVM
    ○ Binary size, build complexity
    ● MIR
    ● YJIT's assembler
    

    View Slide

33. Feasible optimizations
    

    View Slide

34. JIT code dispatch
    ● call vs jmp (direct threading)
    ○ jmp is hard for MJIT
    ○ But fortunately call seems faster, even in YJIT
    

    View Slide

35. Deoptimization
    ● On-stack replacement
    ○ mprotect + SEGV handler
    ○ Code patching
    ● It’s hard for MJIT to manipulate low-level information
    

    View Slide

36. Method frame skip
    ● We already have one since Ruby 2.7
    ○ We also supported frame skip of more methods in Ruby 3.0
    ● Next: Lazy method frame push
    ○ This is probably feasible in MJIT as well
    

    View Slide

37. Conclusion
    ● JIT's architecture may impact:
    ○ When you can use it
    ○ Warmup speed
    ○ Performance of VM and JIT
    ○ Build and runtime dependencies
    

    View Slide