$30 off During Our Annual Pro Sale. View Details »

Why Ruby's JIT was slow / RubyKaigi Takeout 2021

Takashi Kokubun
September 08, 2021

Why Ruby's JIT was slow / RubyKaigi Takeout 2021

RubyKaigi Takeout 2021

Takashi Kokubun

September 08, 2021
Tweet

More Decks by Takashi Kokubun

Other Decks in Programming

Transcript

  1. Why Ruby's JIT was slow
    RubyKaigi Takeout 2021
    @k0kubun / Takashi Kokubun

    View Slide

  2. Self introduction
    ● GitHub, Twitter: @k0kubun
    ● Ruby committer
    ○ JIT
    ○ IRB: Color, ls, show_source
    ○ Struct keyword_init
    ● Treasure Data

    View Slide

  3. My first RubyKaigi: 2015

    View Slide

  4. Hamlit will be Haml 6 (?)
    (Manually merged)

    View Slide

  5. Why Ruby's JIT was slow

    View Slide

  6. View Slide

  7. View Slide

  8. https://gist.github.com/k0kubun/cbc5251be1c19e36b7b7f786db302465

    View Slide

  9. https://gist.github.com/k0kubun/cbc5251be1c19e36b7b7f786db302465

    View Slide

  10. Why was Ruby's JIT slow?
    ● The "MJIT" architecture was making Rails slow
    ○ MJIT: C compiler + dlopen
    ○ A lot of duplications in generated codes
    ■ Ruby 3.0 fixed it
    ■ Ruby 3.1 will have a better default config for it

    View Slide

  11. Other drawbacks of MJIT
    ● Too slow compilation
    ○ 5 min to fully warm up 1,000 methods on Railsbench
    ● Too large compilation overhead
    ● PIC is slower by several cycles

    View Slide

  12. JIT's architecture matters!
    ● It took 3 years to fix MJIT's bottleneck
    ○ and still other drawbacks remain
    ● It impacts your daily Ruby usage
    ○ MJIT will make your Rails app slower during long warm-up
    ○ MJIT requires a C compiler on runtime

    View Slide

  13. MJIT for competitive programming?
    How about supporting JIT in AtCoder?
    (competitive programming website)

    View Slide

  14. MJIT for competitive programming?
    https://docs.google.com/spreadsheets/d/1PmsqufkF3wjKN6g1L0STS80yP4a6u-VdGiEv5uOHe0M/edit
    Removed --jit because it actually makes 2s of use slower.

    View Slide

  15. vs Golang
    https://youtu.be/mMwC0QenvcA?t=5188

    View Slide

  16. vs Python
    https://youtu.be/vucLAqv7qpc

    View Slide

  17. The goal of this talk
    ● Discuss the JIT architecture of Ruby
    ○ It will impact your future use of Ruby
    ○ JIT authors could reduce development effort

    View Slide

  18. Layers of concerns
    VM
    JIT
    compiler
    Codegen
    RTL
    YARV
    RTL-MJIT
    MJIT (C compiler)
    YARV-MJIT (mjit_compile.c)
    MIR
    YJIT
    yjit_codegen.c
    MIR-based
    JIT
    Ruby 2.6~3.0 Feature #12589
    MIR
    YJIT

    View Slide

  19. Discussion points
    ● Maintainability
    ● Internal Representation
    ● How to compile and generate code
    ● What optimization is feasible

    View Slide

  20. Maintainability

    View Slide

  21. Why did we choose MJIT?
    ● One reason: maintainability
    ○ You can use gdb and see C code while debugging JIT-ed methods

    View Slide

  22. Automatic support of new instructions
    ● Support new instructions automatically
    ○ Koichi's idea
    ○ This may be helpful for any JIT
    ● People think MJIT pastes C code and lets GCC do everything,
    but it’s wrong
    ○ GCC alone can’t perform most of Ruby-specific optimizations

    View Slide

  23. Language to implement JIT: C vs Ruby
    ● Use Ruby to write the JIT compiler?
    ○ Ractor: always multi-ractor or create and stop every time?
    ○ Inter-process communication with a JIT process?

    View Slide

  24. Internal Representation

    View Slide

  25. YARV vs RTL
    ● YARV-MJIT is no longer slower than RTL-MJIT
    ○ We didn’t need to rewrite the VM for JIT’s performance.

    View Slide

  26. What RTL had
    ● Register-based instructions
    ● Speculative instructions

    View Slide

  27. Speculative instructions
    ● These work like a profiler of runtime information
    ● Alternatively, we could let JIT generate code for profiling
    ○ The current MJIT generates the most speculative code first, and then
    recompile code with some optimizations disabled when cancelled
    ○ YJIT's basic block versioning also profiles type information, etc.

    View Slide

  28. C method inlining
    ● LLVM, MIR
    ● Rewrite everything in Ruby: YARV
    ● TruffleRuby is both

    View Slide

  29. Compilation and Code Generation

    View Slide

  30. Compilation: Sync vs Async
    ● MJIT: Concurrently JIT-compile methods in an MJIT worker thread
    ● YJIT: Ruby threads JIT-compile methods during execution

    View Slide

  31. Compilation speed
    ● MJIT: 50~200ms for a single compile, and minutes for compaction
    ● YJIT, MIR: < 1ms

    View Slide

  32. Code generation
    ● C compiler
    ○ Slow startup
    ● LLVM
    ○ Binary size, build complexity
    ● MIR
    ● YJIT's assembler

    View Slide

  33. Feasible optimizations

    View Slide

  34. JIT code dispatch
    ● call vs jmp (direct threading)
    ○ jmp is hard for MJIT
    ○ But fortunately call seems faster, even in YJIT

    View Slide

  35. Deoptimization
    ● On-stack replacement
    ○ mprotect + SEGV handler
    ○ Code patching
    ● It’s hard for MJIT to manipulate low-level information

    View Slide

  36. Method frame skip
    ● We already have one since Ruby 2.7
    ○ We also supported frame skip of more methods in Ruby 3.0
    ● Next: Lazy method frame push
    ○ This is probably feasible in MJIT as well

    View Slide

  37. Conclusion
    ● JIT's architecture may impact:
    ○ When you can use it
    ○ Warmup speed
    ○ Performance of VM and JIT
    ○ Build and runtime dependencies

    View Slide