Ruby 3 JIT's roadmap / RubyConf China 2020

RubyConf China 2020 http://www.rubyconfchina.org/

Transcript

1. Ruby 3 JIT's roadmap RubyConf China 2020 Takashi Kokubun / @k0kubun
2. Self introduction • GitHub, Twitter: @k0kubun • Treasure Data, Inc. • Ruby committer • JIT: 2017~2020
3. GitHub Sponsors - Thank you!
4. Agenda 1. What can Ruby's JIT do? 2. Ruby 3 JIT's roadmap 3. Recent progress in Ruby 3 4. Current challenges
5. 1. What can Ruby's JIT do?
6. Ruby JIT's architecture 3VCZ
   JOUFSQSFUFS +*5
   UISFBE 3VCZ
   UISFBE
7. Ruby JIT's architecture 3VCZ
   JOUFSQSFUFS +*5
   UISFBE 3VCZ
   UISFBE .FUIPE D
   pMF
8. Ruby JIT's architecture 3VCZ
   JOUFSQSFUFS +*5
   UISFBE 3VCZ
   UISFBE .FUIPE TP
   pMF $
   DPNQJMFS D
   pMF 3VO
9. Ruby JIT's architecture 3VCZ
   JOUFSQSFUFS +*5
   UISFBE 3VCZ
   UISFBE .FUIPE TP
   pMF $
   DPNQJMFS D
   pMF 3VO -PBE
   BOE
   DBMM
10. What we can do with Ruby's JIT • Optimize Ruby methods to native code for hot spots • Eliminate VM interpretation cost: SP / PC • Optimize based on what C compiler can know • Ruby VM-specific optimizations we implemented
11. What we CAN'T do with Ruby's JIT • Optimize a short-running program • JIT needs time to optimize many methods • Things may be slower while a C compiler is running • Optimization based on native code generated by C compiler • Deoptimization based on native insn pointer / stack pointer
12. Use case: Obviate micro optimizations 4MPXFS 'BTUFS
13. Use case: Obviate micro optimizations
14. Use case: Obviate micro optimizations OVN[FSP OVN
    
     JUFSBUJPO
    
    TFD
    3VCZ
                      7. +*5
15. 2. Ruby 3 JIT's roadmap
16. Ruby JIT's goals • Optcarrot: 3x faster than Ruby 2.0 • Sinatra, Rails: 10% throughput increase vs VM
17. mame/optcarrot 3VCZ
     3VCZ
     3VCZ
     GSBNFT
    
    TFD            7. +*5
18. benchmark-driver/sinatra 3VCZ
     3VCZ
     3VCZ
     SFRVFTUT
    
    TFD                  7. +*5
19. k0kubun/railsbench 3VCZ
     3VCZ
     3VCZ
     SFRVFTUT
    
    TFD            7. +*5
20. What should we do? • Current status: • Programs like Optcarrot run faster • Sinatra, Rails are still slightly slower than no-JIT mode • Let’s take a look at each of major Ruby features and JIT core
21. Ruby 3 JIT's roadmap 1. Variables / Constants 2. Method inlining 3. Constant folding 4. Object allocation 5. Deoptimization 6. Scalability
22. 1. Variables / Constants • Local variables: ⚠ • Instance variables: ✅ • Global variables: ❌ • Constants: ❌
23. 2. Method inlining • Ruby method: ✅ • C method: ✅ • super: ⚠ • yield: ⚠
24. 3. Constant folding • VM-optimized instructions: ⚠ • C method: ⚠
25. 4. Object allocation • Stack allocation: ⚠ • Static allocation: ❌
26. 5. Deoptimization • Reduce safepoints: ✅ • Zero-cost deoptimization: ❌
27. 6. Scalability • Single-page code: ✅ • Code size reduction: ⚠ • JIT dispatch cost: ⚠
28. 3. Recent progress in Ruby 3
29. Decrease ICache misses • ICache: Instruction Cache • Sinatra / Rails spends a lot of time on ICache misses • And the amount is increased by JIT
30. • VTune: VM, JIT VTune: mame/optcarrot - VM
31. • VTune: VM, JIT VTune: mame/optcarrot - JIT
32. VTune: benchmark-driver/sinatra - VM
33. VTune: benchmark-driver/sinatra - JIT
34. Decrease ICache misses • We implemented: • Deduplication of the same code • Hot / cold partitioning
35. Decrease ICache misses
36. Merge type checks on ivar access • Instance variable index is class-specific • We can check class only once per method
37. Merge type checks on ivar access 30CKFDU qBHT
    FNCFE
    GBMTF OVNJW JWQUS
    ˠ
    IFBQ JW@JOEFY@UCM 30CKFDU qBHT
    FNCFE
    USVF BSZ<> BSZ<> BSZ<>
38. Merge type checks on ivar access 3VCZ
     3VCZ
     GSBNFT
    
    TFD          7. +*5
39. Inline C method call • Inlining C method had been hard because of: • Difficulty of detecting whether it’s safe to omit a call frame or not • Lots of indirection between method call and actual C function
40. Inline C method call • We introduced a new type of method definition in CRuby core, called “builtin method”
41. Inline C method call • We also added a way to “annotate” a C function in “builtin method” • Now we can say it’s safe to inline a C function
42. Inline C method call ,FSOFMDMBTT JT      7. +*5 +*5
    JOMJOJOH 1.7x 1.3x
43. Inline C method call Y
    JT
    'JYOVN
    
44. 4. Current challenges
45. Allow exception on "inline" method • A method which raises an exception can't have “inline" • But many methods raise: • TypeError • NoMemoryError • We could lazily update backtrace and others?
46. Optimize VM -> JIT call • VM -> JIT call is slower than VM -> VM call • We might be able to offset icache miss's slowness by this • Prepare a fastpath / VM insn specialized for JIT call
47. Optimize VM -> JIT call def foo3 nil end def foo2 nil end def foo1 nil end Time to call a method returning nil (ns) 0 8 16 24 32 Number of called methods 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 VM JIT
48. Improve inlining decision • Rails has polymorphic methods • If inlined by a specific caller, class can be specific for each caller
49. Improve inlining decision
50. … and many more things in the roadmap • Inline `super` and `yield` • Optimize local variables • Optimize constants • …
51. Summary • We reviewed Ruby 3 JIT's roadmap and what we've implemented. • While it's not useful for Rails yet, we've had progress towards that.