The Method JIT Compiler

RubyKaigi 2018
http://rubykaigi.org/2018

Transcript

1. T R E A S U R E D A T A The Method JIT Compiler for Ruby 2.6 Takashi Kokubun / @k0kubun RubyKaigi 2018
2. T R E A S U R E D A T A Maintainer of ERB, Haml Developing JIT compiler for Ruby 2.6 @k0kubun
4. • 2017 Sep: LLVM JIT (EN) • 2017 Nov: YARV MJIT (EN) • 2017 Dec: YARV MJIT (JA) • 2018 Feb: ERB generation (JA) • 2018 Apr: Preview2 optimizations (EN) .Z
   QBTU
   UBMLT
   BCPVU
   3VCZT
   +*5 https://speakerdeck.com/k0kubun
5. 1. Current Status 2. JIT on Rails 3. Dive Into Native Code 4. Method Inlining 5PEBZT
   UBML
6. 1. CURRENT STATUS
7. • JIT in 2.6.0-preview2 is not production ready yet • Fixing bugs by a race condition • I'll introduce current status of: • Implementation • Portability • Performance $VSSFOU
   TUBUVT
8. IMPLEMENTATION
9. .+*5
   3VCZ`T
   +*5
   BSDIJUFDUVSF Ruby Process Disk Memory MJIT worker Thread Method #1 Bytecode Ruby VM Thread Interpret
10. .+*5
    3VCZ`T
    +*5
    BSDIJUFDUVSF Ruby Process Disk Memory MJIT worker Thread Method #1 Bytecode Ruby VM Thread Interpret Request JIT-ing #1
11. .+*5
    3VCZ`T
    +*5
    BSDIJUFDUVSF Ruby Process Disk Memory Method #1 Bytecode Ruby VM Thread Interpret Request JIT-ing #1 Method #1 C code MJIT worker Thread Generate
12. .+*5
    3VCZ`T
    +*5
    BSDIJUFDUVSF Ruby Process Disk Memory Method #1 Bytecode Ruby VM Thread Interpret Request JIT-ing #1 Method #1 C code MJIT worker Thread Generate Method #1 SO file Run C compiler
13. .+*5
    3VCZ`T
    +*5
    BSDIJUFDUVSF Ruby Process Disk Memory Method #1 Bytecode Ruby VM Thread Interpret Request JIT-ing #1 Method #1 C code MJIT worker Thread Generate Method #1 SO file Run C compiler Method #1 Native code Load
14. .+*5
    3VCZ`T
    +*5
    BSDIJUFDUVSF Ruby Process Disk Memory Method #1 Bytecode Interpret Request JIT-ing #1 Method #1 C code MJIT worker Thread Generate Method #1 SO file Run C compiler Method #1 Native code Load Ruby VM Thread Call
15. )PX
    JT
    UIJT
    JNQMFNFOUFE Ruby Process Disk Memory Method #1 Bytecode Interpret Request JIT-ing #1 Method #1 C code MJIT worker Thread Generate Method #1 SO file Run C compiler Method #1 Native code Load Ruby VM Thread Call
16. )PX
    JT
    UIJT
    JNQMFNFOUFE Ruby Code def three 1 + 2 end Bytecode putobject 1 putobject 2 opt_plus leave =
17. )PX
    JT
    UIJT
    JNQMFNFOUFE Ruby Code def three 1 + 2 end Bytecode putobject 1 putobject 2 opt_plus leave = C code three() { VALUE stack[2]; /* putobject 1 */ stack[0] = 1; }
18. )PX
    JT
    UIJT
    JNQMFNFOUFE Ruby Code def three 1 + 2 end Bytecode putobject 1 putobject 2 opt_plus leave = C code three() { VALUE stack[2]; /* putobject 1 */ stack[0] = 1; /* putobject 2 */ stack[1] = 2; }
19. )PX
    JT
    UIJT
    JNQMFNFOUFE Ruby Code def three 1 + 2 end Bytecode putobject 1 putobject 2 opt_plus leave = C code three() { VALUE stack[2]; /* putobject 1 */ stack[0] = 1; /* putobject 2 */ stack[1] = 2; /* opt_plus */ stack[0] = opt_plus( stack[0], stack[1] ); }
20. )PX
    JT
    UIJT
    JNQMFNFOUFE Ruby Code def three 1 + 2 end Bytecode putobject 1 putobject 2 opt_plus leave = C code three() { VALUE stack[2]; /* putobject 1 */ stack[0] = 1; /* putobject 2 */ stack[1] = 2; /* opt_plus */ stack[0] = opt_plus( stack[0], stack[1] ); /* leave */ return stack[0]; }
21. $
    DPEF
    HFOFSBUPS ERB template: mjit_compile.inc.erb
22. $
    DPEF
    HFOFSBUPS ERB template: mjit_compile.inc.erb VM instructions: insns.def
23. $
    DPEF
    HFOFSBUPS ERB template: mjit_compile.inc.erb VM instructions: insns.def C code generator: mjit_compile.inc Render Copy definition
24. • Based on Ruby 2.5’s Ruby VM • If JIT is disabled, everything must work in 2.6 • JIT implementation is automatically generated • To keep up with frequent Ruby VM changes 3VCZ
    T
    +*5
    EFTJHO
25. PORTABILITY
26. $
    DPNQJMFS
    TVQQPSUT GCC Clang Visual C++ Intel C++ Compiler MJIT worker ◦ ◦ ◦ ◦ JIT header ◦ ◦ × ◦ CLI support ◦ ◦ ◦ × Support plan Done Done Next Later Now MJIT worker (native thread, dynamic loading) runs on Windows and UNIX
27. 1MBUGPSN
    TVQQPSUT
    XJUI
    ($$ Linux MinGW Solaris NetBSD FreeBSD JIT header ◦ ˚ ◦ ◦ ◦ test_jit.rb ◦ ◦ ◦ ? × MinGW header is not minimized and thus compilation speed is slow. I guess NetBSD works but we don’t have NetBSD RubyCI. GCC on FreeBSD is crashing.
28. 1MBUGPSN
    TVQQPSUT
    XJUI
    $MBOH Linux macOS OpenBSD JIT header ◦ ◦ ◦ test_jit.rb ◦ ◦ ? I guess OpenBSD works but we don’t have OpenBSD RubyCI
29. PERFORMANCE
30. 
    3VCZ,BJHJ
    -5
    --7.
    +*5
31. 
    3VCZ,BJHJ
    -5
    --7.
    +*5
32. 
    3VCZ,BJHJ
    5IJT
    UBML https://benchmark-driver.github.io/benchmarks/mjit/commits.html
33. 
    3VCZ,BJHJ
    5IJT
    UBML https://benchmark-driver.github.io/benchmarks/mjit/commits.html 2.6.0 Preview1 2.6.0 Preview2 5.7x faster
34. 0QUDBSSPU GQT      Ruby 2.0 trunk trunk+JIT    1.49x → 2.03x https://gist.github.com/k0kubun/95c81358af6f34b4d0a71425da871178
35. 3BJMT
    %JTDPVSTF
36. 3BJMT
    %JTDPVSTF
37. 2. JIT ON RAILS
38. • Generated code should be faster in general • What's different from Optcarrot? 8IZ
    3BJMT
    CFDPNFT
    TMPX
    XJUI
    +*5
39. 1. longjmp by exception is slow 2. Profiling method calls has overhead 3. JIT-ed call is canceled too often 4. JIT compilation has overhead 5. Calling JIT-ed code has overhead .Z
    IZQPUIFTJT
40. • When a method is returned from its child block, it calls longjmp(3) • VM is implemented with just return statement and may be faster in that case 
    MPOHKNQ
    CZ
    FYDFQUJPO
    JT
    TMPX
41. -FUT
    DIFDL
    JG
    MPOHKNQ
    JT
    DBMMFE • Fortunately, longjmp was not used in this Discourse endpoint
42. • MJIT counts method calls to decide which method to compile with JIT enabled • This was suspected in [Bug #14490] 
    1SPGJMJOH
    NFUIPE
    DBMMT
    IBT
    PWFSIFBE
43. -FUT
    DPVOU
    JU
    FWFO
    JG
    +*5
    JT
    EJTBCMFE
44. /P
    CJH
    EJGGFSFODF
    CZ
    QSPGJMJOH
    NFUIPE
    DBMMT trunk No options modified No options trunk --jit JIT × × ◦ Profiling × ◦ ◦ Percentile: ms GET /: 50: 58.4ms 75: 65.4ms 90: 67.9ms 99: 131.1ms GET /: 50: 58.5ms 75: 64.6ms 90: 67.8ms 99: 127.3ms GET /: 50: 66.3ms 75: 72.3ms 90: 77.0ms 99: 133.3ms `ruby script/simple_bench.rb 1000` with: https://github.com/k0kubun/discourse/tree/20fc03558f16aff94c6c017347783374cf4a0ca8
45. • MJIT has a kind of de-optimization to fallback to VM interpretation when any assumption is not met • ex) Method redefinition, etc. • Such fallback might be an overhead 
    +*5FE
    DBMM
    JT
    DBODFMMFE
    UPP
    PGUFO
46. -FUT
    MPH
    BMM
    +*5
    DBODFMMBUJPO
47. 5IF
    SBUJP
    PG
    +*5
    DBODFMMBUJPO JIT-ed calls Cancel by opt_xxx Cancel by call cache Optcarrot 49,171,765 786,842 (1.60%) 0 (0.00%) Discourse 1,000 requests 168,925,050 19,394,792 (11.5%) 10,092,254 (5.97%) JIT cancel reasons: • opt_xxx: Non-core class is given to +, -, *, /, #[], etc. • call cache: Method redefinition, receiver class is changed
48. 8IZ
    +*5
    DBODFM
    IBQQFOT
    TP
    PGUFO • Current JIT doesn't discard any JIT-ed code whose assumption is not met • opt_xxx is performing badly when a receiver is not a core class like Integer, Float, String, Array, Hash
49. 8IZ
    +*5
    DBODFM
    IBQQFOT
    TP
    PGUFO • Current JIT doesn't discard any JIT-ed code whose assumption is not met • opt_xxx is performing badly when a receiver is not a core class like Integer, Float, String, Array, Hash There are many #[] for non Hash/Array classes in Rails
50. *
    GJYFE
    UIJT
    JTTVF
    GPS
    <>
    S
51. PQU@YYY
    DBODFM
    JT
    EFDSFBTFE
    NVDI JIT-ed calls Cancel by opt_xxx Cancel by call cache Discourse Before 168,925,050 19,394,792 (11.5%) 10,092,254 (5.97%) Discourse After 75,150,482 2,849,825 (3.79%) 3,072,673 (4.09%) #[] has a major impact on Rails. Others are to be improved...
52. • Appending a method to JIT-ed queue may have overhead • GCC or Clang may use the same CPU core, or it may cost to transfer data to another core 
    +*5
    DPNQJMBUJPO
    IBT
    PWFSIFBE
53. 1SFQBSF
    3VCZ7..+*5TUPQ
    BOE Stop JIT compilation
54. +*5
    FOBCMFE
    WT
    +*5
    TUPQQFE JIT enabled 1000 requests JIT enabled 1000 requests JIT stopped 1000 requests RubyVM::MJIT.stop Measure
55. +*5
    DPNQJMBUJPO
    IBE
    PWFSIFBE No options --jit → Stop --jit Code is JIT-ed × ◦ ◦ JIT is going on × × ◦ Percentile: ms GET /: 50: 60.4ms 75: 66.9ms 90: 69.6ms 99: 125.4ms GET /: 50: 65.1ms 75: 72.4ms 90: 75.8ms 99: 145.6ms GET /: 50: 68.4ms 75: 74.8ms 90: 80.0ms 99: 137.2ms But this overhead is excluded from [Bug #14490] degradation…
56. • JIT-ed code behaves slower only on an exception or JIT cancellation, but they weren’t culprit • JIT compilation does not dominate the slowness • Then, calling native code has overhead…? 
    $BMMJOH
    +*5FE
    DPEF
    IBT
    PWFSIFBE
57. -FU`T
    DIFDL
    +*5FE
    DBMM
    PWFSIFBE
58. -FU`T
    DIFDL
    +*5FE
    DBMM
    PWFSIFBE
59. $BMMJOH
    +*5FE
    DPEF
    XBT
    TMPX JIT disabled JIT enabled Duration 2.17s 2.45s
60. -FU`T
    QSPGJMF
    XJUI
    QFSG
61. (VBSE
    GPS
    KJUXBJU
    UBLFT
    UJNF
62. 4LJQ
    KJUXBJU
    DIFDL
    JO
    NBJO
    CSBODI r63480
63. *NQSPWFE
    B
    MJUUMF JIT disabled JIT enabled Duration 2.17s 2.31s (-0.14s)
64. 3FNBJOJOH
    T
    XBT
    GPS Additional memory access here …But it wasn’t a big deal in Rails
65. 8IBU
    JG
    UIFSF
    BSF
    B
    MPU
    PG
    NFUIPET
66. $BMMJOH
    NBOZ
    EJGGFSFOU
    NFUIPET
    JT
    TMPX
    Called methods 1 method 15 methods JIT disabled 3.69s 3.71s JIT enabled 3.79s 5.34s Duration with the same total calls
67. $BMMJOH
    NBOZ
    EJGGFSFOU
    NFUIPET
    JT
    TMPX
    0 1.5 3 4.5 6 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 VM JIT
68. $BMMJOH
    NBOZ
    EJGGFSFOU
    NFUIPET
    JT
    TMPX
    0 1.5 3 4.5 6 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 VM JIT 6 12 19
69. )PU
    NFUIPET
    PG
    0QUDBSSPU Top 6 methods dominate 50%
70. )PU
    NFUIPET
    PG
    %JTDPVSTF Top 6 methods are only 18% They are not so hot.
71. 8IZ
    EPFT
    UIJT
    IBQQFO 0 1.5 3 4.5 6 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 VM JIT 6 12 19
72. -FU`T
    TFF
    lQFSG
    TUBUz 6 methods 40 methods
73. lJOTO
    QFS
    DZDMFz
    JT
    WFSZ
    EJGGFSFOU 6 methods 40 methods
74. Y
    DZDMFT
    GPS
    BMNPTU
    UIF
    TBNF
    JOTOT 6 methods 40 methods
75. 6 methods
76. 40 methods
77. &BDI
    NFUIPE
    TP
    GJMF
    JT
    VTJOH
    POF
    QBHF
    .#
78. 40 methods w/ the same so file PoC
79. • Ongoing JIT compilation may have overhead • JIT cancel is happening frequently (to be fixed) • It stalls to load many different methods (to be fixed) 3FBTPO
    PG
    3BJMT
    TMPXEPXO
    PO
    +*5
80. 3. DIVE INTO NATIVE CODE
81. &YBNQMF
    
    
     Ruby Code def three 1 + 2 end Bytecode putobject 1 putobject 2 opt_plus leave =
82. &YBNQMF
    
    
     Ruby Code def three 1 + 2 end Bytecode putobject 1 putobject 2 opt_plus leave = C code three() { VALUE stack[2]; /* putobject 1 */ stack[0] = 1; /* putobject 2 */ stack[1] = 2; /* opt_plus */ stack[0] = opt_plus( stack[0], stack[1] ); /* leave */ return stack[0]; }
83. More detailed definition before inlining opt_plus
84. opt_plus inlined
85. Native code generated by GCC (output of perf)
86. Integer#+ redefinition check
87. JIT cancel handler Let's ignore this Integer#+ redefinition check
88. Integer#+ redefinition check
89. SET_SP: VM's behavior which can be removed Integer#+ redefinition check
90. Check interrupts like SIGINT, another thread Interruption handler Integer#+ redefinition check SET_SP: VM's behavior which can be removed
91. Interruption handler Check interrupts like SIGINT, another thread Pop VM call frame Integer#+ redefinition check SET_SP: VM's behavior which can be removed
92. Pop VM call frame Interruption handler Check interrupts like SIGINT, another thread Return 3 FIX2INT(0x7) == 3 Integer#+ redefinition check SET_SP: VM's behavior which can be removed
93. So what?
94. Instruction dispatch Instruction dispatch Instruction dispatch Instruction dispatch 1. Instruction dispatch cost is removed
95. Program counter motion Program counter motion Program counter motion Program counter motion 2. No program counter motion
96. Stack pointer motion Stack pointer motion Stack pointer motion Forgot to remove this 3. Stack pointer motion is reduced
97. And also...
98. 4. This optimization is delegated to GCC
99. 8IBU
    JG
    JUT
    NPSF
    DPNQMFY def six 4 + 8 - 3 * 4 / 2 end
101. Fixnum#+ redefinition check Fixnum#* redefinition check Fixnum#/ redefinition check Fixnum#- redefinition check Return 6 FIX2INT(0xd) == 6
102. -BTU
     &YBNQMF
     XIJMF
     MPPQ def while_loop i = 0 while i < 1000000 i += 1 end end i = 0 while i < 2000 while_loop i += 1 end
103. -BTU
     &YBNQMF
     XIJMF
     MPPQ VM: 22.9s JIT: 2.8s Why it becomes so faster? 8.18x faster
104. -FUT
     TFF
     UIF
     OBUJWF
     DPEF
     PG
     XIJMF@MPPQ i = 0 while i < 1000000 i += 1 end
106. j They are JIT cancel handlers Let's ignore them
108. They are interruption handlers Let's ignore them
110. They are write-barrier-related slow paths Let's ignore them
112. It is Bignum promotion handler Let's ignore them
115. "HBJO
     MFUT
     TFF
     UIF
     OBUJWF
     DPEF
     PG i = 0 while i < 1000000 i += 1 end
116. c i = 0
117. i = 0 check interrupts c
118. i = 0 c Fixnum?(i) for #< check interrupts
119. i = 0 c check interrupts Fixnum#< redefined? Fixnum?(i) for #<
120. i = 0 c check interrupts i < 1000000 Fixnum?(i) for #< Fixnum#< redefined?
121. i = 0 c i < 1000000 check interrupts check interrupts Fixnum?(i) for #< Fixnum#< redefined?
122. i = 0 c i < 1000000 check interrupts check interrupts Fixnum#+ redefined? Fixnum?(i) for #< Fixnum#< redefined?
123. i = 0 c i < 1000000 check interrupts check interrupts i + 1 Fixnum?(i) for #< Fixnum#< redefined? Fixnum#+ redefined?
124. i = 0 c i < 1000000 check interrupts check interrupts Int overflow? i + 1 Fixnum?(i) for #< Fixnum#< redefined? Fixnum#+ redefined?
125. i = 0 c i < 1000000 check interrupts check interrupts Int overflow? can't optimize #+ ? i + 1 Fixnum?(i) for #< Fixnum#< redefined? Fixnum#+ redefined?
126. i = 0 c i < 1000000 check interrupts check interrupts Int overflow? can't optimize #+ ? i + 1 Fixnum?(i) for #< Fixnum#< redefined? Fixnum#+ redefined? set i for VM + check WB
127. i = 0 c i < 1000000 check interrupts check interrupts Int overflow? can't optimize #+ ? i + 1 Fixnum?(i) for #< Fixnum#< redefined? Fixnum#+ redefined? set i for VM + check WB set i for JIT
128. i = 0 c i < 1000000 check interrupts check interrupts Int overflow? i + 1 Fixnum?(i) for #< Fixnum#< redefined? Fixnum#+ redefined? can't optimize #+ ? set i for VM + check WB set i for JIT
129. • #+ and #< are performed on not VM stack but registers • #+ and #< share some instructions to check redefinition • Unnecessary type checks are omitted from the loop 8IZ
     XIJMF
     MPPQ
     CFDPNFT
     GBTUFS
130. 4. METHOD INLINING
131. • Many optimizations are possible because C compiler can know definitions • If we could inline methods, C compiler would be able to optimize more -FU
     $
     DPNQJMFS
     XPSL
     IBSE
132. 1. x 2. x 3. x 8IFO
     JT
     NFUIPE
     JOMJOJOH
     QPTTJCMF
133. 1. JIT compiler can know definitions 2. JIT compiler can modify code to call a method 3. Inlined code can be invalidated 8IFO
     JT
     NFUIPE
     JOMJOJOH
     QPTTJCMF
134. 1. JIT compiler can know definitions 2. JIT compiler can modify code to call a method 3. Inlined code can be invalidated 8IFO
     JT
     NFUIPE
     JOMJOJOH
     QPTTJCMF
135. 1. JIT compiler can know definitions 2. JIT compiler can modify code to call a method 3. Inlined code can be invalidated 8IFO
     JT
     NFUIPE
     JOMJOJOH
     QPTTJCMF
136. • Ruby method • called by Ruby method • called by C method • Ruby block • yield-ed by Ruby method • called by C method • C method • called by Ruby method • called by C method .BKPS
     JOMJOF
     UBSHFUT
137. • Ruby method • called by Ruby method => easy • called by C method • Ruby block • yield-ed by Ruby method • called by C method • C method • called by Ruby method • called by C method .BKPS
     JOMJOF
     UBSHFUT JIT compiler can deal with bytecode easily Method cache can be used for invalidation
138. • Ruby method • called by Ruby method => easy • called by C method • Ruby block • yield-ed by Ruby method => medium • called by C method • C method • called by Ruby method => medium • called by C method .BKPS
     JOMJOF
     UBSHFUT yield doesn't have cache Sometimes it's hard to know definitions
139. • Ruby method • called by Ruby method => easy • called by C method => hard • Ruby block • yield-ed by Ruby method => medium • called by C method => hard • C method • called by Ruby method => medium • called by C method => hard .BKPS
     JOMJOF
     UBSHFUT There is no cache key for invalidation How to modify C code?
140. 3VCZ
     ˠ
     $
     ˠ
     3VCZ
     JOMJOJOH
     QSPCMFN ret = 0 1000000.times do |i| ret += i end ret
141. ret = 0 1000000.times do |i| ret += i end ret Ruby -> C method call medium Integer#times is defined with C 3VCZ
     ˠ
     $
     ˠ
     3VCZ
     JOMJOJOH
     QSPCMFN
142. ret = 0 1000000.times do |i| ret += i end ret Ruby -> C method call medium Integer#times is defined with C C -> Ruby block call hard 3VCZ
     ˠ
     $
     ˠ
     3VCZ
     JOMJOJOH
     QSPCMFN
143. What if... Ruby can be faster than C?
144. What if... Ruby can be faster than C?
145. -FUT
     EFGJOF
     *OUFHFSUJNFT
     XJUI
     3VCZ https://github.com/rubinius/rubinius/blob/master/core/integer.rb
146. • Ruby method • called by Ruby method => easy • called by C method => hard • Ruby block • yield-ed by Ruby method => medium • called by C method => hard • C method • called by Ruby method => medium • called by C method => hard *
     JNQMFNFOUFE
     B
     QSPUPUZQF
     UP
     JOMJOF
     UIJT https://github.com/k0kubun/ruby/commits/mjit-inline-send-yield
147. 5JNF
     UP
     CFODINBSL
148. *OUFHFSUJNFT
     CFODINBSL
     SFTVMUT Integer#times in C Integer#times in Ruby VM 145.44s 1.00x 156.38s 0.93x JIT time ruby --disable-gems times_loop.rb
149. *OUFHFSUJNFT
     CFODINBSL
     SFTVMUT Integer#times in C Integer#times in Ruby VM 145.44s 1.00x 156.38s 0.93x JIT 104.80s 1.39x time ruby --disable-gems times_loop.rb
150. *OUFHFSUJNFT
     CFODINBSL
     SFTVMUT Integer#times in C Integer#times in Ruby VM 145.44s 1.00x 156.38s 0.93x JIT 104.80s 1.39x 56.46s 2.56x time ruby --disable-gems times_loop.rb
151. C language is dead
152. • Rails performance is going to be improved • JIT can eliminate many instructions • C language will be useless in the future $PODMVTJPO