Ruby 2.6 JIT / RubyConf 2018

RubyConf 2018 Los Angeles

Transcript

1. Takashi Kokubun
   @k0kubun
   Ruby 2.6
   JIT
   

   View Slide

2. @k0kubun
   Takashi Kokubun / Arm
   Ruby Committer for JIT, ERB
   

   View Slide

3. View Slide

4. View Slide

5. View Slide

6. View Slide

7. Ruby 2.6
   

   View Slide

8. View Slide

9. What's "JIT"?
   

   View Slide

10. JIT: Just In Time Compiler
    

    View Slide

11. View Slide

12. History of Ruby implementation
    

    View Slide

13. def plus(a, b)
    a + b
    end
    +
    a b
    Parse
    Ruby 1.8
    Traverse
    

    View Slide

14. def plus(a, b)
    a + b
    end
    +
    a b
    Compile
    Ruby 1.9~2.5
    Interpret
    getlocal :a
    getlocal :b
    send :+
    

    View Slide

15. def plus(a, b)
    a + b
    end
    +
    a b
    JIT
    Ruby 2.6 (New!)
    Execute
    getlocal :a
    getlocal :b
    send :+
    mov %rdi,%rdx
    mov %rsi,%rax
    add %rdx,%rax
    

    View Slide

16. How can we use it?
    

    View Slide

17. or
    ruby --jit ...
    RUBYOPT="--jit" rails s ...
    

    View Slide

18. Let's benchmark Ruby 2.6 JIT
    

    View Slide

19. View Slide

20. https://gist.github.com/k0kubun/887b9567afa86b326556389ef00e4200
    

    View Slide

21. https://gist.github.com/k0kubun/887b9567afa86b326556389ef00e4200
    Ruby 2.0: 34.3
    Ruby 2.6: 86.7
    2.53x faster
    

    View Slide

22. https://gist.github.com/k0kubun/887b9567afa86b326556389ef00e4200
    Ruby 2.5: 48.3
    Ruby 2.6: 86.7
    1.80x faster
    

    View Slide

23. Achieved Ruby 3x2.5 !!!
    

    View Slide

24. How about other benchmarks?
    

    View Slide

25. View Slide

26. View Slide

27. JIT makes things slower?
    

    View Slide

28. Today's topic
    JIT performance characteristics
    

    View Slide

29. When does Ruby become slow on JIT?
    

    View Slide

30. 1. When there are many JIT-ed methods
    

    View Slide

31. --jit-max-cache (default: 1000)
    

    View Slide

32. JIT by C compiler & dynamic loading
    

    View Slide

33. unused space
    code
    readonly data
    1 method
    Heap
    

    View Slide

34. unused space
    code
    readonly data
    1 method
    Heap
    2MB
    (big)
    about 4KB
    ?KB
    

    View Slide

35. unused space
    code
    readonly data
    unused space
    code
    readonly data
    unused space
    code
    readonly data
    3 methods
    Heap
    

    View Slide

36. unused space
    code
    readonly data
    unused space
    code
    readonly data
    unused space
    code
    readonly data
    L2, ...
    cache
    L1
    cache
    Heap
    

    View Slide

37. unused space
    code
    readonly data
    unused space
    code
    readonly data
    unused space
    code
    readonly data
    L2, ...
    cache
    L1
    cache
    Heap
    

    View Slide

38. unused space
    Heap
    code
    readonly data
    unused space
    code
    readonly data
    unused space
    code
    readonly data
    L2, ...
    cache
    L1
    cache
    

    View Slide

39. 2. When there are still methods to be JIT-ed
    

    View Slide

40. "JIT compaction"
    

    View Slide

41. unused space
    code
    readonly data
    unused space
    code
    readonly data
    unused space
    code
    readonly data
    When it reaches --jit-max-cache,
    it fires "JIT compaction"
    

    View Slide

42. unused space
    code
    readonly data
    unused space
    code
    readonly data
    unused space
    code
    readonly data
    unused space
    code
    code
    code
    readonly data
    readonly data
    readonly data
    

    View Slide

43. unused space
    code
    readonly data
    unused space
    code
    readonly data
    unused space
    code
    readonly data
    unused space
    code
    code
    code
    readonly data
    readonly data
    readonly data
    L2, ...
    cache
    

    View Slide

44. CPU/memory resources
    for C compiler
    

    View Slide

45. Locks on GC, waitpid, ...
    

    View Slide

46. 3. When TracePoint is enabled
    (to be fixed after Ruby 2.6)
    

    View Slide

47. What's TracePoint? When is it enabled?
    • gem 'web-console' on your Rails application
    • byebug or binding.pry (with pry-byebug)
    • Measuring coverage
    

    View Slide

48. Summary: When JIT makes Ruby slow
    Optcarrot Rails
    1. When there are many JIT-ed methods ✓
    2. When there are still methods to be JIT-ed
    
    (Is it a short benchmark?)
    ✓
    3. When TracePoint is enabled
    

    View Slide

49. What is made faster by JIT?
    

    View Slide

50. 1. Almost all methods
    

    View Slide

51. def plus(a, b)
    a + b
    end
    Virtual Machine
    getlocal :a
    getlocal :b
    send :+
    Program Counter
    Stack Pointer
    Computer
    Registers
    Instruction Pointer
    

    View Slide

52. def plus(a, b)
    a + b
    end
    Virtual Machine
    getlocal :a
    getlocal :b
    send :+
    Program Counter
    Stack Pointer
    Computer
    JIT
    Instruction Pointer
    Registers
    mov %rdi,%rdx
    mov %rsi,%rax
    add %rdx,%rax
    

    View Slide

53. 2. Basic operators on core classes
    

    View Slide

54. VM-optimized methods
    Integer, Float +, -, *, /, %
    Array +, <<, [], []=, empty?, size, length, min, max
    Hash [], []=, empty?, size, length
    String +, <<, =~, empty?, size, length, succ
    common !, !=, ==, <, >, <=, >=
    

    View Slide

55. def three
    1 + 2
    end
    putobject 1
    putobject 2
    send :+
    JIT
    mov $0x3,%eax
    retq
    Method inlining on JIT
    

    View Slide

56. 3. Calling Ruby method
    

    View Slide

57. Method dispatch on Ruby VM
    foo.bar
    

    View Slide

58. Method dispatch on Ruby VM
    foo.bar
    Method search
    (slow)
    

    View Slide

59. Method dispatch on Ruby VM
    foo.bar
    Method search
    (slow)
    Ruby
    method
    C function
    attr_reader
    alias
    Method entry
    Foo#bar
    

    View Slide

60. Method dispatch on Ruby VM
    foo.bar
    Method search
    (slow)
    Verify cache
    Ruby
    method
    C function
    attr_reader
    alias
    Method entry
    Foo#bar
    has cache
    

    View Slide

61. Method dispatch on Ruby VM
    foo.bar
    Method search
    (slow)
    Verify cache
    Ruby
    method
    C function
    attr_reader
    alias
    Method entry
    Foo#bar
    has cache
    redefined?
    

    View Slide

62. Method dispatch on Ruby VM
    foo.bar
    Method search
    (slow)
    Verify cache
    Ruby
    method
    C function
    attr_reader
    alias
    Method entry
    Foo#bar
    has cache
    redefined?
    

    View Slide

63. Method dispatch on JIT
    foo.bar
    Verify cache
    Ruby
    method
    Method entry
    Foo#bar
    has cache
    Cancel JIT!
    Less branches
    Less memory access
    redefined?
    some
    inlining
    

    View Slide

64. 4. Instance variable access
    

    View Slide

65. Reading instance variable on VM
    @foo
    

    View Slide

66. Reading instance variable on VM
    @foo
    Search index
    (many branches, slow)
    

    View Slide

67. Reading instance variable on VM
    @foo
    Search index
    (many branches, slow)
    self.class.
    instance_variables[index = 2]
    

    View Slide

68. Reading instance variable on VM
    Search index
    (many branches, slow)
    self.class.
    instance_variables[index = 2]
    @foo
    has cache
    Verify cache
    

    View Slide

69. Reading instance variable on VM
    Search index
    (many branches, slow)
    self.class.
    instance_variables[index = 2]
    @foo different
    class?
    has cache
    Verify cache
    

    View Slide

70. Reading instance variable on VM
    Search index
    (many branches, slow)
    self.class.
    instance_variables[index = 2]
    @foo different
    class?
    has cache
    Verify cache
    

    View Slide

71. Reading instance variable on JIT
    self.class.
    instance_variables[2]
    @foo different
    class?
    has cache
    Verify cache
    Cancel JIT!
    inlined index
    Less branches
    Less memory access
    

    View Slide

72. Summary: When JIT makes Ruby fast
    Optcarrot Rails
    1. Almost all methods ✓ ✓
    2. Basic operators on core classes ✓
    3. Calling Ruby method ✓ ✓
    4. Instance variable access ✓ ?
    

    View Slide

73. Future of Ruby's JIT
    

    View Slide

74. Will Ruby 2.6 be fast only on Optcarrot?
    

    View Slide

75. Future idea (not for 2.6)
    • Stack allocation of objects
    - That requires "escape analysis" (not easy)
    - We haven't estimated its possible impact yet
    

    View Slide

76. Some rooms for improvements in 2.6 (!?)
    • Change heuristics to trigger/compact JIT
    • Profile-guided optimization
    • Method inlining
    

    View Slide

77. How can we experiment them?
    Benchmark!
    

    View Slide

78. View Slide

79. View Slide

80. Create a benchmark for Ruby 3x3?
    

    View Slide

81. benchmark_driver.gem
    

    View Slide

82. View Slide

83. View Slide

84. View Slide

85. View Slide

86. Conclusion
    • Ruby 2.6.0-preview3 JIT is still early days
    - Small --jit-max-cache might be an option for now
    • We still have chance to make Ruby 2.6.0 better
    - Benchmarks are wanted!!!
    

    View Slide