Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Pre-evaluation in Ruby

Pre-evaluation in Ruby

Ruby is historically difficult to optimize due to features that improve flexibility and productivity at the cost of performance. Techniques like Ruby's new JIT compiler and deoptimization code help, but still are limited by techniques like monkey-patching and binding inspection.

Pre-evaluation is another optimization technique that works based on user-defined contracts and assumptions. Users can opt in to optimizations by limiting their use of Ruby's features and thereby allowing further compiler work.

In this talk we'll look at how pre-evaluation works, and what benefits it enables.

Kevin Newton

April 30, 2019
Tweet

More Decks by Kevin Newton

Other Decks in Programming

Transcript

  1. Matz is nice so we are nice. Lexical Analytics •

    Semantic Analysis • Instruction Generation • Optimization
  2. Matz is nice so we are nice . Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  3. is nice so we are nice . Matz Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  4. is nice so we are nice . Matz Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  5. Matz is nice we are nice . so Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  6. Matz is nice so we are nice . Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  7. Matz is nice so we are nice . Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  8. Matz is nice so we are . nice Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  9. Matz is nice so we are nice . Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  10. Matz is nice so we are nice . Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  11. Matz is nice so we are nice . Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  12. is nice are nice Lexical Analytics • Semantic Analysis •

    Instruction Generation • Optimization
  13. is nice are nice Lexical Analytics • Semantic Analysis •

    Instruction Generation • Optimization
  14. is nice verb phrase verb phrase are nice Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  15. Matz is nice verb phrase we verb phrase are nice

    Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  16. subject-phrase : NOUN verb-phrase verb-phrase : VERB ADJECTIVE Grammar Lexical

    Analytics • Semantic Analysis • Instruction Generation • Optimization
  17. Matz is nice verb phrase we verb phrase are nice

    Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  18. Matz is nice subject phrase subject phrase verb phrase we

    verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  19. Matz is nice so subject phrase subject phrase verb phrase

    we verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  20. subject-phrase : NOUN verb-phrase verb-phrase : VERB ADJECTIVE Grammar Lexical

    Analytics • Semantic Analysis • Instruction Generation • Optimization
  21. subordinating-conjunction :
 subject-phrase CONJUNCTION subject-phrase subject-phrase : NOUN verb-phrase verb-phrase

    : VERB ADJECTIVE Grammar Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  22. Matz is nice so subject phrase subject phrase verb phrase

    we verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  23. Matz is nice so subord. conjunction subject phrase subject phrase

    verb phrase we verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  24. Matz is nice so . subord. conjunction subject phrase subject

    phrase verb phrase we verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  25. subordinating-conjunction :
 subject-phrase CONJUNCTION subject-phrase subject-phrase : NOUN verb-phrase verb-phrase

    : VERB ADJECTIVE Grammar Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  26. sentence : subordinating-conjunction PERIOD subordinating-conjunction :
 subject-phrase CONJUNCTION subject-phrase subject-phrase

    : NOUN verb-phrase verb-phrase : VERB ADJECTIVE Grammar Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  27. Matz is nice so . subord. conjunction subject phrase subject

    phrase verb phrase we verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  28. Matz is nice so . sentence subord. conjunction subject phrase

    subject phrase verb phrase we verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  29. sentence : subordinating-conjunction PERIOD subordinating-conjunction :
 subject-phrase CONJUNCTION subject-phrase subject-phrase

    : NOUN verb-phrase verb-phrase : VERB ADJECTIVE Grammar Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  30. class Calcp prechigh nonassoc UMINUS left '*' '/' left '+'

    '-' preclow rule target: exp | /* none */ { result = 0 } exp: exp '+' exp { result += val[2] } | exp '-' exp { result -= val[2] } | exp '*' exp { result *= val[2] } | exp '/' exp { result /= val[2] } | '(' exp ')' { result = val[1] } | '-' NUMBER =UMINUS { result = -val[1] } | NUMBER end https://github.com/tenderlove/racc/blob/master/sample/calc.y Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  31. | arg '+' arg { $$ = call_bin_op(p, $1, '+',

    $3, &@2, &@$); } | arg '-' arg { $$ = call_bin_op(p, $1, '-', $3, &@2, &@$); } | arg '*' arg { $$ = call_bin_op(p, $1, '*', $3, &@2, &@$); } | arg '/' arg { $$ = call_bin_op(p, $1, '/', $3, &@2, &@$); } https://github.com/ruby/ruby/blob/trunk/parse.y Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  32. sentence : subordinating-conjunction PERIOD subordinating-conjunction :
 subject-phrase CONJUNCTION subject-phrase subject-phrase

    : NOUN verb-phrase verb-phrase : VERB ADJECTIVE Grammar Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  33. Matz is nice so . sentence subord. conjunction subject phrase

    subject phrase verb phrase we verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  34. is nice verb phrase verb phrase are nice Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  35. is nice verb phrase verb phrase are nice Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization
  36. Matz is nice subject phrase subject phrase verb phrase we

    verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  37. subject-phrase : NOUN verb-phrase
 verb-phrase : VERB ADJECTIVE
 push-attr($2) Grammar

    Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  38. subject-phrase : NOUN verb-phrase
 save-attr($1, pop-attr) verb-phrase : VERB ADJECTIVE


    push-attr($2) Grammar Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  39. Matz is nice subject phrase subject phrase verb phrase we

    verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  40. Matz is nice so subord. conjunction subject phrase subject phrase

    verb phrase we verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  41. subject-phrase : NOUN verb-phrase
 save-attr($1, pop-attr) verb-phrase : VERB ADJECTIVE


    push-attr($2) Grammar Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  42. subordinating-conjunction :
 subject-phrase CONJUNCTION subject-phrase
 subject-phrase : NOUN verb-phrase
 save-attr($1,

    pop-attr) verb-phrase : VERB ADJECTIVE
 push-attr($2) Grammar Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  43. subordinating-conjunction :
 subject-phrase CONJUNCTION subject-phrase
 cond-skip($1, $2) subject-phrase : NOUN

    verb-phrase
 save-attr($1, pop-attr) verb-phrase : VERB ADJECTIVE
 push-attr($2) Grammar Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  44. Matz is nice so subord. conjunction subject phrase subject phrase

    verb phrase we verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  45. Matz is nice so . sentence subord. conjunction subject phrase

    subject phrase verb phrase we verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  46. subordinating-conjunction :
 subject-phrase CONJUNCTION subject-phrase
 cond-skip($1, $2) subject-phrase : NOUN

    verb-phrase
 save-attr($1, pop-attr) verb-phrase : VERB ADJECTIVE
 push-attr($2) Grammar Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  47. sentence : subordinating-conjunction PERIOD
 subordinating-conjunction :
 subject-phrase CONJUNCTION subject-phrase
 cond-skip($1,

    $2) subject-phrase : NOUN verb-phrase
 save-attr($1, pop-attr) verb-phrase : VERB ADJECTIVE
 push-attr($2) Grammar Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  48. sentence : subordinating-conjunction PERIOD
 trace($1) subordinating-conjunction :
 subject-phrase CONJUNCTION subject-phrase


    cond-skip($1, $2) subject-phrase : NOUN verb-phrase
 save-attr($1, pop-attr) verb-phrase : VERB ADJECTIVE
 push-attr($2) Grammar Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  49. Matz is nice so . sentence subord. conjunction subject phrase

    subject phrase verb phrase we verb phrase are nice Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  50. Matz is nice so . sentence subord. conjunction subject phrase

    subject phrase verb phrase we verb phrase are nice trace(5) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  51. Matz is nice so subord. conjunction subject phrase subject phrase

    verb phrase we verb phrase are nice trace(5) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  52. Matz is nice so subord. conjunction subject phrase subject phrase

    verb phrase we verb phrase are nice trace(5) push-attr(“nice”) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  53. Matz so subord. conjunction subject phrase subject phrase we verb

    phrase are nice trace(5) push-attr(“nice”) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  54. Matz so subord. conjunction subject phrase subject phrase we verb

    phrase are nice trace(5) push-attr(“nice”) save-attr(:Matz, pop-attr) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  55. so subord. conjunction subject phrase we verb phrase are nice

    trace(5) push-attr(“nice”) save-attr(:Matz, pop-attr) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  56. so subord. conjunction subject phrase we verb phrase are nice

    trace(5) push-attr(“nice”) save-attr(:Matz, pop-attr) trace(5) push-attr(“nice”) save-attr(:Matz, pop-attr) cond-skip(2) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  57. subject phrase we verb phrase are nice trace(5) push-attr(“nice”) save-attr(:Matz,

    pop-attr) trace(5) push-attr(“nice”) save-attr(:Matz, pop-attr) cond-skip(2) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  58. subject phrase we verb phrase are nice trace(5) push-attr(“nice”) save-attr(:Matz,

    pop-attr) trace(5) push-attr(“nice”) save-attr(:Matz, pop-attr) cond-skip(2) trace(5) push-attr(“nice”) save-attr(:Matz, pop-attr) cond-skip(2) push-attr(“nice”) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  59. subject phrase we trace(5) push-attr(“nice”) save-attr(:Matz, pop-attr) trace(5) push-attr(“nice”) save-attr(:Matz,

    pop-attr) cond-skip(2) trace(5) push-attr(“nice”) save-attr(:Matz, pop-attr) cond-skip(2) push-attr(“nice”) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  60. subject phrase we trace(5) push-attr(“nice”) save-attr(:Matz, pop-attr) trace(5) push-attr(“nice”) save-attr(:Matz,

    pop-attr) cond-skip(2) trace(5) push-attr(“nice”) save-attr(:Matz, pop-attr) cond-skip(2) push-attr(“nice”) trace(5) push-attr(“nice”) save-attr(:Matz, pop-attr) cond-skip(2) push-attr(“nice”) save-attr(:we, pop-attr) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization
  61. if 5 + 3 == 8 puts 'Hello, world!' end

    Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  62. code = -> { if 5 + 3 == 8

    puts 'Hello, world!' end } puts RubyVM::InstructionSequence.disasm(code) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  63. code = -> { if 5 + 3 == 8

    puts 'Hello, world!' end } puts RubyVM::InstructionSequence.disasm(code) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation == disasm: #<ISeq:block in irb_binding@(irb):1 (1,9)-(5,1)> (catch: FALSE) == catch table | catch type: redo st: 0001 ed: 0023 sp: 0000 cont: 0001 | catch type: next st: 0001 ed: 0023 sp: 0000 cont: 0023 |------------------------------------------------------------------------ 0000 nop ( 1) 0001 putobject 5 ( 2) 0003 putobject 3 0005 opt_plus <callinfo!mid:+, argc:1, ARGS_SIMPLE>, <ca 0008 putobject 8 0010 opt_eq <callinfo!mid:==, argc:1, ARGS_SIMPLE>, <c 0013 branchunless 22 0015 putself ( 3) 0016 putstring "Hello, world!" 0018 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIM 0021 leave ( 5) 0022 putnil ( 3) 0023 nop 0024 leave ( 5)
  64. code = -> { if 5 + 3 == 8

    puts 'Hello, world!' end } puts RubyVM::InstructionSequence.disasm(code) == disasm: #<ISeq:block in irb_binding@(irb):1 (1,9)-(5,1)> (catch: FALSE) == catch table | catch type: redo st: 0001 ed: 0023 sp: 0000 cont: 0001 | catch type: next st: 0001 ed: 0023 sp: 0000 cont: 0023 |------------------------------------------------------------------------ 0000 nop ( 1) 0001 putobject 5 ( 2) 0003 putobject 3 0005 opt_plus <callinfo!mid:+, argc:1, ARGS_SIMPLE>, <ca 0008 putobject 8 0010 opt_eq <callinfo!mid:==, argc:1, ARGS_SIMPLE>, <c 0013 branchunless 22 0015 putself ( 3) 0016 putstring "Hello, world!" 0018 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIM 0021 leave ( 5) 0022 putnil ( 3) 0023 nop 0024 leave ( 5) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  65. code = -> { if 5 + 3 == 8

    puts 'Hello, world!' end } puts RubyVM::InstructionSequence.disasm(code) == disasm: #<ISeq:block in irb_binding@(irb):1 (1,9)-(5,1)> (catch: FALSE) == catch table | catch type: redo st: 0001 ed: 0023 sp: 0000 cont: 0001 | catch type: next st: 0001 ed: 0023 sp: 0000 cont: 0023 |------------------------------------------------------------------------ 0000 nop ( 1) 0001 putobject 5 ( 2) 0003 putobject 3 0005 opt_plus <callinfo!mid:+, argc:1, ARGS_SIMPLE>, <ca 0008 putobject 8 0010 opt_eq <callinfo!mid:==, argc:1, ARGS_SIMPLE>, <c 0013 branchunless 22 0015 putself ( 3) 0016 putstring "Hello, world!" 0018 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIM 0021 leave ( 5) 0022 putnil ( 3) 0023 nop 0024 leave ( 5) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  66. code = -> { if 5 + 3 == 8

    puts 'Hello, world!' end } puts RubyVM::InstructionSequence.disasm(code) == disasm: #<ISeq:block in irb_binding@(irb):1 (1,9)-(5,1)> (catch: FALSE) == catch table | catch type: redo st: 0001 ed: 0023 sp: 0000 cont: 0001 | catch type: next st: 0001 ed: 0023 sp: 0000 cont: 0023 |------------------------------------------------------------------------ 0000 nop ( 1) 0001 putobject 5 ( 2) 0003 putobject 3 0005 opt_plus <callinfo!mid:+, argc:1, ARGS_SIMPLE>, <ca 0008 putobject 8 0010 opt_eq <callinfo!mid:==, argc:1, ARGS_SIMPLE>, <c 0013 branchunless 22 0015 putself ( 3) 0016 putstring "Hello, world!" 0018 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIM 0021 leave ( 5) 0022 putnil ( 3) 0023 nop 0024 leave ( 5) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  67. code = -> { if 5 + 3 == 8

    puts 'Hello, world!' end } puts RubyVM::InstructionSequence.disasm(code) == disasm: #<ISeq:block in irb_binding@(irb):1 (1,9)-(5,1)> (catch: FALSE) == catch table | catch type: redo st: 0001 ed: 0023 sp: 0000 cont: 0001 | catch type: next st: 0001 ed: 0023 sp: 0000 cont: 0023 |------------------------------------------------------------------------ 0000 nop ( 1) 0001 putobject 8 ( 2) 0008 putobject 8 0010 opt_eq <callinfo!mid:==, argc:1, ARGS_SIMPLE>, <c 0013 branchunless 22 0015 putself ( 3) 0016 putstring "Hello, world!" 0018 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIM 0021 leave ( 5) 0022 putnil ( 3) 0023 nop 0024 leave ( 5) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  68. code = -> { if 5 + 3 == 8

    puts 'Hello, world!' end } puts RubyVM::InstructionSequence.disasm(code) == disasm: #<ISeq:block in irb_binding@(irb):1 (1,9)-(5,1)> (catch: FALSE) == catch table | catch type: redo st: 0001 ed: 0023 sp: 0000 cont: 0001 | catch type: next st: 0001 ed: 0023 sp: 0000 cont: 0023 |------------------------------------------------------------------------ 0000 nop ( 1) 0001 putobject 8 ( 2) 0008 putobject 8 0010 opt_eq <callinfo!mid:==, argc:1, ARGS_SIMPLE>, <c 0013 branchunless 22 0015 putself ( 3) 0016 putstring "Hello, world!" 0018 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIM 0021 leave ( 5) 0022 putnil ( 3) 0023 nop 0024 leave ( 5) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  69. code = -> { if 5 + 3 == 8

    puts 'Hello, world!' end } puts RubyVM::InstructionSequence.disasm(code) == disasm: #<ISeq:block in irb_binding@(irb):1 (1,9)-(5,1)> (catch: FALSE) == catch table | catch type: redo st: 0001 ed: 0023 sp: 0000 cont: 0001 | catch type: next st: 0001 ed: 0023 sp: 0000 cont: 0023 |------------------------------------------------------------------------ 0000 nop ( 1) 0001 putobject 8 ( 2) 0008 putobject 8 0010 opt_eq <callinfo!mid:==, argc:1, ARGS_SIMPLE>, <c 0013 branchunless 22 0015 putself ( 3) 0016 putstring "Hello, world!" 0018 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIM 0021 leave ( 5) 0022 putnil ( 3) 0023 nop 0024 leave ( 5) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  70. code = -> { if 5 + 3 == 8

    puts 'Hello, world!' end } puts RubyVM::InstructionSequence.disasm(code) == disasm: #<ISeq:block in irb_binding@(irb):1 (1,9)-(5,1)> (catch: FALSE) == catch table | catch type: redo st: 0001 ed: 0023 sp: 0000 cont: 0001 | catch type: next st: 0001 ed: 0023 sp: 0000 cont: 0023 |------------------------------------------------------------------------ 0000 nop ( 1) 0001 putobject true ( 2) 0013 branchunless 22 0015 putself ( 3) 0016 putstring "Hello, world!" 0018 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIM 0021 leave ( 5) 0022 putnil ( 3) 0023 nop 0024 leave ( 5) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  71. if 5 + 3 == 8 puts 'Hello, world!' end

    Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  72. if true puts 'Hello, world!' end Lexical Analytics • Semantic

    Analysis • Instruction Generation • Optimization • Ruby Compilation
  73. require 'net/http'; require 'uri'; require 'nokogiri' url = 'https://twitter.com/kddeisz/status/1105570552465575937' body

    = Net::HTTP.get_response(URI.parse(url)).body node = Nokogiri::HTML(resp).at_css('.tweet-text') eval(node.text) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  74. require 'net/http'; require 'uri'; require 'nokogiri' url = 'https://twitter.com/kddeisz/status/1105570552465575937' body

    = Net::HTTP.get_response(URI.parse(url)).body node = Nokogiri::HTML(resp).at_css('.tweet-text') eval(node.text) 5 + 3 # => 2 Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  75. code = -> { if 5 + 3 == 8

    puts 'Hello, world!' end } puts RubyVM::InstructionSequence.disasm(code) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation == disasm: #<ISeq:block in irb_binding@(irb):1 (1,9)-(5,1)> (catch: FALSE) == catch table | catch type: redo st: 0001 ed: 0023 sp: 0000 cont: 0001 | catch type: next st: 0001 ed: 0023 sp: 0000 cont: 0023 |------------------------------------------------------------------------ 0000 nop ( 1) 0001 putobject 5 ( 2) 0003 putobject 3 0005 opt_plus <callinfo!mid:+, argc:1, ARGS_SIMPLE>, <ca 0008 putobject 8 0010 opt_eq <callinfo!mid:==, argc:1, ARGS_SIMPLE>, <c 0013 branchunless 22 0015 putself ( 3) 0016 putstring "Hello, world!" 0018 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIM 0021 leave ( 5) 0022 putnil ( 3) 0023 nop 0024 leave ( 5)
  76. code = -> { if 5 + 3 == 8

    puts 'Hello, world!' end } puts RubyVM::InstructionSequence.disasm(code) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation == disasm: #<ISeq:block in irb_binding@(irb):1 (1,9)-(5,1)> (catch: FALSE) == catch table | catch type: redo st: 0001 ed: 0023 sp: 0000 cont: 0001 | catch type: next st: 0001 ed: 0023 sp: 0000 cont: 0023 |------------------------------------------------------------------------ 0000 nop ( 1) 0001 putobject 5 ( 2) 0003 putobject 3 0005 opt_plus <callinfo!mid:+, argc:1, ARGS_SIMPLE>, <ca 0008 putobject 8 0010 opt_eq <callinfo!mid:==, argc:1, ARGS_SIMPLE>, <c 0013 branchunless 22 0015 putself ( 3) 0016 putstring "Hello, world!" 0018 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIM 0021 leave ( 5) 0022 putnil ( 3) 0023 nop 0024 leave ( 5)
  77. code = -> { if 5 + 3 == 8

    puts 'Hello, world!' end } puts RubyVM::InstructionSequence.disasm(code) Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation == disasm: #<ISeq:block in irb_binding@(irb):1 (1,9)-(5,1)> (catch: FALSE) == catch table | catch type: redo st: 0001 ed: 0023 sp: 0000 cont: 0001 | catch type: next st: 0001 ed: 0023 sp: 0000 cont: 0023 |------------------------------------------------------------------------ 0000 nop ( 1) 0001 opt_plus_deopt 8 <callinfo!left:5, mid:+, right:3> ( 2) 0010 opt_eq <callinfo!mid:==, argc:1, ARGS_SIMPLE>, <c 0013 branchunless 22 0015 putself ( 3) 0016 putstring "Hello, world!" 0018 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIM 0021 leave ( 5) 0022 putnil ( 3) 0023 nop 0024 leave ( 5)
  78. WHY WOULD YOU EVER DO THIS!! Lexical Analytics • Semantic

    Analysis • Instruction Generation • Optimization • Ruby Compilation
  79. Ripper.sexp_raw('5 + 3') [:program, [:stmts_add, [:stmts_new], [:binary, [:@int, "5", [1,

    0]], :+, [:@int, "3", [1, 4]]]]] Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  80. module Preval class Parser < Ripper::SexpBuilder def self.parse(source) new(source).parse end

    private SCANNER_EVENTS.each do |event| define_method(:"on_#{event}") do |token| Node.new(:"@#{event}", token, true) end end PARSER_EVENTS.each do |event| define_method(:"on_#{event}") do |*args| Node.new(event, args) end end end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  81. module Preval class Parser < Ripper::SexpBuilder def self.parse(source) new(source).parse end

    private SCANNER_EVENTS.each do |event| define_method(:"on_#{event}") do |token| Node.new(:"@#{event}", token, true) end end PARSER_EVENTS.each do |event| define_method(:"on_#{event}") do |*args| Node.new(event, args) end end end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  82. module Preval class Parser < Ripper::SexpBuilder def self.parse(source) new(source).parse end

    private SCANNER_EVENTS.each do |event| define_method(:"on_#{event}") do |token| Node.new(:"@#{event}", token, true) end end PARSER_EVENTS.each do |event| define_method(:"on_#{event}") do |*args| Node.new(event, args) end end end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  83. module Preval class Parser < Ripper::SexpBuilder def self.parse(source) new(source).parse end

    private SCANNER_EVENTS.each do |event| define_method(:"on_#{event}") do |token| Node.new(:"@#{event}", token, true) end end PARSER_EVENTS.each do |event| define_method(:"on_#{event}") do |*args| Node.new(event, args) end end end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  84. module Preval class Node include Format attr_reader :type, :body, :literal

    def initialize(type, body, literal = false) @type = type @body = body @literal = literal end def to_source return body if literal public_send(:"to_#{type}") end end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  85. module Preval class Node include Format attr_reader :type, :body, :literal

    def initialize(type, body, literal = false) @type = type @body = body @literal = literal end def to_source return body if literal public_send(:"to_#{type}") end end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  86. module Preval module Format def self.to(type, &block) define_method(:"to_#{type}", &block) end

    to(:alias) { "alias #{source(0)} #{source(1)}" } to(:aref) { body[1] ? "#{source(0)}[#{source(1)}]" : "#{source(0)}[]" } to(:aref_field) { "#{source(0)}[#{source(1)}]" } to(:arg_paren) { body[0].nil? ? '' : "(#{source(0)})" } to(:args_add) { starts_with?(:args_new) ? source(1) : join(', ') } to(:args_add_block) do args, block = body parts = args.is?(:args_new) ? [] : [args.to_source] parts << "#{parts.any? ? ', ' : ''}&#{block.to_source}" if block parts.join end to(:args_add_star) { starts_with?(:args_new) ? "*#{source(1)}" : "#{source(0)}, *#{source(1)}" } to(:args_new) { '' } to(:assign) { "#{source(0)} = #{source(1)}" } to(:array) { body[0].nil? ? '[]' : "#{starts_with?(:args_add) ? '[' : ''}#{source(0)}]" } to(:assoc_new) { starts_with?(:@label) ? join(' ') : join(' => ') } to(:assoc_splat) { "**#{source(0)}" } to(:assoclist_from_args) { body[0].map(&:to_source).join(', ') } to(:bare_assoc_hash) { body[0].map(&:to_source).join(', ') } to(:begin) { "begin\n#{join("\n")}\nend" } to(:BEGIN) { "BEGIN {\n#{source(0)}\n}"} to(:binary) { "#{source(0)} #{body[1]} #{source(2)}" } to(:blockarg) { "&#{source(0)}" } to(:block_var) { "|#{source(0)}|" } to(:bodystmt) do source(0).tap do |code| code << "\n#{source(1)}" if body[1] if body[2] stmts = body[2].is?(:else) ? body[2].body[0] : body[2] code << "\nelse\n#{stmts.to_source}" end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  87. to(:string_literal) do content = source(0).map! do |part| part.start_with?('#{') ? part

    : part.gsub(/([^\\])"/) { "#{$1}\\\"" } end "\"#{content.join}\"" end to(:super) { "super#{starts_with?(:arg_paren) ? '' : ' '}#{source(0)}" } to(:symbol) { ":#{source(0)}" } to(:symbol_literal) { source(0) } to(:symbols_add) { join(starts_with?(:symbols_new) ? '' : ' ') } to(:symbols_new) { '%I[' } to(:top_const_field) { "::#{source(0)}" } to(:top_const_ref) { "::#{source(0)}" } to(:unary) { "#{body[0][0]}#{source(1)}" } to(:undef) { "undef #{body[0][0].to_source}" } to(:unless) { "unless #{source(0)}\n#{source(1)}\n#{body[2] ? "#{source(2)}\n" : ''}end" } to(:unless_mod) { "#{source(1)} unless #{source(0)}" } to(:until) { "until #{source(0)}\n#{source(1)}\nend" } to(:until_mod) { "#{source(1)} until #{source(0)}" } to(:var_alias) { "alias #{source(0)} #{source(1)}" } to(:var_field) { join } to(:var_ref) { source(0) } to(:vcall) { join } to(:void_stmt) { '' } to(:when) { "when #{source(0)}\n#{source(1)}#{body[2] ? "\n#{source(2)}" : ''}" } to(:while) { "while #{source(0)}\n#{source(1)}\nend" } to(:while_mod) { "#{source(1)} while #{source(0)}" } to(:word_add) { join } to(:word_new) { '' } to(:words_add) { join(starts_with?(:words_new) ? '' : ' ') } to(:words_new) { '%W[' } to(:xstring_add) { join } to(:xstring_new) { '' } to(:xstring_literal) { "%x[#{source(0)}]" } to(:yield) { "yield#{starts_with?(:paren) ? '' : ' '}#{join}" } to(:yield0) { 'yield' } to(:zsuper) { 'super' } end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  88. module Preval class << self attr_reader :visitors def enable_all! Visitors::Arithmetic.enable!

    Visitors::AttrAccessor.enable! Visitors::Fasterer.enable! Visitors::Loops.enable! end def process(source) visitors.inject(Parser.parse(source)) do |current, visitor| current.tap { |ast| ast.visit(visitor) } end.to_source end end @visitors = [] end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  89. module Preval class << self attr_reader :visitors def enable_all! Visitors::Arithmetic.enable!

    Visitors::AttrAccessor.enable! Visitors::Fasterer.enable! Visitors::Loops.enable! end def process(source) visitors.inject(Parser.parse(source)) do |current, visitor| current.tap { |ast| ast.visit(visitor) } end.to_source end end @visitors = [] end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  90. module Preval class << self attr_reader :visitors def enable_all! Visitors::Arithmetic.enable!

    Visitors::AttrAccessor.enable! Visitors::Fasterer.enable! Visitors::Loops.enable! end def process(source) visitors.inject(Parser.parse(source)) do |current, visitor| current.tap { |ast| ast.visit(visitor) } end.to_source end end @visitors = [] end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  91. module Preval class Node def visit(pass) return if literal handler

    = :"on_#{type}" pass.public_send(handler, self) if pass.respond_to?(handler) return unless body.is_a?(Array) body.each do |child| child.visit(pass) if child.is_a?(Node) end end end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  92. class Preval::Visitors::Arithmetic < Visitor OPERATORS = %i[+ - * /

    % **].freeze def on_binary(node) left, operation, right = node.body return unless OPERATORS.include?(operation) if left.is?(:@int) && right.is?(:@int) value = left.to_int.public_send(operation, right.to_int) node.update(:@int, value.to_s) end end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  93. if defined?(Bootsnap) load_iseq = RubyVM::InstructionSequence.method(:load_iseq) if load_iseq.source_location[0].include?('/bootsnap/') Bootsnap::CompileCache::ISeq.singleton_class.prepend( Module.new do

    def input_to_storage(source, path) source = Preval.process(source) compiled = RubyVM::InstructionSequence.compile( source, path, path ) compiled.to_binary rescue SyntaxError raise Bootsnap::CompileCache::Uncompilable end end ) end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  94. if defined?(Bootsnap) load_iseq = RubyVM::InstructionSequence.method(:load_iseq) if load_iseq.source_location[0].include?('/bootsnap/') Bootsnap::CompileCache::ISeq.singleton_class.prepend( Module.new do

    def input_to_storage(source, path) source = Preval.process(source) compiled = RubyVM::InstructionSequence.compile( source, path, path ) compiled.to_binary rescue SyntaxError raise Bootsnap::CompileCache::Uncompilable end end ) end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  95. if defined?(Bootsnap) load_iseq = RubyVM::InstructionSequence.method(:load_iseq) if load_iseq.source_location[0].include?('/bootsnap/') Bootsnap::CompileCache::ISeq.singleton_class.prepend( Module.new do

    def input_to_storage(source, path) source = Preval.process(source) compiled = RubyVM::InstructionSequence.compile( source, path, path ) compiled.to_binary rescue SyntaxError raise Bootsnap::CompileCache::Uncompilable end end ) end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  96. if defined?(Bootsnap) load_iseq = RubyVM::InstructionSequence.method(:load_iseq) if load_iseq.source_location[0].include?('/bootsnap/') Bootsnap::CompileCache::ISeq.singleton_class.prepend( Module.new do

    def input_to_storage(source, path) source = Preval.process(source) compiled = RubyVM::InstructionSequence.compile( source, path, path ) compiled.to_binary rescue SyntaxError raise Bootsnap::CompileCache::Uncompilable end end ) end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  97. if defined?(Bootsnap) load_iseq = RubyVM::InstructionSequence.method(:load_iseq) if load_iseq.source_location[0].include?('/bootsnap/') Bootsnap::CompileCache::ISeq.singleton_class.prepend( Module.new do

    def input_to_storage(source, path) source = Preval.process(source) compiled = RubyVM::InstructionSequence.compile( source, path, path ) compiled.to_binary rescue SyntaxError raise Bootsnap::CompileCache::Uncompilable end end ) end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  98. if defined?(Bootsnap) load_iseq = RubyVM::InstructionSequence.method(:load_iseq) if load_iseq.source_location[0].include?('/bootsnap/') Bootsnap::CompileCache::ISeq.singleton_class.prepend( Module.new do

    def input_to_storage(source, path) source = Preval.process(source) compiled = RubyVM::InstructionSequence.compile( source, path, path ) compiled.to_binary rescue SyntaxError raise Bootsnap::CompileCache::Uncompilable end end ) end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  99. if defined?(Bootsnap) load_iseq = RubyVM::InstructionSequence.method(:load_iseq) if load_iseq.source_location[0].include?('/bootsnap/') Bootsnap::CompileCache::ISeq.singleton_class.prepend( Module.new do

    def input_to_storage(source, path) source = Preval.process(source) compiled = RubyVM::InstructionSequence.compile( source, path, path ) compiled.to_binary rescue SyntaxError raise Bootsnap::CompileCache::Uncompilable end end ) end end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  100. module Preval class << self attr_reader :visitors def enable_all! Visitors::Arithmetic.enable!

    Visitors::AttrAccessor.enable! Visitors::Fasterer.enable! Visitors::Loops.enable! end def process(source) visitors.inject(Parser.parse(source)) do |current, visitor| current.tap { |ast| ast.visit(visitor) } end.to_source end end @visitors = [] end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  101. module Preval class << self attr_reader :visitors def enable_all! Visitors::Arithmetic.enable!

    Visitors::AttrAccessor.enable! Visitors::Fasterer.enable! Visitors::Loops.enable! end def process(source) visitors.inject(Parser.parse(source)) do |current, visitor| current.tap { |ast| ast.visit(visitor) } end.to_source end end @visitors = [] end Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  102. that can be done now Optimizations Lexical Analytics • Semantic

    Analysis • Instruction Generation • Optimization • Ruby Compilation
  103. that can be done now
 attr_accessor replacement Optimizations Lexical Analytics

    • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  104. that can be done now
 attr_accessor replacement
 instruction elimination Optimizations

    Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  105. that can be done now
 attr_accessor replacement
 instruction elimination that

    can be done with deoptimization Optimizations Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  106. that can be done now
 attr_accessor replacement
 instruction elimination that

    can be done with deoptimization
 for-loop replacement Optimizations Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  107. that can be done now
 attr_accessor replacement
 instruction elimination that

    can be done with deoptimization
 for-loop replacement
 constant folding Optimizations Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  108. that can be done now
 attr_accessor replacement
 instruction elimination that

    can be done with deoptimization
 for-loop replacement
 constant folding that a compiler can never do Optimizations Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  109. that can be done now
 attr_accessor replacement
 instruction elimination that

    can be done with deoptimization
 for-loop replacement
 constant folding that a compiler can never do
 replacing certain APIs with newer or faster APIs Optimizations Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  110. Nothing Senior devs enforce in code review Style Progression Lexical

    Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  111. Nothing Senior devs enforce in code review Style guide is

    developed and linked Style Progression Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  112. Nothing Senior devs enforce in code review Style guide is

    developed and linked Linters are developed and run locally (and in CI) Style Progression Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  113. Nothing Senior devs enforce in code review Style guide is

    developed and linked Linters are developed and run locally (and in CI) Auto-formatters and compilers take care of it Style Progression Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  114. Nothing Senior devs enforce in code review Style guide is

    developed and linked Linters are developed and run locally (and in CI) Auto-formatters and compilers take care of it Style Progression w e’re here Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  115. language: ruby rvm: 2.6.3 branches: only: master before_install: - gem

    update --system - gem install bundler before_script: bundle exec rails db:create cache: bundler script: - bundle exec bundle audit - bin/rails test - bundle exec rubocop --parallel - bin/rails reek - bundle exec brakeman -z - bin/rails fasterer after_success: bin/deploy services: - postgresql - redis Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  116. language: ruby rvm: 2.6.3 branches: only: master before_install: - gem

    update --system - gem install bundler before_script: bundle exec rails db:create cache: bundler script: - bundle exec bundle audit - bin/rails test - bundle exec rubocop --parallel - bin/rails reek - bundle exec brakeman -z - bin/rails fasterer after_success: bin/deploy services: - postgresql - redis Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation WHAT!?
  117. language: ruby rvm: 2.6.3 branches: only: master before_install: - gem

    update --system - gem install bundler before_script: bundle exec rails db:create cache: bundler script: - bundle exec bundle audit - bin/rails test - bundle exec rubocop --parallel - bin/rails reek - bundle exec brakeman -z - bin/rails fasterer after_success: bin/deploy services: - postgresql - redis Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  118. language: ruby rvm: 2.6.3 branches: only: master before_install: - gem

    update --system - gem install bundler before_script: bundle exec rails db:create cache: bundler script: - bundle exec bundle audit - bin/rails test - bundle exec rubocop --parallel - bin/rails reek - bundle exec brakeman -z - bin/rails fasterer after_success: bin/deploy services: - postgresql - redis Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  119. language: ruby rvm: 2.6.3 branches: only: master before_install: - gem

    update --system - gem install bundler before_script: bundle exec rails db:create cache: bundler script: - bundle exec bundle audit - bin/rails test - bundle exec rubocop --parallel - bin/rails reek - bundle exec brakeman -z - bin/rails fasterer after_success: bin/deploy services: - postgresql - redis Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  120. language: ruby rvm: 2.6.3 branches: only: master before_install: - gem

    update --system - gem install bundler before_script: bundle exec rails db:create cache: bundler script: - bundle exec bundle audit - bin/rails test - bundle exec rubocop --parallel - bin/rails reek - bundle exec brakeman -z - bin/rails fasterer after_success: bin/deploy services: - postgresql - redis Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  121. language: ruby rvm: 2.6.3 branches: only: master before_install: - gem

    update --system - gem install bundler before_script: bundle exec rails db:create cache: bundler script: - bundle exec bundle audit - bin/rails test - bundle exec rubocop --parallel - bin/rails reek - bundle exec brakeman -z - bin/rails fasterer after_success: bin/deploy services: - postgresql - redis Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  122. language: ruby rvm: 2.6.3 branches: only: master before_install: - gem

    update --system - gem install bundler before_script: bundle exec rails db:create cache: bundler script: - bin/rails test after_success: bin/deploy services: - postgresql - redis Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  123. Nothing Senior devs enforce in code review Style guide is

    developed and linked Linters are developed and run locally (and in CI) Auto-formatters and compilers take care of it Style Progression Lexical Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  124. Auto-formatters and compilers take care of it Style Progression Lexical

    Analytics • Semantic Analysis • Instruction Generation • Optimization • Ruby Compilation
  125. Write code Test code Lexical Analytics • Semantic Analysis •

    Instruction Generation • Optimization • Ruby Compilation
  126. Write code Test code Deploy code Lexical Analytics • Semantic

    Analysis • Instruction Generation • Optimization • Ruby Compilation