Compiling Ruby

8a66c2a7197be751b21ebd35319ec797?s=47 Kevin Deisz
September 18, 2017

Compiling Ruby

Since Ruby 2.3 and the introduction of RubyVM::InstructionSequence::load_iseq, we've been able to programmatically load ruby bytecode. By divorcing the process of running YARV byte code from the process of compiling ruby code, we can take advantage of the strengths of the ruby virtual machine while simultaneously reaping the benefits of a compiler such as macros, type checking, and instruction sequence optimizations. This can make our ruby faster and more readable! This talk demonstrates how to integrate this into your own workflows and the exciting possibilities this enables.

8a66c2a7197be751b21ebd35319ec797?s=128

Kevin Deisz

September 18, 2017
Tweet

Transcript

  1. Compiling Ruby

  2. Kevin Deisz @kddeisz

  3. # example.rb a = 5 puts a + 5

  4. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $
  5. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ ruby example.rb 10 ~/compiling-ruby $
  6. Ruby execution process

  7. #require

  8. Read source #require

  9. Tokenize Read source #require

  10. Tokenize Build AST Read source #require

  11. Tokenize Build AST Read source #require Interpret AST

  12. Tokenize Build AST Read source Interpret AST #require

  13. Tokenize Build AST Build ISeq Read source Interpret AST #require

  14. Execute ISeq Tokenize Build AST Build ISeq Read source Interpret

    AST #require
  15. Tokenizing the input Tokenize • Build AST • (Interpret AST)

    • Build ISeq • Execute ISeq
  16. # example.rb a = 5 puts a + 5 Tokenize

    • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  17. [comment="# example.rb\n"] a = 5 puts a + 5 Tokenize

    • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  18. [comment="# example.rb\n"] [ignored_nl] a = 5 puts a + 5

    Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  19. [comment="# example.rb\n"] [ignored_nl] [ident=a] = 5 puts a + 5

    Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  20. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] 5 puts a + 5

    Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  21. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5] puts a + 5

    Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  22. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5][nl] puts a + 5

    Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  23. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5][nl] [ident=puts] a + 5

    Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  24. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5][nl] [ident=puts] [ident=a] + 5

    Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  25. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5][nl] [ident=puts] [ident=a] [op+] 5

    Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  26. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5][nl] [ident=puts] [ident=a] [op+] [int=5]

    Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  27. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5][nl] [ident=puts] [ident=a] [op+] [int=5][nl]

    Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  28. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5] [nl] [ident=puts] [ident=a] [op+]

    [int=5] [nl] Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  29. Building the AST Tokenize • Build AST • (Interpret AST)

    • Build ISeq • Execute ISeq
  30. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5] [nl] [ident=puts] [ident=a] [op+]

    [int=5] [nl] Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  31. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5] [nl] [ident=puts] [ident=a] [op+]

    [int=5] [nl] begin Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  32. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5] [nl] [ident=puts] [ident=a] [op+]

    [int=5] [nl] begin Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  33. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5] [nl] [ident=puts] [ident=a] [op+]

    [int=5] [nl] begin lvasgn :a int 5 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  34. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5] [nl] [ident=puts] [ident=a] [op+]

    [int=5] [nl] begin lvasgn :a int 5 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  35. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5] [nl] [ident=puts] [ident=a] [op+]

    [int=5] [nl] begin lvasgn :a int 5 lvar :a int 5 send :+ send :puts Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  36. [comment="# example.rb\n"] [ignored_nl] [ident=a] [op=] [int=5] [nl] [ident=puts] [ident=a] [op+]

    [int=5] [nl] begin lvasgn :a int 5 lvar :a int 5 send :+ send :puts Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  37. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  38. Interpreting the AST (≤ Ruby 1.8) Tokenize • Build AST

    • (Interpret AST) • Build ISeq • Execute ISeq
  39. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  40. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  41. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts local = {} instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  42. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts local = {} instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  43. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts local = {} instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $
  44. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts local = {} instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb
  45. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts local = {} instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb
  46. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 5 local = {} instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb
  47. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts set :a
 5 local = {} instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb
  48. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts local = { a: 5 } instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb
  49. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 5 local = { a: 5 } instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb
  50. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts get :a 5 local = { a: 5 } instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb
  51. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 5 5 local = { a: 5 } instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb
  52. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts send :+ 5 5 local = { a: 5 } instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb
  53. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 10 local = { a: 5 } instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb
  54. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts self
 10 local = { a: 5 } instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb
  55. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts send :puts
 self
 10 local = { a: 5 } instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb
  56. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts local = { a: 5 } instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb 10
  57. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts local = { a: 5 } instance = {} global = {} Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq ~/compiling-ruby $ ruby example.rb 10 ~/compiling-ruby $
  58. Building YARV instruction sequences Tokenize • Build AST • (Interpret

    AST) • Build ISeq • Execute ISeq
  59. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  60. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  61. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  62. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  63. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  64. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  65. 0000 trace 1 0002 putobject 5 begin lvasgn :a int

    5 lvar :a int 5 send :+ send :puts Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  66. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 0004 setlocal 3 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  67. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 0004 setlocal 3 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  68. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  69. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  70. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  71. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  72. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  73. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  74. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  75. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  76. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  77. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache>
  78. begin lvasgn :a int 5 lvar :a int 5 send

    :+ send :puts 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq
  79. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  80. Executing YARV instruction sequences Tokenize • Build AST • (Interpret

    AST) • Build ISeq • Execute ISeq
  81. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  82. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  83. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq local = {} instance = {} global = {} ~/compiling-ruby $ 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  84. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq local = {} instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  85. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq local = {} instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  86. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq 5 local = {} instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  87. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq 
 5 local = {} instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  88. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq local = { 3: 5 } instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  89. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq local = { 3: 5 } instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  90. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq self local = { 3: 5 } instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  91. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq self local = { 3: 5 } instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  92. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq 5
 self local = { 3: 5 } instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  93. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq 5 self local = { 3: 5 } instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  94. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq 5 5 self local = { 3: 5 } instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  95. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq 5 5 self local = { 3: 5 } instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  96. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq 10 self local = { 3: 5 } instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  97. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq 10 self local = { 3: 5 } instance = {} global = {} ~/compiling-ruby $ ruby example.rb 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  98. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq local = { 3: 5 } instance = {} global = {} ~/compiling-ruby $ ruby example.rb 10 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  99. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq local = { 3: 5 } instance = {} global = {} ~/compiling-ruby $ ruby example.rb 10 ~/compiling-ruby $ 0000 trace 1 0002 putobject 5 0004 setlocal 3 0006 trace 1 0008 putself 0009 getlocal 3 0011 putobject 5 0013 opt_plus <callinfo>!mid:+, argc:1, ARGS_SIMPLE>, <callcache> 0016 opt_send_without_block <callinfo!mid:puts, argc:1, FCALL|ARGS_SIMPLE>, <callcache> 0019 leave
  100. Reading/writing YARV instruction sequences (≥ Ruby 2.3) Tokenize • Build

    AST • (Interpret AST) • Build ISeq • Execute ISeq
  101. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq • Read/write ISeq
  102. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq • Read/write ISeq
  103. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq • Read/write ISeq
  104. # example.rb a = 5 puts a + 5 Tokenize

    • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  105. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  106. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  107. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  108. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  109. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  110. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  111. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ ls -l total 16 -rw-r--r-- 1 kddeisz staff 32 Aug 28 15:11 example.rb -rw-r--r-- 1 kddeisz staff 777 Aug 29 11:17 example.yarb ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  112. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ ls -l total 16 -rw-r--r-- 1 kddeisz staff 32 Aug 28 15:11 example.rb -rw-r--r-- 1 kddeisz staff 777 Aug 29 11:17 example.yarb ~/compiling-ruby $ ruby -e "content = File.read('example.yarb'); RubyVM::InstructionSequence.load_from_binary(content).eval" 10 ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  113. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ ls -l total 16 -rw-r--r-- 1 kddeisz staff 32 Aug 28 15:11 example.rb -rw-r--r-- 1 kddeisz staff 777 Aug 29 11:17 example.yarb ~/compiling-ruby $ ruby -e "content = File.read('example.yarb'); RubyVM::InstructionSequence.load_from_binary(content).eval" 10 ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  114. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ ls -l total 16 -rw-r--r-- 1 kddeisz staff 32 Aug 28 15:11 example.rb -rw-r--r-- 1 kddeisz staff 777 Aug 29 11:17 example.yarb ~/compiling-ruby $ ruby -e "content = File.read('example.yarb'); RubyVM::InstructionSequence.load_from_binary(content).eval" 10 ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  115. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ ls -l total 16 -rw-r--r-- 1 kddeisz staff 32 Aug 28 15:11 example.rb -rw-r--r-- 1 kddeisz staff 777 Aug 29 11:17 example.yarb ~/compiling-ruby $ ruby -e "content = File.read('example.yarb'); RubyVM::InstructionSequence.load_from_binary(content).eval" 10 ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  116. # example.rb a = 5 puts a + 5 ~/compiling-ruby

    $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ ls -l total 16 -rw-r--r-- 1 kddeisz staff 32 Aug 28 15:11 example.rb -rw-r--r-- 1 kddeisz staff 777 Aug 29 11:17 example.yarb ~/compiling-ruby $ ruby -e "content = File.read('example.yarb'); RubyVM::InstructionSequence.load_from_binary(content).eval" 10 ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  117. Programmatically loading YARV instruction sequences (≥ Ruby 2.3) Tokenize •

    Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  118. Execute ISeq Tokenize Build AST Build ISeq Read source Interpret

    AST #require Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  119. Tokenize Build AST Build ISeq Read source Interpret AST #require

    Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq Execute ISeq Write ISeq Read ISeq
  120. /* ruby/load.c */ static int rb_load_internal0(rb_thread_t *th, VALUE fname, int

    wrap) { ... if ((iseq = rb_iseq_load_iseq(fname)) != NULL) { /* OK */ } else { VALUE parser = rb_parser_new(); rb_parser_set_context(parser, NULL, FALSE); node = (NODE *)rb_parser_load_file(parser, fname); iseq = rb_iseq_new_top(node, rb_fstring_cstr("<top (required)>"), fname, rb_realpath_internal(Qnil, fname, 1), NULL); } rb_iseq_eval(iseq); ... } Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  121. /* ruby/load.c */ static int rb_load_internal0(rb_thread_t *th, VALUE fname, int

    wrap) { ... if ((iseq = rb_iseq_load_iseq(fname)) != NULL) { /* OK */ } else { VALUE parser = rb_parser_new(); rb_parser_set_context(parser, NULL, FALSE); node = (NODE *)rb_parser_load_file(parser, fname); iseq = rb_iseq_new_top(node, rb_fstring_cstr("<top (required)>"), fname, rb_realpath_internal(Qnil, fname, 1), NULL); } rb_iseq_eval(iseq); ... } Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  122. /* ruby/iseq.c */ const rb_iseq_t * rb_iseq_load_iseq(VALUE fname) { VALUE

    iseqv = rb_check_funcall(rb_cISeq, rb_intern("load_iseq"), 1, &fname); if (!SPECIAL_CONST_P(iseqv) && RBASIC_CLASS(iseqv) == rb_cISeq) { return iseqw_check(iseqv); } return NULL; } Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  123. /* ruby/iseq.c */ const rb_iseq_t * rb_iseq_load_iseq(VALUE fname) { VALUE

    iseqv = rb_check_funcall(rb_cISeq, rb_intern("load_iseq"), 1, &fname); if (!SPECIAL_CONST_P(iseqv) && RBASIC_CLASS(iseqv) == rb_cISeq) { return iseqw_check(iseqv); } return NULL; } Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  124. /* ruby/iseq.c */ const rb_iseq_t * rb_iseq_load_iseq(VALUE fname) { VALUE

    iseqv = rb_check_funcall(rb_cISeq, rb_intern("load_iseq"), 1, &fname); if (!SPECIAL_CONST_P(iseqv) && RBASIC_CLASS(iseqv) == rb_cISeq) { return iseqw_check(iseqv); } return NULL; } Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  125. /* ruby/iseq.c */ const rb_iseq_t * rb_iseq_load_iseq(VALUE fname) { VALUE

    iseqv = rb_check_funcall(rb_cISeq, rb_intern("load_iseq"), 1, &fname); if (!SPECIAL_CONST_P(iseqv) && RBASIC_CLASS(iseqv) == rb_cISeq) { return iseqw_check(iseqv); } return NULL; } Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  126. class RubyVM::InstructionSequence def self.load_iseq(source_path) iseq_path = File.join( File.expand_path(__dir__, '.iseq'), source_path.gsub(/[^A-Za-z0-9\._-]/)

    { |c| '%02x' % c.ord }.gsub('.rb', '.yarb')) if File.exist?(iseq_path) && (File.mtime(source_path) <= File.mtime(iseq_path)) return RubyVM::InstructionSequence .load_from_binary(File.binread(iseq_path)) end begin iseq = RubyVM::InstructionSequence .compile(File.read(source_path)) File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  127. class RubyVM::InstructionSequence def self.load_iseq(source_path) iseq_path = File.join( File.expand_path(__dir__, '.iseq'), source_path.gsub(/[^A-Za-z0-9\._-]/)

    { |c| '%02x' % c.ord }.gsub('.rb', '.yarb')) if File.exist?(iseq_path) && (File.mtime(source_path) <= File.mtime(iseq_path)) return RubyVM::InstructionSequence .load_from_binary(File.binread(iseq_path)) end begin iseq = RubyVM::InstructionSequence .compile(File.read(source_path)) File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  128. class RubyVM::InstructionSequence def self.load_iseq(source_path) iseq_path = File.join( File.expand_path(__dir__, '.iseq'), source_path.gsub(/[^A-Za-z0-9\._-]/)

    { |c| '%02x' % c.ord }.gsub('.rb', '.yarb')) if File.exist?(iseq_path) && (File.mtime(source_path) <= File.mtime(iseq_path)) return RubyVM::InstructionSequence .load_from_binary(File.binread(iseq_path)) end begin iseq = RubyVM::InstructionSequence .compile(File.read(source_path)) File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  129. class RubyVM::InstructionSequence def self.load_iseq(source_path) iseq_path = File.join( File.expand_path(__dir__, '.iseq'), source_path.gsub(/[^A-Za-z0-9\._-]/)

    { |c| '%02x' % c.ord }.gsub('.rb', '.yarb')) if File.exist?(iseq_path) && (File.mtime(source_path) <= File.mtime(iseq_path)) return RubyVM::InstructionSequence .load_from_binary(File.binread(iseq_path)) end begin iseq = RubyVM::InstructionSequence .compile(File.read(source_path)) File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  130. class RubyVM::InstructionSequence def self.load_iseq(source_path) iseq_path = File.join( File.expand_path(__dir__, '.iseq'), source_path.gsub(/[^A-Za-z0-9\._-]/)

    { |c| '%02x' % c.ord }.gsub('.rb', '.yarb')) if File.exist?(iseq_path) && (File.mtime(source_path) <= File.mtime(iseq_path)) return RubyVM::InstructionSequence .load_from_binary(File.binread(iseq_path)) end begin iseq = RubyVM::InstructionSequence .compile(File.read(source_path)) File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  131. class RubyVM::InstructionSequence def self.load_iseq(source_path) iseq_path = File.join( File.expand_path(__dir__, '.iseq'), source_path.gsub(/[^A-Za-z0-9\._-]/)

    { |c| '%02x' % c.ord }.gsub('.rb', '.yarb')) if File.exist?(iseq_path) && (File.mtime(source_path) <= File.mtime(iseq_path)) return RubyVM::InstructionSequence .load_from_binary(File.binread(iseq_path)) end begin iseq = RubyVM::InstructionSequence .compile(File.read(source_path)) File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  132. Examples Tokenize • Build AST • (Interpret AST) • Build

    ISeq • Execute ISeq • Read/write ISeq
  133. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq • Read/write ISeq /* bootsnap/lib/bootsnap/compile_cache/iseq.rb */ module InstructionSequenceMixin def load_iseq(path) # Having coverage enabled prevents iseq dumping/loading. return nil if defined?(Coverage) && Bootsnap::CompileCache::Native.coverage_running? Bootsnap::CompileCache::Native.fetch( Bootsnap::CompileCache::ISeq.cache_dir, path.to_s, Bootsnap::CompileCache::ISeq ) rescue RuntimeError => e if e.message =~ /unmatched platform/ puts "unmatched platform for file #{path}" end raise end ... end
  134. /* yomikomu/lib/yomikomu.rb */ class RubyVM::InstructionSequence if ENV['YOMIKOMU_YOMIKOMANAI'] != 'true' def

    self.load_iseq fname ::Yomikomu::STORAGE.load_iseq(fname) end end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  135. Now for the fun part Tokenize • Build AST •

    (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq
  136. class RubyVM::InstructionSequence def self.load_iseq(source_path) iseq_path = File.join( File.expand_path(__dir__, '.iseq'), source_path.gsub(/[^A-Za-z0-9\._-]/)

    { |c| '%02x' % c.ord }.gsub('.rb', '.yarb')) if File.exist?(iseq_path) && (File.mtime(source_path) <= File.mtime(iseq_path)) return RubyVM::InstructionSequence .load_from_binary(File.binread(iseq_path)) end begin iseq = RubyVM::InstructionSequence .compile(File.read(source_path)) File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  137. class RubyVM::InstructionSequence def self.load_iseq(source_path) iseq_path = File.join( File.expand_path(__dir__, '.iseq'), source_path.gsub(/[^A-Za-z0-9\._-]/)

    { |c| '%02x' % c.ord }.gsub('.rb', '.yarb')) if File.exist?(iseq_path) && (File.mtime(source_path) <= File.mtime(iseq_path)) return RubyVM::InstructionSequence .load_from_binary(File.binread(iseq_path)) end begin content = File.read(source_path) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  138. content = File.read(source_path) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq Tokenize

    • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  139. content = File.read(source_path) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq THIS

    IS THE FUN PART Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  140. content = File.read(source_path) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq Tokenize

    • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  141. content = File.read(source_path) content.gsub!(/~n\(([\d\s+-\/*\(\)]+?)\)/) do |match| eval(match[3..-2]) end iseq =

    RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  142. content = File.read(source_path) content.gsub!(/~n\(([\d\s+-\/*\(\)]+?)\)/) do |match| eval(match[3..-2]) end iseq =

    RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq module Utils def self.days_to_seconds(days) days * 24 * 60 * 60 end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  143. content = File.read(source_path) content.gsub!(/~n\(([\d\s+-\/*\(\)]+?)\)/) do |match| eval(match[3..-2]) end iseq =

    RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq module Utils def self.days_to_seconds(days) days * ~n(24 * 60 * 60) end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  144. content = File.read(source_path) content.gsub!(/~n\(([\d\s+-\/*\(\)]+?)\)/) do |match| eval(match[3..-2]) end iseq =

    RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part module Utils def self.days_to_seconds(days) days * ~n(24 * 60 * 60) end end
  145. content = File.read(source_path) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq Tokenize

    • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  146. content = File.read(source_path) serial_format = '%FT%T%:z' content.gsub!(/~d\((.+?)\)/) do |match| date

    = Date.parse(match[3..-2]) "Date.strptime('#{date.strftime(serial_format)}', " \ "'#{serial_format}')" end iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  147. content = File.read(source_path) serial_format = '%FT%T%:z' content.gsub!(/~d\((.+?)\)/) do |match| date

    = Date.parse(match[3..-2]) "Date.strptime('#{date.strftime(serial_format)}', " \ "'#{serial_format}')" end iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part module Utils def self.days_since_start(date) date - Date.parse('January 1st, 2017') end end
  148. content = File.read(source_path) serial_format = '%FT%T%:z' content.gsub!(/~d\((.+?)\)/) do |match| date

    = Date.parse(match[3..-2]) "Date.strptime('#{date.strftime(serial_format)}', " \ "'#{serial_format}')" end iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq module Utils def self.days_since_start(date) date - ~d(January 1st, 2017) end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  149. content = File.read(source_path) serial_format = '%FT%T%:z' content.gsub!(/~d\((.+?)\)/) do |match| date

    = Date.parse(match[3..-2]) "Date.strptime('#{date.strftime(serial_format)}', " \ "'#{serial_format}')" end iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part module Utils def self.days_since_start(date) date - ~d(January 1st, 2017) end end
  150. It works! Tokenize • Build AST • (Interpret AST) •

    Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  151. It works! https://github.com/kddeisz/vernacular/blob/ master/lib/vernacular/regex_modifier.rb Tokenize • Build AST • (Interpret

    AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  152. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq • Read/write ISeq • The fun part
  153. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq • Read/write ISeq • The fun part
  154. Tokenize Build AST Build ISeq Read source Interpret AST #require

    Execute ISeq Write ISeq Read ISeq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  155. Tokenize Build AST Build ISeq Read source Interpret AST #require

    Execute ISeq Write ISeq Read ISeq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part Modify source
  156. Let’s go further Tokenize • Build AST • (Interpret AST)

    • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  157. content = File.read(source_path) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq Tokenize

    • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  158. content = File.read(source_path) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq THIS

    IS THE FUN PART Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  159. content = File.read(source_path) require 'parser/current' # class CustomBuilder < Parser::Builders::Default

    builder = CustomBuilder.new # class CustomParser < Parser::CurrentRuby parser = CustomParser.new(builder) # class CustomRewriter < Parser::Rewriter rewriter = CustomRewriter.new rewriter.instance_variable_set(:@parser, parser) buffer = Parser::Base.send(:setup_source_buffer, '(string)', 1, content, parser.default_encoding) ast = parser.parse(content) content = rewriter.rewrite(buffer, ast) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  160. content = File.read(source_path) require 'parser/current' # class CustomBuilder < Parser::Builders::Default

    builder = CustomBuilder.new # class CustomParser < Parser::CurrentRuby parser = CustomParser.new(builder) # class CustomRewriter < Parser::Rewriter rewriter = CustomRewriter.new rewriter.instance_variable_set(:@parser, parser) buffer = Parser::Base.send(:setup_source_buffer, '(string)', 1, content, parser.default_encoding) ast = parser.parse(content) content = rewriter.rewrite(buffer, ast) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq module Utils def self.add(left, right) left + right end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  161. content = File.read(source_path) require 'parser/current' # class CustomBuilder < Parser::Builders::Default

    builder = CustomBuilder.new # class CustomParser < Parser::CurrentRuby parser = CustomParser.new(builder) # class CustomRewriter < Parser::Rewriter rewriter = CustomRewriter.new rewriter.instance_variable_set(:@parser, parser) buffer = Parser::Base.send(:setup_source_buffer, '(string)', 1, content, parser.default_encoding) ast = parser.parse(content) content = rewriter.rewrite(buffer, ast) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq module Utils def self.add(left : Integer, right : Integer) left + right end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  162. content = File.read(source_path) require 'parser/current' # class CustomBuilder < Parser::Builders::Default

    builder = CustomBuilder.new # class CustomParser < Parser::CurrentRuby parser = CustomParser.new(builder) # class CustomRewriter < Parser::Rewriter rewriter = CustomRewriter.new rewriter.instance_variable_set(:@parser, parser) buffer = Parser::Base.send(:setup_source_buffer, '(string)', 1, content, parser.default_encoding) ast = parser.parse(content) content = rewriter.rewrite(buffer, ast) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part module Utils def self.add(left, right) unless left.is_a?(Integer) raise ArgumentError, "Invalid type, " \ "expected Integer, got #{left.class.name}" end unless right.is_a?(Integer) raise ArgumentError, "Invalid type, " \ "expected Integer, got #{right.class.name}" end left + right end end
  163. content = File.read(source_path) require 'parser/current' # class CustomBuilder < Parser::Builders::Default

    builder = CustomBuilder.new # class CustomParser < Parser::CurrentRuby parser = CustomParser.new(builder) # class CustomRewriter < Parser::Rewriter rewriter = CustomRewriter.new rewriter.instance_variable_set(:@parser, parser) buffer = Parser::Base.send(:setup_source_buffer, '(string)', 1, content, parser.default_encoding) ast = parser.parse(content) content = rewriter.rewrite(buffer, ast) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq module Utils def self.add(left : Integer, right : Integer) left + right end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  164. content = File.read(source_path) require 'parser/current' # class CustomBuilder < Parser::Builders::Default

    builder = CustomBuilder.new # class CustomParser < Parser::CurrentRuby parser = CustomParser.new(builder) # class CustomRewriter < Parser::Rewriter rewriter = CustomRewriter.new rewriter.instance_variable_set(:@parser, parser) buffer = Parser::Base.send(:setup_source_buffer, '(string)', 1, content, parser.default_encoding) ast = parser.parse(content) content = rewriter.rewrite(buffer, ast) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq module Utils def self.add(left, right) left + right end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  165. content = File.read(source_path) require 'parser/current' # class CustomBuilder < Parser::Builders::Default

    builder = CustomBuilder.new # class CustomParser < Parser::CurrentRuby parser = CustomParser.new(builder) # class CustomRewriter < Parser::Rewriter rewriter = CustomRewriter.new rewriter.instance_variable_set(:@parser, parser) buffer = Parser::Base.send(:setup_source_buffer, '(string)', 1, content, parser.default_encoding) ast = parser.parse(content) content = rewriter.rewrite(buffer, ast) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq module Utils def self.add(left, right) = Integer left + right end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  166. module Utils def self.add(left, right) result = begin left +

    right end unless result.is_a?(Integer) raise "Invalid return value, expected " \ "Integer got #{result.class.name}" end result end end content = File.read(source_path) require 'parser/current' # class CustomBuilder < Parser::Builders::Default builder = CustomBuilder.new # class CustomParser < Parser::CurrentRuby parser = CustomParser.new(builder) # class CustomRewriter < Parser::Rewriter rewriter = CustomRewriter.new rewriter.instance_variable_set(:@parser, parser) buffer = Parser::Base.send(:setup_source_buffer, '(string)', 1, content, parser.default_encoding) ast = parser.parse(content) content = rewriter.rewrite(buffer, ast) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  167. content = File.read(source_path) require 'parser/current' # class CustomBuilder < Parser::Builders::Default

    builder = CustomBuilder.new # class CustomParser < Parser::CurrentRuby parser = CustomParser.new(builder) # class CustomRewriter < Parser::Rewriter rewriter = CustomRewriter.new rewriter.instance_variable_set(:@parser, parser) buffer = Parser::Base.send(:setup_source_buffer, '(string)', 1, content, parser.default_encoding) ast = parser.parse(content) content = rewriter.rewrite(buffer, ast) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq module Utils def self.add(left, right) = Integer left + right end end Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  168. It works! Tokenize • Build AST • (Interpret AST) •

    Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  169. It works! https://github.com/kddeisz/vernacular/blob/ master/lib/vernacular/ast_modifier.rb Tokenize • Build AST • (Interpret

    AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  170. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq • Read/write ISeq • The fun part
  171. Tokenize Build AST Build ISeq Read source Interpret AST #require

    Execute ISeq Write ISeq Read ISeq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part Modify source
  172. Tokenize Build AST Build ISeq Read source Interpret AST #require

    Execute ISeq Write ISeq Read ISeq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part Modify source Modify AST
  173. Let’s go even further Tokenize • Build AST • (Interpret

    AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  174. ~/compiling-ruby $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ Tokenize •

    Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  175. ~/compiling-ruby $ ruby -e "File.write('example.yarb', RubyVM::InstructionSequence.compile_file('example.rb').to_binary)" ~/compiling-ruby $ hexdump example.yarb

    ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  176. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq • Read/write ISeq • The fun part
  177. 0000000 59 41 52 42 02 00 00 00 03

    00 00 00 09 03 00 00 0000010 00 00 00 00 01 00 00 00 04 00 00 00 08 00 00 00 0000020 dc 01 00 00 e0 01 00 00 e9 02 00 00 78 38 36 5f 0000030 36 34 2d 64 61 72 77 69 6e 31 35 00 2a 00 00 00 0000040 00 00 00 00 01 00 00 00 00 00 00 00 0f 00 00 00 0000050 00 00 00 00 04 00 00 00 00 00 00 00 59 00 00 00 0000060 00 00 00 00 03 00 00 00 00 00 00 00 2a 00 00 00 0000070 00 00 00 00 01 00 00 00 00 00 00 00 0e 00 00 00 0000080 00 00 00 00 57 00 00 00 00 00 00 00 03 00 00 00 0000090 00 00 00 00 0f 00 00 00 00 00 00 00 04 00 00 00 00000a0 00 00 00 00 3d 00 00 00 00 00 00 00 00 00 00 00 00000b0 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 00 00000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000d0 00 00 00 00 33 00 00 00 00 00 00 00 00 00 00 00 00000e0 03 00 00 00 06 00 00 00 04 00 00 00 01 00 00 00 00000f0 00 00 00 00 02 00 00 00 00 00 00 00 10 00 00 00 0000100 01 00 00 00 03 00 00 00 00 00 00 00 14 00 00 00 0000110 01 00 00 00 00 00 00 00 14 00 00 00 3c 00 00 00 0000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000150 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 0000160 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0000170 00 00 00 00 03 00 00 00 00 00 00 00 dc 00 00 00 0000180 00 00 00 00 ec 00 00 00 00 00 00 00 00 00 00 00 0000190 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 00 00001a0 00 00 00 00 00 00 00 00 00 00 00 00 f4 00 00 00 00001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00001c0 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 00001d0 00 00 00 00 02 00 00 00 03 00 00 00 14 01 00 00 00001e0 00 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 00001f0 06 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 0000200 f1 00 00 00 08 00 00 00 00 00 00 00 45 00 00 00 0000210 01 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 0000220 65 78 61 6d 70 6c 65 2e 72 62 45 00 00 00 01 00 0000230 00 00 00 00 00 00 43 00 00 00 00 00 00 00 2f 55 0000240 73 65 72 73 2f 6b 64 64 65 69 73 7a 2f 44 6f 63 0000250 75 6d 65 6e 74 73 2f 70 72 6f 6a 65 63 74 73 2f 0000260 73 61 6e 64 62 6f 78 2f 63 6f 6d 70 69 6c 69 6e 0000270 67 2d 72 75 62 79 2f 65 78 61 6d 70 6c 65 2e 72 0000280 62 45 00 00 00 02 00 00 00 00 00 00 00 06 00 00 0000290 00 00 00 00 00 3c 6d 61 69 6e 3e f5 00 00 00 0b 00002a0 00 00 00 00 00 00 00 45 00 00 00 02 00 00 00 00 00002b0 00 00 00 01 00 00 00 00 00 00 00 61 45 00 00 00 00002c0 02 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00002d0 2b 45 00 00 00 02 00 00 00 00 00 00 00 04 00 00 00002e0 00 00 00 00 00 70 75 74 73 00 02 00 00 0c 02 00 00002f0 00 2a 02 00 00 81 02 00 00 9b 02 00 00 a7 02 00 0000300 00 bc 02 00 00 d1 02 00 00 0000309 struct ibf_header { char magic[4]; /* YARB */ unsigned int major_version; unsigned int minor_version; unsigned int size; unsigned int extra_size; unsigned int iseq_list_size; unsigned int id_list_size; unsigned int object_list_size; ibf_offset_t iseq_list_offset; ibf_offset_t id_list_offset; ibf_offset_t object_list_offset; }; Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  178. 0000000 59 41 52 42 02 00 00 00 03

    00 00 00 09 03 00 00 0000010 00 00 00 00 01 00 00 00 04 00 00 00 08 00 00 00 0000020 dc 01 00 00 e0 01 00 00 e9 02 00 00 78 38 36 5f 0000030 36 34 2d 64 61 72 77 69 6e 31 35 00 2a 00 00 00 0000040 00 00 00 00 01 00 00 00 00 00 00 00 0f 00 00 00 0000050 00 00 00 00 04 00 00 00 00 00 00 00 59 00 00 00 0000060 00 00 00 00 03 00 00 00 00 00 00 00 2a 00 00 00 0000070 00 00 00 00 01 00 00 00 00 00 00 00 0e 00 00 00 0000080 00 00 00 00 57 00 00 00 00 00 00 00 03 00 00 00 0000090 00 00 00 00 0f 00 00 00 00 00 00 00 04 00 00 00 00000a0 00 00 00 00 3d 00 00 00 00 00 00 00 00 00 00 00 00000b0 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 00 00000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000d0 00 00 00 00 33 00 00 00 00 00 00 00 00 00 00 00 00000e0 03 00 00 00 06 00 00 00 04 00 00 00 01 00 00 00 00000f0 00 00 00 00 02 00 00 00 00 00 00 00 10 00 00 00 0000100 01 00 00 00 03 00 00 00 00 00 00 00 14 00 00 00 0000110 01 00 00 00 00 00 00 00 14 00 00 00 3c 00 00 00 0000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000150 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 0000160 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0000170 00 00 00 00 03 00 00 00 00 00 00 00 dc 00 00 00 0000180 00 00 00 00 ec 00 00 00 00 00 00 00 00 00 00 00 0000190 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 00 00001a0 00 00 00 00 00 00 00 00 00 00 00 00 f4 00 00 00 00001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00001c0 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 00001d0 00 00 00 00 02 00 00 00 03 00 00 00 14 01 00 00 00001e0 00 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 00001f0 06 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 0000200 f1 00 00 00 08 00 00 00 00 00 00 00 45 00 00 00 0000210 01 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 0000220 65 78 61 6d 70 6c 65 2e 72 62 45 00 00 00 01 00 0000230 00 00 00 00 00 00 43 00 00 00 00 00 00 00 2f 55 0000240 73 65 72 73 2f 6b 64 64 65 69 73 7a 2f 44 6f 63 0000250 75 6d 65 6e 74 73 2f 70 72 6f 6a 65 63 74 73 2f 0000260 73 61 6e 64 62 6f 78 2f 63 6f 6d 70 69 6c 69 6e 0000270 67 2d 72 75 62 79 2f 65 78 61 6d 70 6c 65 2e 72 0000280 62 45 00 00 00 02 00 00 00 00 00 00 00 06 00 00 0000290 00 00 00 00 00 3c 6d 61 69 6e 3e f5 00 00 00 0b 00002a0 00 00 00 00 00 00 00 45 00 00 00 02 00 00 00 00 00002b0 00 00 00 01 00 00 00 00 00 00 00 61 45 00 00 00 00002c0 02 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00002d0 2b 45 00 00 00 02 00 00 00 00 00 00 00 04 00 00 00002e0 00 00 00 00 00 70 75 74 73 00 02 00 00 0c 02 00 00002f0 00 2a 02 00 00 81 02 00 00 9b 02 00 00 a7 02 00 0000300 00 bc 02 00 00 d1 02 00 00 0000309 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  179. 0000000 59 41 52 42 02 00 00 00 03

    00 00 00 09 03 00 00 0000010 00 00 00 00 01 00 00 00 04 00 00 00 08 00 00 00 0000020 dc 01 00 00 e0 01 00 00 e9 02 00 00 78 38 36 5f 0000030 36 34 2d 64 61 72 77 69 6e 31 35 00 2a 00 00 00 0000040 00 00 00 00 01 00 00 00 00 00 00 00 0f 00 00 00 0000050 00 00 00 00 04 00 00 00 00 00 00 00 59 00 00 00 0000060 00 00 00 00 03 00 00 00 00 00 00 00 2a 00 00 00 0000070 00 00 00 00 01 00 00 00 00 00 00 00 0e 00 00 00 0000080 00 00 00 00 57 00 00 00 00 00 00 00 03 00 00 00 0000090 00 00 00 00 0f 00 00 00 00 00 00 00 04 00 00 00 00000a0 00 00 00 00 3d 00 00 00 00 00 00 00 00 00 00 00 00000b0 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 00 00000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000d0 00 00 00 00 33 00 00 00 00 00 00 00 00 00 00 00 00000e0 03 00 00 00 06 00 00 00 04 00 00 00 01 00 00 00 00000f0 00 00 00 00 02 00 00 00 00 00 00 00 10 00 00 00 0000100 01 00 00 00 03 00 00 00 00 00 00 00 14 00 00 00 0000110 01 00 00 00 00 00 00 00 14 00 00 00 3c 00 00 00 0000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000150 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 0000160 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0000170 00 00 00 00 03 00 00 00 00 00 00 00 dc 00 00 00 0000180 00 00 00 00 ec 00 00 00 00 00 00 00 00 00 00 00 0000190 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 00 00001a0 00 00 00 00 00 00 00 00 00 00 00 00 f4 00 00 00 00001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00001c0 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 00001d0 00 00 00 00 02 00 00 00 03 00 00 00 14 01 00 00 00001e0 00 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 00001f0 06 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 0000200 f1 00 00 00 08 00 00 00 00 00 00 00 45 00 00 00 0000210 01 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 0000220 65 78 61 6d 70 6c 65 2e 72 62 45 00 00 00 01 00 0000230 00 00 00 00 00 00 43 00 00 00 00 00 00 00 2f 55 0000240 73 65 72 73 2f 6b 64 64 65 69 73 7a 2f 44 6f 63 0000250 75 6d 65 6e 74 73 2f 70 72 6f 6a 65 63 74 73 2f 0000260 73 61 6e 64 62 6f 78 2f 63 6f 6d 70 69 6c 69 6e 0000270 67 2d 72 75 62 79 2f 65 78 61 6d 70 6c 65 2e 72 0000280 62 45 00 00 00 02 00 00 00 00 00 00 00 06 00 00 0000290 00 00 00 00 00 3c 6d 61 69 6e 3e f5 00 00 00 0b 00002a0 00 00 00 00 00 00 00 45 00 00 00 02 00 00 00 00 00002b0 00 00 00 01 00 00 00 00 00 00 00 61 45 00 00 00 00002c0 02 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00002d0 2b 45 00 00 00 02 00 00 00 00 00 00 00 04 00 00 00002e0 00 00 00 00 00 70 75 74 73 00 02 00 00 0c 02 00 00002f0 00 2a 02 00 00 81 02 00 00 9b 02 00 00 a7 02 00 0000300 00 bc 02 00 00 d1 02 00 00 0000309 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  180. 0000000 59 41 52 42 02 00 00 00 03

    00 00 00 09 03 00 00 0000010 00 00 00 00 01 00 00 00 04 00 00 00 08 00 00 00 0000020 dc 01 00 00 e0 01 00 00 e9 02 00 00 78 38 36 5f 0000030 36 34 2d 64 61 72 77 69 6e 31 35 00 2a 00 00 00 0000040 00 00 00 00 01 00 00 00 00 00 00 00 0f 00 00 00 0000050 00 00 00 00 04 00 00 00 00 00 00 00 59 00 00 00 0000060 00 00 00 00 03 00 00 00 00 00 00 00 2a 00 00 00 0000070 00 00 00 00 01 00 00 00 00 00 00 00 0e 00 00 00 0000080 00 00 00 00 57 00 00 00 00 00 00 00 03 00 00 00 0000090 00 00 00 00 0f 00 00 00 00 00 00 00 04 00 00 00 00000a0 00 00 00 00 3d 00 00 00 00 00 00 00 00 00 00 00 00000b0 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 00 00000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000d0 00 00 00 00 33 00 00 00 00 00 00 00 00 00 00 00 00000e0 03 00 00 00 06 00 00 00 04 00 00 00 01 00 00 00 00000f0 00 00 00 00 02 00 00 00 00 00 00 00 10 00 00 00 0000100 01 00 00 00 03 00 00 00 00 00 00 00 14 00 00 00 0000110 01 00 00 00 00 00 00 00 14 00 00 00 3c 00 00 00 0000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000150 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 0000160 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0000170 00 00 00 00 03 00 00 00 00 00 00 00 dc 00 00 00 0000180 00 00 00 00 ec 00 00 00 00 00 00 00 00 00 00 00 0000190 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 00 00001a0 00 00 00 00 00 00 00 00 00 00 00 00 f4 00 00 00 00001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00001c0 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 00001d0 00 00 00 00 02 00 00 00 03 00 00 00 14 01 00 00 00001e0 00 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 00001f0 06 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 0000200 f1 00 00 00 08 00 00 00 00 00 00 00 45 00 00 00 0000210 01 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 0000220 65 78 61 6d 70 6c 65 2e 72 62 45 00 00 00 01 00 0000230 00 00 00 00 00 00 43 00 00 00 00 00 00 00 2f 55 0000240 73 65 72 73 2f 6b 64 64 65 69 73 7a 2f 44 6f 63 0000250 75 6d 65 6e 74 73 2f 70 72 6f 6a 65 63 74 73 2f 0000260 73 61 6e 64 62 6f 78 2f 63 6f 6d 70 69 6c 69 6e 0000270 67 2d 72 75 62 79 2f 65 78 61 6d 70 6c 65 2e 72 0000280 62 45 00 00 00 02 00 00 00 00 00 00 00 06 00 00 0000290 00 00 00 00 00 3c 6d 61 69 6e 3e f5 00 00 00 0b 00002a0 00 00 00 00 00 00 00 45 00 00 00 02 00 00 00 00 00002b0 00 00 00 01 00 00 00 00 00 00 00 61 45 00 00 00 00002c0 02 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00002d0 2b 45 00 00 00 02 00 00 00 00 00 00 00 04 00 00 00002e0 00 00 00 00 00 70 75 74 73 00 02 00 00 0c 02 00 00002f0 00 2a 02 00 00 81 02 00 00 9b 02 00 00 a7 02 00 0000300 00 bc 02 00 00 d1 02 00 00 0000309 %w[59 41 52 42].map do |ord| ord.to_i(16).chr end.join # YARB Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  181. 0000000 59 41 52 42 02 00 00 00 03

    00 00 00 09 03 00 00 0000010 00 00 00 00 01 00 00 00 04 00 00 00 08 00 00 00 0000020 dc 01 00 00 e0 01 00 00 e9 02 00 00 78 38 36 5f 0000030 36 34 2d 64 61 72 77 69 6e 31 35 00 2a 00 00 00 0000040 00 00 00 00 01 00 00 00 00 00 00 00 0f 00 00 00 0000050 00 00 00 00 04 00 00 00 00 00 00 00 59 00 00 00 0000060 00 00 00 00 03 00 00 00 00 00 00 00 2a 00 00 00 0000070 00 00 00 00 01 00 00 00 00 00 00 00 0e 00 00 00 0000080 00 00 00 00 57 00 00 00 00 00 00 00 03 00 00 00 0000090 00 00 00 00 0f 00 00 00 00 00 00 00 04 00 00 00 00000a0 00 00 00 00 3d 00 00 00 00 00 00 00 00 00 00 00 00000b0 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 00 00000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000d0 00 00 00 00 33 00 00 00 00 00 00 00 00 00 00 00 00000e0 03 00 00 00 06 00 00 00 04 00 00 00 01 00 00 00 00000f0 00 00 00 00 02 00 00 00 00 00 00 00 10 00 00 00 0000100 01 00 00 00 03 00 00 00 00 00 00 00 14 00 00 00 0000110 01 00 00 00 00 00 00 00 14 00 00 00 3c 00 00 00 0000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000150 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 0000160 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0000170 00 00 00 00 03 00 00 00 00 00 00 00 dc 00 00 00 0000180 00 00 00 00 ec 00 00 00 00 00 00 00 00 00 00 00 0000190 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 00 00001a0 00 00 00 00 00 00 00 00 00 00 00 00 f4 00 00 00 00001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00001c0 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 00001d0 00 00 00 00 02 00 00 00 03 00 00 00 14 01 00 00 00001e0 00 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 00001f0 06 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 0000200 f1 00 00 00 08 00 00 00 00 00 00 00 45 00 00 00 0000210 01 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 0000220 65 78 61 6d 70 6c 65 2e 72 62 45 00 00 00 01 00 0000230 00 00 00 00 00 00 43 00 00 00 00 00 00 00 2f 55 0000240 73 65 72 73 2f 6b 64 64 65 69 73 7a 2f 44 6f 63 0000250 75 6d 65 6e 74 73 2f 70 72 6f 6a 65 63 74 73 2f 0000260 73 61 6e 64 62 6f 78 2f 63 6f 6d 70 69 6c 69 6e 0000270 67 2d 72 75 62 79 2f 65 78 61 6d 70 6c 65 2e 72 0000280 62 45 00 00 00 02 00 00 00 00 00 00 00 06 00 00 0000290 00 00 00 00 00 3c 6d 61 69 6e 3e f5 00 00 00 0b 00002a0 00 00 00 00 00 00 00 45 00 00 00 02 00 00 00 00 00002b0 00 00 00 01 00 00 00 00 00 00 00 61 45 00 00 00 00002c0 02 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00002d0 2b 45 00 00 00 02 00 00 00 00 00 00 00 04 00 00 00002e0 00 00 00 00 00 70 75 74 73 00 02 00 00 0c 02 00 00002f0 00 2a 02 00 00 81 02 00 00 9b 02 00 00 a7 02 00 0000300 00 bc 02 00 00 d1 02 00 00 0000309 %w[59 41 52 42].map do |ord| ord.to_i(16).chr end.join # YARB %w{78 38 36 5f 36 34 2d 64 61 72 77 69 6e 31 35}.map do |ord| ord.to_i(16).chr end.join # x86_64-darwin15 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  182. 0000000 59 41 52 42 02 00 00 00 03

    00 00 00 09 03 00 00 0000010 00 00 00 00 01 00 00 00 04 00 00 00 08 00 00 00 0000020 dc 01 00 00 e0 01 00 00 e9 02 00 00 78 38 36 5f 0000030 36 34 2d 64 61 72 77 69 6e 31 35 00 2a 00 00 00 0000040 00 00 00 00 01 00 00 00 00 00 00 00 0f 00 00 00 0000050 00 00 00 00 04 00 00 00 00 00 00 00 59 00 00 00 0000060 00 00 00 00 03 00 00 00 00 00 00 00 2a 00 00 00 0000070 00 00 00 00 01 00 00 00 00 00 00 00 0e 00 00 00 0000080 00 00 00 00 57 00 00 00 00 00 00 00 03 00 00 00 0000090 00 00 00 00 0f 00 00 00 00 00 00 00 04 00 00 00 00000a0 00 00 00 00 3d 00 00 00 00 00 00 00 00 00 00 00 00000b0 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 00 00000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000d0 00 00 00 00 33 00 00 00 00 00 00 00 00 00 00 00 00000e0 03 00 00 00 06 00 00 00 04 00 00 00 01 00 00 00 00000f0 00 00 00 00 02 00 00 00 00 00 00 00 10 00 00 00 0000100 01 00 00 00 03 00 00 00 00 00 00 00 14 00 00 00 0000110 01 00 00 00 00 00 00 00 14 00 00 00 3c 00 00 00 0000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000150 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 0000160 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0000170 00 00 00 00 03 00 00 00 00 00 00 00 dc 00 00 00 0000180 00 00 00 00 ec 00 00 00 00 00 00 00 00 00 00 00 0000190 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 00 00001a0 00 00 00 00 00 00 00 00 00 00 00 00 f4 00 00 00 00001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00001c0 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 00001d0 00 00 00 00 02 00 00 00 03 00 00 00 14 01 00 00 00001e0 00 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 00001f0 06 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 0000200 f1 00 00 00 08 00 00 00 00 00 00 00 45 00 00 00 0000210 01 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 0000220 65 78 61 6d 70 6c 65 2e 72 62 45 00 00 00 01 00 0000230 00 00 00 00 00 00 43 00 00 00 00 00 00 00 2f 55 0000240 73 65 72 73 2f 6b 64 64 65 69 73 7a 2f 44 6f 63 0000250 75 6d 65 6e 74 73 2f 70 72 6f 6a 65 63 74 73 2f 0000260 73 61 6e 64 62 6f 78 2f 63 6f 6d 70 69 6c 69 6e 0000270 67 2d 72 75 62 79 2f 65 78 61 6d 70 6c 65 2e 72 0000280 62 45 00 00 00 02 00 00 00 00 00 00 00 06 00 00 0000290 00 00 00 00 00 3c 6d 61 69 6e 3e f5 00 00 00 0b 00002a0 00 00 00 00 00 00 00 45 00 00 00 02 00 00 00 00 00002b0 00 00 00 01 00 00 00 00 00 00 00 61 45 00 00 00 00002c0 02 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00002d0 2b 45 00 00 00 02 00 00 00 00 00 00 00 04 00 00 00002e0 00 00 00 00 00 70 75 74 73 00 02 00 00 0c 02 00 00002f0 00 2a 02 00 00 81 02 00 00 9b 02 00 00 a7 02 00 0000300 00 bc 02 00 00 d1 02 00 00 0000309 Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  183. 0000000 59 41 52 42 02 00 00 00 03

    00 00 00 09 03 00 00 0000010 00 00 00 00 01 00 00 00 04 00 00 00 08 00 00 00 0000020 dc 01 00 00 e0 01 00 00 e9 02 00 00 78 38 36 5f 0000030 36 34 2d 64 61 72 77 69 6e 31 35 00 2a 00 00 00 0000040 00 00 00 00 01 00 00 00 00 00 00 00 0f 00 00 00 0000050 00 00 00 00 04 00 00 00 00 00 00 00 59 00 00 00 0000060 00 00 00 00 03 00 00 00 00 00 00 00 2a 00 00 00 0000070 00 00 00 00 01 00 00 00 00 00 00 00 0e 00 00 00 0000080 00 00 00 00 57 00 00 00 00 00 00 00 03 00 00 00 0000090 00 00 00 00 0f 00 00 00 00 00 00 00 04 00 00 00 00000a0 00 00 00 00 3d 00 00 00 00 00 00 00 00 00 00 00 00000b0 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 00 00000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000d0 00 00 00 00 33 00 00 00 00 00 00 00 00 00 00 00 00000e0 03 00 00 00 06 00 00 00 04 00 00 00 01 00 00 00 00000f0 00 00 00 00 02 00 00 00 00 00 00 00 10 00 00 00 0000100 01 00 00 00 03 00 00 00 00 00 00 00 14 00 00 00 0000110 01 00 00 00 00 00 00 00 14 00 00 00 3c 00 00 00 0000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000150 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 0000160 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0000170 00 00 00 00 03 00 00 00 00 00 00 00 dc 00 00 00 0000180 00 00 00 00 ec 00 00 00 00 00 00 00 00 00 00 00 0000190 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 00 00001a0 00 00 00 00 00 00 00 00 00 00 00 00 f4 00 00 00 00001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00001c0 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 00001d0 00 00 00 00 02 00 00 00 03 00 00 00 14 01 00 00 00001e0 00 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 00001f0 06 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 0000200 f1 00 00 00 08 00 00 00 00 00 00 00 45 00 00 00 0000210 01 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 0000220 65 78 61 6d 70 6c 65 2e 72 62 45 00 00 00 01 00 0000230 00 00 00 00 00 00 43 00 00 00 00 00 00 00 2f 55 0000240 73 65 72 73 2f 6b 64 64 65 69 73 7a 2f 44 6f 63 0000250 75 6d 65 6e 74 73 2f 70 72 6f 6a 65 63 74 73 2f 0000260 73 61 6e 64 62 6f 78 2f 63 6f 6d 70 69 6c 69 6e 0000270 67 2d 72 75 62 79 2f 65 78 61 6d 70 6c 65 2e 72 0000280 62 45 00 00 00 02 00 00 00 00 00 00 00 06 00 00 0000290 00 00 00 00 00 3c 6d 61 69 6e 3e f5 00 00 00 0b 00002a0 00 00 00 00 00 00 00 45 00 00 00 02 00 00 00 00 00002b0 00 00 00 01 00 00 00 00 00 00 00 61 45 00 00 00 00002c0 02 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00002d0 2b 45 00 00 00 02 00 00 00 00 00 00 00 04 00 00 00002e0 00 00 00 00 00 70 75 74 73 00 02 00 00 0c 02 00 00002f0 00 2a 02 00 00 81 02 00 00 9b 02 00 00 a7 02 00 0000300 00 bc 02 00 00 d1 02 00 00 0000309 header Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  184. 0000000 59 41 52 42 02 00 00 00 03

    00 00 00 09 03 00 00 0000010 00 00 00 00 01 00 00 00 04 00 00 00 08 00 00 00 0000020 dc 01 00 00 e0 01 00 00 e9 02 00 00 78 38 36 5f 0000030 36 34 2d 64 61 72 77 69 6e 31 35 00 2a 00 00 00 0000040 00 00 00 00 01 00 00 00 00 00 00 00 0f 00 00 00 0000050 00 00 00 00 04 00 00 00 00 00 00 00 59 00 00 00 0000060 00 00 00 00 03 00 00 00 00 00 00 00 2a 00 00 00 0000070 00 00 00 00 01 00 00 00 00 00 00 00 0e 00 00 00 0000080 00 00 00 00 57 00 00 00 00 00 00 00 03 00 00 00 0000090 00 00 00 00 0f 00 00 00 00 00 00 00 04 00 00 00 00000a0 00 00 00 00 3d 00 00 00 00 00 00 00 00 00 00 00 00000b0 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 00 00000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000d0 00 00 00 00 33 00 00 00 00 00 00 00 00 00 00 00 00000e0 03 00 00 00 06 00 00 00 04 00 00 00 01 00 00 00 00000f0 00 00 00 00 02 00 00 00 00 00 00 00 10 00 00 00 0000100 01 00 00 00 03 00 00 00 00 00 00 00 14 00 00 00 0000110 01 00 00 00 00 00 00 00 14 00 00 00 3c 00 00 00 0000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000150 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 0000160 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0000170 00 00 00 00 03 00 00 00 00 00 00 00 dc 00 00 00 0000180 00 00 00 00 ec 00 00 00 00 00 00 00 00 00 00 00 0000190 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 00 00001a0 00 00 00 00 00 00 00 00 00 00 00 00 f4 00 00 00 00001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00001c0 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 00001d0 00 00 00 00 02 00 00 00 03 00 00 00 14 01 00 00 00001e0 00 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 00001f0 06 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 0000200 f1 00 00 00 08 00 00 00 00 00 00 00 45 00 00 00 0000210 01 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 0000220 65 78 61 6d 70 6c 65 2e 72 62 45 00 00 00 01 00 0000230 00 00 00 00 00 00 43 00 00 00 00 00 00 00 2f 55 0000240 73 65 72 73 2f 6b 64 64 65 69 73 7a 2f 44 6f 63 0000250 75 6d 65 6e 74 73 2f 70 72 6f 6a 65 63 74 73 2f 0000260 73 61 6e 64 62 6f 78 2f 63 6f 6d 70 69 6c 69 6e 0000270 67 2d 72 75 62 79 2f 65 78 61 6d 70 6c 65 2e 72 0000280 62 45 00 00 00 02 00 00 00 00 00 00 00 06 00 00 0000290 00 00 00 00 00 3c 6d 61 69 6e 3e f5 00 00 00 0b 00002a0 00 00 00 00 00 00 00 45 00 00 00 02 00 00 00 00 00002b0 00 00 00 01 00 00 00 00 00 00 00 61 45 00 00 00 00002c0 02 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00002d0 2b 45 00 00 00 02 00 00 00 00 00 00 00 04 00 00 00002e0 00 00 00 00 00 70 75 74 73 00 02 00 00 0c 02 00 00002f0 00 2a 02 00 00 81 02 00 00 9b 02 00 00 a7 02 00 0000300 00 bc 02 00 00 d1 02 00 00 0000309 header RUBY_VERSION Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  185. 0000000 59 41 52 42 02 00 00 00 03

    00 00 00 09 03 00 00 0000010 00 00 00 00 01 00 00 00 04 00 00 00 08 00 00 00 0000020 dc 01 00 00 e0 01 00 00 e9 02 00 00 78 38 36 5f 0000030 36 34 2d 64 61 72 77 69 6e 31 35 00 2a 00 00 00 0000040 00 00 00 00 01 00 00 00 00 00 00 00 0f 00 00 00 0000050 00 00 00 00 04 00 00 00 00 00 00 00 59 00 00 00 0000060 00 00 00 00 03 00 00 00 00 00 00 00 2a 00 00 00 0000070 00 00 00 00 01 00 00 00 00 00 00 00 0e 00 00 00 0000080 00 00 00 00 57 00 00 00 00 00 00 00 03 00 00 00 0000090 00 00 00 00 0f 00 00 00 00 00 00 00 04 00 00 00 00000a0 00 00 00 00 3d 00 00 00 00 00 00 00 00 00 00 00 00000b0 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 00 00000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000d0 00 00 00 00 33 00 00 00 00 00 00 00 00 00 00 00 00000e0 03 00 00 00 06 00 00 00 04 00 00 00 01 00 00 00 00000f0 00 00 00 00 02 00 00 00 00 00 00 00 10 00 00 00 0000100 01 00 00 00 03 00 00 00 00 00 00 00 14 00 00 00 0000110 01 00 00 00 00 00 00 00 14 00 00 00 3c 00 00 00 0000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000150 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 0000160 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0000170 00 00 00 00 03 00 00 00 00 00 00 00 dc 00 00 00 0000180 00 00 00 00 ec 00 00 00 00 00 00 00 00 00 00 00 0000190 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 00 00001a0 00 00 00 00 00 00 00 00 00 00 00 00 f4 00 00 00 00001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00001c0 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 00001d0 00 00 00 00 02 00 00 00 03 00 00 00 14 01 00 00 00001e0 00 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 00001f0 06 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 0000200 f1 00 00 00 08 00 00 00 00 00 00 00 45 00 00 00 0000210 01 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 0000220 65 78 61 6d 70 6c 65 2e 72 62 45 00 00 00 01 00 0000230 00 00 00 00 00 00 43 00 00 00 00 00 00 00 2f 55 0000240 73 65 72 73 2f 6b 64 64 65 69 73 7a 2f 44 6f 63 0000250 75 6d 65 6e 74 73 2f 70 72 6f 6a 65 63 74 73 2f 0000260 73 61 6e 64 62 6f 78 2f 63 6f 6d 70 69 6c 69 6e 0000270 67 2d 72 75 62 79 2f 65 78 61 6d 70 6c 65 2e 72 0000280 62 45 00 00 00 02 00 00 00 00 00 00 00 06 00 00 0000290 00 00 00 00 00 3c 6d 61 69 6e 3e f5 00 00 00 0b 00002a0 00 00 00 00 00 00 00 45 00 00 00 02 00 00 00 00 00002b0 00 00 00 01 00 00 00 00 00 00 00 61 45 00 00 00 00002c0 02 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00002d0 2b 45 00 00 00 02 00 00 00 00 00 00 00 04 00 00 00002e0 00 00 00 00 00 70 75 74 73 00 02 00 00 0c 02 00 00002f0 00 2a 02 00 00 81 02 00 00 9b 02 00 00 a7 02 00 0000300 00 bc 02 00 00 d1 02 00 00 0000309 header RUBY_VERSION iseq offset list Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  186. 0000000 59 41 52 42 02 00 00 00 03

    00 00 00 09 03 00 00 0000010 00 00 00 00 01 00 00 00 04 00 00 00 08 00 00 00 0000020 dc 01 00 00 e0 01 00 00 e9 02 00 00 78 38 36 5f 0000030 36 34 2d 64 61 72 77 69 6e 31 35 00 2a 00 00 00 0000040 00 00 00 00 01 00 00 00 00 00 00 00 0f 00 00 00 0000050 00 00 00 00 04 00 00 00 00 00 00 00 59 00 00 00 0000060 00 00 00 00 03 00 00 00 00 00 00 00 2a 00 00 00 0000070 00 00 00 00 01 00 00 00 00 00 00 00 0e 00 00 00 0000080 00 00 00 00 57 00 00 00 00 00 00 00 03 00 00 00 0000090 00 00 00 00 0f 00 00 00 00 00 00 00 04 00 00 00 00000a0 00 00 00 00 3d 00 00 00 00 00 00 00 00 00 00 00 00000b0 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 00 00000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000d0 00 00 00 00 33 00 00 00 00 00 00 00 00 00 00 00 00000e0 03 00 00 00 06 00 00 00 04 00 00 00 01 00 00 00 00000f0 00 00 00 00 02 00 00 00 00 00 00 00 10 00 00 00 0000100 01 00 00 00 03 00 00 00 00 00 00 00 14 00 00 00 0000110 01 00 00 00 00 00 00 00 14 00 00 00 3c 00 00 00 0000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000150 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 0000160 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0000170 00 00 00 00 03 00 00 00 00 00 00 00 dc 00 00 00 0000180 00 00 00 00 ec 00 00 00 00 00 00 00 00 00 00 00 0000190 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 00 00001a0 00 00 00 00 00 00 00 00 00 00 00 00 f4 00 00 00 00001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00001c0 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 00001d0 00 00 00 00 02 00 00 00 03 00 00 00 14 01 00 00 00001e0 00 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 00001f0 06 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 0000200 f1 00 00 00 08 00 00 00 00 00 00 00 45 00 00 00 0000210 01 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 0000220 65 78 61 6d 70 6c 65 2e 72 62 45 00 00 00 01 00 0000230 00 00 00 00 00 00 43 00 00 00 00 00 00 00 2f 55 0000240 73 65 72 73 2f 6b 64 64 65 69 73 7a 2f 44 6f 63 0000250 75 6d 65 6e 74 73 2f 70 72 6f 6a 65 63 74 73 2f 0000260 73 61 6e 64 62 6f 78 2f 63 6f 6d 70 69 6c 69 6e 0000270 67 2d 72 75 62 79 2f 65 78 61 6d 70 6c 65 2e 72 0000280 62 45 00 00 00 02 00 00 00 00 00 00 00 06 00 00 0000290 00 00 00 00 00 3c 6d 61 69 6e 3e f5 00 00 00 0b 00002a0 00 00 00 00 00 00 00 45 00 00 00 02 00 00 00 00 00002b0 00 00 00 01 00 00 00 00 00 00 00 61 45 00 00 00 00002c0 02 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00002d0 2b 45 00 00 00 02 00 00 00 00 00 00 00 04 00 00 00002e0 00 00 00 00 00 70 75 74 73 00 02 00 00 0c 02 00 00002f0 00 2a 02 00 00 81 02 00 00 9b 02 00 00 a7 02 00 0000300 00 bc 02 00 00 d1 02 00 00 0000309 header RUBY_VERSION iseq offset list id offset list Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  187. 0000000 59 41 52 42 02 00 00 00 03

    00 00 00 09 03 00 00 0000010 00 00 00 00 01 00 00 00 04 00 00 00 08 00 00 00 0000020 dc 01 00 00 e0 01 00 00 e9 02 00 00 78 38 36 5f 0000030 36 34 2d 64 61 72 77 69 6e 31 35 00 2a 00 00 00 0000040 00 00 00 00 01 00 00 00 00 00 00 00 0f 00 00 00 0000050 00 00 00 00 04 00 00 00 00 00 00 00 59 00 00 00 0000060 00 00 00 00 03 00 00 00 00 00 00 00 2a 00 00 00 0000070 00 00 00 00 01 00 00 00 00 00 00 00 0e 00 00 00 0000080 00 00 00 00 57 00 00 00 00 00 00 00 03 00 00 00 0000090 00 00 00 00 0f 00 00 00 00 00 00 00 04 00 00 00 00000a0 00 00 00 00 3d 00 00 00 00 00 00 00 00 00 00 00 00000b0 00 00 00 00 00 00 00 00 00 00 00 00 30 00 00 00 00000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00000d0 00 00 00 00 33 00 00 00 00 00 00 00 00 00 00 00 00000e0 03 00 00 00 06 00 00 00 04 00 00 00 01 00 00 00 00000f0 00 00 00 00 02 00 00 00 00 00 00 00 10 00 00 00 0000100 01 00 00 00 03 00 00 00 00 00 00 00 14 00 00 00 0000110 01 00 00 00 00 00 00 00 14 00 00 00 3c 00 00 00 0000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 * 0000150 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 0000160 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0000170 00 00 00 00 03 00 00 00 00 00 00 00 dc 00 00 00 0000180 00 00 00 00 ec 00 00 00 00 00 00 00 00 00 00 00 0000190 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 00 00 00001a0 00 00 00 00 00 00 00 00 00 00 00 00 f4 00 00 00 00001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00001c0 00 00 00 00 01 00 00 00 00 00 00 00 02 00 00 00 00001d0 00 00 00 00 02 00 00 00 03 00 00 00 14 01 00 00 00001e0 00 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 00001f0 06 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 0000200 f1 00 00 00 08 00 00 00 00 00 00 00 45 00 00 00 0000210 01 00 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 0000220 65 78 61 6d 70 6c 65 2e 72 62 45 00 00 00 01 00 0000230 00 00 00 00 00 00 43 00 00 00 00 00 00 00 2f 55 0000240 73 65 72 73 2f 6b 64 64 65 69 73 7a 2f 44 6f 63 0000250 75 6d 65 6e 74 73 2f 70 72 6f 6a 65 63 74 73 2f 0000260 73 61 6e 64 62 6f 78 2f 63 6f 6d 70 69 6c 69 6e 0000270 67 2d 72 75 62 79 2f 65 78 61 6d 70 6c 65 2e 72 0000280 62 45 00 00 00 02 00 00 00 00 00 00 00 06 00 00 0000290 00 00 00 00 00 3c 6d 61 69 6e 3e f5 00 00 00 0b 00002a0 00 00 00 00 00 00 00 45 00 00 00 02 00 00 00 00 00002b0 00 00 00 01 00 00 00 00 00 00 00 61 45 00 00 00 00002c0 02 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00002d0 2b 45 00 00 00 02 00 00 00 00 00 00 00 04 00 00 00002e0 00 00 00 00 00 70 75 74 73 00 02 00 00 0c 02 00 00002f0 00 2a 02 00 00 81 02 00 00 9b 02 00 00 a7 02 00 0000300 00 bc 02 00 00 d1 02 00 00 0000309 header RUBY_VERSION iseq offset list id offset list object offset list Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  188. ~/compiling-ruby $ ruby -e "RubyVM::InstructionSequence.compile_file('example.rb').to_binary" [ 0 - 3] magic

    = YARB [ 4 - 7] major_version = 2 [ 8 - 11] minor_version = 3 [12 - 15] size = 801 [16 - 19] extra_size = 0 [20 - 23] iseq_list_size = 1 [24 - 27] id_list_size = 4 [28 - 31] object_list_size = 9 [32 - 35] iseq_list_offset = 468 [36 - 39] id_list_offset = 472 [40 - 43] object_list_offset = 765 [44 - 60] RUBY_PLATFORM = x86_64-darwin15 [468 - 471] iseq 0 = 276 [472 - 479] id 0 = a [480 - 487] id 1 = + [488 - 495] id 2 = puts iseqs = [{ location.pathobj = ["example.rb", true] location.base_label = "<main>" location.label = "<main>" iseq_encoded = param.opt_table = NULL param.keyword = NULL line_info_table = [{ position: 0, line_no: 3 }, { position: 6, line_no: 4 }] local_table = [36945] catch_table = NULL parent_iseq = -1 local_iseq = is_entries = NULL ci_entries = [{ mid: 43, flag: 16, orig_argc: 1 }, { mid: 15617, flag: 20, orig_argc: 1 }] cc_entries = NULL mark_ary = 0 body size = 20 }] ~/compiling-ruby $ Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  189. #load_iseq allows us to provide our own bytecode. Tokenize •

    Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  190. #load_iseq allows us to provide our own bytecode. We can

    inject debugging code into Ruby to learn the binary structure of a YARV bytecode dump. Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  191. #load_iseq allows us to provide our own bytecode. We can

    inject debugging code into Ruby to learn the binary structure of a YARV bytecode dump. So, we can write our own code that compiles down to YARV bytecode. Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  192. class RubyVM::InstructionSequence def self.load_iseq(source_path) Tokenize • Build AST • (Interpret

    AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part begin iseq = RubyVM::InstructionSequence .compile(File.read(source_path)) File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end iseq_path = File.join( File.expand_path(__dir__, '.iseq'), source_path.gsub(/[^A-Za-z0-9\._-]/) { |c| '%02x' % c.ord }.gsub('.rb', '.yarb')) if File.exist?(iseq_path) && (File.mtime(source_path) <= File.mtime(iseq_path)) return RubyVM::InstructionSequence .load_from_binary(File.binread(iseq_path)) end
  193. class RubyVM::InstructionSequence def self.load_iseq(source_path) Tokenize • Build AST • (Interpret

    AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part begin iseq = RubyVM::InstructionSequence .compile(File.read(source_path)) File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end ...
  194. class RubyVM::InstructionSequence def self.load_iseq(source_path) Tokenize • Build AST • (Interpret

    AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part begin content = File.read(source_path) iseq = RubyVM::InstructionSequence.compile(content) File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end ...
  195. class RubyVM::InstructionSequence def self.load_iseq(source_path) Tokenize • Build AST • (Interpret

    AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part begin content = File.read(source_path) iseq = if File.basename(source_path).end_with?('.tb.rb') Tuby.compile(content) else RubyVM::InstructionSequence.compile(content) end File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end ...
  196. class RubyVM::InstructionSequence def self.load_iseq(source_path) Tokenize • Build AST • (Interpret

    AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part begin content = File.read(source_path) iseq = if File.basename(source_path).end_with?('.tb.rb') Tuby.compile(content) else RubyVM::InstructionSequence.compile(content) end File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end ...
  197. class RubyVM::InstructionSequence def self.load_iseq(source_path) Tokenize • Build AST • (Interpret

    AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part begin content = File.read(source_path) iseq = if File.basename(source_path).end_with?('.tb.rb') Tuby.compile(content) else RubyVM::InstructionSequence.compile(content) end File.binwrite(iseq_path, iseq.to_binary) iseq rescue SyntaxError, RuntimeError nil end end end ... Contract = String → #to_binary Tuby::compile RubyVM::InstructionSequence::compile
  198. Tokenize Build AST Build ISeq Read source Interpret AST #require

    Execute ISeq Write ISeq Read ISeq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part Modify source Modify AST
  199. Tokenize Build AST Build ISeq Read source Interpret AST #require

    Execute ISeq Write ISeq Read ISeq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part Modify source Modify AST Build Tuby
  200. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq • Read/write ISeq • The fun part Because we’re writing YARV bytecode, we maintain interoperability with Ruby.
  201. Because we’re writing YARV bytecode, we maintain interoperability with Ruby.

    We can get the benefits of a compiler and instruction elimination without having to write a native extension. Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  202. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq • Read/write ISeq • The fun part /* formulae.tb.rb */ impl Formulae::sphere_volume(radius) { (4 / 3) * 3.14159 * (radius ** 3) }
  203. Tokenize • Build AST • (Interpret AST) • Build ISeq

    • Execute ISeq • Read/write ISeq • The fun part /* formulae.tb.rb */ impl Formulae::sphere_volume(radius) { (4 / 3) * 3.14159 * (radius ** 3) } /* formulae.rb */ Formulae.sphere_volume(5) # 392.68987500000003
  204. Does it work? Tokenize • Build AST • (Interpret AST)

    • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  205. Does it work? Kind of… Tokenize • Build AST •

    (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  206. Does it work? Kind of… https://github.com/kddeisz/tuby Tokenize • Build AST

    • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  207. Tokenize Build AST Build ISeq Read source Interpret AST #require

    Execute ISeq Write ISeq Read ISeq Tokenize • Build AST • (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part Modify source Modify AST Build Tuby
  208. #require Execute ISeq Read ISeq Tokenize • Build AST •

    (Interpret AST) • Build ISeq • Execute ISeq • Read/write ISeq • The fun part
  209. Why would you ever do this?

  210. Programming is using human language to represent computer lexicon.

  211. Programming is using human language to represent computer lexicon. Code

    gets read many more times than it gets written.
  212. Programming is using human language to represent computer lexicon. Code

    gets read many more times than it gets written. Optimize for reading code.
  213. What’s next?

  214. #require Execute ISeq Read ISeq

  215. #require Execute ISeq Read ISeq JIT Compile ISeq

  216. None
  217. #require Execute ISeq Read ISeq

  218. #require Execute ISeq Read ISeq Queue ISeq

  219. #require Execute ISeq Read ISeq Queue ISeq Start worker

  220. #require Execute ISeq Read ISeq Queue ISeq Start worker Dequeue

    ISeq
  221. #require Execute ISeq Read ISeq Queue ISeq Start worker Generate

    C Dequeue ISeq
  222. #require Execute ISeq Read ISeq Queue ISeq Start worker Generate

    C Dequeue ISeq Compile native
  223. #require Execute ISeq Read ISeq Queue ISeq Start worker Generate

    C Dequeue ISeq Compile native Call native
  224. None
  225. https://github.com/ruby/ruby/pull/1782

  226. What’s next?

  227. Compiling Ruby github.com/kddeisz/compiling-ruby Kevin Deisz @kddeisz