Tweet
Tweet

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

/1+2/

Slide 4

Slide 4 text

match /1+2/

Slide 5

Slide 5 text

match /1+2 /

Slide 6

Slide 6 text

match /1+2 #/

Slide 7

Slide 7 text

/\d+/ =~ "3" match /1+2 #/

Slide 8

Slide 8 text

/(?\d+)/ =~ "3" match /1+2 #/

Slide 9

Slide 9 text

* = <<~EOF, /(?\d+)/ =~ "3" matchy match EOF match /1+2 #/

Slide 10

Slide 10 text

* = <<~EOF, /(?\d+)/ =~ "3" match /1+2 #/

Slide 11

Slide 11 text

What does this code do?

Slide 12

Slide 12 text

What does this code do?

Slide 13

Slide 13 text

What does this code do? How would you fi nd out?

Slide 14

Slide 14 text

Dependencies

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

Applications

Slide 17

Slide 17 text

External libraries Applications

Slide 18

Slide 18 text

Standard libraries External libraries Applications

Slide 19

Slide 19 text

Byte code/IR Standard libraries External libraries Applications

Slide 20

Slide 20 text

Compiler Byte code/IR Standard libraries External libraries Applications

Slide 21

Slide 21 text

Parser Compiler Byte code/IR Standard libraries External libraries Applications

Slide 22

Slide 22 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications

Slide 23

Slide 23 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications

Slide 24

Slide 24 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications

Slide 25

Slide 25 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications def ❨╯°□°❩╯︵┻━┻ puts "Calm down, yo." end

Slide 26

Slide 26 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications def ❨╯°□°❩╯︵┻━┻ puts "Calm down, yo." end

Slide 27

Slide 27 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications [[KEYWORD_DEF, "def"], [IDENTIFIER, "❨╯°□°❩╯︵┻━┻"], [NEWLINE, "\n"], [IDENTIFIER, "puts"], [STRING_BEGIN, "\""], [STRING_CONTENT, "Calm down, yo."], [STRING_END, "\""], [NEWLINE, "\n"], [KEYWORD_END, "end"], [NEWLINE, "\n"], [EOF, ""]]

Slide 28

Slide 28 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications @ DefNode (location: (1,0)-(3,3)) ├── name: :❨╯°□°❩╯︵┻━┻ └── body: @ StatementsNode (location: (2,2)-(2,23)) └── body: (length: 1) └── @ CallNode (location: (2,2)-(2,23)) ├── arguments: │ @ ArgumentsNode (location: (2,7)-(2,23)) │ └── arguments: (length: 1) │ └── @ StringNode (location: (2,7)-(2,23)) │ └── unescaped: "Calm down, yo." └── name: :puts

Slide 29

Slide 29 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications @ DefNode (location: (1,0)-(3,3)) ├── name: :❨╯°□°❩╯︵┻━┻ └── body: @ StatementsNode (location: (2,2)-(2,23)) └── body: (length: 1) └── @ CallNode (location: (2,2)-(2,23)) ├── arguments: │ @ ArgumentsNode (location: (2,7)-(2,23)) │ └── arguments: (length: 1) │ └── @ StringNode (location: (2,7)-(2,23)) │ └── unescaped: "Calm down, yo." └── name: :puts

Slide 30

Slide 30 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications @ DefNode (location: (1,0)-(3,3)) ├── name: :❨╯°□°❩╯︵┻━┻ └── body: @ StatementsNode (location: (2,2)-(2,23)) └── body: (length: 1) └── @ CallNode (location: (2,2)-(2,23)) ├── arguments: │ @ ArgumentsNode (location: (2,7)-(2,23)) │ └── arguments: (length: 1) │ └── @ StringNode (location: (2,7)-(2,23)) │ └── unescaped: "Calm down, yo." └── name: :puts

Slide 31

Slide 31 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications @ DefNode (location: (1,0)-(3,3)) ├── name: :❨╯°□°❩╯︵┻━┻ └── body: @ StatementsNode (location: (2,2)-(2,23)) └── body: (length: 1) └── @ CallNode (location: (2,2)-(2,23)) ├── arguments: │ @ ArgumentsNode (location: (2,7)-(2,23)) │ └── arguments: (length: 1) │ └── @ StringNode (location: (2,7)-(2,23)) │ └── unescaped: "Calm down, yo." └── name: :puts putself

Slide 32

Slide 32 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications @ DefNode (location: (1,0)-(3,3)) ├── name: :❨╯°□°❩╯︵┻━┻ └── body: @ StatementsNode (location: (2,2)-(2,23)) └── body: (length: 1) └── @ CallNode (location: (2,2)-(2,23)) ├── arguments: │ @ ArgumentsNode (location: (2,7)-(2,23)) │ └── arguments: (length: 1) │ └── @ StringNode (location: (2,7)-(2,23)) │ └── unescaped: "Calm down, yo." └── name: :puts putself

Slide 33

Slide 33 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications @ DefNode (location: (1,0)-(3,3)) ├── name: :❨╯°□°❩╯︵┻━┻ └── body: @ StatementsNode (location: (2,2)-(2,23)) └── body: (length: 1) └── @ CallNode (location: (2,2)-(2,23)) ├── arguments: │ @ ArgumentsNode (location: (2,7)-(2,23)) │ └── arguments: (length: 1) │ └── @ StringNode (location: (2,7)-(2,23)) │ └── unescaped: "Calm down, yo." └── name: :puts putself putstring "Calm down, yo.”

Slide 34

Slide 34 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications @ DefNode (location: (1,0)-(3,3)) ├── name: :❨╯°□°❩╯︵┻━┻ └── body: @ StatementsNode (location: (2,2)-(2,23)) └── body: (length: 1) └── @ CallNode (location: (2,2)-(2,23)) ├── arguments: │ @ ArgumentsNode (location: (2,7)-(2,23)) │ └── arguments: (length: 1) │ └── @ StringNode (location: (2,7)-(2,23)) │ └── unescaped: "Calm down, yo." └── name: :puts putself putstring "Calm down, yo.”

Slide 35

Slide 35 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications @ DefNode (location: (1,0)-(3,3)) ├── name: :❨╯°□°❩╯︵┻━┻ └── body: @ StatementsNode (location: (2,2)-(2,23)) └── body: (length: 1) └── @ CallNode (location: (2,2)-(2,23)) ├── arguments: │ @ ArgumentsNode (location: (2,7)-(2,23)) │ └── arguments: (length: 1) │ └── @ StringNode (location: (2,7)-(2,23)) │ └── unescaped: "Calm down, yo." └── name: :puts putself putstring "Calm down, yo.” send :puts

Slide 36

Slide 36 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications @ DefNode (location: (1,0)-(3,3)) ├── name: :❨╯°□°❩╯︵┻━┻ └── body: @ StatementsNode (location: (2,2)-(2,23)) └── body: (length: 1) └── @ CallNode (location: (2,2)-(2,23)) ├── arguments: │ @ ArgumentsNode (location: (2,7)-(2,23)) │ └── arguments: (length: 1) │ └── @ StringNode (location: (2,7)-(2,23)) │ └── unescaped: "Calm down, yo." └── name: :puts putself putstring "Calm down, yo.” send :puts

Slide 37

Slide 37 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications @ DefNode (location: (1,0)-(3,3)) ├── name: :❨╯°□°❩╯︵┻━┻ └── body: @ StatementsNode (location: (2,2)-(2,23)) └── body: (length: 1) └── @ CallNode (location: (2,2)-(2,23)) ├── arguments: │ @ ArgumentsNode (location: (2,7)-(2,23)) │ └── arguments: (length: 1) │ └── @ StringNode (location: (2,7)-(2,23)) │ └── unescaped: "Calm down, yo." └── name: :puts putself putstring "Calm down, yo.” send :puts leave

Slide 38

Slide 38 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications putself putstring "Calm down, yo.” send :puts leave

Slide 39

Slide 39 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications putself putstring "Calm down, yo.” send :puts leave

Slide 40

Slide 40 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications == disasm: # 0000 putself 0001 putstring "Calm down, yo." 0003 send 0005 leave

Slide 41

Slide 41 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications == disasm: # 0000 putself 0001 putstring "Calm down, yo." 0003 send 0005 leave

Slide 42

Slide 42 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications == disasm: # 0000 putself 0001 putstring "Calm down, yo." 0003 send 0005 leave

Slide 43

Slide 43 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications == disasm: # 0000 putself 0001 putstring "Calm down, yo." 0003 send 0005 leave

Slide 44

Slide 44 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications == disasm: # 0000 putself 0001 putstring "Calm down, yo." 0003 send 0005 leave

Slide 45

Slide 45 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications == disasm: # 0000 putself 0001 putstring "Calm down, yo." 0003 send 0005 leave

Slide 46

Slide 46 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications

Slide 47

Slide 47 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications

Slide 48

Slide 48 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications Sidekiq.❨╯°□°❩╯︵┻━┻

Slide 49

Slide 49 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications

Slide 50

Slide 50 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications

Slide 51

Slide 51 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y

Slide 52

Slide 52 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c

Slide 53

Slide 53 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV

Slide 54

Slide 54 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib

Slide 55

Slide 55 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org

Slide 56

Slide 56 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org

Slide 57

Slide 57 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org

Slide 58

Slide 58 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org RubyParser.y

Slide 59

Slide 59 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org RubyParser.y IRBuilder.java

Slide 60

Slide 60 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org RubyParser.y IRBuilder.java JRuby IR

Slide 61

Slide 61 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org RubyParser.y IRBuilder.java JRuby IR JRuby stdlib

Slide 62

Slide 62 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org RubyParser.y IRBuilder.java JRuby IR JRuby stdlib rubygems.org*

Slide 63

Slide 63 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org RubyParser.y IRBuilder.java JRuby IR JRuby stdlib rubygems.org* Applications*

Slide 64

Slide 64 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org

Slide 65

Slide 65 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org RubyParser.y

Slide 66

Slide 66 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org RubyParser.y Translator.java

Slide 67

Slide 67 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org RubyParser.y Translator.java Tru ff l e

Slide 68

Slide 68 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org RubyParser.y Translator.java Tru ff l e TRuby stdlib

Slide 69

Slide 69 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org RubyParser.y Translator.java Tru ff l e TRuby stdlib rubygems.org*

Slide 70

Slide 70 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org RubyParser.y Translator.java Tru ff l e TRuby stdlib rubygems.org* Applications*

Slide 71

Slide 71 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org

Slide 72

Slide 72 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org natalie_parser

Slide 73

Slide 73 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org natalie_parser pass1.rb

Slide 74

Slide 74 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org natalie_parser pass1.rb instructions/*.rb

Slide 75

Slide 75 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org natalie_parser pass1.rb instructions/*.rb Natalie stdlib

Slide 76

Slide 76 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org natalie_parser pass1.rb instructions/*.rb Natalie stdlib rubygems.org*

Slide 77

Slide 77 text

Ruby source Parser Compiler Byte code/IR Standard libraries External libraries Applications parse.y compile.c YARV CRuby stdlib rubygems.org natalie_parser pass1.rb instructions/*.rb Natalie stdlib rubygems.org* Applications*

Slide 78

Slide 78 text

Ruby source

Slide 79

Slide 79 text

Ruby source parse.y compile.c YARV

Slide 80

Slide 80 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR

Slide 81

Slide 81 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e

Slide 82

Slide 82 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 83

Slide 83 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 84

Slide 84 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 85

Slide 85 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 86

Slide 86 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 87

Slide 87 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 88

Slide 88 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 89

Slide 89 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 90

Slide 90 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 91

Slide 91 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 92

Slide 92 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 93

Slide 93 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 94

Slide 94 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 95

Slide 95 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 96

Slide 96 text

Ruby source parse.y compile.c YARV RubyParser.y IRBuilder.java JRuby IR RubyParser.y Translator.java Tru ffl e natalie_parser pass1.rb instructions/*.rb

Slide 97

Slide 97 text

Ruby source

Slide 98

Slide 98 text

No content

Slide 99

Slide 99 text

Ruby source

Slide 100

Slide 100 text

Ruby source Parser

Slide 101

Slide 101 text

Ruby source Parser Compiler

Slide 102

Slide 102 text

Ruby source Parser Compiler Byte code/IR

Slide 103

Slide 103 text

Ruby source Parser Compiler Byte code/IR

Slide 104

Slide 104 text

Ruby source Parser Compiler Byte code/IR parse.y

Slide 105

Slide 105 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c

Slide 106

Slide 106 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV

Slide 107

Slide 107 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV

Slide 108

Slide 108 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV parser gem

Slide 109

Slide 109 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV parser gem rubocop

Slide 110

Slide 110 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV

Slide 111

Slide 111 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV ruby_parser gem

Slide 112

Slide 112 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV ruby_parser gem fasterer

Slide 113

Slide 113 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV

Slide 114

Slide 114 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV tree-sitter-ruby

Slide 115

Slide 115 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV tree-sitter-ruby vscode-ruby

Slide 116

Slide 116 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV

Slide 117

Slide 117 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV ripper

Slide 118

Slide 118 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV ripper syntax_tree

Slide 119

Slide 119 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV

Slide 120

Slide 120 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV

Slide 121

Slide 121 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV RubyVM::ISeq

Slide 122

Slide 122 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV RubyVM::ISeq typeprof

Slide 123

Slide 123 text

Ruby source Parser Compiler Byte code/IR parse.y compile.c YARV RubyVM::ISeq tenderjit

Slide 124

Slide 124 text

Ruby source

Slide 125

Slide 125 text

Ruby source parser gem

Slide 126

Slide 126 text

Ruby source parser gem ruby_parser gem

Slide 127

Slide 127 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby

Slide 128

Slide 128 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper

Slide 129

Slide 129 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop

Slide 130

Slide 130 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph

Slide 131

Slide 131 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser

Slide 132

Slide 132 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next

Slide 133

Slide 133 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next standard

Slide 134

Slide 134 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next standard fasterer

Slide 135

Slide 135 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next standard fl og fasterer

Slide 136

Slide 136 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next standard fl og fl ay fasterer

Slide 137

Slide 137 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next standard fl og fl ay debride fasterer

Slide 138

Slide 138 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next standard fl og fl ay debride fasterer vscode-ruby

Slide 139

Slide 139 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next standard fl og fl ay debride fasterer vscode-ruby syntax_tree

Slide 140

Slide 140 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next standard fl og fl ay debride fasterer vscode-ruby syntax_tree rubyfmt

Slide 141

Slide 141 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next standard fl og fl ay debride fasterer vscode-ruby syntax_tree rubyfmt irb

Slide 142

Slide 142 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next standard fl og fl ay debride fasterer vscode-ruby syntax_tree rubyfmt irb rufo

Slide 143

Slide 143 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next standard fl og fl ay debride fasterer vscode-ruby syntax_tree rubyfmt irb rufo prettier

Slide 144

Slide 144 text

Ruby source parser gem ruby_parser gem tree-sitter-ruby ripper rubocop solargraph unparser ruby-next standard fl og fl ay debride fasterer vscode-ruby syntax_tree rubyfmt irb rufo prettier

Slide 145

Slide 145 text

Ruby source

Slide 146

Slide 146 text

No content

Slide 147

Slide 147 text

Di ff erent representations lead to di ff erent assumptions.

Slide 148

Slide 148 text

Different representations

Slide 149

Slide 149 text

Different representations • Every tool represents Ruby di ff erently

Slide 150

Slide 150 text

Different representations • Every tool represents Ruby di ff erently • Every tool requires contributors to relearn

Slide 151

Slide 151 text

Different representations • Every tool represents Ruby di ff erently • Every tool requires contributors to relearn • None of this knowledge translates to contributing to CRuby

Slide 152

Slide 152 text

Maintaining a Ruby parser is a massive maintenance burden.

Slide 153

Slide 153 text

Maintenance burden

Slide 154

Slide 154 text

Maintenance burden • Every parser has to update every time new syntax is introduced

Slide 155

Slide 155 text

Maintenance burden • Every parser has to update every time new syntax is introduced • Very minor changes have very major consequences

Slide 156

Slide 156 text

Maintenance burden • Every parser has to update every time new syntax is introduced • Very minor changes have very major consequences • Helping Ruby can inadvertently hurt the Ruby ecosystem

Slide 157

Slide 157 text

Fractured ecosystem

Slide 158

Slide 158 text

Fractured ecosystem

Slide 159

Slide 159 text

Fractured ecosystem Ruby p a rsers

Slide 160

Slide 160 text

Fractured ecosystem • ruby/ruby/parse.y Ruby p a rsers

Slide 161

Slide 161 text

Fractured ecosystem • ruby/ruby/parse.y • ruby/ruby/ext/ripper/parse.y Ruby p a rsers

Slide 162

Slide 162 text

Fractured ecosystem • ruby/ruby/parse.y • ruby/ruby/ext/ripper/parse.y • lib-ruby-parser/lib-ruby-parser Ruby p a rsers

Slide 163

Slide 163 text

Fractured ecosystem • ruby/ruby/parse.y • ruby/ruby/ext/ripper/parse.y • lib-ruby-parser/lib-ruby-parser • mruby/mruby Ruby p a rsers

Slide 164

Slide 164 text

Fractured ecosystem • ruby/ruby/parse.y • ruby/ruby/ext/ripper/parse.y • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet Ruby p a rsers

Slide 165

Slide 165 text

Fractured ecosystem • ruby/ruby/parse.y • ruby/ruby/ext/ripper/parse.y • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser Ruby p a rsers

Slide 166

Slide 166 text

Fractured ecosystem • ruby/ruby/parse.y • ruby/ruby/ext/ripper/parse.y • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser • seattlerb/ruby_parser Ruby p a rsers

Slide 167

Slide 167 text

Fractured ecosystem • ruby/ruby/parse.y • ruby/ruby/ext/ripper/parse.y • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser • seattlerb/ruby_parser Ruby p a rsers • jruby/jruby

Slide 168

Slide 168 text

Fractured ecosystem • ruby/ruby/parse.y • ruby/ruby/ext/ripper/parse.y • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser • seattlerb/ruby_parser Ruby p a rsers • jruby/jruby • oracle/tru ff l eruby

Slide 169

Slide 169 text

Fractured ecosystem • ruby/ruby/parse.y • ruby/ruby/ext/ripper/parse.y • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser • seattlerb/ruby_parser Ruby p a rsers • jruby/jruby • oracle/tru ff l eruby • tree-sitter/tree-sitter-ruby

Slide 170

Slide 170 text

Fractured ecosystem • ruby/ruby/parse.y • ruby/ruby/ext/ripper/parse.y • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser • seattlerb/ruby_parser Ruby p a rsers • jruby/jruby • oracle/tru ff l eruby • tree-sitter/tree-sitter-ruby • sisshiki1969/ruruby

Slide 171

Slide 171 text

Fractured ecosystem • ruby/ruby/parse.y • ruby/ruby/ext/ripper/parse.y • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser • seattlerb/ruby_parser Ruby p a rsers • jruby/jruby • oracle/tru ff l eruby • tree-sitter/tree-sitter-ruby • sisshiki1969/ruruby • natalie-lang/natalie_parser

Slide 172

Slide 172 text

Fractured ecosystem Synt a x trees

Slide 173

Slide 173 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree Synt a x trees

Slide 174

Slide 174 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} Synt a x trees

Slide 175

Slide 175 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree Synt a x trees

Slide 176

Slide 176 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser Synt a x trees

Slide 177

Slide 177 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby Synt a x trees

Slide 178

Slide 178 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet Synt a x trees

Slide 179

Slide 179 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees

Slide 180

Slide 180 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor

Slide 181

Slide 181 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast

Slide 182

Slide 182 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e

Slide 183

Slide 183 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter

Slide 184

Slide 184 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby

Slide 185

Slide 185 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby • natalie-lang/natalie_parser

Slide 186

Slide 186 text

No content

Slide 187

Slide 187 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby • natalie-lang/natalie_parser

Slide 188

Slide 188 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby • natalie-lang/natalie_parser • ruby/prism

Slide 189

Slide 189 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby • natalie-lang/natalie_parser • ruby/prism

Slide 190

Slide 190 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby • natalie-lang/natalie_parser • ruby/prism

Slide 191

Slide 191 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby • natalie-lang/natalie_parser • ruby/prism

Slide 192

Slide 192 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby • natalie-lang/natalie_parser • ruby/prism

Slide 193

Slide 193 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby • natalie-lang/natalie_parser • ruby/prism

Slide 194

Slide 194 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby • natalie-lang/natalie_parser • ruby/prism

Slide 195

Slide 195 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby • natalie-lang/natalie_parser • ruby/prism

Slide 196

Slide 196 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby • natalie-lang/natalie_parser • ruby/prism

Slide 197

Slide 197 text

Fractured ecosystem • NODE, RubyVM::AbstractSyntaxTree • Ripper::SexpBuilder{,PP} • ruby-syntax-tree/syntax_tree • lib-ruby-parser/lib-ruby-parser • mruby/mruby • typedruby/typedruby, sorbet/sorbet • whitequark/parser, rubocop/rubocop-ast Synt a x trees • seattlerb/sexp_processor • org.jruby.ast • Tru ff l e • tree-sitter/tree-sitter • sisshiki1969/ruruby • natalie-lang/natalie_parser • ruby/prism

Slide 198

Slide 198 text

Fractured ecosystem • lib-ruby-parser/lib-ruby-parser • mruby/mruby Synt a x trees • tree-sitter/tree-sitter • sisshiki1969/ruruby • ruby/prism

Slide 199

Slide 199 text

Fractured ecosystem • ruby/prism • lib-ruby-parser/lib-ruby-parser • mruby/mruby • tree-sitter/tree-sitter • sisshiki1969/ruruby Synt a x trees

Slide 200

Slide 200 text

Fractured ecosystem • ruby/prism • lib-ruby-parser/lib-ruby-parser • mruby/mruby • tree-sitter/tree-sitter • sisshiki1969/ruruby Synt a x trees

Slide 201

Slide 201 text

ecosystem • ruby/prism • lib-ruby-parser/lib-ruby-parser • mruby/mruby • tree-sitter/tree-sitter • sisshiki1969/ruruby Synt a x trees

Slide 202

Slide 202 text

Prism

Slide 203

Slide 203 text

Prism

Slide 204

Slide 204 text

Prism • Compatibility

Slide 205

Slide 205 text

Prism • Compatibility • Maintainability

Slide 206

Slide 206 text

Prism • Compatibility • Maintainability • Error tolerance

Slide 207

Slide 207 text

Prism • Compatibility • Maintainability • Error tolerance • Portability

Slide 208

Slide 208 text

Prism • Compatibility • Maintainability • Error tolerance • Portability • Performance

Slide 209

Slide 209 text

Timeline

Slide 210

Slide 210 text

Timeline

Slide 211

Slide 211 text

Timeline 2022/01/01 2023/01/01 2024/01/01 2022/07/01 2023/07/01

Slide 212

Slide 212 text

Timeline 2022/01/01 2023/01/01 2024/01/01 2022/07/01 2023/07/01 Prototype fi rst commit

Slide 213

Slide 213 text

Timeline 2022/01/01 2023/01/01 2024/01/01 2022/07/01 2023/07/01 Prototype fi rst commit First call with CRuby team

Slide 214

Slide 214 text

Timeline 2022/01/01 2023/01/01 2024/01/01 2022/07/01 2023/07/01 Prototype fi rst commit First call with CRuby team YARP fi rst commit

Slide 215

Slide 215 text

Timeline 2022/01/01 2023/01/01 2024/01/01 2022/07/01 2023/07/01 Prototype fi rst commit First call with CRuby team YARP fi rst commit 👶

Slide 216

Slide 216 text

Timeline 2022/01/01 2023/01/01 2024/01/01 2022/07/01 2023/07/01 Prototype fi rst commit First call with CRuby team YARP fi rst commit Tru ff l eRuby fi rst commit 👶

Slide 217

Slide 217 text

Timeline 2022/01/01 2023/01/01 2024/01/01 2022/07/01 2023/07/01 Prototype fi rst commit First call with CRuby team YARP fi rst commit Tru ff l eRuby fi rst commit JRuby fi rst commit 👶

Slide 218

Slide 218 text

Timeline 2022/01/01 2023/01/01 2024/01/01 2022/07/01 2023/07/01 Prototype fi rst commit First call with CRuby team YARP fi rst commit Tru ff l eRuby fi rst commit JRuby fi rst commit CRuby fi rst commit 👶

Slide 219

Slide 219 text

Timeline 2022/01/01 2023/01/01 2024/01/01 2022/07/01 2023/07/01 Prototype fi rst commit First call with CRuby team YARP fi rst commit Tru ff l eRuby fi rst commit JRuby fi rst commit CRuby fi rst commit Syntax Tree fi rst commit 👶

Slide 220

Slide 220 text

Timeline 2022/01/01 2023/01/01 2024/01/01 2022/07/01 2023/07/01 Prototype fi rst commit First call with CRuby team YARP fi rst commit Tru ff l eRuby fi rst commit JRuby fi rst commit CRuby fi rst commit Syntax Tree fi rst commit Parser fi rst commit 👶

Slide 221

Slide 221 text

Timeline 2022/01/01 2023/01/01 2024/01/01 2022/07/01 2023/07/01 Prototype fi rst commit First call with CRuby team YARP fi rst commit Tru ff l eRuby fi rst commit JRuby fi rst commit CRuby fi rst commit Syntax Tree fi rst commit Parser fi rst commit Rename to prism 👶

Slide 222

Slide 222 text

How was it built?

Slide 223

Slide 223 text

Design

Slide 224

Slide 224 text

Design Appro a ch

Slide 225

Slide 225 text

Design • Comparative analysis with existing trees Appro a ch

Slide 226

Slide 226 text

Design • Comparative analysis with existing trees • "Semantic" versus "syntactic" trees Appro a ch

Slide 227

Slide 227 text

Design • Comparative analysis with existing trees • "Semantic" versus "syntactic" trees • Provide enough value for all consumers Appro a ch

Slide 228

Slide 228 text

Design Implement a tions

Slide 229

Slide 229 text

Design Implement a tions • Make compilation as simple as possible

Slide 230

Slide 230 text

Design Implement a tions • Make compilation as simple as possible • Never have to check child to compile current node

Slide 231

Slide 231 text

Design Implement a tions • Make compilation as simple as possible • Never have to check child to compile current node • Cover as much common functionality as possible

Slide 232

Slide 232 text

Design Tooling

Slide 233

Slide 233 text

Design Tooling • Named fi elds and known types

Slide 234

Slide 234 text

Design Tooling • Named fi elds and known types • Nodes with enough information to round-trip

Slide 235

Slide 235 text

Design Tooling • Named fi elds and known types • Nodes with enough information to round-trip • Location information for as much as possible

Slide 236

Slide 236 text

Design M a inten a nce

Slide 237

Slide 237 text

Design M a inten a nce • Template as much as possible

Slide 238

Slide 238 text

Design M a inten a nce • Template as much as possible • Provide as much documentation as possible

Slide 239

Slide 239 text

Design M a inten a nce • Template as much as possible • Provide as much documentation as possible • Build up as big of a test suite as possible

Slide 240

Slide 240 text

Challenges

Slide 241

Slide 241 text

Challenges

Slide 242

Slide 242 text

Challenges Encoding

Slide 243

Slide 243 text

Challenges Encoding • # -*- encoding: Shift_JIS -*-

Slide 244

Slide 244 text

Challenges Encoding • # -*- encoding: Shift_JIS -*- • 90 possible encodings for source fi les

Slide 245

Slide 245 text

Challenges Encoding • # -*- encoding: Shift_JIS -*- • 90 possible encodings for source fi les • 154 aliases for those possible encodings

Slide 246

Slide 246 text

Challenges Encoding • # -*- encoding: Shift_JIS -*- • 90 possible encodings for source fi les • 154 aliases for those possible encodings • => we must support them all

Slide 247

Slide 247 text

Challenges Esc a ping

Slide 248

Slide 248 text

Challenges Esc a ping • \\, \a, \b, \e, \f, \n, \r, \r\n, \s, \t, \v (character(s))

Slide 249

Slide 249 text

Challenges Esc a ping • \\, \a, \b, \e, \f, \n, \r, \r\n, \s, \t, \v (character(s)) • \000 (octal)

Slide 250

Slide 250 text

Challenges Esc a ping • \\, \a, \b, \e, \f, \n, \r, \r\n, \s, \t, \v (character(s)) • \000 (octal) • \x00 (hex, can change encoding)

Slide 251

Slide 251 text

Challenges Esc a ping • \\, \a, \b, \e, \f, \n, \r, \r\n, \s, \t, \v (character(s)) • \000 (octal) • \x00 (hex, can change encoding) • \u0000 \u{00 00} (hex, can change encoding)

Slide 252

Slide 252 text

Challenges Esc a ping • \\, \a, \b, \e, \f, \n, \r, \r\n, \s, \t, \v (character(s)) • \000 (octal) • \x00 (hex, can change encoding) • \u0000 \u{00 00} (hex, can change encoding) • \cx, \C-x, \M-x, \C-\M-x, \M-\C-x

Slide 253

Slide 253 text

Challenges Esc a ping • \\, \a, \b, \e, \f, \n, \r, \r\n, \s, \t, \v (character(s)) • \000 (octal) • \x00 (hex, can change encoding) • \u0000 \u{00 00} (hex, can change encoding) • \cx, \C-x, \M-x, \C-\M-x, \M-\C-x • => brute force testing

Slide 254

Slide 254 text

Challenges Regul a r Expressions

Slide 255

Slide 255 text

Challenges Regul a r Expressions • You must know local variables to parse correctly

Slide 256

Slide 256 text

Challenges Regul a r Expressions • You must know local variables to parse correctly • /(?\d+)/ can make local variables

Slide 257

Slide 257 text

Challenges Regul a r Expressions • You must know local variables to parse correctly • /(?\d+)/ can make local variables • => we need a regex parser

Slide 258

Slide 258 text

Challenges Heredocs

Slide 259

Slide 259 text

Challenges Heredocs • Declarations function as any other expression

Slide 260

Slide 260 text

Challenges Heredocs • Declarations function as any other expression • => every newline, you must check for heredoc contents

Slide 261

Slide 261 text

What’s the current status?

Slide 262

Slide 262 text

Current status

Slide 263

Slide 263 text

Current status • C API • CRuby, Natalie, Rust tooling • Sorbet, Artichoke, C/C++/Rust/Zig tools

Slide 264

Slide 264 text

Current status void Init_ISeq(void) { rb_define_singleton_method( rb_cISeq, "compile_prism", iseqw_s_compile_prism, -1 ); } CRuby

Slide 265

Slide 265 text

Current status class Natalie::Parser def ast Prism .parse(@code_str, filepath: @path) .value .accept(PathVisitor.new(@path)) end end N a t a lie

Slide 266

Slide 266 text

Current status • C API • CRuby, Natalie, Rust tooling • Sorbet, Artichoke, C/C++/Rust/Zig tools

Slide 267

Slide 267 text

Current status • C API • CRuby, Natalie, Rust tooling • Sorbet, Artichoke, C/C++/Rust/Zig tools • Ruby API • Natalie, Ruby tools

Slide 268

Slide 268 text

Current status def parse(source_buffer) @source_buffer = source_buffer build_ast( ::Prism.parse( source_buffer.source, source_buffer.name ).value ) end p a rser-prism

Slide 269

Slide 269 text

Current status • C API • CRuby, Natalie, Rust tooling • Sorbet, Artichoke, C/C++/Rust/Zig tools • Ruby API • Natalie, Ruby tools

Slide 270

Slide 270 text

Current status • C API • CRuby, Natalie, Rust tooling • Sorbet, Artichoke, C/C++/Rust/Zig tools • Ruby API • Natalie, Ruby tools • Serialization API • JRuby, Tru ffl eRuby, JavaScript

Slide 271

Slide 271 text

public interface ParserBindingPrism { void pm_serialize_parse( Buffer buffer, @In byte[] source, int size, @In byte[] metadata ); } Current status JRuby

Slide 272

Slide 272 text

JNIEXPORT jbyteArray JNICALL Java_org_prism_Parser_parseAndSerialize(...) { ... pm_parse_serialize((uint8_t*) bytes, size, &buffer, NULL); ... } Current status Truf f leRuby

Slide 273

Slide 273 text

Current status J a v a Script export async function loadPrism() { const fileURL = new URL("prism.wasm", import.meta.url); const wasm = await WebAssembly.compile( await readFile(fileURLToPath(fileURL)) ); const wasi = new WASI({ version: "preview1" }); const imports = wasi.getImportObject(); const instance = await WebAssembly.instantiate(wasm, imports); wasi.initialize(instance); return function parse(source) { return parsePrism(instance.exports, source); } }

Slide 274

Slide 274 text

What’s next?

Slide 275

Slide 275 text

What’s next?

Slide 276

Slide 276 text

What’s next? • Improved error tolerance and messaging

Slide 277

Slide 277 text

What’s next? • Improved error tolerance and messaging • Performance enhancements

Slide 278

Slide 278 text

What’s next? • Improved error tolerance and messaging • Performance enhancements • Multi-version support

Slide 279

Slide 279 text

What’s next? • Improved error tolerance and messaging • Performance enhancements • Multi-version support • Further integrations into the ecosystem

Slide 280

Slide 280 text

What’s next? • Improved error tolerance and messaging • Performance enhancements • Multi-version support • Further integrations into the ecosystem • Improved Ruby library for external tools

Slide 281

Slide 281 text

What’s next? • Improved error tolerance and messaging • Performance enhancements • Multi-version support • Further integrations into the ecosystem • Improved Ruby library for external tools • Contributor community around a single tool

Slide 282

Slide 282 text

I have worked on parsing Ruby for 6 years.

Slide 283

Slide 283 text

Parsing Ruby is not fun.

Slide 284

Slide 284 text

Building Ruby tools is very fun.

Slide 285

Slide 285 text

No content

Slide 286

Slide 286 text

No content

Slide 287

Slide 287 text

No content

Slide 288

Slide 288 text

We can have all of these tools, and more.

Slide 289

Slide 289 text

* = <<~EOF, /(?\d+)/ =~ "3" match /1+2 #/

Slide 290

Slide 290 text

The Future of Understanding Ruby

Slide 291

Slide 291 text

• An ecosystem of runtimes, that get to work on what makes each one unique. The Future of Understanding Ruby

Slide 292

Slide 292 text

• An ecosystem of runtimes, that get to work on what makes each one unique. • An ecosystem of tools, that all agree on Ruby representations. The Future of Understanding Ruby

Slide 293

Slide 293 text

• An ecosystem of runtimes, that get to work on what makes each one unique. • An ecosystem of tools, that all agree on Ruby representations. • Lower barriers to entry for contributions around the ecosystem. The Future of Understanding Ruby

Slide 294

Slide 294 text

• An ecosystem of runtimes, that get to work on what makes each one unique. • An ecosystem of tools, that all agree on Ruby representations. • Lower barriers to entry for contributions around the ecosystem. • A community that works together to empower every tool and everyone. The Future of Understanding Ruby

Slide 295

Slide 295 text

Let’s build it together.

Slide 296

Slide 296 text

No content