Ruby Parser Basics
@y
d
a
h
Ky
ob
a
sh
i
.r
b Of
f l
i
ne
#1
2
02
4
.0
2
.0
5 (Mo
n
.)
Slide 2
Slide 2 text
ɾ Name: Yudai Takada
ɾ GitHub: @ydah / X: @ydah_
ɾ Software Engineer at ANDPAD, Inc.
ɾ Co-Founder of Kyobashi.rb
ɾ Member of RuboCop RSpec team
ɾ Contributor of Lrama
ɾ Implementation of Parameterizing rules
self.inspect
Slide 3
Slide 3 text
Today’s talk
Slide 4
Slide 4 text
Ruby Parser Basics
Slide 5
Slide 5 text
Age of Parser!
Slide 6
Slide 6 text
In Ruby 3.3.0
release notes
Slide 7
Slide 7 text
Two new parsers?!
Slide 8
Slide 8 text
• Lrama
• A pure Ruby parser generator, not a parser
• Created as a CRuby parser generator to
replace GNU Bison
• By adding new features, Lrama aims to make
parse.y simpler and more maintainable
Two new Ruby parsers (1)
Slide 9
Slide 9 text
• Prism
• A Ruby parser library written by hand from scratch
• It is under development to replace the Ruby
parser with Prism
• For node structure, AST is also designed from
scratch
Two new Ruby parsers (2)
Slide 10
Slide 10 text
• Lexer
• Reads the source code character by character and returns a single token when
it is meaningfully cohesive
• Parser
• Check the Lexer output token for grammatical conformance and create an AST
• Appropriate error handling is done for grammatically incorrect programs
• Compiler
• Ruby compiles into machine language for virtual machines(YARV), and then
the virtual machines execute
Components related to parsing
Slide 11
Slide 11 text
the process by which code becomes executablE
Source
Code
Slide 12
Slide 12 text
the process by which code becomes executablE
Token
Source
Code
Lexical analysis
Slide 13
Slide 13 text
the process by which code becomes executablE
Token AST
Source
Code
Parsing
Slide 14
Slide 14 text
the process by which code becomes executablE
0101
0101
Token AST ISeq
Source
Code
Compile
Slide 15
Slide 15 text
• A type of bottom-up parsing
• Lookahead Left to Right, Rightmost
derivation parser
• Parser generators such as racc, Bison, and
Lrama generate parsers of this type
What is LALR(1) parser?
Slide 16
Slide 16 text
def method_name(parameters)
"method body"
end
Parsing procedure
Slide 17
Slide 17 text
def method_name lparen parameter rparen
string end
Parsing procedure
Slide 18
Slide 18 text
method_name lparen parameter rparen
↓
Parsing procedure
def
string end
Slide 19
Slide 19 text
↓
method_name lparen parameter rparen
Parsing procedure
def
string end
Slide 20
Slide 20 text
↓
lparen parameter rparen
Parsing procedure
def method_name
string end
Slide 21
Slide 21 text
↓
lparen parameter rparen
Parsing procedure
def method_name
string end
Slide 22
Slide 22 text
↓
lparen parameter rparen
Parsing procedure
def method_name
string end
Slide 23
Slide 23 text
↓
lparen parameter rparen
Parsing procedure
def method_name
string end
Slide 24
Slide 24 text
def method_name lparen parameter rparen
string end
↓
Parsing procedure
Slide 25
Slide 25 text
def method_name(parameters)
"method body"
end
Parsing procedure
Slide 26
Slide 26 text
def method_name(parameters)
"method body"
end
Method de
fi
nition
Parsing procedure
Slide 27
Slide 27 text
• Ruby parsers are one of the hottest topics
• Maybe, there will be sessions at RubyKaigi
2024 where the progress of each (Lrama and
Prism) is presented (No, there must be.)
• Please enjoy the Parser session!
Finally…