How does Lrama make the Ruby parser grammar G.O.A.T.?

Slide 1

Slide 1 text

How does Lrama make the Ruby parser grammar G.O.A.T.? EuRuKo 2024—Sarajevo, Bosnia & Herzegovina Hotel Hills: Congress & Thermal Spa Resort 13 September 2024 Yudai Takada (@ydah)

Slide 2

Slide 2 text

Dobro jutro!

Slide 3

Slide 3 text

Veoma sam sretan što mogu doći u Bosnu!

Slide 4

Slide 4 text

EURUKO

Slide 5

Slide 5 text

This is my fi rst time visiting Bosnia

Slide 6

Slide 6 text

I LOVE Ruby and Beer!

Slide 7

Slide 7 text

So I love Bosnia as well!

Slide 8

Slide 8 text

RubyKaigi RubyKaigi is an international conference on the Ruby programming language in Japan. 8FMPPLGPSXBSEUPZPVSBUUFOEBODF

Slide 9

Slide 9 text

RubyKaigi 2025 April 16th - 18th, 2025 Matsuyama City, Ehime Prefecture

Slide 10

Slide 10 text

Supported by

Slide 11

Slide 11 text

Yudai Takada https://ydah.net X: @ydah_ GitHub: @ydah Mastodon: @[email protected]

Slide 12

Slide 12 text

Osaka, Japan

Slide 13

Slide 13 text

RuboCop Headquarters RuboCop RSpec team

Slide 14

Slide 14 text

interagent/committee Committee maintainers team

Slide 15

Slide 15 text

ruby/lrama Committer

Slide 16

Slide 16 text

These Slides

Slide 17

Slide 17 text

Today's talk ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 18

Slide 18 text

ruby/lrama ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 19

Slide 19 text

Confidential “It is hard, if not possible, to build a robust compiler out of "hacks."” — Ravi Sethi Beginning of the journey ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 20

Slide 20 text

• LALR (1) parser generator written by Ruby • Output a LALR parser written by C with “parse.y” as input • Replace GNU Bison with Lrama in Ruby 3.3.0 ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ What is Lrama? ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 21

Slide 21 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Why use Lrama instead of Bison? • We can move away from supporting multiple versions • Its easy to add new functions and grammar. ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 22

Slide 22 text

@ydah @yui-knk @junk0612 Author ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Lrama Committers ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 23

Slide 23 text

Maybe your first impression would be ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 24

Slide 24 text

PRONUNCIATION IS SO DIFFICULT

Slide 25

Slide 25 text

laa·muh ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

Why not llama? ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 28

Slide 28 text

^^ LALR (1) parser generator ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 29

Slide 29 text

LRama ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 30

Slide 30 text

What is LALR(1) parser? ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 31

Slide 31 text

LA LR (1) parser ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Look-Ahead Left-to-right Rightmost derivation Number of tokens to Look-Ahead ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 32

Slide 32 text

• Lrama is a Pure Ruby LALR parser generator • In Ruby 3.3.0 replaces GNU Bison with Lrama • LALR (1) parser refers to a Lookahead Left-to- Right, Rightmost derivation parser. ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Summary ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 33

Slide 33 text

“Computers are good at following instructions, but not at reading your mind.” — Donald E. Knuth Components Related to Parsing ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 34

Slide 34 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Lexer Parser Compiler Text Tokens AST ISeq ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 35

Slide 35 text

Slide 36

Slide 36 text

• Converts source code into tokens, the building blocks for further compilation ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Lexer, Lexical analysis ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 37

Slide 37 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ def method_name(pa r am) puts pa r am end Lexer, Lexical analysis ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 38

Slide 38 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ def method_name(pa r am) puts pa r am end Lexer, Lexical analysis def method_name ( param ) ... ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 39

Slide 39 text

Slide 40

Slide 40 text

• Check the Lexer output token for grammatical conformance and create an AST • Appropriate error handling is done for grammatically incorrect programs ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Parser ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 41

Slide 41 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ def method_name(pa r am) puts pa r am end Parser def method_name ( param ) ... defn args body fcall puts args ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 42

Slide 42 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ❯ r uby - - dump=pa r set r ee - e 'p "EuRuKo"' : (snip) # @ NODE_SCOPE (id: 3, line: 1, location: (1,0)-(1,10)) # +- nd_tbl: (empty) # +- nd_a r gs: # | (null node) # +- nd_body: # @ NODE_FCALL (id: 0, line: 1, location: (1,0)-(1,10))* # +- nd_mid: :p # +- nd_a r gs: # @ NODE_LIST (id: 2, line: 1, location: (1,2)-(1,10)) # +- nd_alen: 1 # +- nd_head: # | @ NODE_STR (id: 1, line: 1, location: (1,2)-(1,10)) # | +- nd_lit: "EuRuKo" # +- nd_next: # (null node) Parser ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 43

Slide 43 text

Slide 44

Slide 44 text

• Ruby compiles into machine language for virtual machines(YARV), and then the virtual machines execute ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Compiler ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 45

Slide 45 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 46

Slide 46 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ i r b(main) : 001> iseq = RubyVM : : Inst r uctionSequence.compile("p 'EuRuKo'") i r b(main) : 002> puts iseq.disasm = = disasm: #@ : 1 (1,0)-(1,10)> 0000 putself ( 1)[Li] 0001 putst r ing "EuRuKo" 0003 opt_send_without_block 0005 leave Compiler ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 47

Slide 47 text

Parser Generator? ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 48

Slide 48 text

• Generates a parser from a formal grammar speci fi cation • Takes a set of rules de fi ning the syntax of a programming language or data format and produces code that can parse strings according to those rules ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Parser Genarator ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 49

Slide 49 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Parser Generator Rules Parser ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 50

Slide 50 text

• Components related to parsing are lexer, parser and compiler • Parser generater generates a parser from a set of rules de fi ning the syntax of a programming language or data format ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Summary ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 51

Slide 51 text

“The only languages which do not change are dead ones.” — David Crystal What is parser grammar? ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 52

Slide 52 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Recursively eunmrable Context-sensitive Context-free Regular Turing machine Linear-bounded non-deterministic Turing machine non-deterministic pushdown auto-maton fi nite-state automaton Chomsky hierarchy ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 53

Slide 53 text

In order for Bison to parse a language, it must be described by a context-free grammar. ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ In Bison https://www.gnu.org/software/bison/manual/html_node/Language-and-Grammar.html ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 54

Slide 54 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Recursively eunmrable Context-sensitive Context-free Regular Chomsky hierarchy ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 55

Slide 55 text

• BNF (Backus-Naur form) is a notation for expressing Context-free grammar • Standard format for describing context-free grammars and is used to de fi ne the syntax of programming languages and protocols ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Notation for representing CFG ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 56

Slide 56 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ BNF expr : NUMBER { $$ = $1; } | expr '+' expr { $$ = $1 + $3; } ; ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 57

Slide 57 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ BNF expr : NUMBER { $$ = $1; } | expr '+' expr { $$ = $1 + $3; } ; LHS (Left Hand Side) RHS (Right Hand Side) ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 58

Slide 58 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ BNF expr : NUMBER { $$ = $1; } | expr '+' expr { $$ = $1 + $3; } ; Nonterminal Symbol Terminal Symbol ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 59

Slide 59 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ BNF expr : NUMBER { $$ = $1; } | expr '+' expr { $$ = $1 + $3; } ; (Semantic) Actions ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 60

Slide 60 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ BNF expr : NUMBER { $$ = $1; } | expr '+' expr { $$ = $1 + $3; } ; Numbered Parameter Numbered Parameter ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 61

Slide 61 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ BNF expr : NUMBER { $$ = $1; } | expr '+' expr { $$ = $1 + $3; } ; Refers to the value of the token ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 62

Slide 62 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ BNF expr : NUMBER { $$ = $1; } | expr '+' expr { $$ = $1 + $3; } ; Behaves like a return value ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 63

Slide 63 text

• Includes Grammar Rules, Actions, Token De fi nitions and Error Handling • Like a blueprint that tells Ruby how to understand and execute programs by breaking down code into its fundamental components and rules ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ parse.y ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 64

Slide 64 text

• LR parsers can parse Context-free grammar • Context-free grammar can be expressed using Backus-Naur form • Ruby uses BNF in a fi le called parse.y to de fi ne Ruby grammar and behavior ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Summary ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 65

Slide 65 text

“The best programs are written so that computing machines can perform them quickly and so that human beings can understand them clearly.” — Donald E. Knuth How does Lrama make the Ruby parser grammar G.O.A.T.? ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 66

Slide 66 text

• Maintainability • “parse.y” is dif fi cult • Be likened to a Demon Castle, Hell and Monstrous • What's the dif fi culty? • Grammar provided by Bison is primitive ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ What is issue? ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 67

Slide 67 text

Difficulty with Numbered Parameters ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 68

Slide 68 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ expr : NUMBER { $$ = $1; } | expr '+' expr { $$ = $1 + $3; } ; Numbered Parameters ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 69

Slide 69 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ lambda : tLAMBDA { : (snip) } max_numpa r am numpa r am it_id allow_exits f_la r glist { : (snip) } lambda_body { $$ = NEW_LAMBDA($5, $7, &loc); } ; Numbered Parameters Which are the 5th and 7th tokens? ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 70

Slide 70 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ lambda : tLAMBDA { : (snip) } max_numpa r am numpa r am it_id allow_exits f_la r glist < - - - - - - 5th { : (snip) } lambda_body < - - - - - 7th { $$ = NEW_LAMBDA($5, $7, &loc); } ; Numbered Parameters ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 71

Slide 71 text

Named References ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 72

Slide 72 text

• This is a Bison's feature • Original symbol names may be used as named reference • Enclosing with [] in rule descriptions allows for assigning alias name for reference ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Named References ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 73

Slide 73 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ expr : NUMBER { $$ = $1; } | expr '+' expr { $$ = $1 + $3; } ; befo r e ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 74

Slide 74 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ expr[result] : NUMBER { $result = $NUMBER; } | expr[left] '+' expr[right] { $result = $left + $right; } ; Afte r ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 75

Slide 75 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ expr[result] : NUMBER { $result = $NUMBER; } | expr[left] '+' expr[right] { $result = $left + $right; } ; Refer to the NUMBER on the RHS ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 76

Slide 76 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ expr[result] : NUMBER { $result = $NUMBER; } | expr[left] '+' expr[right] { $result = $left + $right; } ; De fi ne Alias Use Alias Use Alias De fi ne Alias ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 77

Slide 77 text

What if there are duplicate code or logic? ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 78

Slide 78 text

Ruby ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 79

Slide 79 text

🧐 Modularization: Break down systems into reusable components (e.g., methods, modules, classes) to avoid duplication. ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 80

Slide 80 text

Bison ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 81

Slide 81 text

Nothing! ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 82

Slide 82 text

YOU MEAN, DOES NO CONCEPT OF A METHOD EXIST? AHHHHHHHHH!!!

Slide 83

Slide 83 text

Slide 84

Slide 84 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ f_block_kwa r g: f_block_kw { $$ = $1; / * % r ippe r : r b_a r y_new3(1, get_value($ : 1)) % * / } | f_block_kwa r g ',' f_block_kw { $$ = kwd_append($1, $3); / * % r ippe r : r b_a r y_push(get_value($ : 1), get_value($ : 3)) % * / } ; foo { |ba r :, baz:| puts " # { ba r }- # { baz}" } ^^^^^^^^^^^^^ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 85

Slide 85 text

Parameterizing Rules ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 86

Slide 86 text

• The de fi nition of a Nonterminal symbol can be parameterized with other (Terminal or Nonterminal) symbols • Idea of Parameterizing Rules comes from Menhir LR(1) parser generator • https://gallium.inria.fr/~fpottier/menhir/manual.html#sec32 parser generator ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Parameterizing Rules ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 87

Slide 87 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ f_kwa r g : f_kw { $$ = $1; / * % r ippe r : r b_a r y_new3(1, get_value($ : 1)) % * / } | f_kwa r g ',' f_kw { $$ = kwd_append($1, $3); / * % r ippe r : r b_a r y_push(get_value($ : 1), get_value($ : 3)) % * / } ; f_block_kwa r g: f_block_kw { $$ = $1; / * % r ippe r : r b_a r y_new3(1, get_value($ : 1)) % * / } | f_block_kwa r g ',' f_block_kw { $$ = kwd_append($1, $3); / * % r ippe r : r b_a r y_push(get_value($ : 1), get_value($ : 3)) % * / } ; befo r e ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 88

Slide 88 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ % r ules kwa r g(kw) : kw { $$ = $1; / * % r ippe r : r b_a r y_new3(1, get_value($ : 1)) % * / } | kwa r g(kw) ',' kw { $$ = kwd_append($1, $3); / * % r ippe r : r b_a r y_push(get_value($ : 1), get_value($ : 3)) % * / } ; f_kwa r g : kwa r g(f_kw); f_block_kwa r g : kwa r g(f_block_kw); afte r ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 89

Slide 89 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ % r ules option(X) : / * empty * / | X ; Rule name Parameter Parameter is expanded here ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 90

Slide 90 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ opt_nl : / * empty * / | '\n' ; ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 91

Slide 91 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ % r ules option(X) : / * empty * / | X ; opt_nl : option('\n') ; ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 92

Slide 92 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ % r ules option('\n') : / * empty * / | X ; opt_nl : option('\n') ; ^^^^^^ Be expanded ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 93

Slide 93 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ % r ules option('\n') : / * empty * / | '\n' ; opt_nl : option('\n') ; ^^^^^^ ^^^^^^ Be expanded Be expanded ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 94

Slide 94 text

Standard library ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 95

Slide 95 text

• Provides a Standard Library of rules with a general structure, such as `option` or `list` • Stored in a fi le lib/lrama/grammar/stdlib.y, which is embedded inside grammar fi le when before parsing ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Standard library ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 96

Slide 96 text

The following three rules can be created using the familiar regular expression-like syntax: ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Standard library Names Recognizes Alias option(X) є | X X? list(X) a possibly empty sequence of X’s X* nonempty_list(X) a nonempty sequence of X’s X+ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 97

Slide 97 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ % r ules option(X) : / * empty * / | X ; opt_nl : option('\n') ; befo r e ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 98

Slide 98 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ opt_nl : '\n'? ; afte r ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 99

Slide 99 text

Ruby parser's grammar is now G.O.A.T., right? ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 100

Slide 100 text

• Currently, with the replacement of the parser generator from Bison to Lrama, we have more powerful and richer grammars • Named References solved the problem of easily hard-to-read references • Parameterizing Rules gave us a great power to abstract structures • The syntax continues to evolve and the G.O.A.T. continues to be updated on a daily basis ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Summary ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 101

Slide 101 text

“People think that computer science is the art of geniuses but the actual reality is the opposite, just many people doing things that build on eachother, like a wall of mini stones.” — Donald E. Knuth Conclusion ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 102

Slide 102 text

• One of the problems with parse.y was the issue of maintainability • It was suggested that just extending the grammar would bring signi fi cant improvements to the Ruby parser grammar • The syntax continues to evolve throughout the development of Lrama, and the G.O.A.T. continues to be updated daily ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Conclusion ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 103

Slide 103 text

ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ And more https://docs.google.com/present a tion/d/1E4v9WPHBLjtvkN7QqulHPGJzKkwIweVfc a MsIQ984_Q/edit#slide=id.p ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ

Slide 104

Slide 104 text

Stargaze at github.com/ruby/lrama

Slide 105

Slide 105 text

• Lrama LALR (1) parser generator https://github.com/ruby/lrama • Menhir Reference Manual(version 20231231) https://gallium.inria.fr/~fpottier/menhir/manual.html • Language and Grammar (Bison 3.8.1) https://www.gnu.org/software/bison/manual/html_node/ Language-and-Grammar.html • Chomsky hierarchy - Wikipedia https://en.wikipedia.org/wiki/Chomsky_hierarchy • Named References (Bison 3.8.1) https://www.gnu.org/software/bison/manual/html_node/Named- References.html • Ruby Parser Roadmap - Google slide https://docs.google.com/presentation/d/ 1E4v9WPHBLjtvkN7QqulHPGJzKkwIweVfcaMsIQ984_Q/edit#slide=id.p ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ Reference ydah | https://speakerdeck.com/ydah/how-does-lrama-make-the-ruby-parser-grammar-goatɹɹ