The Sequel to a Dream of Ruby Parser's Grammar

Slide 1

Slide 1 text

The Sequel to a Dream of Ruby Parser's Grammar RubyKaigi 2024 follow up 31 August 2024 Yudai Takada (@ydah)

Slide 2

Slide 2 text

Yudai Takada https://ydah.net X: @ydah_ GitHub: @ydah Mastodon: @[email protected] photo by: @hachi

Slide 3

Slide 3 text

Osaka, Japan

Slide 4

Slide 4 text

Live Tour 2024 •2024.8.31 (ONLINE) RubyKaigi 2024 followup •2024.9.13 (Sarajevo) EuRuKo 2024 •2024.10.25-26 (Tokyo) Kaigi on Rails 2024

Slide 5

Slide 5 text

RuboCop Headquarters RuboCop RSpec team

Slide 6

Slide 6 text

interagent/committee Committee maintainers team

Slide 7

Slide 7 text

ruby/lrama Committer

Slide 8

Slide 8 text

kyobashi.rb Co-Founder

Slide 9

Slide 9 text

Thank you for coming!

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

These Slides

Slide 12

Slide 12 text

Today’s talk ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 13

Slide 13 text

The Sequel to a Dream ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 14

Slide 14 text

夢の続き ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 15

Slide 15 text

ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar Previous Episode

Slide 16

Slide 16 text

Ruby Parser's Grammar ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 17

Slide 17 text

パーサーの文法? ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 18

Slide 18 text

expr : NUMBER { $$ = $1; } | expr '+' expr { $$ = $1 + $3; } | expr '-' expr { $$ = $1 - $3; } | expr '*' expr { $$ = $1 * $3; } | expr '/' expr { $$ = $1 / $3; } | '(' expr ')' { $$ = $2; } ; BNF ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 19

Slide 19 text

parse.y ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar •File de fi nes the Ruby grammar and processing. •Extremely di ff i cult to understand. •Be likened to a Demon Castle, Hell, Monstrous

Slide 20

Slide 20 text

What's di ffi culty? ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar •Grammar provided by Bison is primitive. •Parser and Lexer are tightly coupled. •Di ff i cult to resolve S/R or R/R con fl icts.

Slide 21

Slide 21 text

Bison provides primitive grammar ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 22

Slide 22 text

What if there are duplicate codes? ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 23

Slide 23 text

Ruby ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 24

Slide 24 text

Extract to method ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 25

Slide 25 text

Bison ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 26

Slide 26 text

Nothing! ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 27

Slide 27 text

えっ??? ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 28

Slide 28 text

opt_args_tail : ',' args_tail { $$ = $2; / * % ripper: get_value($ : 2); % * / } | / * none * / { $$ = new_args_tail(p, 0, 0, 0, &@0); / * % ripper: rb_ary_new_from_args(3, Qnil, Qnil, Qnil); % * / } ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 29

Slide 29 text

opt_block_args_tail : ',' block_args_tail { $$ = $2; / * % ripper: get_value($ : 2); % * / } | / * none * / { $$ = new_args_tail(p, 0, 0, 0, &@0); / * % ripper: rb_ary_new_from_args(3, Qnil, Qnil, Qnil); % * / } ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 30

Slide 30 text

何がちがうの... ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar Seek the Savior…

Slide 34

Slide 34 text

Parameterizing Rules ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 35

Slide 35 text

%rules args_tail(tail) : ',' tail { $$ = $2; ɹɹ ɹ / * % ripper: get_value($ : 2); % * / } | / * none * / { $$ = new_args_tail(p, 0, 0, 0, &@0); / * % ripper: rb_ary_new_from_args(3, Qnil, Qnil, Qnil); % * / } ; % % opt_args_tail: args_tail(arg_tail) ; opt_block_args_tail : args_tail(blk_tail) ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

Slide 40

Slide 40 text

べんり!! ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 41

Slide 41 text

Bison provides primitive grammar ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 42

Slide 42 text

Conditionals ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 43

Slide 43 text

Ruby ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 44

Slide 44 text

if down? if right - lower? if right? if P? puts 'Hadouken' end end end end ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 45

Slide 45 text

Bison ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 46

Slide 46 text

Nothing! ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 47

Slide 47 text

はい。 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 48

Slide 48 text

Is it necessary? ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar •Yes!!! •Cases where the syntax is valid for certain conditions but invalid for others.

Slide 49

Slide 49 text

irb(main) : 001> - > (a) {} = > # irb(main) : 002> - > (a=1) {} = > # irb(main) : 003> - > ( . . . ) {} : 187:in `loop': (irb) : 3 : syntax error, unexpected . . . , expecting ')' (SyntaxError) - > ( . . . ) {} ^ ~ ~ Example ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 50

Slide 50 text

https://github.com/ruby/lrama/pull/418 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 51

Slide 51 text

%rule def i ned_rule(X, condition) : %if(condition) X %endif X { $$ = $1; } ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; Idea 1 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 52

Slide 52 text

%rule def i ned_rule(X, condition) : %if(condition) X %endif X { $$ = $1; } ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar Idea 1 trueを渡す %if(true)

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Slide 56

Slide 56 text

Rejected ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 57

Slide 57 text

Why? ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar •Di ff i cult to understand that `%if` and `%endif` are mixed in the terminal, non-terminal symbols.

Slide 58

Slide 58 text

%rule def i ned_rule(X, condition) : X { $$ = $1; } %if(condition) ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; Idea 2 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 59

Slide 59 text

Slide 60

Slide 60 text

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Slide 63

Slide 63 text

Approved! ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 64

Slide 64 text

だけど、改善点はある ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < 構文をもう少し改善したい

Slide 65

Slide 65 text

%rule if_example(foo, cond) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, %true) ; Point 1 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 66

Slide 66 text

%rule if_example(foo, cond) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, %true) ; Point 1 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar ここ < foo と cond は構文を分けたい

Slide 67

Slide 67 text

%rule if_example(foo, cond) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, %true) ; Point 2 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 68

Slide 68 text

%rule if_example(foo, cond) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, %true) ; Point 2 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar ここ < 条件引数って “式” の一種なのでは?

Slide 69

Slide 69 text

A Few Moments Later… ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 70

Slide 70 text

%rule if_example(foo, cond: Conditional) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, { %true }) ; Proposal ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < OsakaRubyKaigi04を準備しつつ考えてました

Slide 71

Slide 71 text

Slide 72

Slide 72 text

%rule if_example(foo, cond: Conditional) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, { %true }) ; Proposal ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < 式なので、 { foo==bar }の様に書けると良い?

Slide 73

Slide 73 text

Slide 74

Slide 74 text

%rule if_example(foo, cond: Boolean) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, { %true }) ; Other Proposal ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < これだと型に見える??

Slide 75

Slide 75 text

%rule if_example(foo, @cond) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, { %true }) ; Other Proposal ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < 接頭語をつける案。いい記号があればいいケド

Slide 76

Slide 76 text

%rule if_example(foo, cond: Conditional) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, { %true }) ; Proposal ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < という訳でこれが最有力候補です。WDYT?

Slide 77

Slide 77 text

Further Sequel to a Dream ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar

Slide 78

Slide 78 text

Wish List ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar •Release v0.6.10 of Lrama. •Adding location information to the node structure. < がんばるか、超がんばるかしかない!!!

Slide 79

Slide 79 text

end ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar