The Sequel to a Dream of Ruby Parser's Grammar

RubyKaigi 2024 follow up「The Sequel to a Dream of Ruby Parser's Grammar」の発表スライド。
#rubykaigi_followup https://rhc.connpass.com/event/320709/

Transcript

1. The Sequel to a Dream of Ruby Parser's Grammar RubyKaigi 2024 follow up 31 August 2024 Yudai Takada (@ydah)
2. Yudai Takada https://ydah.net X: @ydah_ GitHub: @ydah Mastodon: @ydah@ruby.social photo by: @hachi
3. Osaka, Japan
4. Live Tour 2024 •2024.8.31 (ONLINE) RubyKaigi 2024 followup •2024.9.13 (Sarajevo) EuRuKo 2024 •2024.10.25-26 (Tokyo) Kaigi on Rails 2024
5. RuboCop Headquarters RuboCop RSpec team
6. interagent/committee Committee maintainers team
7. ruby/lrama Committer
8. kyobashi.rb Co-Founder
9. Thank you for coming!
11. These Slides
12. Today’s talk ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
13. The Sequel to a Dream ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
14. 夢の続き ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
15. ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar Previous Episode
16. Ruby Parser's Grammar ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
17. パーサーの文法? ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
18. expr : NUMBER { $$ = $1; } | expr '+' expr { $$ = $1 + $3; } | expr '-' expr { $$ = $1 - $3; } | expr '*' expr { $$ = $1 * $3; } | expr '/' expr { $$ = $1 / $3; } | '(' expr ')' { $$ = $2; } ; BNF ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
19. parse.y ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar •File de fi nes the Ruby grammar and processing. •Extremely di ff i cult to understand. •Be likened to a Demon Castle, Hell, Monstrous
20. What's di ffi culty? ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar •Grammar provided by Bison is primitive. •Parser and Lexer are tightly coupled. •Di ff i cult to resolve S/R or R/R con fl icts.
21. Bison provides primitive grammar ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
22. What if there are duplicate codes? ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
23. Ruby ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
24. Extract to method ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
25. Bison ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
26. Nothing! ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
27. えっ??? ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
28. opt_args_tail : ',' args_tail { $$ = $2; / * % ripper: get_value($ : 2); % * / } | / * none * / { $$ = new_args_tail(p, 0, 0, 0, &@0); / * % ripper: rb_ary_new_from_args(3, Qnil, Qnil, Qnil); % * / } ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
29. opt_block_args_tail : ',' block_args_tail { $$ = $2; / * % ripper: get_value($ : 2); % * / } | / * none * / { $$ = new_args_tail(p, 0, 0, 0, &@0); / * % ripper: rb_ary_new_from_args(3, Qnil, Qnil, Qnil); % * / } ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
30. 何がちがうの... ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
31. opt_args_tail : ',' args_tail { $$ = $2; / * % ripper: get_value($ : 2); % * / } | / * none * / { $$ = new_args_tail(p, 0, 0, 0, &@0); / * % ripper: rb_ary_new_from_args(3, Qnil, Qnil, Qnil); % * / } ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar ???????
32. opt_block_args_tail : ',' block_args_tail { $$ = $2; / * % ripper: get_value($ : 2); % * / } | / * none * / { $$ = new_args_tail(p, 0, 0, 0, &@0); / * % ripper: rb_ary_new_from_args(3, Qnil, Qnil, Qnil); % * / } ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar ????????????
33. ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar Seek the Savior…
34. Parameterizing Rules ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
35. %rules args_tail(tail) : ',' tail { $$ = $2; ɹɹ ɹ / * % ripper: get_value($ : 2); % * / } | / * none * / { $$ = new_args_tail(p, 0, 0, 0, &@0); / * % ripper: rb_ary_new_from_args(3, Qnil, Qnil, Qnil); % * / } ; % % opt_args_tail: args_tail(arg_tail) ; opt_block_args_tail : args_tail(blk_tail) ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
36. %rules args_tail(tail) : ',' tail { $$ = $2; ɹɹ ɹ / * % ripper: get_value($ : 2); % * / } | / * none * / { $$ = new_args_tail(p, 0, 0, 0, &@0); / * % ripper: rb_ary_new_from_args(3, Qnil, Qnil, Qnil); % * / } ; % % opt_args_tail: args_tail(arg_tail) ; opt_block_args_tail : args_tail(blk_tail) ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar 共通的な構造の定義
37. %rules args_tail(tail) : ',' tail { $$ = $2; ɹɹ ɹ / * % ripper: get_value($ : 2); % * / } | / * none * / { $$ = new_args_tail(p, 0, 0, 0, &@0); / * % ripper: rb_ary_new_from_args(3, Qnil, Qnil, Qnil); % * / } ; % % opt_args_tail: args_tail(arg_tail) ; opt_block_args_tail : args_tail(blk_tail) ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar 定義されたルールを使う
38. %rules args_tail(tail) : ',' tail { $$ = $2; ɹɹ ɹ / * % ripper: get_value($ : 2); % * / } | / * none * / { $$ = new_args_tail(p, 0, 0, 0, &@0); / * % ripper: rb_ary_new_from_args(3, Qnil, Qnil, Qnil); % * / } ; % % opt_args_tail: args_tail(arg_tail) ; opt_block_args_tail : args_tail(blk_tail) ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar 引数 これが引数として渡る
39. %rules args_tail(tail) : ',' tail { $$ = $2; ɹɹ ɹ / * % ripper: get_value($ : 2); % * / } | / * none * / { $$ = new_args_tail(p, 0, 0, 0, &@0); / * % ripper: rb_ary_new_from_args(3, Qnil, Qnil, Qnil); % * / } ; % % opt_args_tail: args_tail(arg_tail) ; opt_block_args_tail : args_tail(blk_tail) ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar 展開される 展開される
40. べんり!! ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
41. Bison provides primitive grammar ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
42. Conditionals ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
43. Ruby ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
44. if down? if right - lower? if right? if P? puts 'Hadouken' end end end end ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
45. Bison ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
46. Nothing! ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
47. はい。 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
48. Is it necessary? ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar •Yes!!! •Cases where the syntax is valid for certain conditions but invalid for others.
49. irb(main) : 001> - > (a) {} = > #<Proc:0 x 000000012316e698 (irb) : 1 (lambda)> irb(main) : 002> - > (a=1) {} = > #<Proc:0 x 0000000123119918 (irb) : 2 (lambda)> irb(main) : 003> - > ( . . . ) {} <internal:kernel > : 187:in `loop': (irb) : 3 : syntax error, unexpected . . . , expecting ')' (SyntaxError) - > ( . . . ) {} ^ ~ ~ Example ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
50. https://github.com/ruby/lrama/pull/418 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
51. %rule def i ned_rule(X, condition) : %if(condition) X %endif X { $$ = $1; } ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; Idea 1 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
52. %rule def i ned_rule(X, condition) : %if(condition) X %endif X { $$ = $1; } ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar Idea 1 trueを渡す %if(true)
53. %rule def i ned_rule(X, condition) : %if(condition) X %endif X { $$ = $1; } ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar =>X X { $$ = $1; } Idea 1 ????????????????????????????????
54. %rule def i ned_rule(X, condition) : %if(condition) X %endif X { $$ = $1; } ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar Idea 1 falseを渡す %if(false)
55. %rule def i ned_rule(X, condition) : %if(condition) X %endif X { $$ = $1; } ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar => X { $$ = $1; } Idea 1 ????????????
56. Rejected ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
57. Why? ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar •Di ff i cult to understand that `%if` and `%endif` are mixed in the terminal, non-terminal symbols.
58. %rule def i ned_rule(X, condition) : X { $$ = $1; } %if(condition) ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; Idea 2 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
59. %rule def i ned_rule(X, condition) : X { $$ = $1; } %if(condition) ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; Idea 2 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar %if(true) trueを渡す
60. %rule def i ned_rule(X, condition) : X { $$ = $1; } %if(condition) ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; Idea 2 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar ????????????? => X { $$ = $1; }
61. %rule def i ned_rule(X, condition) : X { $$ = $1; } %if(condition) ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; Idea 2 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar falseを渡す %if(false)
62. %rule def i ned_rule(X, condition) : X { $$ = $1; } %if(condition) ; % % r_true : def i ned_rule(number, %true) ; r_false : def i ned_rule(number, %false) ; Idea 2 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar ??????????????????????????? => /* empty */
63. Approved! ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
64. だけど、 改善点はある ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < 構文をもう少し改善したい
65. %rule if_example(foo, cond) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, %true) ; Point 1 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
66. %rule if_example(foo, cond) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, %true) ; Point 1 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar ここ < foo と cond は構文を分けたい
67. %rule if_example(foo, cond) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, %true) ; Point 2 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
68. %rule if_example(foo, cond) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, %true) ; Point 2 ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar ここ < 条件引数って “式” の一種なのでは?
69. A Few Moments Later… ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
70. %rule if_example(foo, cond: Conditional) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, { %true }) ; Proposal ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < OsakaRubyKaigi04を準備しつつ考えてました
71. %rule if_example(foo, cond: Conditional) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, { %true }) ; Proposal ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
72. %rule if_example(foo, cond: Conditional) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, { %true }) ; Proposal ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < 式なので、 { foo==bar }の様に書けると良い?
73. %rule if_example(foo, cond: Conditional) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, { %true }) ; Proposal ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < まだ少しなやんでいます。。。
74. %rule if_example(foo, cond: Boolean) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, { %true }) ; Other Proposal ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < これだと型に見える??
75. %rule if_example(foo, @cond) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, { %true }) ; Other Proposal ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < 接頭語をつける案。いい記号があればいいケド
76. %rule if_example(foo, cond: Conditional) : foo { $$ = $1; } %if(cond) ; % % example : if_example(value, { %true }) ; Proposal ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar < という訳でこれが最有力候補です。WDYT?
77. Further Sequel to a Dream ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar
78. Wish List ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar •Release v0.6.10 of Lrama. •Adding location information to the node structure. < がんばるか、 超がんばるかしかない!!!
79. end ydah | https://speakerdeck.com/ydah/the-sequel-to-a-dream-of-ruby-parsers-grammar