Tweet
Tweet

Slide 1

Slide 1 text

My favorite script, "dsl.rb" May 31, 2024 in RubyKaigi 2024 事後勉強会 @yui-knk Yuichiro Kaneko

Slide 2

Slide 2 text

About me Yuichiro Kaneko yui-knk (GitHub) / spikeolaf (Twitter) Treasure Data Engineering Manager of Applications Backend CRuby committer, mainly develop parser generator and parser Lrama LALR (1) parser generator (2023, Ruby 3.3) Love LR parser

Slide 3

Slide 3 text

May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan The Bison Slayer The parser monster Parser界の黎明卿 The world is now in the great age of parsers. People are setting sail into the vast sea of parsers. - RubyKaigi 2023 LT- Yuichiro Kaneko https://twitter.com/kakutani/status/1657762294431105025/ NEW !!! NEW !!!

Slide 4

Slide 4 text

May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan

Slide 5

Slide 5 text

May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Turn back the clock

Slide 6

Slide 6 text

May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan EXTRA STAGE “Ripper” CHALLENGE 19

Slide 7

Slide 7 text

May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan How Ripper is implemented?

Slide 8

Slide 8 text

How parse.y is used parse.y is a source of ripper.y parse.y parse.c parse.h Lrama ripper.y tool/id2token.rb tools/preproc.rb Lrama ripper.c

Slide 9

Slide 9 text

Comments in parse.y is transformed to C codes in ripper.y Comments are not comments !! parse.y is two-faced parse.y ripper.y

Slide 10

Slide 10 text

“dsl.rb” is fantastic “ext/ripper/tools/dsl.rb” “tools/preproc.rb” requires “dsl.rb” This is my favorite script !! Less than 100 lines but very hacky

Slide 11

Slide 11 text

May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Today’s main topic

Slide 12

Slide 12 text

How parse.y is used parse.y is a source of ripper.y parse.y parse.c parse.h Lrama ripper.y tool/id2token.rb tools/preproc.rb Lrama ripper.c tools/dsl.rb

Slide 13

Slide 13 text

preproc.rb preproc.rb processes input file line by line /*% ripper: stmts_add!($:1, $:3) %*/ match /*% ripper: stmts_add!($:1, $:3) %*/ processed

Slide 14

Slide 14 text

Ruby codes in C comments

Slide 15

Slide 15 text

In parse.y, C codes and Ruby codes in C’s comment exist In ripper.y, Ruby codes are transformed to C codes Mixture of C and Ruby parse.y ripper.y C codes Ruby codes C codes

Slide 16

Slide 16 text

Of course, they are here Of course, method_missing and const_missing are used

Slide 17

Slide 17 text

“dsl.rb” is fantastic !!! “p = p” !? undef lambda, hash, class

Slide 18

Slide 18 text

May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan What's happening?

Slide 19

Slide 19 text

#1. Split the code Split a comment into options and “@code” Generate DSL instance class DSL options @error @brace @ fi nal @code

Slide 20

Slide 20 text

#2. eval @code Eval @code, then method_missing and/or const_missing will be called const_missing method_missing method_missing

Slide 21

Slide 21 text

#3. method_missing method_missing generates C code #stmts_new! call with no arguments then translated to dispatch0 function call

Slide 22

Slide 22 text

#3. method_missing #stmts_add! call with 2 arguments then translated to variable setup codes and dispatch2 function call

Slide 23

Slide 23 text

#4. DSL#generate Finally, the last value is assigned to "p->s_lvalue" by #generate

Slide 24

Slide 24 text

Tips #1 p = p = "p" As you know, "p" is "struct parser_params *p” What is Kernel#p method? Double assignments is for warning suppression "warning: assigned but unused variable - p"

Slide 25

Slide 25 text

Tips #2 undef is needed for lambda, hash, class events Otherwise method_missing is not called

Slide 26

Slide 26 text

May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan After RubyKaigi…

Slide 27

Slide 27 text

PR #10766 PR was created by nobu on May 12 RubyKaigi was May 15-17 I wan in Okinawa https://github.com/ruby/ruby/pull/10766

Slide 28

Slide 28 text

No more get_value No need to write get_value anymore!

Slide 29

Slide 29 text

DSL#initialize get_value is automatically inserted by DSL

Slide 30

Slide 30 text

Support [$:n, ...] No need to write rb_ary_new_from_args or rb_ary_new […] can be used instead of that

Slide 31

Slide 31 text

Support [$:n, ...] Actually […] is expanded to rb_ary_new_from_args

Slide 32

Slide 32 text

Re fi ne Array !! Redefine Array#to_s Impact is limited by refinements

Slide 33

Slide 33 text

$:n[i] and $:n[i..j] to splat head in ripper is an array with 3 elements Want to splat the array when invoke defs event

Slide 34

Slide 34 text

No magic Simply define #[] method

Slide 35

Slide 35 text

p = p = "p" is improved Remove double assignments by using default value of argument Intention is more clear than previous codes

Slide 36

Slide 36 text

Summary “ext/ripper/tools/dsl.rb” Is less than 200 lines but very powerful Contains real world examples of #eval #method_missing #const_missing #binding Refinements (<- New!)

Slide 37

Slide 37 text

Conclusions It’s fun to use black magic !!

Slide 38

Slide 38 text

May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Thank you !!!