Slide 1
Slide 1 text
My favorite script,
"dsl.rb"
May 31, 2024 in RubyKaigi 2024
事後勉強会
@yui-knk
Yuichiro Kaneko
Slide 2
Slide 2 text
About me
Yuichiro Kaneko
yui-knk (GitHub) / spikeolaf (Twitter)
Treasure Data
Engineering Manager of Applications Backend
CRuby committer, mainly develop parser generator and
parser
Lrama LALR (1) parser generator (2023, Ruby 3.3)
Love LR parser
Slide 3
Slide 3 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
The Bison Slayer
The parser monster
Parser界の黎明卿
The world is now in the great age
of parsers. People are setting sail
into the vast sea of parsers.
- RubyKaigi 2023 LT-
Yuichiro Kaneko
https://twitter.com/kakutani/status/1657762294431105025/
NEW !!!
NEW !!!
Slide 4
Slide 4 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
Slide 5
Slide 5 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
Turn back the clock
Slide 6
Slide 6 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
EXTRA STAGE “Ripper”
CHALLENGE 19
Slide 7
Slide 7 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
How Ripper is
implemented?
Slide 8
Slide 8 text
How parse.y is used
parse.y is a source of ripper.y
parse.y
parse.c
parse.h
Lrama
ripper.y
tool/id2token.rb tools/preproc.rb
Lrama
ripper.c
Slide 9
Slide 9 text
Comments in parse.y
is transformed to C
codes in ripper.y
Comments are not
comments !!
parse.y is two-faced
parse.y
ripper.y
Slide 10
Slide 10 text
“dsl.rb” is fantastic
“ext/ripper/tools/dsl.rb”
“tools/preproc.rb” requires “dsl.rb”
This is my favorite script !!
Less than 100 lines but very hacky
Slide 11
Slide 11 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
Today’s main topic
Slide 12
Slide 12 text
How parse.y is used
parse.y is a source of ripper.y
parse.y
parse.c
parse.h
Lrama
ripper.y
tool/id2token.rb tools/preproc.rb
Lrama
ripper.c
tools/dsl.rb
Slide 13
Slide 13 text
preproc.rb
preproc.rb processes input file line by line
/*% ripper: stmts_add!($:1, $:3) %*/
match
/*% ripper: stmts_add!($:1, $:3) %*/
processed
Slide 14
Slide 14 text
Ruby codes in C comments
Slide 15
Slide 15 text
In parse.y, C codes
and Ruby codes in C’s
comment exist
In ripper.y, Ruby codes
are transformed to C
codes
Mixture of C and Ruby
parse.y
ripper.y
C codes
Ruby codes
C codes
Slide 16
Slide 16 text
Of course, they are here
Of course, method_missing and const_missing are used
Slide 17
Slide 17 text
“dsl.rb” is fantastic !!!
“p = p” !?
undef lambda, hash, class
Slide 18
Slide 18 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
What's happening?
Slide 19
Slide 19 text
#1. Split the code
Split a comment into options and “@code”
Generate DSL instance
class DSL
options
@error
@brace
@
fi
nal
@code
Slide 20
Slide 20 text
#2. eval @code
Eval @code, then method_missing and/or
const_missing will be called
const_missing
method_missing method_missing
Slide 21
Slide 21 text
#3. method_missing
method_missing generates C code
#stmts_new! call with no arguments then translated to
dispatch0 function call
Slide 22
Slide 22 text
#3. method_missing
#stmts_add! call with 2 arguments then translated to
variable setup codes and dispatch2 function call
Slide 23
Slide 23 text
#4. DSL#generate
Finally, the last value is assigned to "p->s_lvalue" by
#generate
Slide 24
Slide 24 text
Tips #1
p = p = "p"
As you know, "p" is "struct parser_params *p”
What is Kernel#p method?
Double assignments is for warning suppression
"warning: assigned but unused variable - p"
Slide 25
Slide 25 text
Tips #2
undef is needed for lambda, hash, class events
Otherwise method_missing is not called
Slide 26
Slide 26 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
After RubyKaigi…
Slide 27
Slide 27 text
PR #10766
PR was created by nobu on May 12
RubyKaigi was May 15-17
I wan in Okinawa
https://github.com/ruby/ruby/pull/10766
Slide 28
Slide 28 text
No more get_value
No need to write get_value anymore!
Slide 29
Slide 29 text
DSL#initialize
get_value is automatically inserted by DSL
Slide 30
Slide 30 text
Support [$:n, ...]
No need to write rb_ary_new_from_args or
rb_ary_new
[…] can be used instead of that
Slide 31
Slide 31 text
Support [$:n, ...]
Actually […] is expanded to rb_ary_new_from_args
Slide 32
Slide 32 text
Re
fi
ne Array !!
Redefine Array#to_s
Impact is limited by refinements
Slide 33
Slide 33 text
$:n[i] and $:n[i..j] to splat
head in ripper is an array with 3 elements
Want to splat the array when invoke defs event
Slide 34
Slide 34 text
No magic
Simply define #[] method
Slide 35
Slide 35 text
p = p = "p" is improved
Remove double assignments by using default value of
argument
Intention is more clear than previous codes
Slide 36
Slide 36 text
Summary
“ext/ripper/tools/dsl.rb”
Is less than 200 lines but very powerful
Contains real world examples of
#eval
#method_missing
#const_missing
#binding
Refinements (<- New!)
Slide 37
Slide 37 text
Conclusions
It’s fun to use black magic !!
Slide 38
Slide 38 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
Thank you !!!