Slide 1
Slide 1 text
Menhir is here!
@ydah
Fukuoka.rb #333 Ninja Talks
2023.11.29 (Wed.)
Slide 2
Slide 2 text
ɾ Name: Yudai Takada
ɾ GitHub: @ydah / X: @ydah_
ɾ Software Engineer at ANDPAD, Inc.
ɾ Member of RuboCop RSpec team
ɾ Contributor of Lrama
ɾ Implementation of Parameterizing rules
ɾ Optimization of Lexer, etc.
self.inspect
Slide 3
Slide 3 text
The world is about
witness a Great Era
of Parser!
Slide 4
Slide 4 text
But…
Slide 5
Slide 5 text
Don't touch the GC,
the parser and the
code generator.
— Matz #rubyworld2023
Slide 6
Slide 6 text
That’s not true!
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
You should try!
Slide 9
Slide 9 text
Today's talk
Slide 10
Slide 10 text
Menhir is here!
Slide 11
Slide 11 text
• Highly functional LR(1) parser generator for the
OCaml
• OCaml also has a parser generator called ocamlyacc,
Menhir is 90% compatible with ocamlyacc
• Designed and implemented by François Pottier
and Yann Régis-Gianas
What is Menhir?
Slide 12
Slide 12 text
• Single, upright megalithic
monument
• The word menhir was
adopted by 19th century
archaeologists via the French
• There is also famous menhir in
Japan. There are many in Gifu
By the Way
Slide 13
Slide 13 text
Why talking about
OCaml's parser?
Slide 14
Slide 14 text
6 MONTHS
EARLIER…
Slide 15
Slide 15 text
No content
Slide 16
Slide 16 text
https://bugs.ruby-lang.org/issues/19637
Slide 17
Slide 17 text
• LALR (1) parser generator written in Ruby
• Created as a CRuby parser generator to
replace GNU Bison.
• The author is Yuichiro Kaneko
• Ref: https://yui-knk.hatenablog.com/entry/2023/03/13/101951
• Ref: https://youtu.be/IhfDsLx784g?si=I3DG-ccG2rxbqxHd
What is Lrama?
Slide 18
Slide 18 text
• Before replacing it with Lrama, Ruby had
to support multiple Bison
• It is not easy to maintain for multiple
version
• New features are added, but they are not
available
One of reasons replaced Bison
Slide 19
Slide 19 text
• Can break away from multiple versions of
maintenance!
• It will can use new features in Bison! (If we
implement it)
• Parser generator features other than Bison
may be used! (if we implement it)
The world after replacing Lrama
Slide 20
Slide 20 text
• Can break away from multiple versions of
maintenance!
• It will can use new features in Bison! (If we
implement it)
• Parser generator features other than Bison
may be used! (if we implement it)
The world after replacing Lrama
Slide 21
Slide 21 text
• We have the goal of making parse.y more
readable
• We need richer DSL for Lrama grammar
fi
les
Provide rich syntax extensions
Slide 22
Slide 22 text
https://docs.google.com/presentation/d/
1E4v9WPHBLjtvkN7QqulHPGJzKkwIweVfcaMsIQ984_Q
Slide 23
Slide 23 text
So
Slide 24
Slide 24 text
Menhir is here!
Slide 25
Slide 25 text
• Parameterizing rules
• Rules can be parameterized with any number of
symbols, called formal parameters
• The standard library
• A library of standard parameterization de
fi
nitions
• Inlining
• The %inline keyword helps avoid or resolve con
fl
icts
without arti
fi
cially changing the grammar
Useful Menhir Features
Slide 26
Slide 26 text
Useful Menhir Features
• Parameterizing rules
• Rules can be parameterized with any number of
symbols, called formal parameters
• The standard library
• A library of standard parameterization de
fi
nitions
• Inlining
• The %inline keyword helps avoid or resolve con
fl
icts
without arti
fi
cially changing the grammar
Slide 27
Slide 27 text
Parameterizing rules
Slide 28
Slide 28 text
• The de
fi
nition of a non-terminal symbol
can be parameterized with other
(terminal or non-terminal) symbols
• Provide syntax sugar for common pattern
for grammar rules
Parameterizing rules
Slide 29
Slide 29 text
Parameterizing rules
function_name(arg1, arg2)
args : arg
| args ',' arg
;
Slide 30
Slide 30 text
Parameterizing rules
function_name(arg1, arg2)
args : arg
| args ',' arg
;
Slide 31
Slide 31 text
Parameterizing rules
function_name(arg)
args : arg
| args ',' arg
;
Slide 32
Slide 32 text
Parameterizing rules
function_name(arg1, arg2)
args : arg
| args ',' arg
;
Slide 33
Slide 33 text
Parameterizing rules
function_name(arg)
args : arg
| args ',' arg
;
Slide 34
Slide 34 text
Parameterizing rules
function_name(arg1, arg2)
args : arg
| args ',' arg
;
Slide 35
Slide 35 text
Parameterizing rules
function_name(arg1, arg2, …)
args : arg
| args ',' arg
;
Slide 36
Slide 36 text
Parameterizing rules
args : arg
| args ',' arg
;
Slide 37
Slide 37 text
Parameterizing rules
args : arg
| args ',' arg
;
args : separated_list(',', arg)
;
Slide 38
Slide 38 text
Parameterizing rules
Name Recognizes Alias
option(X) є | X X?
list(X) a possibly empty sequence of X’s X*
nonempty_list(X) a nonempty sequence of X’s X+
separated_list(sep, X)
a possibly empty sequence of X’s
separated with sep’s
separated_nonempty_list(sep, X)
a nonempty sequence of X’s
separated with sep’s
Slide 39
Slide 39 text
Tag for
Parameterizing rules
Slide 40
Slide 40 text
Tag for Parameterizing rules
%union {
int i;
char *str;
}
%
t
oken NUMBER
%
t
oken
S
T
RING
Slide 41
Slide 41 text
Tag for Parameterizing rules
program: option(foo)
program: option_foo
option_foo: ε
option_foo: foo
Slide 42
Slide 42 text
program: option(foo) { $$ =
$
1
}
program: option_foo { $$ =
$
1
}
option_foo: ε
option_foo: foo
Tag for Parameterizing rules
Slide 43
Slide 43 text
%union {
int i;
}
program : option(foo)
;
Tag for Parameterizing rules
Slide 44
Slide 44 text
• Support for users to de
fi
ne rules
• Support for nesting of parameterizing
rules. e.g. `option(list(arg))`
• Support for %inline
Future Vision of Parameterizing rules
Slide 45
Slide 45 text
end