Share
Embed
The grand strategy of Ruby Parser
RubyKaigi 2024
https://rubykaigi.org/2024/presentations/spikeolaf.html
ruby/lrama
https://github.com/ruby/lrama
yui-knk
May 15, 2024
Transcript
124 slides
-
The grand strategy of Ruby Parser May 15, 2024 in
RubyKaigi 2024 @yui-knk Yuichiro Kaneko -
About me Yuichiro Kaneko yui-knk (GitHub) / spikeolaf (Twitter) Treasure
Data Engineering Manager of Applications Backend -
TD and Ruby committers twitter: @nalsh GitHub: @nurse twitter: @k_tsj
GitHub: @k-tsj twitter: @ spikeolaf GitHub: @yui-knk twitter: @mineroaoki GitHub: @aamine twitter: @nahi GitHub: @nahi Applications Backend -
Attendees from TD @spikeolaf @nalsh @k_tsj @makimoto @ citystar (GH)
@ybiquitous @a_ksi19 @tomog105 -
About me Yuichiro Kaneko yui-knk (GitHub) / spikeolaf (Twitter) CRuby
committer, mainly develop parser generator and parser Lrama LALR (1) parser generator (2023, Ruby 3.3) The Bison Slayer Ripper Rearchitecture (2024, Ruby 3.4) Code positions to RNode (2018, Ruby 2.6) RubyVM::AbstractSyntaxTree (2018, Ruby 2.6) -
May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt,
Okinawa, Japan Introduction “ཱྀ͍ߦʹඞཁͳͷେ͖ͳΧόϯ͡Όͳ͘ɺޱͣ͞ΊΔҰ ͭͷՎ͞” - εφϑΩϯ - -
Parser in Ruby Converting input script into Abstract Syntax Tree
CRuby’s parser is LALR parser CRuby uses GNU Bison Lrama to generate parser codes -
How to create parser? Use parser generator Lrama (CRuby) Bison
(Perl, PHP, PostgreSQL) ANTLR (Hive, Trino) Hand written parser Go, Rust, C# Prism -
What is parser generator? The tool generates parser from grammar
file, “parse.y”. parse.c Lrama parse.y -
May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt,
Okinawa, Japan The grand strategy of Ruby Parser “ੜ͖Δͬͯ͜ͱɺฏͳͷ͡Όͳ͍ΜͰ͢Α” - εφϑΩϯ - -
Grand Strategy > Grand strategy or high strategy is a
state's strategy of how means (military and nonmilitary) can be used to advance and achieve national interests in the long- term. https://en.wikipedia.org/wiki/Grand_strategy -
Interests in the long-term Provide platform for LSP and other
tools Provide Universal parser Keep both Ruby grammar and parser to be maintainable -
Why LR parser is the best? LR parser Can handle
large range of languages Major parser algorithm To be precise, LR-attributed grammar I believe grammar easy for human is close to LR grammar LL parser Has has less power than LR parser PEG It’s difficult to create Error Tolerant parser A rule failure doesn’t imply a parsing failure like in context free grammars -
Why LR parser generator is the best? LR parser generator
gives accurate feedback for grammar BNF is very declarative No gap between grammar and parser implementation LR parser is based on theory of computer science -
It’s possible to implement > but nobu said it's hard
to support because of parse.y limitation. No, it’s possible!! https://github.com/yui-knk/ruby/tree/bugs_18080 -
Need to consider these patterns There is an argument or
not The arguments are sounded by parentheses or not There is block or not The symbol of pattern matching, `in` or `=>` -
Con fl icts with existing grammar There is no block
The arguments are not sounded by parentheses The symbol of pattern matching is `=>` -
LR parser generator knows this issue S/R or R/R conflict
detection is a friend for programming language designer -
Why this issue is dif fi cult to detect? Need
to check all combination of grammar rules Discussion of grammar and implementation of parser are localized -
Combination of grammar rules A lot of rules are optional
Argument is optional Parentheses around arguments are optional Block is optional (The symbol of pattern matching, `in` or `=>) Need to discuss grammar rules as group E.g. “a == b”, “1 + 2” and “1..2” are in same “arg” group If change “arg” rules, need to consider the impact on “expr” and “stmt” too -
Localized discussion & implementation Examples in a ticket is simple
Parser implementation is a combination of parts Parser generator: combination of rules Recursive Descent Parser: combination of functions, e.g. “parse_pattern_matching”, “parse_arguments” -
Localized discussion & implementation Localized discussion and implementation are good
practice Divide the difficulties However it requires mechanism to integrate these parts LR parser generator has the mechanism, conflict detection Hand written parser doesn’t have such mechanism -
Ruby grammer evolves Ruby grammar will change New syntax will
be added Existing syntax will change by feedback Parser generator works as checker/linter for grammar Can not keep soundness of grammar without the help from computer science -
BNF is very declarative parse.y: 28 lines Logic for parsing
and others are separated prism.c: 127 lines Logic are mixed up Commit 96710a3 (Sat May 4 18:03:52 2024 +0900) -
3. Gap between grammar and parser With parser generator, parser
follows grammar With hand written parser, grammar follows parser implementation -
Precedence difference The result of this code is different between
parse.y and prism How modifier rescue & if is combined is different Commit 68b6fe7 (Sat May 11 02:19:38 2024 +0900) -
Is this bug or intentional? With parser generator, let’s discuss
about grammar With hand written parser, it’s impossible to distinguish parser’s bug from grammar problem -
4. Based on theory of computer science Many books are
published for LR parser, formal language and automaton -
We can share the knowledge Joel Denny. “PSLR(1): Pseudo-Scannerless Minimal
LR(1) for the Deterministic Parsing of Composite Languages”, May 2010. https://tigerprints.clemson.edu/cgi/ viewcontent.cgi?article=1519&context=all_dissertations Lukas Diekmann and Laurence Tratt. “Don’t Panic! Better, Fewer, Syntax Errors for LR Parsers”, July 2020. https:// arxiv.org/pdf/1804.07133.pdf Joe Zimmerman “Practical LR Parser Generation”, Sep 2022 https://arxiv.org/pdf/2209.08383.pdf -
Guidebooks for beginners Yuichiro Kaneko. “Ruby Parser։ൃࢽ (14) - LR
parserશʹཧղ͠ ͨ”, December 2023. https://yui-knk.hatenablog.com/entry/ 2023/12/06/082203 shioimm/coe401_. “ͨͷ͍͠RubyͷߏจղੳπΞʔ”, March 2023. https://speakerdeck.com/coe401_/tanosiirubynogou-wen-jie-xi- tua aamine. “Rubyιʔείʔυશղઆ” ୈ 2 ෦ʮߏจղੳʯ, July 2004. https://i.loveruby.net/ja/rhg/book/ [JA] https://ruby-hacking-guide.github.io/ [EN] -
History of LR parser 1965: Donald E. Knuth invents LR
parsing. “On the translation of languages from left to right” 1975: Yacc is published 1985: GNU Bison initial release 1989: Berkeley Yacc initial release 2006: GCC migrates it’s parser from Bison to hand- written recursive-descent parsers (C++ was 2004) 2015: Go migrates it’s parser from Bison to hand- written recursive-descent parsers -
History of LR parser 2020: “Don't Panic! Better, Fewer, Syntax
Errors for LR Parsers” 2022: “Practical LR Parser Generation” 2023: “The future vision of Ruby Parser” in RubyKaigi 2023 2023: Lrama replaces Bison in CRuby 2023: New era of LR parser !! 2024: “The grand strategy of Ruby Parser” in RubyKaigi 2024 -
What is Lrama? Lrama is LR parser generator written by
Ruby Lrama (Llama) inherits the name from Yacc (Yak) and Bison It’s LR parser generator, then not “LL”ama but “LR”ama Ruby uses Lrama to generate parser from 3.3 -
Why Lrama is needed? Bison is not perfect Therefore we
hack parse.y We need more and more features Bison is not easy to enhance new features Ruby build system depends on Bison installed on your machine Lrama is installed into ruby/ruby tool directory then we can use latest features Bison is difficult to manage It was broken even though we didn't do anything when we released Ruby 2.7.7 Especially installing Bison on Windows is not easy task -
Summary The grand strategy of Ruby Parser Long term goals
Provide platform for LSP and other tools Provide Universal parser Keep both Ruby grammar and parser to be maintainable Solution LR parser and parser generator are the best friends for Ruby Lrama is new foundation for Ruby parser instead of Bison -
May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt,
Okinawa, Japan LSP “ੈͷதͬͯɺ΄Μͱʹ͓͠Ζ͍ͷͶɻۜͷ͓΅Μͷ͍ ํҰ͖ͭΓͩͱɺΈΜͳ͕৴͖ͯͨ͡ͷʹɺͥΜͥΜͭ ͷɺͣͬͱ͍͍͍ํ͕͋ͬͨͷͶ” - ϜʔϛϯϚϚ - -
Many tools need Syntax Tree Many tools uses Abstract Syntax
Tree LSP, TypeProf, RuboCop … LSP TypeProf RuboCop Abstract Syntax Tree -
User friendly node structure Current CRuby node structure was designed
as Object. Therefore some types of node, e.g. NODE_OP_ASGN2, has difficult structure. Need to re-design each node structure to be more easily understandable. More information is needed More location information Comments But there is a blocker of this goal… -
Delete parser level optimization Before VM and InstructionSequence were introduced,
CRuby interpreted AST nodes directly. Therefore some optimizations happen on parser even today. Need to delete such optimizations so that the result of parse keeps the structure of input text. But there is a blocker of this goal too… -
Union to Struct (Node) Any kinds of nodes share the
single struct definition There is no flexibility to add new field to specific type of node It’s not straightforward to cast each field based on node type Need to change data structure from union base struct to dedicated struct for each node -
Union to Struct (Node) ✅ This milestone is already achieved
on master branch -
Why Error-tolerant parser is need? LSP (Language Server Protocol) requires
parser to parse invalid code as far as possible Just raising syntax error is not enough in this case -
Error tolerance parser CPCT+ algorithm solves the problem Insert/Delete/Shift operations
based error recovery Lukas Diekmann and Laurence Tratt. “Don’t Panic! Better, Fewer, Syntax Errors for LR Parsers”, July 2020 https://arxiv.org/pdf/1804.07133.pdf -
Insert (or delete) tokens to make input script to be
syntax valid Leverage the fact LR parser has clear automaton structure How CPCT+ works? IF … END IF expr_value THEN compstmt if_tail END IF expr_value THEN … END true then -
Error tolerance parser Integration to parse.y Delete operation support Efficient
data structure (Cactuses) More accurate recovery -
Universal Parser LSP parse.y for Under graduate Error tolerance Parser
Generator (Lrama) Parser -
LSP Delete parser level optimization Union to Struct (Node) User
friendly node structure Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Error tolerance Parser Generator (Lrama) Parser ✅ 💪 Universal Parser parse.y for Under graduate -
May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt,
Okinawa, Japan parse.y for Undergraduate “Ͳ͜ʹͳ͔ͬͨΒࣗͰ࡞ͬͨΒͲ͏͍ͩʁ Ͱ͖Δ͔Ͳ͏͔ͬͯΈͳ͍ͱΘ͔Βͳ͍Μ ͡Όͳ͍͔ͳ” - εφϑΩϯ - -
Rumors about parse.y Monstrous lex_state (2017) Demon Castle parse.y (2017)
parse.y is “hell” (2019) The current parse.y is a hell (2021) -
It is one-sided match because the theory only covers one
of The Big Five parse.y calamities Gap between theory and practice Theory Practice LR parser Lex State Semantic Analysis Primitive BNF Ripper -
parse.y for Under graduate Developers, who has textbook level knowledge
of parser theory, can understand parse.y More declarative parser More expressive grammar -
More declarative parser “Scanner state update syntax” v.s. Semantic Analysis
“Scannerless parser” v.s. Lex State Practice LR parser Lex State Semantic Analysis Primitive BNF Ripper -
Is parse.y declarative? If parse.y is like below, it’s very
declarative and easy to understand Grammar rule and logic for class node creation -
Where return can be written “return” can not be in
the class context but can be in method context Invalid return in class/ module body (SyntaxError) OK -
What is lex_context Parser manages current context by lex_context These
logic are written in action by C in_class: 1 in_def: 0 in_class: 1 in_def: 1 -
Scanner state update syntax These contexts are managed by push/pop
operations They are effective in a rule, e.g. class or method definition Embed state management into grammar rule POC is implemented Want to introduce this feature to Ruby 3.4 https://github.com/ruby/lrama/pull/231 -
Why lex_state is needed In general lexer check input text
in the longest match manner otherwise longer one never matches E.g. Check “||” then check “|” -
Why lex_state is needed However in some cases, shorter token
should be returned “|” for block parameter is two “|” -
EXPR_BEG or not If lex state is EXPR_BEG then “|”
is retuned otherwise “||” is retuned A lot of conditional branches based on lex state Too complecated “|” “||” Check lex state -
Scannerless parser Is this explicit communication between parser and lexer
really needed? Parser knows current situation Parser knows “||” is not accepted after “do” -
PSLR (1) It seems good idea to integrate parser and
lexer then change to manage states on parser side Joel E. Denny. “PSLR(1): Pseudo-Scannerless Minimal LR(1) for the Deterministic Parsing of Composite Languages”, May 2010. https://tigerprints.clemson.edu/cgi/ viewcontent.cgi?article=1519&context=all_dissertations -
PSLR (1) > Nevertheless, traditional scanner and parser generators attempt
to generate loosely coupled scanners and parsers, so the user must maintain these tightly coupled scanner and parser specifications separately but consistently. > Scanner and parser specifications would be significantly more maintainable if all sub-language transitions were instead computed from a grammar by a parser generator and recognized automatically by the scanner using the parser’s stack. -
IELR PSLR is an extension of IELR Both PSLR and
IELR are invented by Joel E. Denny IELR is more powerful than LALR -
More expressive grammar “Parameterizing rules” v.s. Primitive BNF Practice LR
parser Lex State Semantic Analysis Primitive BNF Ripper -
Primitive BNF In parse.y, we need to write all grammar
rules by hand There is “common pattern” in grammar rules e.g. Optional part, list No way to abstract it -
Parameterizing rules If you are familiar with parser generator, Menhir
in OCaml is a good example -
Refactoring Ripper ✅ “User define stack” v.s. Ripper Practice LR
parser Lex State Semantic Analysis Primitive BNF Ripper -
Replace hand written parser with Racc ✅ Need to enhance
new features to Lrama Need to expand Lrama grammar Parser generator is the best friend https://github.com/ruby/lrama/pull/62 -
Summary Defeat calamities by more powerful theory, abstraction and Refactoring
Practice LR parser Lex State Semantic Analysis Primitive BNF Ripper Scanner state update syntax PSLR / IELR Parameterizing rules Refactoring Ripper -
LSP Delete parser level optimization Union to Struct (Node) User
friendly node structure Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Error tolerance Parser Generator (Lrama) Parser ✅ 💪 Universal Parser parse.y for Under graduate -
Refactoring Ripper LSP Delete parser level optimization Union to Struct
(Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ 💪 💪 💪 💪 Universal Parser -
Deep forest - parse.y - yui-knk Lv. 5 HP 122
MP 16 ydah Lv. 5 HP 114 MP 20 junk0612 Lv. 5 HP 140 MP 19 ydah & junk0612 join to the party 🎉 -
May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt,
Okinawa, Japan Universal Parser “ॱংਖ਼࢝͘͠ΊΔͱ͍͏͜ͱɺຊ͕ୈҰߦ͔Β ࢝·Δͷͱಉ͡Α͏ʹɺେͳ͜ͱ͔ͩΒͶɻສ ࣄ͕ͦΕͰܾ·ΔΜͩΑ” - Ϝʔϛϯύύ - -
Universal Parser Everyone wants to use CRuby parser mruby, PicoRuby:
Other Ruby implementation by C JRuby, TruffleRuby, ruruby: Other Ruby implementation by non-C Implementing 100 % compatible Ruby parser is a bit difficult Managing parser for each version is difficult -
What’s is the challenge? CRuby parser depends on other CRuby
functionaries !!! lexer & parser GC RString RArray RHash … rb_mRubyVMFrozen Core struct rb_iseq_struct * Ruby -
Decouple AST from imemo Ruby AST structure is managed by
GC as imemo object imemo is “Internal memo object” managed by GC Ruby’s GC is useful. It frees memory which is not used anymore Before this goal, it needs to remove objects from nodes Objects on nodes are GC marked via AST structure -
Remove Object from Node Many kinds of nodes refer to
Ruby Object MATCH, LIT, STR, DSTR, XSTR, DXSTR, DREGX, DSYM This challenge depends on “Refactoring Ripper” challenge Node RIPPER and RIPPER_VALUES also refer to Ruby Object -
The rest of the work ID ID is a number
having a one-to-one association with a string. Warnings, error messages rb_str_vcatf -
Refactoring Ripper LSP Delete parser level optimization Union to Struct
(Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ 💪 💪 💪 💪 -
Universal Parser Decouple AST from imemo Remove Object from Node
Refactoring Ripper LSP Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ ✅ ✅ 💪 💪 💪 💪 💪 -
Dark cave - Universal Parser - yui-knk Lv. 23 HP
394 MP 52 ydah Lv. 23 HP 361 MP 51 junk0612 Lv. 23 HP 340 MP 56 hasumikin Lv. 22 HP 355 MP 54 S-H-GAMELINKS Lv. 22 HP 425 MP 46 hasumikin & S-H-GAMELINKS join to the party 🎉 -
May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt,
Okinawa, Japan Other challenges “ͳʹ͔ͨΊͯ͠ΈΑ͏ͬͯͱ͖ʹɺͲ͏ͨͬͯ͠ةݥ͕ͱ ͳ͏Μͩ” - εφϑΩϯ - -
Optimize Node memory management ✅ In the past all types
of nodes used single struct Some nodes, NODE_TRUE, allocated more memory than needed This issue was solved when "Union to Struct” is achieved -
RBS Adding RBS files to Lrama parser generator so that
the implementation is more clear Led by Yla Aioi (Little-Rubyist) https://github.com/ruby/lrama/pull/417 -
Universal Parser Decouple AST from imemo Remove Object from Node
Refactoring Ripper LSP Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ ✅ ✅ 💪 💪 💪 💪 💪 -
Universal Parser Decouple AST from imemo Remove Object from Node
Refactoring Ripper LSP Optimize Node memory management Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR RBS Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ ✅ ✅ ✅ 💪 💪 💪 💪 💪 💪 -
The End Of Time yui-knk Lv. 31 HP 562 MP
68 ydah Lv. 30 HP 514 MP 67 junk0612 Lv. 31 HP 578 MP 64 hasumikin Lv. 29 HP 448 MP 68 S-H-GAMELINKS Lv. 28 HP 565 MP 60 Little-Rubyist Lv. 28 HP 442 MP 66 Little-Rubyist joins to the party 🎉 -
May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt,
Okinawa, Japan -
Universal Parser Decouple AST from imemo Remove Object from Node
Refactoring Ripper LSP Optimize Node memory management Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR RBS Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ ✅ ✅ ✅ 💪 💪 💪 💪 💪 💪 -
Is it really “Universal” Parser? Current Universal Parser is implemented
by C Each programing language needs to integrated it Parser implemented by C C Java JavaScript Dynamic link JNI/JNA FFI Other languages wasm -
True Universal Parser Higher order Universal Parser will provide parser
written by each programing language Grammar C Java JavaScript Parser (C) Parser (Java) Parser (JavaScript) Parser Generator Generate Other languages Parser (Other languages) -
Ruby parser by Ruby We can use the latest Ruby
syntax in *.rbinc source files if parse.y can be transformed to Ruby parser array.rb array.rbinc (C fi le) mk_builtin_loader.rb (ripper) + baseruby parse.y parse.rb Lrama + baseruby array.rb array.rbinc (C fi le) parse.rb + baseruby -
Programing language for action Remove C codes from actions Replace
them with simple new language -
Type inference for action It obvious that “$$” in action
has same type with “tail” The type of “tail” is same with “block_args_tail” The type of “block_args_tail” is “node_args” -
May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt,
Okinawa, Japan Conclusion “৺ͷܨ͕ͬͨؒͦ͜ɺϧϏʔʹউΔඒ͍͠ϧϏʔ͞” - εφϑΩϯ - -
The grand strategy of Ruby Parser The grand strategy of
Ruby Parser Long term goals Provide platform for LSP and other tools Provide Universal parser Keep both Ruby grammar and parser to be maintainable Solution LR parser and parser generator are the best approach for Ruby -
Development cycle with parser generator Designer can focus on grammar
Parser generator gives correct feedback Parser generator evolves Independently from grammar Programming Language Designer Grammar Parser Generator Parser Cactuses data structures Comopact data structures Panic Mode CPCT+ LALR IELR PSLR Design Input Generate Feedback Develop new features -
Universal Parser LSP parse.y for Under graduate Error tolerance Parser
Generator (Lrama) Parser -
Universal Parser Decouple AST from imemo Remove Object from Node
Refactoring Ripper LSP Optimize Node memory management Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR RBS Error tolerance Parser Generator (Lrama) Parser -
Universal Parser Decouple AST from imemo Remove Object from Node
Refactoring Ripper LSP Optimize Node memory management Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR RBS Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ ✅ ✅ ✅ 💪 💪 💪 💪 💪 💪 💪 -
Universal Parser Decouple AST from imemo Remove Object from Node
Refactoring Ripper LSP Optimize Node memory management Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR RBS Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ ✅ ✅ ✅ 💪 💪 💪 💪 💪 💪 💪 -
Ruby 3.4 Universal Parser Fine-grained Error Tolerance parser Compatibility layer
for Prism interfaces -
Acknowledgements @nobu, @nurse and other committers ESM, Inc. Parser Club
Dragon Book study group -
Acknowledgements Contributors and supporters for Lrama and parse.y Junichi Kobayashi
(@junk0612) Yudai Takada (@ydah_) Hitoshi HASUMI (@hasumikin) S.H. (@S-H-GAMELINKS) Yla Aioi (@Little_Rubyist) -
References Lrama LALR (1) parser generator https://github.com/ruby/lrama Yuichiro Kaneko. “Ruby
Parser Roadmap”, https://docs.google.com/presentation/d/ 1E4v9WPHBLjtvkN7QqulHPGJzKkwIweVfcaMsIQ984_Q Yuichiro Kaneko. “Ruby Parser։ൃࢽ (12) - LR parser generatorͷఏڙ͢Δจ๏ͷ݈શੑ”, September 2023. https://yui-knk.hatenablog.com/entry/2023/09/19/191135 Yuichiro Kaneko. “Ruby Parser։ൃࢽ (14) - LR parserશʹཧղͨ͠”, December 2023. https://yui-knk.hatenablog.com/entry/2023/12/06/082203 Lukas Diekmann and Laurence Tratt. “Don’t Panic! Better, Fewer, Syntax Errors for LR Parsers”, July 2020. https://arxiv.org/pdf/1804.07133.pdf Joel E. Denny. “PSLR(1): Pseudo-Scannerless Minimal LR(1) for the Deterministic Parsing of Composite Languages”, May 2010. https://tigerprints.clemson.edu/cgi/viewcontent.cgi? article=1519&context=all_dissertations -
May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt,
Okinawa, Japan Thank you !!!