The grand strategy of Ruby Parser

RubyKaigi 2024
https://rubykaigi.org/2024/presentations/spikeolaf.html

ruby/lrama
https://github.com/ruby/lrama

Transcript

1. The grand strategy of Ruby Parser May 15, 2024 in RubyKaigi 2024 @yui-knk Yuichiro Kaneko
2. About me Yuichiro Kaneko yui-knk (GitHub) / spikeolaf (Twitter) Treasure Data Engineering Manager of Applications Backend
3. PR: We are Gold sponsor!
4. TD and Ruby committers twitter: @nalsh GitHub: @nurse twitter: @k_tsj GitHub: @k-tsj twitter: @ spikeolaf GitHub: @yui-knk twitter: @mineroaoki GitHub: @aamine twitter: @nahi GitHub: @nahi Applications Backend
5. Attendees from TD @spikeolaf @nalsh @k_tsj @makimoto @ citystar (GH) @ybiquitous @a_ksi19 @tomog105
6. About me Yuichiro Kaneko yui-knk (GitHub) / spikeolaf (Twitter) CRuby committer, mainly develop parser generator and parser Lrama LALR (1) parser generator (2023, Ruby 3.3) The Bison Slayer Ripper Rearchitecture (2024, Ruby 3.4) Code positions to RNode (2018, Ruby 2.6) RubyVM::AbstractSyntaxTree (2018, Ruby 2.6)
7. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Introduction “௕ཱྀ͍ߦʹඞཁͳͷ͸େ͖ͳΧόϯ͡Όͳ͘ɺޱͣ͞ΊΔҰ ͭͷՎ͞” - εφϑΩϯ -
8. Parser in Ruby Converting input script into Abstract Syntax Tree CRuby’s parser is LALR parser CRuby uses GNU Bison Lrama to generate parser codes
9. How to create parser? Use parser generator Lrama (CRuby) Bison (Perl, PHP, PostgreSQL) ANTLR (Hive, Trino) Hand written parser Go, Rust, C# Prism
10. What is parser generator? The tool generates parser from grammar file, “parse.y”. parse.c Lrama parse.y
11. “parse.y” Write grammar rule with BNF (Backus–Naur form)
12. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan The grand strategy of Ruby Parser “ੜ͖Δͬͯ͜ͱ͸ɺฏ࿨ͳ΋ͷ͡Όͳ͍ΜͰ͢Α” - εφϑΩϯ -
13. Grand Strategy > Grand strategy or high strategy is a state's strategy of how means (military and nonmilitary) can be used to advance and achieve national interests in the long- term. https://en.wikipedia.org/wiki/Grand_strategy
14. Interests in the long-term Provide platform for LSP and other tools Provide Universal parser Keep both Ruby grammar and parser to be maintainable
15. Why LR parser is the best? LR parser Can handle large range of languages Major parser algorithm To be precise, LR-attributed grammar I believe grammar easy for human is close to LR grammar LL parser Has has less power than LR parser PEG It’s difficult to create Error Tolerant parser A rule failure doesn’t imply a parsing failure like in context free grammars
16. Why LR parser generator is the best? LR parser generator gives accurate feedback for grammar BNF is very declarative No gap between grammar and parser implementation LR parser is based on theory of computer science
17. 1. Accurate feedback for grammar https://bugs.ruby-lang.org/issues/18080
18. It’s possible to implement > but nobu said it's hard to support because of parse.y limitation. No, it’s possible!! https://github.com/yui-knk/ruby/tree/bugs_18080
19. Need to consider these patterns There is an argument or not The arguments are sounded by parentheses or not There is block or not The symbol of pattern matching, `in` or `=>`
20. There is one combination which is suspicious Need to consider these patterns
21. Con fl icts with existing grammar There is no block The arguments are not sounded by parentheses The symbol of pattern matching is `=>`
22. LR parser generator knows this issue S/R or R/R conflict detection is a friend for programming language designer
23. Why this issue is dif fi cult to detect? Need to check all combination of grammar rules Discussion of grammar and implementation of parser are localized
24. Combination of grammar rules A lot of rules are optional Argument is optional Parentheses around arguments are optional Block is optional (The symbol of pattern matching, `in` or `=>) Need to discuss grammar rules as group E.g. “a == b”, “1 + 2” and “1..2” are in same “arg” group If change “arg” rules, need to consider the impact on “expr” and “stmt” too
25. Localized discussion & implementation Examples in a ticket is simple Parser implementation is a combination of parts Parser generator: combination of rules Recursive Descent Parser: combination of functions, e.g. “parse_pattern_matching”, “parse_arguments”
26. Localized discussion & implementation Localized discussion and implementation are good practice Divide the difficulties However it requires mechanism to integrate these parts LR parser generator has the mechanism, conflict detection Hand written parser doesn’t have such mechanism
27. Ruby grammer evolves Ruby grammar will change New syntax will be added Existing syntax will change by feedback Parser generator works as checker/linter for grammar Can not keep soundness of grammar without the help from computer science
28. 2. BNF is very declarative parse_conditional in prism.c
29. BNF is very declarative parse.y: 28 lines Logic for parsing and others are separated prism.c: 127 lines Logic are mixed up Commit 96710a3 (Sat May 4 18:03:52 2024 +0900)
30. 3. Gap between grammar and parser With parser generator, parser follows grammar With hand written parser, grammar follows parser implementation
31. Precedence difference The result of this code is different between parse.y and prism How modifier rescue & if is combined is different Commit 68b6fe7 (Sat May 11 02:19:38 2024 +0900)
32. Is this bug or intentional? With parser generator, let’s discuss about grammar With hand written parser, it’s impossible to distinguish parser’s bug from grammar problem
33. 4. Based on theory of computer science Many books are published for LR parser, formal language and automaton
34. We can share the knowledge Joel Denny. “PSLR(1): Pseudo-Scannerless Minimal LR(1) for the Deterministic Parsing of Composite Languages”, May 2010. https://tigerprints.clemson.edu/cgi/ viewcontent.cgi?article=1519&context=all_dissertations Lukas Diekmann and Laurence Tratt. “Don’t Panic! Better, Fewer, Syntax Errors for LR Parsers”, July 2020. https:// arxiv.org/pdf/1804.07133.pdf Joe Zimmerman “Practical LR Parser Generation”, Sep 2022 https://arxiv.org/pdf/2209.08383.pdf
35. Guidebooks for beginners Yuichiro Kaneko. “Ruby Parser։ൃ೔ࢽ (14) - LR parser׬શʹཧղ͠ ͨ”, December 2023. https://yui-knk.hatenablog.com/entry/ 2023/12/06/082203 shioimm/coe401_. “ͨͷ͍͠RubyͷߏจղੳπΞʔ”, March 2023. https://speakerdeck.com/coe401_/tanosiirubynogou-wen-jie-xi- tua aamine. “Rubyιʔείʔυ׬શղઆ” ୈ 2 ෦ʮߏจղੳʯ, July 2004. https://i.loveruby.net/ja/rhg/book/ [JA] https://ruby-hacking-guide.github.io/ [EN]
36. History of LR parser 1965: Donald E. Knuth invents LR parsing. “On the translation of languages from left to right” 1975: Yacc is published 1985: GNU Bison initial release 1989: Berkeley Yacc initial release 2006: GCC migrates it’s parser from Bison to hand- written recursive-descent parsers (C++ was 2004) 2015: Go migrates it’s parser from Bison to hand- written recursive-descent parsers
37. History of LR parser 2020: “Don't Panic! Better, Fewer, Syntax Errors for LR Parsers” 2022: “Practical LR Parser Generation” 2023: “The future vision of Ruby Parser” in RubyKaigi 2023 2023: Lrama replaces Bison in CRuby 2023: New era of LR parser !! 2024: “The grand strategy of Ruby Parser” in RubyKaigi 2024
38. What is Lrama? Lrama is LR parser generator written by Ruby Lrama (Llama) inherits the name from Yacc (Yak) and Bison It’s LR parser generator, then not “LL”ama but “LR”ama Ruby uses Lrama to generate parser from 3.3
39. Why Lrama is needed? Bison is not perfect Therefore we hack parse.y We need more and more features Bison is not easy to enhance new features Ruby build system depends on Bison installed on your machine Lrama is installed into ruby/ruby tool directory then we can use latest features Bison is difficult to manage It was broken even though we didn't do anything when we released Ruby 2.7.7 Especially installing Bison on Windows is not easy task
40. Win “16th Fukuoka Ruby Award”
41. Summary The grand strategy of Ruby Parser Long term goals Provide platform for LSP and other tools Provide Universal parser Keep both Ruby grammar and parser to be maintainable Solution LR parser and parser generator are the best friends for Ruby Lrama is new foundation for Ruby parser instead of Bison
42. What’s the challenge? LSP parse.y for Undergraduate Universal Parser
43. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan LSP “ੈͷதͬͯɺ΄Μͱʹ͓΋͠Ζ͍΋ͷͶɻۜͷ͓΅Μͷ࢖͍ ํ͸Ұ͖ͭΓͩͱɺΈΜͳ͕৴͖ͯͨ͡ͷʹɺͥΜͥΜ΂ͭ ͷɺͣͬͱ͍͍࢖͍ํ͕͋ͬͨͷͶ” - ϜʔϛϯϚϚ -
44. Many tools need Syntax Tree Many tools uses Abstract Syntax Tree LSP, TypeProf, RuboCop … LSP TypeProf RuboCop Abstract Syntax Tree
45. User friendly node structure Current CRuby node structure was designed as Object. Therefore some types of node, e.g. NODE_OP_ASGN2, has difficult structure. Need to re-design each node structure to be more easily understandable. More information is needed More location information Comments But there is a blocker of this goal…
46. Delete parser level optimization Before VM and InstructionSequence were introduced, CRuby interpreted AST nodes directly. Therefore some optimizations happen on parser even today. Need to delete such optimizations so that the result of parse keeps the structure of input text. But there is a blocker of this goal too…
47. Union to Struct (Node) Any kinds of nodes share the single struct definition There is no flexibility to add new field to specific type of node It’s not straightforward to cast each field based on node type Need to change data structure from union base struct to dedicated struct for each node
48. Union to Struct (Node) ✅ This milestone is already achieved on master branch
49. Why Error-tolerant parser is need? LSP (Language Server Protocol) requires parser to parse invalid code as far as possible Just raising syntax error is not enough in this case
50. Error tolerance parser CPCT+ algorithm solves the problem Insert/Delete/Shift operations based error recovery Lukas Diekmann and Laurence Tratt. “Don’t Panic! Better, Fewer, Syntax Errors for LR Parsers”, July 2020 https://arxiv.org/pdf/1804.07133.pdf
51. Insert (or delete) tokens to make input script to be syntax valid Leverage the fact LR parser has clear automaton structure How CPCT+ works? IF … END IF expr_value THEN compstmt if_tail END IF expr_value THEN … END true then
52. Error tolerance parser Integration to parse.y Delete operation support Efficient data structure (Cactuses) More accurate recovery
53. Universal Parser LSP parse.y for Under graduate Error tolerance Parser Generator (Lrama) Parser
54. LSP Delete parser level optimization Union to Struct (Node) User friendly node structure Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Error tolerance Parser Generator (Lrama) Parser ✅ 💪 Universal Parser parse.y for Under graduate
55. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan parse.y for Undergraduate “Ͳ͜ʹ΋ͳ͔ͬͨΒࣗ෼Ͱ࡞ͬͨΒͲ͏͍ͩʁ Ͱ͖Δ͔Ͳ͏͔͸΍ͬͯΈͳ͍ͱΘ͔Βͳ͍Μ ͡Όͳ͍͔ͳ” - εφϑΩϯ -
56. Rumors about parse.y Monstrous lex_state (2017) Demon Castle parse.y (2017) parse.y is “hell” (2019) The current parse.y is a hell (2021)
57. Gap between theory and practice Practice Theory vs.
58. It is one-sided match because the theory only covers one of The Big Five parse.y calamities Gap between theory and practice Theory Practice LR parser Lex State Semantic Analysis Primitive BNF Ripper
59. parse.y for Under graduate Developers, who has textbook level knowledge of parser theory, can understand parse.y More declarative parser More expressive grammar
60. More declarative parser “Scanner state update syntax” v.s. Semantic Analysis “Scannerless parser” v.s. Lex State Practice LR parser Lex State Semantic Analysis Primitive BNF Ripper
61. Is parse.y declarative? If parse.y is like below, it’s very declarative and easy to understand Grammar rule and logic for class node creation
62. Life is hard Manage local variables tables and contexts (lex_context)
63. Where return can be written “return” can not be in the class context but can be in method context Invalid return in class/ module body (SyntaxError) OK
64. What is lex_context Parser manages current context by lex_context These logic are written in action by C in_class: 1 in_def: 0 in_class: 1 in_def: 1
65. Scanner state update syntax These contexts are managed by push/pop operations They are effective in a rule, e.g. class or method definition Embed state management into grammar rule POC is implemented Want to introduce this feature to Ruby 3.4 https://github.com/ruby/lrama/pull/231
66. Monstrous lex_state Usually lexer is state less Ruby’s lexer has 13 state bits!
67. Why lex_state is needed In general lexer check input text in the longest match manner otherwise longer one never matches E.g. Check “||” then check “|”
68. Why lex_state is needed However in some cases, shorter token should be returned “|” for block parameter is two “|”
69. EXPR_BEG or not If lex state is EXPR_BEG then “|” is retuned otherwise “||” is retuned A lot of conditional branches based on lex state Too complecated “|” “||” Check lex state
70. Scannerless parser Is this explicit communication between parser and lexer really needed? Parser knows current situation Parser knows “||” is not accepted after “do”
71. PSLR (1) It seems good idea to integrate parser and lexer then change to manage states on parser side Joel E. Denny. “PSLR(1): Pseudo-Scannerless Minimal LR(1) for the Deterministic Parsing of Composite Languages”, May 2010. https://tigerprints.clemson.edu/cgi/ viewcontent.cgi?article=1519&context=all_dissertations
72. PSLR (1) > Nevertheless, traditional scanner and parser generators attempt to generate loosely coupled scanners and parsers, so the user must maintain these tightly coupled scanner and parser specifications separately but consistently. > Scanner and parser specifications would be significantly more maintainable if all sub-language transitions were instead computed from a grammar by a parser generator and recognized automatically by the scanner using the parser’s stack.
73. IELR PSLR is an extension of IELR Both PSLR and IELR are invented by Joel E. Denny IELR is more powerful than LALR
74. Day 3: 14:10 - 14:40
75. More expressive grammar “Parameterizing rules” v.s. Primitive BNF Practice LR parser Lex State Semantic Analysis Primitive BNF Ripper
76. Primitive BNF In parse.y, we need to write all grammar rules by hand There is “common pattern” in grammar rules e.g. Optional part, list No way to abstract it
77. Parameterizing rules If you are familiar with parser generator, Menhir in OCaml is a good example
78. Day 2: 11:30 - 12:00
79. Refactoring Ripper ✅ “User define stack” v.s. Ripper Practice LR parser Lex State Semantic Analysis Primitive BNF Ripper
80. Day 2: 17:20 - 18:20 (LT)
81. Replace hand written parser with Racc ✅ Need to enhance new features to Lrama Need to expand Lrama grammar Parser generator is the best friend https://github.com/ruby/lrama/pull/62
82. Summary Defeat calamities by more powerful theory, abstraction and Refactoring Practice LR parser Lex State Semantic Analysis Primitive BNF Ripper Scanner state update syntax PSLR / IELR Parameterizing rules Refactoring Ripper
83. LSP Delete parser level optimization Union to Struct (Node) User friendly node structure Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Error tolerance Parser Generator (Lrama) Parser ✅ 💪 Universal Parser parse.y for Under graduate
84. Refactoring Ripper LSP Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ 💪 💪 💪 💪 Universal Parser
85. Deep forest - parse.y - yui-knk Lv. 5 HP 122 MP 16 ydah Lv. 5 HP 114 MP 20 junk0612 Lv. 5 HP 140 MP 19 ydah & junk0612 join to the party 🎉
86. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Universal Parser “ॱংਖ਼࢝͘͠ΊΔͱ͍͏͜ͱ͸ɺຊ͕ୈҰߦ͔Β ࢝·Δͷͱಉ͡Α͏ʹɺେ੾ͳ͜ͱ͔ͩΒͶɻສ ࣄ͕ͦΕͰܾ·ΔΜͩΑ” - Ϝʔϛϯύύ -
87. Universal Parser Everyone wants to use CRuby parser mruby, PicoRuby: Other Ruby implementation by C JRuby, TruffleRuby, ruruby: Other Ruby implementation by non-C Implementing 100 % compatible Ruby parser is a bit difficult Managing parser for each version is difficult
88. What’s is the challenge? CRuby parser depends on other CRuby functionaries !!! lexer & parser GC RString RArray RHash … rb_mRubyVMFrozen Core struct rb_iseq_struct * Ruby
89. Decouple AST from imemo Ruby AST structure is managed by GC as imemo object imemo is “Internal memo object” managed by GC Ruby’s GC is useful. It frees memory which is not used anymore Before this goal, it needs to remove objects from nodes Objects on nodes are GC marked via AST structure
90. Dependencies of GC mark “str” :sym
91. Remove Object from Node Many kinds of nodes refer to Ruby Object MATCH, LIT, STR, DSTR, XSTR, DXSTR, DREGX, DSYM This challenge depends on “Refactoring Ripper” challenge Node RIPPER and RIPPER_VALUES also refer to Ruby Object
92. Both of them are resolved
93. Day 2: 14:10 - 14:40
94. Day 2: 17:20 - 18:20 (LT)
95. The rest of the work ID ID is a number having a one-to-one association with a string. Warnings, error messages rb_str_vcatf
96. Refactoring Ripper LSP Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ 💪 💪 💪 💪
97. Universal Parser Decouple AST from imemo Remove Object from Node Refactoring Ripper LSP Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ ✅ ✅ 💪 💪 💪 💪 💪
98. Dark cave - Universal Parser - yui-knk Lv. 23 HP 394 MP 52 ydah Lv. 23 HP 361 MP 51 junk0612 Lv. 23 HP 340 MP 56 hasumikin Lv. 22 HP 355 MP 54 S-H-GAMELINKS Lv. 22 HP 425 MP 46 hasumikin & S-H-GAMELINKS join to the party 🎉
99. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Other challenges “ͳʹ͔ͨΊͯ͠ΈΑ͏ͬͯͱ͖ʹ͸ɺͲ͏ͨͬͯ͠ةݥ͕ͱ ΋ͳ͏Μͩ” - εφϑΩϯ -
100. Optimize Node memory management ✅ In the past all types of nodes used single struct Some nodes, NODE_TRUE, allocated more memory than needed This issue was solved when "Union to Struct” is achieved
101. RBS Adding RBS files to Lrama parser generator so that the implementation is more clear Led by Yla Aioi (Little-Rubyist) https://github.com/ruby/lrama/pull/417
102. Universal Parser Decouple AST from imemo Remove Object from Node Refactoring Ripper LSP Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ ✅ ✅ 💪 💪 💪 💪 💪
103. Universal Parser Decouple AST from imemo Remove Object from Node Refactoring Ripper LSP Optimize Node memory management Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR RBS Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ ✅ ✅ ✅ 💪 💪 💪 💪 💪 💪
104. The End Of Time yui-knk Lv. 31 HP 562 MP 68 ydah Lv. 30 HP 514 MP 67 junk0612 Lv. 31 HP 578 MP 64 hasumikin Lv. 29 HP 448 MP 68 S-H-GAMELINKS Lv. 28 HP 565 MP 60 Little-Rubyist Lv. 28 HP 442 MP 66 Little-Rubyist joins to the party 🎉
105. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
106. Universal Parser Decouple AST from imemo Remove Object from Node Refactoring Ripper LSP Optimize Node memory management Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR RBS Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ ✅ ✅ ✅ 💪 💪 💪 💪 💪 💪
107. Is it really “Universal” Parser? Current Universal Parser is implemented by C Each programing language needs to integrated it Parser implemented by C C Java JavaScript Dynamic link JNI/JNA FFI Other languages wasm
108. True Universal Parser Higher order Universal Parser will provide parser written by each programing language Grammar C Java JavaScript Parser (C) Parser (Java) Parser (JavaScript) Parser Generator Generate Other languages Parser (Other languages)
109. Ruby parser by Ruby We can use the latest Ruby syntax in *.rbinc source files if parse.y can be transformed to Ruby parser array.rb array.rbinc (C fi le) mk_builtin_loader.rb (ripper) + baseruby parse.y parse.rb Lrama + baseruby array.rb array.rbinc (C fi le) parse.rb + baseruby
110. Programing language for action Remove C codes from actions Replace them with simple new language
111. Type inference for action It obvious that “$$” in action has same type with “tail” The type of “tail” is same with “block_args_tail” The type of “block_args_tail” is “node_args”
112. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Conclusion “৺ͷܨ͕ͬͨ஥ؒͦ͜ɺϧϏʔʹ΋উΔඒ͍͠ϧϏʔ͞” - εφϑΩϯ -
113. The grand strategy of Ruby Parser The grand strategy of Ruby Parser Long term goals Provide platform for LSP and other tools Provide Universal parser Keep both Ruby grammar and parser to be maintainable Solution LR parser and parser generator are the best approach for Ruby
114. Development cycle with parser generator Designer can focus on grammar Parser generator gives correct feedback Parser generator evolves Independently from grammar Programming Language Designer Grammar Parser Generator Parser Cactuses data structures Comopact data structures Panic Mode CPCT+ LALR IELR PSLR Design Input Generate Feedback Develop new features
115. Universal Parser LSP parse.y for Under graduate Error tolerance Parser Generator (Lrama) Parser
116. Universal Parser Decouple AST from imemo Remove Object from Node Refactoring Ripper LSP Optimize Node memory management Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR RBS Error tolerance Parser Generator (Lrama) Parser
117. Universal Parser Decouple AST from imemo Remove Object from Node Refactoring Ripper LSP Optimize Node memory management Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR RBS Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ ✅ ✅ ✅ 💪 💪 💪 💪 💪 💪 💪
118. Universal Parser Decouple AST from imemo Remove Object from Node Refactoring Ripper LSP Optimize Node memory management Delete parser level optimization Union to Struct (Node) User friendly node structure parse.y for Under graduate More declarative parser Ef fi cient data structure (Cactuses) Delete operation support Integration to parse.y More accurate recovery Parameterizing rules Replace hand written parser with Racc User de fi ned stack Scanner state update syntax Scannerless parser IELR RBS Error tolerance Parser Generator (Lrama) Parser ✅ ✅ ✅ ✅ ✅ ✅ ✅ 💪 💪 💪 💪 💪 💪 💪
119. Ruby 3.4 Universal Parser Fine-grained Error Tolerance parser Compatibility layer for Prism interfaces
120. Need your help ! LSP milestones Compatibility layer for Prism interfaces
121. Acknowledgements @nobu, @nurse and other committers ESM, Inc. Parser Club Dragon Book study group
122. Acknowledgements Contributors and supporters for Lrama and parse.y Junichi Kobayashi (@junk0612) Yudai Takada (@ydah_) Hitoshi HASUMI (@hasumikin) S.H. (@S-H-GAMELINKS) Yla Aioi (@Little_Rubyist)
123. References Lrama LALR (1) parser generator https://github.com/ruby/lrama Yuichiro Kaneko. “Ruby Parser Roadmap”, https://docs.google.com/presentation/d/ 1E4v9WPHBLjtvkN7QqulHPGJzKkwIweVfcaMsIQ984_Q Yuichiro Kaneko. “Ruby Parser։ൃ೔ࢽ (12) - LR parser generatorͷఏڙ͢Δจ๏ͷ݈શੑ”, September 2023. https://yui-knk.hatenablog.com/entry/2023/09/19/191135 Yuichiro Kaneko. “Ruby Parser։ൃ೔ࢽ (14) - LR parser׬શʹཧղͨ͠”, December 2023. https://yui-knk.hatenablog.com/entry/2023/12/06/082203 Lukas Diekmann and Laurence Tratt. “Don’t Panic! Better, Fewer, Syntax Errors for LR Parsers”, July 2020. https://arxiv.org/pdf/1804.07133.pdf Joel E. Denny. “PSLR(1): Pseudo-Scannerless Minimal LR(1) for the Deterministic Parsing of Composite Languages”, May 2010. https://tigerprints.clemson.edu/cgi/viewcontent.cgi? article=1519&context=all_dissertations
124. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Thank you !!!