Converting AST June 25, 2024 After RubyKaigi 2024 LR Parser Night w/ Asakusa.rb @yui-knk Yuichiro Kaneko

About me Yuichiro Kaneko yui-knk (GitHub) / spikeolaf (Twitter) Treasure Data Engineering Manager of Applications Backend CRuby committer, mainly develop parser generator and parser Lrama LALR (1) parser generator (2023, Ruby 3.3) Love LR parser

May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan The Bison Slayer The parser monster Parser界の黎明卿 The world is now in the great age of parsers. People are setting sail into the vast sea of parsers. - RubyKaigi 2023 LT- Yuichiro Kaneko https://twitter.com/kakutani/status/1657762294431105025/ NEW !!! NEW !!!

May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan

ast_to_prism https://github.com/yui-knk/ast_to_prism Converts RubyVM::AST to Prism::Node Motivations Provide the compatible layer Clarify the difference between current Node and Prism Node Send feedbacks to current struct RNode Writing codes is easier than writing docs It may heavily depend on ruby master branch

They are watching me 2 watching and 3 stars !?

Who are Watchers and Stargazers

The position of ast_to_prism This is a missing piece Source code RubyVM::AST::Node parse.y struct rb_ast_struct struct RNode ast.c Prism::Node ast_to_prism Parser::AST::Node Prism::Translation::Parser RuboCop <>

How ast_to_prism works “parse” the source code then “convert” nodes to prism nodes

How ast_to_prism works “convert” method is a large case statement Some node types are supported, others are not

Milestones 1. Add insufficient information to Node 2. Change AST structures 3. Keep some nodes

Insuf fi cient information Location information E.g. “if”, “then”, “end” locations are not supported

Insuf fi cient information Simply, add them to nodes RNode is already free from 3 fields restriction

Change AST structures NODE_RESCUE wraps begin statement

Change AST structures NODE_ENSURE wraps NODE_RESCUE

Change AST structures Change it to be more simple structure

Change AST structures RNode is already free from 3 fields restriction !!!

Another example: UNDEF undef with list is expanded to multiple undef(s)

Another example: UNDEF Change NODE_UNDEF to holds all items

Keep some nodes NODE_BLOCK is omitted for single line body

Keep some nodes Right now, “convert_stmts” checks the node is BLOCK or not

Tips: range argument Why range optional argument is needed?

How “e” is managed Need to manage “e” variable

Prism::RescueNode Manages “e” in “reference“ field

NODE_RESBODY “e” is managed as the first line of nd_body

Need to split nd_body reference = nd_body.children[0] statements = convert_stmts(nd_body, 1..-1)

Expand NODE_RESCUE RNode is already free from 3 fields restriction !!!

How to develop Add test cases to “spec/basic_spec.rb” Implement it on “lib/ast_to_prism/parser.rb” If test passes OK. Go next NG. “pending” test case Create PR to ruby/ruby

“spec/basic_spec.rb” Add test cases

“spec/basic_spec.rb” Remove “not_supported” then implement convert logic

If you can not implement pending test case Investigate the reason why it can not Create PR to ruby/ruby Implement the conversion logic after merge

Next Steps Cover all simple cases in ast_to_prism Cover “test/*” files under ruby/ruby

May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Thank you !!!