Contributing to the Ruby Parser

by S.H.

Slide 1

Slide 1 text

Contributing to the Ruby Parser S.H.

Slide 2

Slide 2 text

About Me ● S.H. ● Software engineer at ESM, Inc. ● Hamada.rb Organizer ● I’m hosting Ruby Hacking Challenge in Hamada.rb

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

ruby/ruby contributors

Slide 5

Slide 5 text

ruby/ruby contributors I’m here!

Slide 6

Slide 6 text

parse.y contributors

Slide 7

Slide 7 text

parse.y contributors I’m here!

Slide 8

Slide 8 text

Today Topic is… ● Why I got interested in the Ruby Parser ● How I started contributing to the Ruby Parser ● How I started contributing to the Universal Parser

Slide 9

Slide 9 text

In RubyKaigi 2021 Takeout

Slide 10

Slide 10 text

In RubyKaigi 2021 Takeout ● parse.y is difficult to maintain ○ No one but nobu can maintain parse.y ● parse.y seems hard to understand

Slide 11

Slide 11 text

Is parse.y really that hard to maintain?

Slide 12

Slide 12 text

I’m worried

Slide 13

Slide 13 text

I started to reading parse.y

Slide 14

Slide 14 text

My first impression of parse.y ● The codebase is large ● It’s hard to figure out what to begin reading ● I have a no clue about BNF

Slide 15

Slide 15 text

What I read to study to parse.y ● Ruby Under the Microscope ● Ruby Hacking Guide ● ruby trunk changes

Slide 16

Slide 16 text

I’ve started to understand parse.y a bit

Slide 17

Slide 17 text

Difficulties I noticed while reading ● Ruby Parser and Ripper are interwind ● The overall codebase is large

Slide 18

Slide 18 text

Difficulties I noticed while reading ● Ruby Parser and Ripper are interwind ● The overall codebase is large It’s too hard

Slide 19

Slide 19 text

Difficulties I noticed while reading ● Ruby Parser and Ripper are interwind ● The overall codebase is large May be able to do something here?

Slide 20

Slide 20 text

Let’s do it!

Slide 21

Slide 21 text

Action ● Reusing BNF ● Introducing some macros ● Introducing Ruby C API

Slide 22

Slide 22 text

Reusing BNF (1)

Slide 23

Slide 23 text

Reusing BNF (1)

Slide 24

Slide 24 text

Reusing BNF (2)

Slide 25

Slide 25 text

Reusing BNF (2)

Slide 26

Slide 26 text

Reusing BNF (3)

Slide 27

Slide 27 text

Reusing BNF (3)

Slide 28

Slide 28 text

Introducing macro (1)

Slide 29

Slide 29 text

Introducing macro (2)

Slide 30

Slide 30 text

Introducing Ruby C API

Slide 31

Slide 31 text

Results ● Cut down codebase ● Added dependencies by introducing macros and Ruby C API

Slide 32

Slide 32 text

In RubyKaigi 2023

Slide 33

Slide 33 text

In RubyKaigi 2023 ● I feel the future of the Ruby Parser in “The Future of the Ruby Parser” at spikeolaf ● I realized that added macros and Ruby C API are make dependencies for Universal Parser

Slide 34

Slide 34 text

After RubyKaigi 2023

Slide 35

Slide 35 text

After RubyKaigi 2023 ● I read “Started parse.y Refactoring Challenge” ○ https://yui-knk.hatenablog.com/entry/202 3/06/18/162100 ● It was written want to remove the generation of literal objects in parse.y

Slide 36

Slide 36 text

Why? ● parse.y needs Ruby C API for generation of literal objects ● Will obtain a simpler AST ○ e.g. Fixnum, Bignum, and Float…

Slide 37

Slide 37 text

Let’s do it!

Slide 38

Slide 38 text

Introducing new NODE (1)

Slide 39

Slide 39 text

Introducing new NODE (2)

Slide 40

Slide 40 text

Reducing C API dependencies (1)

Slide 41

Slide 41 text

Results ● Obtained a simpler AST for some literal ○ e.g. Numeric and __ENCODING__ ● Reducing Ruby C API dependencies for Universal Parser

Slide 42

Slide 42 text

KPT Keep motivation Should more disscuss Share my experience Increase Parser Contributor Keep Problem Try

Slide 43

Slide 43 text

parse.y is NOT Hell! We can hack parse.y!