最高の構文木の設計 2024年版

by yui-knk

Slide 1

Slide 1 text

Osaka RubyKaigi 04 2024/08/24 in Osaka RubyKaigi 04 @yui-knk Yuichiro Kaneko ࠷ߴͷߏจ໦ͷઃܭ 2024೥൛

Slide 2

Slide 2 text

Osaka RubyKaigi 04 ࣗݾ঺հ

Slide 3

Slide 3 text

About me • Yuichiro Kaneko • yui-knk (GitHub) / spikeolaf (Twitter) • Treasure Data • Engineering Manager of Applications Backend

Slide 4

Slide 4 text

TD and Ruby committers twitter: @nalsh GitHub: @nurse twitter: @k_tsj GitHub: @k-tsj twitter: @ spikeolaf GitHub: @yui-knk twitter: @mineroaoki GitHub: @aamine twitter: @nahi GitHub: @nahi Applications Backend

Slide 5

Slide 5 text

About me • CRuby committer, mainly develop parser generator and parser • Lrama LALR (1) parser generator (2023, Ruby 3.3) • The Bison Slayer • The parser monster • Parserքͷᴈ໌ᷭ • Ripper Rearchitecture (2024, Ruby 3.4) • Code positions to RNode (2018, Ruby 2.6) • RubyVM::AbstractSyntaxTree (2018, Ruby 2.6)

Slide 6

Slide 6 text

Osaka RubyKaigi 04 ࢲͱେࡕ

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

https://rubykansai.doorkeeper.jp/events/8078/participants https://shinosakarb.doorkeeper.jp/events/8604/participants ؔ੢ͷRuby ίϛϡχςΟ

Slide 9

Slide 9 text

ߏจ໦ʹਅཧ͸͋Δͷ͔ • Rubyͷparserͷ࣮૷͸ෳ਺͋Δ • ͦΕͧΕͷߏจ໦͸ඍົʹҟͳΔ • Ϣʔεέʔεʹ߹ΘͤͯΞυϗοΫʹ֦ு͍ͯ͠ΔΑ͏ʹࢥ͑Δ • Ͳ͏ਐΊΔ͔? • ϢʔεέʔεΛूΊΔ • ઃܭΛߟ͑Δ • ઃܭ͕ϢʔεέʔεΛͲͷ͘Β͍ΧόʔͰ͖͍ͯΔ͔ධՁ͢Δ

Slide 10

Slide 10 text

Osaka RubyKaigi 04 ߏจ໦ͷ࢖ΘΕํ ͷมભ

Slide 11

Slide 11 text

Rubyͷίʔυ͕࣮ߦ͞ΕΔ·Ͱ • ೖྗ͞ΕͨRuby Script͸ParserʹΑͬͯந৅ߏจ໦(Abstract Syntax Tree: AST)ʹม׵͞ΕΔ • Compiler͸ASTΛByte Codeʹม׵͢Δ • Virtual Machine͕Byte CodeΛ࣮ߦ͢Δ

Slide 12

Slide 12 text

ݪ࢝తͳݴޠ -ి୎- • ి୎ͷ࣮૷Ͱ͸ɺߏจղੳ͠ͳ͕Β࣮ࡍͷܭࢉΛߦ͏͜ͱ΋͋Δ • ྫ: expr '+' term ͷΞΫγϣϯͰ࣮ࡍʹ਺஋ͷՃࢉΛ͓͜ͳ͍ͬͯΔ

Slide 13

Slide 13 text

LR parser 101 • https://github.com/yui-knk/lr-parser-101

Slide 14

Slide 14 text

ݱ࣮͸ෳࡶ -ϝιου- • ྫ͑͹ϝιου • 5ߦ໨ͷ`m`ͷݺͼग़࣌͠ʹ͸ϝιου`m`ͷఆٛ͸طʹύʔεࡁΈ • Ұճ໨ͷύʔε࣌ʹ͸จࣈྻͱͯ͠`m`ͷఆٛΛ͓͍࣋ͬͯͯɺݺͼग़͠ ࣌ʹ࠶౓ύʔεͯ͠ܭࢉ͢Δ!? (ͨ͘͠ͳ͍)

Slide 15

Slide 15 text

ݱ࣮͸ෳࡶ -ܕγεςϜ- • ྫ͑͹ܕγεςϜ • `{…}`ͷ෦෼ʹܕγεςϜͷ࣮૷ΛೖΕΔ!? (͍Εͨ͘ͳ͍)

Slide 16

Slide 16 text

ߏจ໦ͷಋೖ • ύʔεͱͦͷଞͷܭࢉΛ෼཭͢ΔͨΊʹதؒͷσʔλߏ଄Λಋೖ͢Δ • Parser͸ߏจ໦ͷ࡞੒ʹूத͢Δ͜ͱ͕Ͱ͖Δ

Slide 17

Slide 17 text

ߏจ໦ͷಋೖ • ߏจ໦ͱ͸ϓϩάϥϜͷจ๏ߏ଄Λ໦ߏ଄ͱͯ͠දͨ͠΋ͷ Class Method Method Assignment @name Call name capitalize

Slide 18

Slide 18 text

Osaka RubyKaigi 04 ίʔυΛ࣮ߦ͢ΔͨΊ ͷந৅ߏจ໦

Slide 19

Slide 19 text

͞·͟·ͳ if • RubyͰ͸ifʹ͍͍͔ͭͯͭ͘ͷҟͳΔॻ͖ํ͕͋Δ • ී௨ͷif • ޙஔif • ࡾ߲ԋࢉࢠ

Slide 20

Slide 20 text

if Λ࣮ߦͯ͠ΈΔ • ͜ͷ3ͭͷϓϩάϥϜΛ࣮ߦ͢Δ͜ͱΛߟ͑Δ • ৚݅ࣜ (`a == 1`) Λ࣮ߦ͢Δ • ࣮ߦ݁Ռ͕truthyͳΒ :t Λදࣔ͢Δ • ࣮ߦ݁Ռ͕falsyͳΒ :f Λදࣔ͢Δ • ஫: ޙஔifͰ͸else۟͸ଘࡏ͠ͳ͍ • ࣮ߦͷ࢓ํ͸ॻ͖ํʹґଘ͠ͳ͍

Slide 21

Slide 21 text

• ී௨ͷifͱࡾ߲ԋࢉࢠͷif͸શ͘ಉ͡໦ߏ଄ͰදݱͰ͖Δ 3ͭͷ if ͱ1ͭͷදݱ NODE_IF NODE_OPCALL "a == 1" NODE_FCALL "p :t" NODE_FCALL "p :f” cond body else

Slide 22

Slide 22 text

• ޙஔif͸else͕ৗʹnullʹͳΔ͕ɺ͜Ε΋ଞͷifͱಉ͡໦ߏ଄ͰදݱͰ͖Δ 3ͭͷ if ͱ1ͭͷදݱ NODE_IF NODE_OPCALL "a == 1" NODE_FCALL "p :t" null cond body else

Slide 23

Slide 23 text

if ͷٯ - unless - • bodyͱelseΛೖΕସ͑Δ͜ͱͰunless΋ಉ͡໦ߏ଄ͰදݱͰ͖Δ NODE_IF NODE_OPCALL "a == 1" NODE_FCALL "p :f” NODE_FCALL "p :t” cond body else NODE_UNLESS NODE_OPCALL "a == 1" NODE_FCALL "p :t" NODE_FCALL "p :f” cond body else

Slide 24

Slide 24 text

͔ͭͯNODE_UNLESS͸ͳ͔ͬͨ • ࣮͸RubyʹNODE_UNLESS͕ಋೖ͞Εͨͷ͸Ruby 2.5͔Β https://github.com/ruby/ruby/commit/3c8c17d30defb41399eed65f2cf0f30e5469fa07

Slide 25

Slide 25 text

࣮ߦͷͨΊͷ"ந৅"ߏจ໦ • ίʔυΛ࣮ߦ͢Δ͜ͱʹ஫໨͢Δͱҙຯ͕ಉ͡Ͱจ๏্ͷදݱ͕ҟͳΔཁ ૉΛ1ͭͷϊʔυͰදͤΔ • ந৅ߏจ໦ͱΑͿ • ந৅Խ͢ΔͱམͪΔཁૉ • ΋ͱͷίʔυͰͷSyntax (if͔ޙஔif͔ࡾ߲ԋࢉࢠ͔(unless͔)) • ΧοίͳͲߏจ໦Ͱදݱ͞Ε͍ͯΔ΋ͷ • ۭനɺվߦͳͲ

Slide 26

Slide 26 text

ந৅ߏจ໦ͷา͖ํ • ͲͷΑ͏ʹߏจ໦ΛḷΔ͔΋ͻͱͭͷϢʔεέʔε nd_args NODE_CALL mid: :m NODE_VCALL nd_mid: :obj NODE_INTEGER val: 1 NODE_LIST nd_recv NODE_INTEGER val: 2

Slide 27

Slide 27 text

ந৅ߏจ໦ͷา͖ํ • ߏจ໦Λ࣮ߦ͢Δ͍͞ʹ͸্(਌)͔ΒԼ(ࢠ)΁ḷΔ (2) nd_args NODE_CALL mid: :m NODE_VCALL nd_mid: :obj NODE_INTEGER val: 1 NODE_LIST (1) nd_recv NODE_INTEGER val: 2 (1) ϨγʔόʔͷίϯύΠϧ (2) Ҿ਺ͷίϯύΠϧ (3) ϝιου m ͷݺͼग़͠ (3)

Slide 28

Slide 28 text

·ͱΊ • ਌ϊʔυ͔ΒࢠϊʔυΛḷΓ͍ͨ • ifͷΑ͏ʹॻ͖ํ͕ෳ਺͋Δ৔߹Ͱ΋ɺͦͷҙຯʹ஫໨ͯ͠1ͭͷϊʔυʹ ·ͱΊͨ΄͏͕ѻ͍΍͍͢ • τʔΫϯ΍ίϝϯτͷҐஔ৘ใɺ಺༰ͳͲ͸ѻΘͳ͍ • దͨ͠σʔλߏ଄ͱͯ͠ந৅ߏจ໦͕͋Δ

Slide 29

Slide 29 text

Osaka RubyKaigi 04 ίʔυΛղੳ͢ΔͨΊ ͷߏจ໦

Slide 30

Slide 30 text

ίʔυղੳͷ۩ମྫ • LSP • ruby-lsp gem (Shopify/ruby-lsp) • Linter • RuboCop gem (rubocop/rubocop) • Code Formatter • RuboCop gem (rubocop/rubocop)

Slide 31

Slide 31 text

LSPͷ৔߹ • τʔΫϯΛղੳ͢Δ • ίϝϯτΛղੳ͢Δ • ίʔυΛॻ͖׵͑Δ • ߏจ໦ͷḷΓํ

Slide 32

Slide 32 text

ͬ͘͟ΓLSP • ΫϥΠΞϯτ (Language Client) • VS CodeͳͲ • αʔόʔ (Language Server) • ruby-lspͳͲ

Slide 33

Slide 33 text

τʔΫϯΛղੳ͢Δ • Semantic Tokens • def, foo, + ͳͲʹ৭Λ͚ͭΔػೳ • αʔόʔ͸֤τʔΫϯͷҐஔ৘ใͱछྨΛฦ͢ • ਖ਼֬ʹ͸Τϯίʔυͨ͠৘ใΛฦ͢ • ֤τʔΫϯͷҐஔ৘ใ΍छผΛऔಘ͍ͨ͠

Slide 34

Slide 34 text

ίϝϯτΛղੳ͢Δ • DocumentLink • ίϝϯτͷҰ෦Λlinkʹ͢Δػೳ • αʔόʔ͸linkʹ͢ΔൣғͱURIΛฦ͢ • ίϝϯτͷ಺༰ͱҐஔ৘ใΛऔಘ͍ͨ͠ • Ϋϥεఆٛ΍ϝιουఆٛͷίϝϯτ͚ͩΛର৅ʹ͢ΔͳΒɺίϝϯτͱ ϊʔυΛඥ͚ͮΔඞཁ͕͋Δ

Slide 35

Slide 35 text

ίʔυΛॻ͖׵͑Δ • CodeActionResolve • “Extract Variable”ͳͲͷϦϑΝΫλϦϯάػೳ • ίʔυΛબ୒ͯ͠ఏҊ͞ΕͨΞΫγϣϯΛ࣮ߦ

Slide 36

Slide 36 text

ίʔυΛॻ͖׵͑Δ • ࣮ࡍͷίʔυͷॻ͖׵͑͸ΫϥΠΞϯτ͕ߦ͏ • αʔόʔ͸ॻ͖׵͑Δൣғ(range)ͱॻ͖׵͑ޙͷςΩετ(newText)Λฦ ͢

Slide 37

Slide 37 text

ίʔυΛॻ͖׵͑Δ • ࢦఆ͞Εͨൣғ͔ΒϊʔυΛಛఆ͢Δ • ϊʔυ΋͘͠͸ࢦఆ͞ΕͨൣғʹରԠ͢ΔίʔυΛऔಘ͢Δ • ίϝϯτ΍ۭനͳͲ΋ѻ͏ඞཁ͕͋Δ • ৽͍͠ίʔυΛੜ੒͢Δ 1 + 1 1 1 NODE_OPCALL (0, 0)-(0, 5) new_variable = 1 + 1 new_variable

Slide 38

Slide 38 text

ॻ͖׵͑ͷ೉͠͞ • “Extract Variable”ͷΞϧΰϦζϜΛߟ͑ͯΈΔ • "var = ࢦఆ͞Εͨൣғ" ͱ͍͏ίʔυΛҰ্ͭͷߦʹ௥Ճ͢Δ • ࢦఆ͞ΕͨൣғΛ "var" Ͱஔ͖׵͑Δ 1 + 1 var = 1 + 1 var var = 1 + 1 Λ1ߦ্ʹ௥Ճ 1 + 1 Λ var Ͱஔ͖׵͑Δ

Slide 39

Slide 39 text

ॻ͖׵͑ͷ೉͠͞ • ϫϯϥΠφʔͷϒϩοΫ಺෦ͷϦϑΝΫλϦϯάͰ͸1ߦ্ʹม਺Λఆٛ͠ ͯ΄͍͠Θ͚Ͱ͸ͳ͍ • "i + 1" ͱ͍͏ࣜͷલʹม਺Λఆٛͯ͠΄͍͠ • ϊʔυͷछྨ΍Ґஔ৘ใ͕ඞཁ 10.times {|i| i + 1 } var = i + 1 10.times {|i| var } i + 1 Λ var Ͱஔ͖׵͑Δ var = i + 1 Λ1ߦ্ʹ௥Ճ 10.times {|i| i + 1 } 10.times {|i| var = i + 1; var } i + 1 Λ var Ͱஔ͖׵͑Δ var = i + 1 Λ௚લʹ௥Ճ ⭕ ❌

Slide 40

Slide 40 text

ॻ͖׵͑ͷ೉͠͞ • ϫϯϥΠφʔͩͬͨΒͱ͍͏ͷ΋࣮͸ۂऀ • ൑ఆ͢΂͖͸ϒϩοΫͷ։࢝ߦͱબ୒ൣғͷߦ͕ಉ͔͡Ͳ͏͔ • ruby-lsp: v.0.7.14 ❌ ⭕

Slide 41

Slide 41 text

ॻ͖׵͑ͷ೉͠͞ • ෳ਺ߦ͕བྷΉͱ೉͍͠ • ruby-lsp: v.0.7.14 • ͜ͷઌ͸܅ࣗ਎ͷ໨Ͱ͔֬Ίͯ͘Ε!! • ςΩετϑΝΠϧΛ͍͡ΔΑ͏ʹίʔυΛॻ͖׵͑Δͷ͸ҙ֎ͱ೉͍͠

Slide 42

Slide 42 text

༨ஊ: ίϝϯτΛͲ͏ѻ͍͍͔ͨ • ෳ਺ͷࣜͷ্ʹॻ͍ͯ͋Δίϝϯτ͸Ͳ͜ʹ͔͔Δͷ͔? • 1ͭͷࣜʹ͔͔Δέʔεͱෳ਺ࣜʹ͔͔Δέʔε͕͋Γͦ͏ • બ୒ൣғʹΑͬͯ੾Γग़͠ํΛม͑Δͷ͕͍͍͔ʁ def count # @obj can be nil until # #compute is called @obj&.count || 0 end def count # Default value is 0 @obj&.count || 0 end def count # @obj can be nil until # #compute is called var = @obj&.count var || 0 end def count var = @obj&.count # Default value is 0 var || 0 end

Slide 43

Slide 43 text

ߏจ໦ɺԼ͔ΒݟΔ͔ʁ্͔ΒݟΔ͔ʁ • ౴: Ͳͬͪ΋ඞཁ • ্͔ΒԼ΁ḷΔέʔε • FoldingRanges • CodeLens • … • Լ͔Β্΁ḷΔέʔε • SelectionRange • Completion • …

Slide 44

Slide 44 text

্͔ΒԼ΁ • FoldingRanges • ίʔυΛ͋Δ୯ҐͰંΓͨͨΉػೳ • ෳ਺ͷંΓ৞ΉൣғΛαʔόʔ͕ฦ͢ • ߏจ໦Λ্͔ΒԼ΁ḷ͍͖ͬͯંΓ৞Ήର৅ͷϊʔυ Λݟ͚ͭΔͨͼʹϨεϙϯεʹؚΊΔ ΫϦοΫ͢Δ

Slide 45

Slide 45 text

্͔ΒԼ΁ NODE_CLASS NODE_DEF NODE_IF p :t NODE_DEF expr

Slide 46

Slide 46 text

Լ͔Β্΁ • SelectionRange • ίʔυͷબ୒ൣғΛม͑Δػೳ • ΫϥΠΞϯτ͔ΒҐஔ৘ใ͕͓͘ΒΕͯ͘ΔͷͰɺରԠ͢ΔൣғΛαʔό ʔ͕ฦ͢

Slide 47

Slide 47 text

Լ͔Β্΁ NODE_CLASS NODE_DEF obj.m1

Slide 48

Slide 48 text

Լ͔Β্ɺ্͔ͦͯ͠ΒԼ • DocumentHighlight • બ୒͞Εͨม਺ͳͲΛϋΠϥΠτ͢ΔͨΊͷػೳ • ϋΠϥΠτ͢Δൣғ • ϩʔΧϧม਺: ಉҰϝιου಺෦ • Πϯελϯεม਺: ಉҰΫϥε಺෦

Slide 49

Slide 49 text

ϩʔΧϧม਺ͷϋΠϥΠτ • 4ߦ໨ͷbarΛબΜͩ࣌ • 4, 5ߦ໨ͷbar͕ϋΠϥΠτ͞ΕΔ NODE_CLASS NODE_DEF bar = 0 NODE_DEF @foo = 1 bar += 1 bar = 0 @foo = 1 bar += 1

Slide 50

Slide 50 text

Πϯελϯεม਺ͷϋΠϥΠτ • 3ߦ໨ͷ@fooΛબΜͩ࣌ • 3, 9ߦ໨ͷ@foo͕ϋΠϥΠτ͞ΕΔ NODE_CLASS NODE_DEF bar = 0 NODE_DEF @foo = 1 bar += 1 bar = 0 @foo = 1 bar += 1

Slide 51

Slide 51 text

·ͱΊ: LSPͷϢʔεέʔε • ਌ϊʔυ͔Βࢠϊʔυɺࢠϊʔυ͔Β਌ϊʔυͷ྆ํ޲ʹḷΓ͍ͨ • τʔΫϯͷҐஔ৘ใΛऔಘ͍ͨ͠ • ίϝϯτͷ಺༰΍Ґஔ৘ใΛऔಘ͍ͨ͠ • ίʔυͷॻ͖׵͑͸ࢥͬͨҎ্ʹ೉͍͠ • ίʔυͷॻ͖׵͑ʹ͍ͭͯ͸͜ͷ͋ͱ࠶౓৮ΕΔ

Slide 52

Slide 52 text

Linterͷ৔߹ • Style::IfInsideElse ͱ͍͏CopΛΈͯΈΔ • ωετͨ͠ifͷ಺ଆʹ͋ΔifΛ֎ଆͷelsifʹ͢Δ https://github.com/rubocop/rubocop/blob/v1.65.1/lib/rubocop/cop/style/if_inside_else.rb#L10-L29

Slide 53

Slide 53 text

Style::IfInsideElse ͷ࣮૷ • parser gemͰ͸unless΍ࡾ߲ԋࢉࢠ΍ޙஔif΋if nodeʹͳΔ • “if” nodeΛݕࡧͨ͠͏͑Ͱࡉ͔͘νΣοΫͯ͠໨త֎ͷϊʔυΛ஄͘ • #3. AllowIfModifier͕trueͳΒωετͨ͠ޙஔifͷέʔεΛ஄͘ 1. “if” nodeΛݕࡧ͢Δ 2. ࡾ߲ԋࢉࢠͱunlessΛ஄͘ 3. ޙஔifΛ஄͘ https://github.com/rubocop/rubocop/blob/v1.65.1/lib/rubocop/cop/style/if_inside_else.rb

Slide 54

Slide 54 text

RuboCop::AST::Node • `#ternary?`΍`#modifier_form?`͸parser gemͷNodeʹ͸ੜ͑ͯͳ͍ • Parser::AST::Node Λܧঝͨ͠ RuboCop::AST::Node Λఆٛͯ͠ɺͦ͜ʹ helperϝιουΛ࣮૷͍ͯ͠Δ

Slide 55

Slide 55 text

Style::IfInsideElseͷέʔε • ݕࡧ͍ͨ͠ifϊʔυ • ී௨ͷif • ࡾ߲ԋࢉࢠ • unless • ޙஔif (֎ଆͷϊʔυ) • `else_branch`͕nilʹͳ͍ͬͯΔ • ifϊʔυΛࡉ͔͘෼͚ͨ΄͏͕ݕࡧʹศར ??

Slide 56

Slide 56 text

NestedTernaryOperator • ωετͨ͠ࡾ߲ԋࢉࢠΛࢦఠ͢ΔCop • ifϊʔυΛࡉ͔͘෼͚ͨํ͕ศརͳέʔε https://github.com/rubocop/rubocop/blob/v1.65.1/lib/rubocop/cop/style/nested_ternary_operator.rb 1. “if” nodeΛݕࡧ͢Δ 2. ࡾ߲ԋࢉࢠҎ֎Λ஄͘

Slide 57

Slide 57 text

LiteralAsCondition • ৚͕݅ࣜϦςϥϧͷ৔߹ʹࢦఠ͢ΔCop • ifϊʔυΛࡉ͔͘෼͚ͳ͍ํ͕ศརͳέʔε https://github.com/rubocop/rubocop/blob/v1.65.1/lib/rubocop/cop/lint/literal_as_condition.rb 1. “if” nodeΛݕࡧ͢Δ 2. Կ΋஄͔ͳ͍

Slide 58

Slide 58 text

·ͱΊ: LinterͷϢʔεέʔε • LinterͷϧʔϧΛ࣮૷͢Δͱ͖ʹ͸ɺ2ͭͷҟͳΔϊʔυͷ෼ྨ͕ඞཁ • Syntax͔ΒΈͨͱ͖ͷϊʔυͷछྨ • ifϊʔυΛࡉ͔͘෼͚Δ • Semantics͔ΒΈͨͱ͖ͷϊʔυͷछྨ • IfϊʔυΛࡉ͔͘෼͚ͳ͍ • Ϣʔεέʔε͕ଟ༷Ͱ໘ന͍Ͱ͢Ͷ Θ͕··͕͗͢·ͤΜ͔…?

Slide 59

Slide 59 text

Code Formatterͷ৔߹ • Style::IfInsideElse ͷ #autocorrect ΛΈͯΈΔ • RuboCop::Cop::Corrector < Parser::Source::TreeRewriter ʹରͯ͠ૢ࡞͢Δ • ॻ͖׵͑Δൣғ (range)ͱ৽͍͠ςΩετ (content)Λࢦఆ͢Δ • #replace(range, content) • #remove(range) • #insert_before(range, content) • #insert_after(range, content)

Slide 60

Slide 60 text

ίʔυॻ͖׵͑ͷ༷ࢠ • IfInsideElse#correct_to_elsif_from_if_inside_else_form ʹΑΔॻ͖׵͑ https://github.com/rubocop/rubocop/blob/v1.65.1/lib/rubocop/cop/style/if_inside_else.rb#L10-L29

Slide 61

Slide 61 text

ίʔυॻ͖׵͑ͷ༷ࢠ if condition_a action_a else if condition_b action_b else action_c end end if condition_a action_a elsif condition_b if condition_b action_b else action_c end end if condition_a action_a elsif condition_b action_b action_b else action_c end end if condition_a action_a elsif condition_b action_b action_b else action_c end if condition_a action_a elsif condition_b action_b else action_c end 1. elseΛelsif΁ 2. if condition_b Λ࡟আ 3. ༨෼ͳendΛ ࡟আ 4.ॏෳ͍ͯ͠Δaction_b Λ࡟আ

Slide 62

Slide 62 text

TreeRewriterͷ໰୊఺ #1 • ࣮૷͕ෳࡶ • TreeRewriter͕௚઀จࣈྻΛॻ͖׵͑ΔΘ͚Ͱ͸ͳ͍ • TreeRewriter::ActionͷΠϯελϯεΛͭͬͯ͘ɺ࠷ޙʹҰؾʹมߋΛՃ͑ Δ Action. :replace (2, 0)-(2, 4) “elsif condition_b” Action. :replace (3, 2)-(3, 16) “action_b” Action. :replace (7, 0)-(7, 6) “” Action. :replace (4, 0)-(4, 13) “”

Slide 63

Slide 63 text

ActionΛ༻͍Δཧ༝ #1 • จࣈྻΛ౎౓ॻ͖׵͑Δͱίετ͕ߴ͍͔Β • ͲͪΒͷέʔε΋elseҎ߱ͷจࣈྻΛҠಈ(ίϐʔ)͠ͳ͍ͱ͍͚ͳ͍ if condition_a action_a else action_b end if condition_a action_a action_b end elseΛ࡟আ if condition_a action_a else action_b end if condition_a action_a elsif action_b end elsifʹஔ׵

Slide 64

Slide 64 text

ͦ͏͍͑͹… • จࣈྻͷ෦෼ฤूͱ͍͑͹ςΩετΤσΟλ • ςΩετΤσΟλͱ͍͑͹…?

Slide 65

Slide 65 text

ActionΛ༻͍Δཧ༝ #2 • จࣈྻΛ௚઀ॻ͖׵͑ΔͱଞͷϊʔυʹӨڹ͢Δ͔Β if condition_a action_a else action_b end Parser::Source::Bu ff er if condition_a action_a action_b end Parser::Source::Bu ff er NODE_VCALL action_b Range (3, 2)-(3, 10) elseΛ࡟আ

Slide 66

Slide 66 text

TreeRewriterͷ໰୊఺ #2 • ॻ͖׵͑࣌ͷૢ࡞͕൥ࡶ • ֤ૢ࡞ͰࠓͲͷΑ͏ͳঢ়ଶ͔ཧղ͠ͳ͍ͱ͍͚ͳ͍ if condition_a action_a else if condition_b action_b else action_c end end if condition_a action_a elsif condition_b if condition_b action_b else action_c end end if condition_a action_a elsif condition_b action_b action_b else action_c end end if condition_a action_a elsif condition_b action_b action_b else action_c end if condition_a action_a elsif condition_b action_b else action_c end 1. elseΛelsif΁ 2. if condition_b Λ࡟আ 3. ༨෼ͳendΛ ࡟আ 4.ॏෳ͍ͯ͠Δaction_b Λ࡟আ

Slide 67

Slide 67 text

ߏจ໦Λॻ͖׵͑ͨ৔߹ • #autocorrectΛߏจ໦ͷॻ͖׵͑Ͱදݱ͢Δ • ߏจ໦ͷߏ଄Λੜ͔͍ͨ͠ • NODE_IFΛNODE_ELSIFʹͯ͠ɺNODE_ELSEΛ͚͢ NODE_IF condition_a action_a NODE_ELSE NODE_IF condition_b action_b NODE_ELSE action_c NODE_IF condition_a action_a NODE_ELSIF condition_b action_b action_c

Slide 68

Slide 68 text

ߏจ໦Λॻ͖׵͑ͨ৔߹ • CopΛϊʔυॻ͖׵͑Ͱ࣮૷͢Δ৔߹ͷίʔυ • ςΩετΛฤू͢ΔΑΓ΋ߏ଄తͰΘ͔Γ΍͍͢ͷͰ͸?

Slide 69

Slide 69 text

Extract Variable ࠶ߟ • ߏจ໦ϕʔεͷ৔߹ͷΞϧΰϦζϜ • "var = ࢦఆ͞Εͨൣғ" ͱ͍͏ίʔυΛҰͭखલͷࣜͱͯ͠௥Ճ͢Δ • ࢦఆ͞ΕͨൣғΛ "var" Ͱஔ͖׵͑Δ ❌ NODE_BLOCK NODE_OPCALL i + 1 NODE_BLOCK NODE_LVAR new_variable NODE_LASGN new_variable = i + 1 ⭕

Slide 70

Slide 70 text

ߏจ໦͔Βιʔείʔυ΁ • ͋ͱ͸ߏจ໦͔ΒιʔείʔυΛੜ੒͢Δ͚ͩ • ۭന΍վߦͳͲASTͰ͸͚͍ܽͯΔ৘ใ͕͋Δ NODE_IF condition_a action_a NODE_ELSIF condition_b action_b action_c

Slide 71

Slide 71 text

Ґஔ৘ใΛ༩͑Δ • NodeͷΠϯελϯε࡞੒࣌ʹRangeΛ౉͢ඞཁ͕͋Δ • ੜ੒͞ΕΔίʔυΛΠϝʔδ͠ͳ͕ΒҐஔ৘ใΛߟ͑ͳ͍ͱ͍͚ͳ͍ͷͰ ൥ࡶ if condition_a action_a else if condition_b action_b else action_c end end if condition_a action_a elsif condition_b action_b else action_c end

Slide 72

Slide 72 text

Ґஔ৘ใͷमਖ਼ • มߋͨ͠ߏจ໦Ҏ߱ͷશͯͷϊʔυͷҐஔ৘ใ͕มԽ͢Δ • ؔ৺ͷ͋ΔIF NODEͷܑఋཁૉʹ΋Өڹ͢Δ NODE_CLASS condition_a action_a NODE_ELSIF NODE_DEF NODE_IF NODE_DEF expr1 expr2 expr2 expr2 condition_b action_b action_c

Slide 73

Slide 73 text

Migration Tool • ίʔυͷॻ͖׵͑͸LSP΍Code Formatter͚ͩͷ࢓ࣄͰ͸ͳ͍ • TranspecͳͲͷmigration toolͰ΋ίʔυͷॻ͖׵͑Λߦ͏ඞཁ͕͋Δ http://yujinakayama.me/transpec/

Slide 74

Slide 74 text

·ͱΊ: ͞·͟·ͳϢʔεέʔε • ਌ϊʔυ͔Βࢠϊʔυɺࢠϊʔυ͔Β਌ϊʔυͷ྆ํ޲ʹḷΓ͍ͨ • τʔΫϯͷҐஔ৘ใΛऔಘ͍ͨ͠ • ίϝϯτͷ಺༰΍Ґஔ৘ใΛऔಘ͍ͨ͠ • ֤छϊʔυʹରͯ͠Syntax͔ΒΈͨͱ͖ͷϊʔυͷछྨͱɺSemantics͔ΒΈͨͱ͖ ͷϊʔυͷछྨͷ2ͭͷ৘ใΛ͍࣋ͨͤͨ • ϊʔυͷॻ͖׵͑Λߦ͏͜ͱͰɺίʔυͷॻ͖׵͑Λߦ͍͍ͨ • ϊʔυͷ͍࣋ͬͯΔ৘ใͷΈΛ༻͍ͯݩͷίʔυΛ׬શʹ෮ݩ͍ͨ͠ • ϊʔυͷॻ͖׵͑ʹΑͬͯ΄͔ͷϊʔυͷҐஔ৘ใΛߋ৽͢ΔΑ͏ͳࣄଶ͸ආ͚ ͍ͨ

Slide 75

Slide 75 text

Osaka RubyKaigi 04 ந৅ߏจ໦Λ ௒͑ͯ

Slide 76

Slide 76 text

Osaka RubyKaigi 04 ந৅ߏจ໦͔Β ۩৅ߏจ໦΁ - ߏจ໦ͷઃܭ -

Slide 77

Slide 77 text

۩৅ߏจ໦ͱ͸ • ۩৅ߏจ໦ (Concrete Syntax Tree: CST) • ׅހͳͲASTͰ͸ࣦΘΕͯ͠·͏৘ใ΋࢒ͨ͠ߏจ໦ • AST͕ҙຯ(Semantics)ʹ஫໨͍ͯ͠Δͷʹରͯ͠ɺCST͸ߏจ(Syntax)ʹ஫໨ ͍ͯ͠Δ • ಛ௃ 1.τʔΫϯΛද͢σʔλߏ଄Λಋೖ͢Δ 2.lexerͰམͱͯ͠͠·͏৘ใΛτʔΫϯʹඥ͚ͮΔ 3.ϊʔυ͕τʔΫϯΛ࣋ͭΑ͏ʹ͢Δ • ߏจ໦͔Β΋ͱͷίʔυΛ׬શʹ෮ݩͰ͖ΔΑ͏ʹ͢Δ

Slide 78

Slide 78 text

τʔΫϯ༻ͷσʔλߏ଄ • ߏจ໦͔ΒίʔυΛ෮ݩ͢ΔͨΊʹ͸τʔΫϯͷ৘ใ΋ඞཁ • τʔΫϯछผΛද͢`type`ͱτʔΫϯͷࣈ໘Λද͢`text`Λ΋ͬͨߏ଄ • ࠷దԽͷ༨஍͸͍Ζ͍Ζ͋Δ͕͜͜Ͱ͸γϯϓϧͳߏ଄ʹ͢Δ

Slide 79

Slide 79 text

Trivia • LexerͰམͱͯ͠͠·͏৘ใΛTriviaͱΑͿ • ۭന, վߦ, ίϝϯτͳͲ͕Trivia Trivia (comment) Trivia (spaces) Trivia (new line)

Slide 80

Slide 80 text

Trivia • Triviaͷࣈ໘Λද͢`text`Λ΋ͬͨߏ଄ • ͜Ε΋࠷దԽͷ༨஍͕͋Δ • Trivia͸τʔΫϯʹඥ͚ͮͯ؅ཧ͢Δ • τʔΫϯ͸ͦͷલޙʹTriviaΛ΋ͭ͜ͱ͕Ͱ͖Δ

Slide 81

Slide 81 text

ϊʔυͱτʔΫϯ • ϊʔυ͕τʔΫϯΛ࣋ͯΔΑ͏ʹ͢Δ • ͍ͭ͘τʔΫϯΛ͔࣋ͭ͸ϊʔυͷछྨʹΑΔ

Slide 82

Slide 82 text

Node, Token and Trivia NODE_IF IF cond action_a END Token NODE ຌྫ space (1) NL (1) + space (2) NL (1) Trivia • ϊʔυ͸ϊʔυ / τʔΫϯΛ΋ͭ • τʔΫϯ͸લޙʹTriviaΛ΋ͭ

Slide 83

Slide 83 text

ߏจ໦͔Βίʔυ΁ • ਂ͞༏ઌ୳ࡧͭͭ͠จࣈྻΛdump͢Δͱ΋ͱͷίʔυʹͳΔ Token NODE ຌྫ NODE_IF IF cond action_a END space (1) NL (1) + space (2) NL (1) Trivia

Slide 84

Slide 84 text

ߏจղੳͱͷ૬ੑ • Trivia (ۭന΍վߦ) ͸ੜ੒نଇͷ͋ΒΏΔͱ͜Ζʹग़ͯ͘Δ • TriviaΛτʔΫϯ (ऴ୺ه߸)ͱͯ͠ѻ͏ͷ͸൥ࡶ • ࠓճͷํ๏ͳΒߏจղੳ্͸طଘͷτʔΫϯͷଐੑͱͯ͠ѻ͑Δ TriviaΛτʔΫϯͱͯ͠ѻ͏৔߹ TriviaΛτʔΫϯͷଐੑͱͯ͠ѻ͏৔߹

Slide 85

Slide 85 text

Error Recovery • parser͕error recoveryΛ͢Δͱ͖ͷಈ࡞ • TokenΛແࢹ͢Δ • TokenΛ௥Ճ͢Δ • ௥Ճ͞ΕͨTokenΛ۩৅ߏจ໦ͰදݱͰ͖Δ NODE_IF IF true (a + b) END Token NODE ຌྫ Missing Token

Slide 86

Slide 86 text

Lexer, Parser and Trivia • ͍··Ͱ͸LexerͰམ͍ۭͪͯͨനͳͲͷ৘ใ͕Triviaͱͯ͠τʔΫϯʹඥ ͮ͘ • Parser͸tokenͱnodeΛ૊Έ߹Θͤͯߏจ໦Λͭ͘Δ Ruby Script Lexer Parser Token Trivia Syntax Tree Token Node

Slide 87

Slide 87 text

Triviaͷৄࡉ • “cond\n action_a”ΛΈͯΈΔͱվߦ(1)ͱۭന(2)͕͋Δ • ͜ΕΛcondͷtrailing_triviaʹ͢Δ͔ɺaction_aͷleading_triviaʹ͢Δ͔ if cond action_a end if cond action_a end if cond action_a end

Slide 88

Slide 88 text

Triviaͷϧʔϧ • ϧʔϧ1: τʔΫϯ͸ࣗ਎ͷ͋ͱʹଓ͘TriviaΛอ࣋͢Δɻ͜ͷͱ͖վߦจࣈ͸ؚ·ͳ ͍ • ϧʔϧ 2: 1Ͱॴଐͷܾ·Βͳ͍Trivia͸ͦͷ௚ޙͷτʔΫϯʹॴଐ͢Δ • Ifͷ͋ͱͷۭന͸ifʹଐ͢Δ (ϧʔϧ1) • action_a ͸௚લͷվߦͱ2ͭͷۭനΛ΋ͭ (ϧʔϧ1&2) • end͸௚લͷվߦΛ΋ͭ (ϧʔϧ1&2) if cond\n action_a\n end if cond\n action_a\n end if cond\n action_a\n end

Slide 89

Slide 89 text

ϝϦοτ1: ίϝϯτͷѻ͍ • ͜ͷϧʔϧʹै͏ͱίϝϯτ͕ظ଴ͨ͠τʔΫϯʹඥͮ͘ • ϝιουͷલͷϒϩοΫίϝϯτ͸defʹɺࣜͷ͋ͱͷίϝϯτ͸ࣜͷ຤ඌ ͷτʔΫϯʹඥͮ͘

Slide 90

Slide 90 text

ϝϦοτ2: ߦ಄ͷτʔΫϯ • leading_trivia͕վߦ͔Β࢝·Δ͔ΛνΣοΫ͢Δ͜ͱͰɺͦͷτʔΫϯ͕ ߦ಄ͷτʔΫϯ͔Ͳ͏͔൑ผͰ͖Δ • IndentΛܭࢉ͢Δͱ͖ʹߦ಄ͷτʔΫϯ͔Ͳ͏͔͕ॏཁ Leading (nl + sp + sp) Leading (nl)

Slide 91

Slide 91 text

·ͱΊ: ۩৅ߏจ໦ • ۩৅ߏจ໦ (Concrete Syntax Tree: CST) • ۭനͳͲΛtriviaͱͯ͠τʔΫϯʹඥ͚ͮΔ • ϊʔυ͸ϊʔυ͚ͩͰͳ͘τʔΫϯ΋࣋ͭ

Slide 92

Slide 92 text

·ͱΊ: ۩৅ߏจ໦ • ਌ϊʔυ͔Βࢠϊʔυɺࢠϊʔυ͔Β਌ϊʔυͷ྆ํ޲ʹḷΓ͍ͨ • τʔΫϯͷҐஔ৘ใΛऔಘ͍ͨ͠ • ίϝϯτͷ಺༰΍Ґஔ৘ใΛऔಘ͍ͨ͠ • ֤छϊʔυʹରͯ͠Syntax͔ΒΈͨͱ͖ͷϊʔυͷछྨͱɺSemantics͔ΒΈͨͱ͖ ͷϊʔυͷछྨͷ2ͭͷ৘ใΛ͍࣋ͨͤͨ • ϊʔυͷॻ͖׵͑Λߦ͏͜ͱͰɺίʔυͷॻ͖׵͑Λߦ͍͍ͨ • ϊʔυͷ͍࣋ͬͯΔ৘ใͷΈΛ༻͍ͯݩͷίʔυΛ׬શʹ෮ݩ͍ͨ͠ • ϊʔυͷॻ͖׵͑ʹΑͬͯ΄͔ͷϊʔυͷҐஔ৘ใΛߋ৽͢ΔΑ͏ͳࣄଶ͸ආ͚ ͍ͨ

Slide 93

Slide 93 text

Osaka RubyKaigi 04 Red Green Tree - ߏจ໦ͷ࣮૷ -

Slide 94

Slide 94 text

Red Green Treeͱ͸ • C# (Roslyn)ͷൃ໌ • Swift (SwiftSyntax)΍ rust-analyzer (LSP)Ͱ΋࢖ΘΕ͍ͯΔ • ߏจ໦ΛRed NodeͱGreen Nodeͱ͍͏2ͭͷσʔλߏ଄Ͱදݱ͢Δ • ͜ͷൃදͰ͸swift-syntaxͷ࣮૷Λओʹࢀߟʹ͍ͯ͠Δ • https://github.com/swiftlang/swift-syntax

Slide 95

Slide 95 text

ͳͥRedͱGreenͳͷ͔ • https://learn.microsoft.com/en-us/archive/blogs/ericlippert/persistence- facades-and-roslyns-red-green-trees Incidentally, these are called "red/green trees" because those were the whiteboard marker colours we used to draw the data structure in the design meeting. There's no other meaning to the colours. • σʔλߏ଄ʹ͍ͭͯٞ࿦͍ͯ͠Δ࣌ʹ࢖͍ͬͯͨϚʔΧʔ͕੺ͱ྘ͩͬ ͨɻͦΕҎ্ͷཧ༝͸ͳ͍ɻ

Slide 96

Slide 96 text

Green Node • ࢠϊʔυ(Green Node)͓ΑͼτʔΫϯ΁ͷࢀরΛ΋ͭ • ࣗ਎ͷ෯(width)Λ͍࣋ͬͯΔ • Offset(ϑΝΠϧઌ಄͔ΒͷҐஔ৘ใ)Λ࣋ͨͳ͍

Slide 97

Slide 97 text

Green Nodeͷߏ଄ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7

Slide 98

Slide 98 text

Green Nodeͷಛ௃(1) • ࢠϊʔυ(Green Node)͓ΑͼτʔΫϯ΁ͷࢀরΛ΋ͭ Token NODE ຌྫ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7

Slide 99

Slide 99 text

Green Nodeͷಛ௃(2) • ࣗ਎ͷ෯(width)Λ͍࣋ͬͯΔ • IF tokenͷ෯͕2͡Όͳ͍ͷ͸trivia͕͋Δ͔Β Green Node ຌྫ ௨ৗͷNode࣮૷ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 NODE_IF o ff set: 0 NODE_IF o ff set: 30 Token

Slide 100

Slide 100 text

OffsetΛ͍࣋ͬͯΔ৔߹ • มԽͷ͋ͬͨϊʔυ/τʔΫϯͷޙଓͷ͢΂ͯͷཁૉʹӨڹ͢Δ • action_bͷ຤ඌʹۭന͕ೖͬͨ࣌ ӨڹΛड͚Δཁૉ NODE_IF o ff set: 0 IF o ff set: 0 NODE_IF o ff set: 30 condition_a o ff set: 3 action_a o ff set: 14 NODE_ELSE o ff set: 25 END o ff set: 86 ELSE o ff set: 25 IF o ff set: 30 condition_b o ff set: 35 action_b o ff set: 46 -> 47 END o ff set: 79 NODE_ELSE o ff set: 59 action_c o ff set: 66 ELSE o ff set: 59

Slide 101

Slide 101 text

෯Λ͍࣋ͬͯΔ৔߹ • มԽͷ͋ͬͨϊʔυ/τʔΫϯͷ਌ཁૉʹӨڹ͕ݶΒΕΔ • ໦ͷߴ͞(log n)ͰࡁΉ ӨڹΛड͚Δཁૉ NODE_IF width: 90 -> 91 IF width: 3 NODE_IF width: 56 -> 57 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 -> 62 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 -> 14 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7

Slide 102

Slide 102 text

Red Node • ࢠϊʔυ(Green Node / τʔΫϯ)ͱ਌ϊʔυ(Red Node)΁ͷࢀরΛ΋ͭ • ϑΝΠϧͷઌ಄͔ΒͷҐஔ৘ใ(offset)Λ΋ͭ • Red Nodeͷߏங͸ඞཁʹԠͯ͡lazyʹߦ͏

Slide 103

Slide 103 text

Red Nodeͷߏ଄ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 NODE_IF o ff set: 30

Slide 104

Slide 104 text

Red Nodeͷಛ௃(1) • ࢠϊʔυ(Green Node)ͱ਌ϊʔυ(Red Node)΁ͷࢀরΛ΋ͭ • ࢠ͔Β਌΁ͷΞΫηε͕؆୯ Token Green NODE ຌྫ Red NODE NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 NODE_IF o ff set: 30

Slide 105

Slide 105 text

Red Nodeͷಛ௃(2) • ϑΝΠϧͷઌ಄͔ΒͷҐஔ৘ใ(offset)Λ΋ͭ Token Green NODE ຌྫ Red NODE NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 NODE_IF o ff set: 30

Slide 106

Slide 106 text

offsetͷܭࢉํ๏ • offset͸਌Red NodeͷoffsetͱࢠͷGreen Nodeͷwidth͔ΒܭࢉͰ͖Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 (3 + 11 + 11) NODE_IF o ff set: 30 25 + (5)

Slide 107

Slide 107 text

Red Nodeͷಛ௃(3) • Red Nodeͷߏங͸ඞཁʹԠͯ͡lazyʹߦ͏ • Green Node͸ߏจղੳͷ݁ՌͳͷͰৗʹ࡞Δ͕ɺRed Node͸ඞཁʹͳΔ ·Ͱੜ੒Λ஗ԆͰ͖Δ

Slide 108

Slide 108 text

Red/Green Nodeͷ࢖͍෼͚ • Green Node • ϊʔυʹہॴԽ͞Εͨ(ࣗ਎ͱࢠཁૉ͚ͩͰܭࢉͰ͖Δ)৘ใΛ࣋ͨͤΔ • Red Node • ͦ͏Ͱͳ͍৘ใΛ࣋ͨͤΔ • ྫ: offsetͱߦ൪߸ͷmap͸Red Nodeʹ࣋ͨͤΔ

Slide 109

Slide 109 text

Red Node & Green Node • Red Node΋Green Node΋immutableͰpersistentͳσʔλߏ଄ͱ͢Δ • ߏจ໦ͷϢʔβʔʹ͸Red NodeͷΈΛެ։͠ɺGreen Node͸internalͳσ ʔλߏ଄ͱ͢Δ

Slide 110

Slide 110 text

Immutable • Ұ౓࡞ͬͨNodeͷଐੑΛޙ͔Βߋ৽͠ͳ͍/Ͱ͖ͳ͍ • มߋΛՃ͍͑ͨͱ͖͸ίϐʔΛͭͬͯ͘Ұ෦ͷଐੑΛॻ͖׵͑Δ • େ෦෼ͷNode΍Token͸ڞ༗Ͱ͖Δ IF width: 3 END width: 4 NODE_IF width: 90 NODE_IF width: 90 IF width: 3 END width: 4 NODE_IF width: 91 IF width: 4 If ͷ͋ͱʹۭനΛ௥Ճ

Slide 111

Slide 111 text

Ӭଓσʔλߏ଄ (Persistent) • ߏจ໦ͷมߋͷલޙɺͲͪΒͷόʔδϣϯʹ΋ΞΫηεͰ͖Δ NODE_IF (ver.1) width: 90 IF width: 3 END width: 4 NODE_IF (ver.1) width: 90 IF width: 3 END width: 4 NODE_IF (ver.2) width: 91 IF width: 4

Slide 112

Slide 112 text

ύʔε݁Ռ͸Red Node • ιʔείʔυΛύʔεͨ͠ͱ͖͸rootͷRed Node͕ฦͬͯ͘Δ parse ͜Ε͕ฦͬͯ͘Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 NODE_IF o ff set: 0

Slide 113

Slide 113 text

ࢠཁૉ΋Red Node • ࢠϊʔυɺτʔΫϯʹΞΫηεͨ͠ͱ͖͸Red Nodeʹwrap͞Εͨཁૉ͕ ฦΔ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 NODE_IF o ff set: 0 TOKEN_IF o ff set: 0 NODE_ELSE o ff set: 25 node#if_token node#else_node

Slide 114

Slide 114 text

৽ن࡞੒࣌΋Red Node • NodeΛ৽͘͠࡞Δͱ͖͸Red NodeΛ࡞Δ • ৽نʹelsifͱelseͷτʔΫϯΛ࡞Δ͕ɺ͜Ε΋Red Node (Red Token) • ৽͘͠࡞ͬͨRed Node͸Ҿ਺ͷRed Node͔ΒGreen NodeΛऔΓग़ͯ͠ɺ ࣗ෼ͷGreen Nodeʹඥ͚ͮΔ # good if condition_a action_a elsif condition_b action_b else action_c end

Slide 115

Slide 115 text

Node࡞੒ͷཪଆ NODE_ELSE width: 61 NODE_IF width: 56 NODE_IF o ff set: 25 condition_b o ff set: 30 action_b o ff set: 41 action_c o ff set: 47 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7

Slide 116

Slide 116 text

Node࡞੒ͷཪଆ NODE_ELSE width: 61 NODE_IF width: 56 NODE_IF o ff set: 25 condition_b o ff set: 30 action_b o ff set: 41 action_c o ff set: 47 ELSIF width: 7 ELSE width: 5 ELSIF o ff set: 0 ELSE o ff set: 0 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 Tokenͷੜ੒

Slide 117

Slide 117 text

Node࡞੒ͷཪଆ NODE_ELSE width: 61 NODE_IF width: 56 NODE_IF o ff set: 25 condition_b o ff set: 30 action_b o ff set: 41 action_c o ff set: 47 ELSIF width: 7 ELSE width: 5 ELSIF o ff set: 0 ELSE o ff set: 0 ຌྫ NODE_ELSIFͷҾ਺ NODE_ELSIF o ff set: 0 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ৽͍͠Node

Slide 118

Slide 118 text

Node࡞੒ͷཪଆ NODE_ELSE width: 61 NODE_IF width: 56 NODE_IF o ff set: 25 condition_b o ff set: 30 action_b o ff set: 41 action_c o ff set: 47 NODE_ELSIF width: 49 ELSIF width: 7 ELSE width: 5 ELSIF o ff set: 0 ELSE o ff set: 0 ຌྫ NODE_ELSIFͷҾ਺ NODE_ELSIF o ff set: 0 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ৽͍͠Node width: 11 + 13 + 13 + 7 + 5 = 49

Slide 119

Slide 119 text

ߏจ໦ͷҰ෦Λॻ͖׵͑Δ • ֎ଆͷifͷelseΛ࡞ͬͨelsif nodeͰஔ͖׵͑Δ # bad if condition_a action_a else if condition_b action_b else action_c end end

Slide 120

Slide 120 text

ॻ͖׵͑ͷཪଆ NODE_IF width: 90 IF width: 3 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 • ॻ͖׵͑લͷঢ়ଶ

Slide 121

Slide 121 text

ॻ͖׵͑ͷཪଆ NODE_IF width: 90 IF width: 3 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 NODE_ELSIF width: 49 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 NODE_ELSIF o ff set: 0 • ઌ΄Ͳͭͬͨ͘NODE_ELSIFΛ΋ͬͯ͘Δ

Slide 122

Slide 122 text

ॻ͖׵͑ͷཪଆ NODE_IF width: 90 IF width: 3 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 NODE_ELSIF width: 49 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 NODE_IF o ff set: 0 NODE_IF width: 78 NODE_ELSIF o ff set: 0 • ৽͘͠NODE_IF (Red Node & Green Node)Λ ͭ͘Δ

Slide 123

Slide 123 text

ਂ͍஍఺ͷॻ͖׵͑ • ྫ: action_c ͷtrivia͕มΘͬͨͱ͖ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 if condition_a action_a else if condition_b action_b else action_c ; end end ߋ৽

Slide 124

Slide 124 text

ਂ͍஍఺ͷॻ͖׵͑ • ߋ৽લͷߏจ໦ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ߋ৽ action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF

Slide 125

Slide 125 text

ਂ͍஍఺ͷॻ͖׵͑ • ৽͘͠action_cͷnodeΛͭ͘Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ߋ৽ action_c width: 15 action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF

Slide 126

Slide 126 text

ਂ͍஍఺ͷॻ͖׵͑ • action_cͷRed NodeΛͭ͘Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ߋ৽ action_c width: 15 action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF action_c

Slide 127

Slide 127 text

ਂ͍஍఺ͷॻ͖׵͑ • ࢠཁૉ΋෯΋มΘΔͷͰGreen NodeΛͭ͘Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ߋ৽ action_c width: 15 action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF NODE_ELSE width: 22 action_c

Slide 128

Slide 128 text

ਂ͍஍఺ͷॻ͖׵͑ • NODE_ELSEͷRed NodeΛͭ͘Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ߋ৽ action_c width: 15 action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF NODE_ELSE width: 22 action_c NODE_ELSE

Slide 129

Slide 129 text

ਂ͍஍఺ͷॻ͖׵͑ • ಉ༷ʹͯ͠root·ͰͷશͯͷRed NodeΛ࡞Γ௚͢ඞཁ͕͋Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ߋ৽ action_c width: 15 action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF NODE_ELSE width: 22 NODE_IF width: 58 NODE_ELSE width: 63 NODE_IF width: 92 action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF

Slide 130

Slide 130 text

Rewriterͷඞཁੑ • ߏจ໦ʹରͯ͠਺ճ͔͠ॻ͖׵͑ΛߦΘͳ͍ͳΒ͜ͷίετ͸ແࢹͰ͖Δ • RuboCopͳͲͰ͸ߏจ໦ͷ͋ͪͪ͜Λॻ͖׵͑Δ͜ͱʹͳΔ • ౎౓root node·ͰͷRed NodeΛ࡞Δͷ͸ແବ • ޮ཰తͳॻ͖׵͑Λࢧԉ͢Δػೳ͕ඞཁ • RewriterΛ࣮૷͢Δ

Slide 131

Slide 131 text

Rewriterͷ࣮૷ • ֤ϊʔυʹରԠͨ͠ϝιουΛఆٛ͢Δ (#on_xxx) • ࢠཁૉΛϝιουʹ౉ͯ͠ɺҰͭͰ΋มԽ͍ͯͨ͠Βࣗ਎Λ࠶࡞੒͢Δ • ࡞Γ௚͢৔߹͸root nodeͱͯ͠࡞Γ௚͢

Slide 132

Slide 132 text

Rewriterͷܧঝ • ؔ৺ͷ͋ΔϊʔυʹରԠͨ͠ϝιουΛ্ॻ͖͢Δ • ඞཁʹԠͯ͡৽͍͠ϊʔυΛroot nodeͱͯ͠࡞ͬͯฦ͢

Slide 133

Slide 133 text

Rewriterͷ࢓૊Έ • ਂ͞༏ઌͰϊʔυΛḷͬͯɺมԽ͕͋Ε͹ؼΓ͕͚ʹϊʔυΛ࠶ߏங͢Δ NODE_CLASS ver. 1 NODE_DEF ver. 1 NODE_DEF ver. 1 NODE_IF ver. 1 NODE_CALL ver. 1 NODE_CASE ver. 1 ߋ৽ NODE_CLASS ver. 1 NODE_DEF ver. 2 NODE_DEF ver. 1 NODE_IF ver. 2 NODE_CALL ver. 1 NODE_CASE ver. 2 ߋ৽ ৽ن ৽ن ࡞Γ௚͢લʹӈͷDEF΁͍͘

Slide 134

Slide 134 text

Rewriterͷ࢓૊Έ • 1ϊʔυʹରͯ͠࠷େ1૊ͷϊʔυΛ࡞Δ͚ͩͰࡁΉ • ͦΕͧΕ͸root nodeͱͯ͠ϊʔυΛ࡞ΔͷͰ਌ͷߋ৽Λ͠ͳͯ͘ࡁΉ NODE_CLASS ver. 1 NODE_DEF ver. 2 NODE_DEF ver. 1 NODE_IF ver. 2 NODE_CALL ver. 1 NODE_CASE ver. 2 ߋ৽ NODE_CLASS ver. 1 NODE_DEF ver. 2 NODE_DEF ver. 2 NODE_IF ver. 2 NODE_CALL ver. 2 NODE_CASE ver. 2 NODE_CLASS ver. 2 NODE_DEF ver. 2 NODE_DEF ver. 2 NODE_IF ver. 2 NODE_CALL ver. 2 NODE_CASE ver. 2 ৽ن ৽ن

Slide 135

Slide 135 text

Rewriter ଞݴޠͰͷ࣮૷ • Rewriter͸൚༻తͳ࣮૷ͳͷͰίΞʹ͍ۙͱ͜ΖͰఏڙ͢Δͷ͕Α͍ͷͰ ͸ͳ͍͔ • Roslyn: CSharpSyntaxRewriter • https://github.com/dotnet/roslyn/blob/v4.2.0/src/Compilers/CSharp/ Portable/Syntax/CSharpSyntaxRewriter.cs • SwiftSyntax: SyntaxRewriter • https://github.com/swiftlang/swift-syntax/blob/600.0.0- prerelease-2024-07-30/Sources/SwiftSyntax/generated/ SyntaxRewriter.swift

Slide 136

Slide 136 text

Ӭଓσʔλߏ଄ͷརศੑ • ݩͷ໦ɺॻ͖׵͑ޙͷ෦෼໦ɺॻ͖׵͑ޙͷશମͷ໦͕ͦΕͧΕ҆શʹख ʹೖΔ

Slide 137

Slide 137 text

Ӭଓσʔλߏ଄ͷརศੑ • Code FormatterͷRule (Cop)͸ߏจ໦Λड͚औͬͯɺ৽͍͠෦෼໦Λੜ੒ ͯ͠ɺߏจ໦શମΛߋ৽͢Δ • Rule͸ߋ৽લޙͷ໦Λ΋ͯΔ • มߋલޙͷίʔυΛදࣔͰ͖Δ IfInsideElse if condition_a action_a else if condition_b action_b else action_c end end if condition_a action_a elsif condition_b action_b else action_c end NODE_IF width: 78 NODE_ELSE width: 49 NODE_IF o ff set: 0 NODE_ELSIF o ff set: 0 NODE_IF width: 90 NODE_ELSE width: 61 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25

Slide 138

Slide 138 text

·ͱΊ: Red Green Tree • ߏจ໦ΛRed Tree/NodeͱGreen Tree/Nodeͱ͍͏2ͭͷσʔλߏ଄Ͱද͢ • Green Node͸ࣗ਎ͱࢠཁૉʹดͨ͡৘ใΛ΋ͭ • Red Node͸ͦΕҎ֎ͷ৘ใΛ΋ͭ • Red Node΋Green Node΋immutableͰpersistentͳσʔλߏ଄ • ߏจ໦ͷϢʔβʔʹ͸Red NodeͷΈΛެ։͠ɺGreen Node͸internalͳσ ʔλߏ଄ͱ͢Δ

Slide 139

Slide 139 text

·ͱΊ: Red Green Tree • ਌ϊʔυ͔Βࢠϊʔυɺࢠϊʔυ͔Β਌ϊʔυͷ྆ํ޲ʹḷΓ͍ͨ • τʔΫϯͷҐஔ৘ใΛऔಘ͍ͨ͠ • ίϝϯτͷ಺༰΍Ґஔ৘ใΛऔಘ͍ͨ͠ • ֤छϊʔυʹରͯ͠Syntax͔ΒΈͨͱ͖ͷϊʔυͷछྨͱɺSemantics͔ΒΈͨͱ͖ ͷϊʔυͷछྨͷ2ͭͷ৘ใΛ͍࣋ͨͤͨ • ϊʔυͷॻ͖׵͑Λߦ͏͜ͱͰɺίʔυͷॻ͖׵͑Λߦ͍͍ͨ • ϊʔυͷ͍࣋ͬͯΔ৘ใͷΈΛ༻͍ͯݩͷίʔυΛ׬શʹ෮ݩ͍ͨ͠ • ϊʔυͷॻ͖׵͑ʹΑͬͯ΄͔ͷϊʔυͷҐஔ৘ใΛߋ৽͢ΔΑ͏ͳࣄଶ͸ආ͚ͨ ͍

Slide 140

Slide 140 text

Osaka RubyKaigi 04 ۩৅ߏจ໦ͱ ந৅ߏจ໦ͷཱ྆

Slide 141

Slide 141 text

ϊʔυʹ྆ํͷ৘ใΛ΋ͨͤΔ • Green NodeʹSyntaxͱSemanticsͷ2ͭͷtypeΛ࣋ͨͤΔ • Semanticsʹ஫໨ͯ͠ΞΫηε͍ͨ͠ͱ͖ͷͨΊʹhelperؔ਺Λఆٛ͢Δ

Slide 142

Slide 142 text

ϊʔυͷߏ଄ • enum node_syntax_type • ϊʔυ͝ͱʹҟͳΔ • enum node_semantics_type • semantics͕ಉ͡ͳΒಉ͡ (ྫ: NODE_SEMANTICS_IF)

Slide 143

Slide 143 text

semantics nodeͱͯ͠ΞΫηε͢Δ • if nodeͷछผʹΑΒͣ৚݅۟ (condition) ͕΄͍͠ • node_syntax_typeͰ෼ذͨ͠ίʔυΛॻ͍ͯɺؔ਺ͱͯ͠ఏڙ͢Δ

Slide 144

Slide 144 text

Slide 145

Slide 145 text

Osaka RubyKaigi 04 Rubyʹ࣮૷͢Δʹ͋ͨͬͯ Open Problems

Slide 146

Slide 146 text

Կ͕trivia͔ • Rubyʹ͓͍ͯ͸վߦ͸τʔΫϯͱͯ͠ѻΘΕΔέʔεͱlexerͰམͱ͞ΕΔ έʔε (tIGNORED_NL) ͕͋Δ • ํ๏1: τʔΫϯʹͳͬͨվߦ͸τʔΫϯɺͦ͏Ͱͳ͍΋ͷ͸triviaʹ͢Δ • ํ๏2: શͯͷվߦΛtriviaʹ͢Δ • ηϛίϩϯ͸trivialʹͨ͠ํ͕ѻ͍΍͍͢ͷͰ͸ͳ͍͔ʁ • ()͸ϊʔυͷ΄͏͕͍͍ͷ͔ʁ

Slide 147

Slide 147 text

ώΞυΩϡϝϯτ • ώΞυΩϡϝϯτ͸1ߦʹෳ਺ݸॻ͘͜ͱ͕Ͱ͖Δ • ͜ͷΑ͏ͳέʔεͰ΋ݩͷίʔυ͕෮ݩͰ͖ΔΑ͏ͳϊʔυʹ͠ͳ͍ͱ͍ ͚ͳ͍ • Green Tree͸۪௚ʹͭͬͯ͘ɺRed Treeͷoffset΍ςΩετʹ໭͢ͱ͖ʹ޻ ෉͢Δͷ͕Α͍ͷͰ͸ͳ͍͔ puts <<~STR1 + <<~STR2 111 STR1 aaa STR2 # => # 111 # aaa NODE_HEREDOC width: 90 <<~ width: 3 111 width: 4 STR1 width: 4 STR1 width: 5

Slide 148

Slide 148 text

Lexer interface • ͍··Ͱ͸τʔΫϯΛฦ͢ΠϯλʔϑΣΠεͱϊʔυΛฦ͢ΠϯλʔϑΣ Πε͕ผͩͬͨ • ۩৅ߏจ໦ʹͳͬͯϊʔυ͕τʔΫϯΛؚΉΑ͏ʹͳͬͨ͋ͱͰ΋͜ͷೋ ͭͷΠϯλʔϑΣΠε͸ผͷ΄͏͕͍͍ͷ͔ʁ • irbʹ͓͚Δ৭෇͚΍minifierͷ࣮૷ʹ͓͍ͯ͸ͲͪΒͷํ͕ศརͳͷ͔ʁ • ϊʔυͷ΋ͭߏ଄ʹؔ͢Δ৘ใ͕ඞཁͳͷ͔ʁ

Slide 149

Slide 149 text

Osaka RubyKaigi 04 ·ͱΊ

Slide 150

Slide 150 text

ߏจ໦ઃܭͷݪཧݪଇΛٻΊͯ • ߏจ໦͸ͦͷ౎౓ΞυϗοΫʹ֦ு͞Ε͖ͯͨΑ͏ʹݟ͑Δ • ͦͷઃܭʹݪཧݪଇ͸ͳ͍ͷͩΖ͏͔ʁ • ·ͣϢʔεέʔεΛௐ΂Δ • ίʔυΛղੳ͠Α͏ͱ͢Δͱίʔυͷ࣮ߦʹൺ΂ͯΑΓଟ͘ͷ৘ใ͕ඞ ཁʹͳΔ • ίʔυͷॻ͖׵͑ΛςΩετϕʔεͰߦ͏ͷ͸೉͍͠ • طଘͷߏจ໦࣮૷Ͱ͸ߏจ໦ͷॻ͖׵͕͑େม

Slide 151

Slide 151 text

ߏจ໦΁ͷཁ๬ • ਌ϊʔυ͔Βࢠϊʔυɺࢠϊʔυ͔Β਌ϊʔυͷ྆ํ޲ʹḷΓ͍ͨ • τʔΫϯͷҐஔ৘ใΛऔಘ͍ͨ͠ • ίϝϯτͷ಺༰΍Ґஔ৘ใΛऔಘ͍ͨ͠ • ֤छϊʔυʹରͯ͠Syntax͔ΒΈͨͱ͖ͷϊʔυͷछྨͱɺSemantics͔ΒΈͨͱ͖ ͷϊʔυͷछྨͷ2ͭͷ৘ใΛ͍࣋ͨͤͨ • ϊʔυͷॻ͖׵͑Λߦ͏͜ͱͰɺίʔυͷॻ͖׵͑Λߦ͍͍ͨ • ϊʔυͷ͍࣋ͬͯΔ৘ใͷΈΛ༻͍ͯݩͷίʔυΛ׬શʹ෮ݩ͍ͨ͠ • ϊʔυͷॻ͖׵͑ʹΑͬͯ΄͔ͷϊʔυͷҐஔ৘ใΛߋ৽͢ΔΑ͏ͳࣄଶ͸ආ͚ ͍ͨ

Slide 152

Slide 152 text

ߏจ໦ͷઃܭͱ࣮૷ • ߏจ໦ͷઃܭ: • ۩৅ߏจ໦ • ߏจ໦ͷ࣮૷: • Red Green Tree • ந৅ߏจͷ৘ใΛ΋ͬͨ۩৅ߏจ໦ • ߏจ໦΁ͷཁ๬Λୡ੒Ͱ͖ͨʂ • ࠓޙ΍Δ͜ͱ • Rubyݻ༗ͷ໰୊Λղ͘ • ࣮૷Λ͢Δ

Slide 153

Slide 153 text

Osaka RubyKaigi 04 Thank You !!!