最高の構文木の設計 2024年版

Osaka RubyKaigi 04
https://regional.rubykaigi.org/osaka04/

Transcript

1. Osaka RubyKaigi 04 2024/08/24 in Osaka RubyKaigi 04 @yui-knk Yuichiro Kaneko ࠷ߴͷߏจ໦ͷઃܭ 2024೥൛
2. Osaka RubyKaigi 04 ࣗݾ঺հ
3. About me • Yuichiro Kaneko • yui-knk (GitHub) / spikeolaf (Twitter) • Treasure Data • Engineering Manager of Applications Backend
4. TD and Ruby committers twitter: @nalsh GitHub: @nurse twitter: @k_tsj GitHub: @k-tsj twitter: @ spikeolaf GitHub: @yui-knk twitter: @mineroaoki GitHub: @aamine twitter: @nahi GitHub: @nahi Applications Backend
5. About me • CRuby committer, mainly develop parser generator and parser • Lrama LALR (1) parser generator (2023, Ruby 3.3) • The Bison Slayer • The parser monster • Parserքͷᴈ໌ᷭ • Ripper Rearchitecture (2024, Ruby 3.4) • Code positions to RNode (2018, Ruby 2.6) • RubyVM::AbstractSyntaxTree (2018, Ruby 2.6)
6. Osaka RubyKaigi 04 ࢲͱେࡕ
8. https://rubykansai.doorkeeper.jp/events/8078/participants https://shinosakarb.doorkeeper.jp/events/8604/participants ؔ੢ͷRuby ίϛϡχςΟ
9. ߏจ໦ʹਅཧ͸͋Δͷ͔ • Rubyͷparserͷ࣮૷͸ෳ਺͋Δ • ͦΕͧΕͷߏจ໦͸ඍົʹҟͳΔ • Ϣʔεέʔεʹ߹ΘͤͯΞυϗοΫʹ֦ு͍ͯ͠ΔΑ͏ʹࢥ͑Δ • Ͳ͏ਐΊΔ͔? • ϢʔεέʔεΛूΊΔ • ઃܭΛߟ͑Δ • ઃܭ͕ϢʔεέʔεΛͲͷ͘Β͍ΧόʔͰ͖͍ͯΔ͔ධՁ͢Δ
10. Osaka RubyKaigi 04 ߏจ໦ͷ࢖ΘΕํ ͷมભ
11. Rubyͷίʔυ͕࣮ߦ͞ΕΔ·Ͱ • ೖྗ͞ΕͨRuby Script͸ParserʹΑͬͯந৅ߏจ໦(Abstract Syntax Tree: AST)ʹม׵͞ΕΔ • Compiler͸ASTΛByte Codeʹม׵͢Δ • Virtual Machine͕Byte CodeΛ࣮ߦ͢Δ
12. ݪ࢝తͳݴޠ -ి୎- • ి୎ͷ࣮૷Ͱ͸ɺߏจղੳ͠ͳ͕Β࣮ࡍͷܭࢉΛߦ͏͜ͱ΋͋Δ • ྫ: expr '+' term ͷΞΫγϣϯͰ࣮ࡍʹ਺஋ͷՃࢉΛ͓͜ͳ͍ͬͯΔ
13. LR parser 101 • https://github.com/yui-knk/lr-parser-101
14. ݱ࣮͸ෳࡶ -ϝιου- • ྫ͑͹ϝιου • 5ߦ໨ͷ`m`ͷݺͼग़࣌͠ʹ͸ϝιου`m`ͷఆٛ͸طʹύʔεࡁΈ • Ұճ໨ͷύʔε࣌ʹ͸จࣈྻͱͯ͠`m`ͷఆٛΛ͓͍࣋ͬͯͯɺݺͼग़͠ ࣌ʹ࠶౓ύʔεͯ͠ܭࢉ͢Δ!? (ͨ͘͠ͳ͍)
15. ݱ࣮͸ෳࡶ -ܕγεςϜ- • ྫ͑͹ܕγεςϜ • `{…}`ͷ෦෼ʹܕγεςϜͷ࣮૷ΛೖΕΔ!? (͍Εͨ͘ͳ͍)
16. ߏจ໦ͷಋೖ • ύʔεͱͦͷଞͷܭࢉΛ෼཭͢ΔͨΊʹதؒͷσʔλߏ଄Λಋೖ͢Δ • Parser͸ߏจ໦ͷ࡞੒ʹूத͢Δ͜ͱ͕Ͱ͖Δ
17. ߏจ໦ͷಋೖ • ߏจ໦ͱ͸ϓϩάϥϜͷจ๏ߏ଄Λ໦ߏ଄ͱͯ͠දͨ͠΋ͷ Class Method Method Assignment @name Call name capitalize
18. Osaka RubyKaigi 04 ίʔυΛ࣮ߦ͢ΔͨΊ ͷந৅ߏจ໦
19. ͞·͟·ͳ if • RubyͰ͸ifʹ͍͍͔ͭͯͭ͘ͷҟͳΔॻ͖ํ͕͋Δ • ී௨ͷif • ޙஔif • ࡾ߲ԋࢉࢠ
20. if Λ࣮ߦͯ͠ΈΔ • ͜ͷ3ͭͷϓϩάϥϜΛ࣮ߦ͢Δ͜ͱΛߟ͑Δ • ৚݅ࣜ (`a == 1`) Λ࣮ߦ͢Δ • ࣮ߦ݁Ռ͕truthyͳΒ :t Λදࣔ͢Δ • ࣮ߦ݁Ռ͕falsyͳΒ :f Λදࣔ͢Δ • ஫: ޙஔifͰ͸else۟͸ଘࡏ͠ͳ͍ • ࣮ߦͷ࢓ํ͸ॻ͖ํʹґଘ͠ͳ͍
21. • ී௨ͷifͱࡾ߲ԋࢉࢠͷif͸શ͘ಉ͡໦ߏ଄ͰදݱͰ͖Δ 3ͭͷ if ͱ1ͭͷදݱ NODE_IF NODE_OPCALL "a == 1" NODE_FCALL "p :t" NODE_FCALL "p :f” cond body else
22. • ޙஔif͸else͕ৗʹnullʹͳΔ͕ɺ͜Ε΋ଞͷifͱಉ͡໦ߏ଄ͰදݱͰ͖Δ 3ͭͷ if ͱ1ͭͷදݱ NODE_IF NODE_OPCALL "a == 1" NODE_FCALL "p :t" null cond body else
23. if ͷٯ - unless - • bodyͱelseΛೖΕସ͑Δ͜ͱͰunless΋ಉ͡໦ߏ଄ͰදݱͰ͖Δ NODE_IF NODE_OPCALL "a == 1" NODE_FCALL "p :f” NODE_FCALL "p :t” cond body else NODE_UNLESS NODE_OPCALL "a == 1" NODE_FCALL "p :t" NODE_FCALL "p :f” cond body else
24. ͔ͭͯNODE_UNLESS͸ͳ͔ͬͨ • ࣮͸RubyʹNODE_UNLESS͕ಋೖ͞Εͨͷ͸Ruby 2.5͔Β https://github.com/ruby/ruby/commit/3c8c17d30defb41399eed65f2cf0f30e5469fa07
25. ࣮ߦͷͨΊͷ"ந৅"ߏจ໦ • ίʔυΛ࣮ߦ͢Δ͜ͱʹ஫໨͢Δͱҙຯ͕ಉ͡Ͱจ๏্ͷදݱ͕ҟͳΔཁ ૉΛ1ͭͷϊʔυͰදͤΔ • ந৅ߏจ໦ͱΑͿ • ந৅Խ͢ΔͱམͪΔཁૉ • ΋ͱͷίʔυͰͷSyntax (if͔ޙஔif͔ࡾ߲ԋࢉࢠ͔(unless͔)) • ΧοίͳͲߏจ໦Ͱදݱ͞Ε͍ͯΔ΋ͷ • ۭനɺվߦͳͲ
26. ந৅ߏจ໦ͷา͖ํ • ͲͷΑ͏ʹߏจ໦ΛḷΔ͔΋ͻͱͭͷϢʔεέʔε nd_args NODE_CALL mid: :m NODE_VCALL nd_mid: :obj NODE_INTEGER val: 1 NODE_LIST nd_recv NODE_INTEGER val: 2
27. ந৅ߏจ໦ͷา͖ํ • ߏจ໦Λ࣮ߦ͢Δ͍͞ʹ͸্(਌)͔ΒԼ(ࢠ)΁ḷΔ (2) nd_args NODE_CALL mid: :m NODE_VCALL nd_mid: :obj NODE_INTEGER val: 1 NODE_LIST (1) nd_recv NODE_INTEGER val: 2 (1) ϨγʔόʔͷίϯύΠϧ (2) Ҿ਺ͷίϯύΠϧ (3) ϝιου m ͷݺͼग़͠ (3)
28. ·ͱΊ • ਌ϊʔυ͔ΒࢠϊʔυΛḷΓ͍ͨ • ifͷΑ͏ʹॻ͖ํ͕ෳ਺͋Δ৔߹Ͱ΋ɺͦͷҙຯʹ஫໨ͯ͠1ͭͷϊʔυʹ ·ͱΊͨ΄͏͕ѻ͍΍͍͢ • τʔΫϯ΍ίϝϯτͷҐஔ৘ใɺ಺༰ͳͲ͸ѻΘͳ͍ • దͨ͠σʔλߏ଄ͱͯ͠ந৅ߏจ໦͕͋Δ
29. Osaka RubyKaigi 04 ίʔυΛղੳ͢ΔͨΊ ͷߏจ໦
30. ίʔυղੳͷ۩ମྫ • LSP • ruby-lsp gem (Shopify/ruby-lsp) • Linter • RuboCop gem (rubocop/rubocop) • Code Formatter • RuboCop gem (rubocop/rubocop)
31. LSPͷ৔߹ • τʔΫϯΛղੳ͢Δ • ίϝϯτΛղੳ͢Δ • ίʔυΛॻ͖׵͑Δ • ߏจ໦ͷḷΓํ
32. ͬ͘͟ΓLSP • ΫϥΠΞϯτ (Language Client) • VS CodeͳͲ • αʔόʔ (Language Server) • ruby-lspͳͲ
33. τʔΫϯΛղੳ͢Δ • Semantic Tokens • def, foo, + ͳͲʹ৭Λ͚ͭΔػೳ • αʔόʔ͸֤τʔΫϯͷҐஔ৘ใͱछྨΛฦ͢ • ਖ਼֬ʹ͸Τϯίʔυͨ͠৘ใΛฦ͢ • ֤τʔΫϯͷҐஔ৘ใ΍छผΛऔಘ͍ͨ͠
34. ίϝϯτΛղੳ͢Δ • DocumentLink • ίϝϯτͷҰ෦Λlinkʹ͢Δػೳ • αʔόʔ͸linkʹ͢ΔൣғͱURIΛฦ͢ • ίϝϯτͷ಺༰ͱҐஔ৘ใΛऔಘ͍ͨ͠ • Ϋϥεఆٛ΍ϝιουఆٛͷίϝϯτ͚ͩΛର৅ʹ͢ΔͳΒɺίϝϯτͱ ϊʔυΛඥ͚ͮΔඞཁ͕͋Δ
35. ίʔυΛॻ͖׵͑Δ • CodeActionResolve • “Extract Variable”ͳͲͷϦϑΝΫλϦϯάػೳ • ίʔυΛબ୒ͯ͠ఏҊ͞ΕͨΞΫγϣϯΛ࣮ߦ
36. ίʔυΛॻ͖׵͑Δ • ࣮ࡍͷίʔυͷॻ͖׵͑͸ΫϥΠΞϯτ͕ߦ͏ • αʔόʔ͸ॻ͖׵͑Δൣғ(range)ͱॻ͖׵͑ޙͷςΩετ(newText)Λฦ ͢
37. ίʔυΛॻ͖׵͑Δ • ࢦఆ͞Εͨൣғ͔ΒϊʔυΛಛఆ͢Δ • ϊʔυ΋͘͠͸ࢦఆ͞ΕͨൣғʹରԠ͢ΔίʔυΛऔಘ͢Δ • ίϝϯτ΍ۭനͳͲ΋ѻ͏ඞཁ͕͋Δ • ৽͍͠ίʔυΛੜ੒͢Δ 1 + 1 1 1 NODE_OPCALL (0, 0)-(0, 5) new_variable = 1 + 1 new_variable
38. ॻ͖׵͑ͷ೉͠͞ • “Extract Variable”ͷΞϧΰϦζϜΛߟ͑ͯΈΔ • "var = ࢦఆ͞Εͨൣғ" ͱ͍͏ίʔυΛҰ্ͭͷߦʹ௥Ճ͢Δ • ࢦఆ͞ΕͨൣғΛ "var" Ͱஔ͖׵͑Δ 1 + 1 var = 1 + 1 var var = 1 + 1 Λ1ߦ্ʹ௥Ճ 1 + 1 Λ var Ͱஔ͖׵͑Δ
39. ॻ͖׵͑ͷ೉͠͞ • ϫϯϥΠφʔͷϒϩοΫ಺෦ͷϦϑΝΫλϦϯάͰ͸1ߦ্ʹม਺Λఆٛ͠ ͯ΄͍͠Θ͚Ͱ͸ͳ͍ • "i + 1" ͱ͍͏ࣜͷલʹม਺Λఆٛͯ͠΄͍͠ • ϊʔυͷछྨ΍Ґஔ৘ใ͕ඞཁ 10.times {|i| i + 1 } var = i + 1 10.times {|i| var } i + 1 Λ var Ͱஔ͖׵͑Δ var = i + 1 Λ1ߦ্ʹ௥Ճ 10.times {|i| i + 1 } 10.times {|i| var = i + 1; var } i + 1 Λ var Ͱஔ͖׵͑Δ var = i + 1 Λ௚લʹ௥Ճ ⭕ ❌
40. ॻ͖׵͑ͷ೉͠͞ • ϫϯϥΠφʔͩͬͨΒͱ͍͏ͷ΋࣮͸ۂऀ • ൑ఆ͢΂͖͸ϒϩοΫͷ։࢝ߦͱબ୒ൣғͷߦ͕ಉ͔͡Ͳ͏͔ • ruby-lsp: v.0.7.14 ❌ ⭕
41. ॻ͖׵͑ͷ೉͠͞ • ෳ਺ߦ͕བྷΉͱ೉͍͠ • ruby-lsp: v.0.7.14 • ͜ͷઌ͸܅ࣗ਎ͷ໨Ͱ͔֬Ίͯ͘Ε!! • ςΩετϑΝΠϧΛ͍͡ΔΑ͏ʹίʔυΛॻ͖׵͑Δͷ͸ҙ֎ͱ೉͍͠
42. ༨ஊ: ίϝϯτΛͲ͏ѻ͍͍͔ͨ • ෳ਺ͷࣜͷ্ʹॻ͍ͯ͋Δίϝϯτ͸Ͳ͜ʹ͔͔Δͷ͔? • 1ͭͷࣜʹ͔͔Δέʔεͱෳ਺ࣜʹ͔͔Δέʔε͕͋Γͦ͏ • બ୒ൣғʹΑͬͯ੾Γग़͠ํΛม͑Δͷ͕͍͍͔ʁ def count # @obj can be nil until # #compute is called @obj&.count || 0 end def count # Default value is 0 @obj&.count || 0 end def count # @obj can be nil until # #compute is called var = @obj&.count var || 0 end def count var = @obj&.count # Default value is 0 var || 0 end
43. ߏจ໦ɺԼ͔ΒݟΔ͔ʁ্͔ΒݟΔ͔ʁ • ౴: Ͳͬͪ΋ඞཁ • ্͔ΒԼ΁ḷΔέʔε • FoldingRanges • CodeLens • … • Լ͔Β্΁ḷΔέʔε • SelectionRange • Completion • …
44. ্͔ΒԼ΁ • FoldingRanges • ίʔυΛ͋Δ୯ҐͰંΓͨͨΉػೳ • ෳ਺ͷંΓ৞ΉൣғΛαʔόʔ͕ฦ͢ • ߏจ໦Λ্͔ΒԼ΁ḷ͍͖ͬͯંΓ৞Ήର৅ͷϊʔυ Λݟ͚ͭΔͨͼʹϨεϙϯεʹؚΊΔ ΫϦοΫ͢Δ
45. ্͔ΒԼ΁ NODE_CLASS NODE_DEF NODE_IF p :t NODE_DEF expr
46. Լ͔Β্΁ • SelectionRange • ίʔυͷબ୒ൣғΛม͑Δػೳ • ΫϥΠΞϯτ͔ΒҐஔ৘ใ͕͓͘ΒΕͯ͘ΔͷͰɺରԠ͢ΔൣғΛαʔό ʔ͕ฦ͢
47. Լ͔Β্΁ NODE_CLASS NODE_DEF obj.m1
48. Լ͔Β্ɺ্͔ͦͯ͠ΒԼ • DocumentHighlight • બ୒͞Εͨม਺ͳͲΛϋΠϥΠτ͢ΔͨΊͷػೳ • ϋΠϥΠτ͢Δൣғ • ϩʔΧϧม਺: ಉҰϝιου಺෦ • Πϯελϯεม਺: ಉҰΫϥε಺෦
49. ϩʔΧϧม਺ͷϋΠϥΠτ • 4ߦ໨ͷbarΛબΜͩ࣌ • 4, 5ߦ໨ͷbar͕ϋΠϥΠτ͞ΕΔ NODE_CLASS NODE_DEF bar = 0 NODE_DEF @foo = 1 bar += 1 bar = 0 @foo = 1 bar += 1
50. Πϯελϯεม਺ͷϋΠϥΠτ • 3ߦ໨ͷ@fooΛબΜͩ࣌ • 3, 9ߦ໨ͷ@foo͕ϋΠϥΠτ͞ΕΔ NODE_CLASS NODE_DEF bar = 0 NODE_DEF @foo = 1 bar += 1 bar = 0 @foo = 1 bar += 1
51. ·ͱΊ: LSPͷϢʔεέʔε • ਌ϊʔυ͔Βࢠϊʔυɺࢠϊʔυ͔Β਌ϊʔυͷ྆ํ޲ʹḷΓ͍ͨ • τʔΫϯͷҐஔ৘ใΛऔಘ͍ͨ͠ • ίϝϯτͷ಺༰΍Ґஔ৘ใΛऔಘ͍ͨ͠ • ίʔυͷॻ͖׵͑͸ࢥͬͨҎ্ʹ೉͍͠ • ίʔυͷॻ͖׵͑ʹ͍ͭͯ͸͜ͷ͋ͱ࠶౓৮ΕΔ
52. Linterͷ৔߹ • Style::IfInsideElse ͱ͍͏CopΛΈͯΈΔ • ωετͨ͠ifͷ಺ଆʹ͋ΔifΛ֎ଆͷelsifʹ͢Δ https://github.com/rubocop/rubocop/blob/v1.65.1/lib/rubocop/cop/style/if_inside_else.rb#L10-L29
53. Style::IfInsideElse ͷ࣮૷ • parser gemͰ͸unless΍ࡾ߲ԋࢉࢠ΍ޙஔif΋if nodeʹͳΔ • “if” nodeΛݕࡧͨ͠͏͑Ͱࡉ͔͘νΣοΫͯ͠໨త֎ͷϊʔυΛ஄͘ • #3. AllowIfModifier͕trueͳΒωετͨ͠ޙஔifͷέʔεΛ஄͘ 1. “if” nodeΛݕࡧ͢Δ 2. ࡾ߲ԋࢉࢠͱunlessΛ஄͘ 3. ޙஔifΛ஄͘ https://github.com/rubocop/rubocop/blob/v1.65.1/lib/rubocop/cop/style/if_inside_else.rb
54. RuboCop::AST::Node • `#ternary?`΍`#modifier_form?`͸parser gemͷNodeʹ͸ੜ͑ͯͳ͍ • Parser::AST::Node Λܧঝͨ͠ RuboCop::AST::Node Λఆٛͯ͠ɺͦ͜ʹ helperϝιουΛ࣮૷͍ͯ͠Δ
55. Style::IfInsideElseͷέʔε • ݕࡧ͍ͨ͠ifϊʔυ • ී௨ͷif • ࡾ߲ԋࢉࢠ • unless • ޙஔif (֎ଆͷϊʔυ) • `else_branch`͕nilʹͳ͍ͬͯΔ • ifϊʔυΛࡉ͔͘෼͚ͨ΄͏͕ݕࡧʹศར ??
56. NestedTernaryOperator • ωετͨ͠ࡾ߲ԋࢉࢠΛࢦఠ͢ΔCop • ifϊʔυΛࡉ͔͘෼͚ͨํ͕ศརͳέʔε https://github.com/rubocop/rubocop/blob/v1.65.1/lib/rubocop/cop/style/nested_ternary_operator.rb 1. “if” nodeΛݕࡧ͢Δ 2. ࡾ߲ԋࢉࢠҎ֎Λ஄͘
57. LiteralAsCondition • ৚͕݅ࣜϦςϥϧͷ৔߹ʹࢦఠ͢ΔCop • ifϊʔυΛࡉ͔͘෼͚ͳ͍ํ͕ศརͳέʔε https://github.com/rubocop/rubocop/blob/v1.65.1/lib/rubocop/cop/lint/literal_as_condition.rb 1. “if” nodeΛݕࡧ͢Δ 2. Կ΋஄͔ͳ͍
58. ·ͱΊ: LinterͷϢʔεέʔε • LinterͷϧʔϧΛ࣮૷͢Δͱ͖ʹ͸ɺ2ͭͷҟͳΔϊʔυͷ෼ྨ͕ඞཁ • Syntax͔ΒΈͨͱ͖ͷϊʔυͷछྨ • ifϊʔυΛࡉ͔͘෼͚Δ • Semantics͔ΒΈͨͱ͖ͷϊʔυͷछྨ • IfϊʔυΛࡉ͔͘෼͚ͳ͍ • Ϣʔεέʔε͕ଟ༷Ͱ໘ന͍Ͱ͢Ͷ Θ͕··͕͗͢·ͤΜ͔…?
59. Code Formatterͷ৔߹ • Style::IfInsideElse ͷ #autocorrect ΛΈͯΈΔ • RuboCop::Cop::Corrector < Parser::Source::TreeRewriter ʹରͯ͠ૢ࡞͢Δ • ॻ͖׵͑Δൣғ (range)ͱ৽͍͠ςΩετ (content)Λࢦఆ͢Δ • #replace(range, content) • #remove(range) • #insert_before(range, content) • #insert_after(range, content)
60. ίʔυॻ͖׵͑ͷ༷ࢠ • IfInsideElse#correct_to_elsif_from_if_inside_else_form ʹΑΔॻ͖׵͑ https://github.com/rubocop/rubocop/blob/v1.65.1/lib/rubocop/cop/style/if_inside_else.rb#L10-L29
61. ίʔυॻ͖׵͑ͷ༷ࢠ if condition_a action_a else if condition_b action_b else action_c end end if condition_a action_a elsif condition_b if condition_b action_b else action_c end end if condition_a action_a elsif condition_b action_b action_b else action_c end end if condition_a action_a elsif condition_b action_b action_b else action_c end if condition_a action_a elsif condition_b action_b else action_c end 1. elseΛelsif΁ 2. if condition_b Λ࡟আ 3. ༨෼ͳendΛ ࡟আ 4.ॏෳ͍ͯ͠Δaction_b Λ࡟আ
62. TreeRewriterͷ໰୊఺ #1 • ࣮૷͕ෳࡶ • TreeRewriter͕௚઀จࣈྻΛॻ͖׵͑ΔΘ͚Ͱ͸ͳ͍ • TreeRewriter::ActionͷΠϯελϯεΛͭͬͯ͘ɺ࠷ޙʹҰؾʹมߋΛՃ͑ Δ Action. :replace (2, 0)-(2, 4) “elsif condition_b” Action. :replace (3, 2)-(3, 16) “action_b” Action. :replace (7, 0)-(7, 6) “” Action. :replace (4, 0)-(4, 13) “”
63. ActionΛ༻͍Δཧ༝ #1 • จࣈྻΛ౎౓ॻ͖׵͑Δͱίετ͕ߴ͍͔Β • ͲͪΒͷέʔε΋elseҎ߱ͷจࣈྻΛҠಈ(ίϐʔ)͠ͳ͍ͱ͍͚ͳ͍ if condition_a action_a else action_b end if condition_a action_a action_b end elseΛ࡟আ if condition_a action_a else action_b end if condition_a action_a elsif action_b end elsifʹஔ׵
64. ͦ͏͍͑͹… • จࣈྻͷ෦෼ฤूͱ͍͑͹ςΩετΤσΟλ • ςΩετΤσΟλͱ͍͑͹…?
65. ActionΛ༻͍Δཧ༝ #2 • จࣈྻΛ௚઀ॻ͖׵͑ΔͱଞͷϊʔυʹӨڹ͢Δ͔Β if condition_a action_a else action_b end Parser::Source::Bu ff er if condition_a action_a action_b end Parser::Source::Bu ff er NODE_VCALL action_b Range (3, 2)-(3, 10) elseΛ࡟আ
66. TreeRewriterͷ໰୊఺ #2 • ॻ͖׵͑࣌ͷૢ࡞͕൥ࡶ • ֤ૢ࡞ͰࠓͲͷΑ͏ͳঢ়ଶ͔ཧղ͠ͳ͍ͱ͍͚ͳ͍ if condition_a action_a else if condition_b action_b else action_c end end if condition_a action_a elsif condition_b if condition_b action_b else action_c end end if condition_a action_a elsif condition_b action_b action_b else action_c end end if condition_a action_a elsif condition_b action_b action_b else action_c end if condition_a action_a elsif condition_b action_b else action_c end 1. elseΛelsif΁ 2. if condition_b Λ࡟আ 3. ༨෼ͳendΛ ࡟আ 4.ॏෳ͍ͯ͠Δaction_b Λ࡟আ
67. ߏจ໦Λॻ͖׵͑ͨ৔߹ • #autocorrectΛߏจ໦ͷॻ͖׵͑Ͱදݱ͢Δ • ߏจ໦ͷߏ଄Λੜ͔͍ͨ͠ • NODE_IFΛNODE_ELSIFʹͯ͠ɺNODE_ELSEΛ͚͢ NODE_IF condition_a action_a NODE_ELSE NODE_IF condition_b action_b NODE_ELSE action_c NODE_IF condition_a action_a NODE_ELSIF condition_b action_b action_c
68. ߏจ໦Λॻ͖׵͑ͨ৔߹ • CopΛϊʔυॻ͖׵͑Ͱ࣮૷͢Δ৔߹ͷίʔυ • ςΩετΛฤू͢ΔΑΓ΋ߏ଄తͰΘ͔Γ΍͍͢ͷͰ͸?
69. Extract Variable ࠶ߟ • ߏจ໦ϕʔεͷ৔߹ͷΞϧΰϦζϜ • "var = ࢦఆ͞Εͨൣғ" ͱ͍͏ίʔυΛҰͭखલͷࣜͱͯ͠௥Ճ͢Δ • ࢦఆ͞ΕͨൣғΛ "var" Ͱஔ͖׵͑Δ ❌ NODE_BLOCK NODE_OPCALL i + 1 NODE_BLOCK NODE_LVAR new_variable NODE_LASGN new_variable = i + 1 ⭕
70. ߏจ໦͔Βιʔείʔυ΁ • ͋ͱ͸ߏจ໦͔ΒιʔείʔυΛੜ੒͢Δ͚ͩ • ۭന΍վߦͳͲASTͰ͸͚͍ܽͯΔ৘ใ͕͋Δ NODE_IF condition_a action_a NODE_ELSIF condition_b action_b action_c
71. Ґஔ৘ใΛ༩͑Δ • NodeͷΠϯελϯε࡞੒࣌ʹRangeΛ౉͢ඞཁ͕͋Δ • ੜ੒͞ΕΔίʔυΛΠϝʔδ͠ͳ͕ΒҐஔ৘ใΛߟ͑ͳ͍ͱ͍͚ͳ͍ͷͰ ൥ࡶ if condition_a action_a else if condition_b action_b else action_c end end if condition_a action_a elsif condition_b action_b else action_c end
72. Ґஔ৘ใͷमਖ਼ • มߋͨ͠ߏจ໦Ҏ߱ͷશͯͷϊʔυͷҐஔ৘ใ͕มԽ͢Δ • ؔ৺ͷ͋ΔIF NODEͷܑఋཁૉʹ΋Өڹ͢Δ NODE_CLASS condition_a action_a NODE_ELSIF NODE_DEF NODE_IF NODE_DEF expr1 expr2 expr2 expr2 condition_b action_b action_c
73. Migration Tool • ίʔυͷॻ͖׵͑͸LSP΍Code Formatter͚ͩͷ࢓ࣄͰ͸ͳ͍ • TranspecͳͲͷmigration toolͰ΋ίʔυͷॻ͖׵͑Λߦ͏ඞཁ͕͋Δ http://yujinakayama.me/transpec/
74. ·ͱΊ: ͞·͟·ͳϢʔεέʔε • ਌ϊʔυ͔Βࢠϊʔυɺࢠϊʔυ͔Β਌ϊʔυͷ྆ํ޲ʹḷΓ͍ͨ • τʔΫϯͷҐஔ৘ใΛऔಘ͍ͨ͠ • ίϝϯτͷ಺༰΍Ґஔ৘ใΛऔಘ͍ͨ͠ • ֤छϊʔυʹରͯ͠Syntax͔ΒΈͨͱ͖ͷϊʔυͷछྨͱɺSemantics͔ΒΈͨͱ͖ ͷϊʔυͷछྨͷ2ͭͷ৘ใΛ͍࣋ͨͤͨ • ϊʔυͷॻ͖׵͑Λߦ͏͜ͱͰɺίʔυͷॻ͖׵͑Λߦ͍͍ͨ • ϊʔυͷ͍࣋ͬͯΔ৘ใͷΈΛ༻͍ͯݩͷίʔυΛ׬શʹ෮ݩ͍ͨ͠ • ϊʔυͷॻ͖׵͑ʹΑͬͯ΄͔ͷϊʔυͷҐஔ৘ใΛߋ৽͢ΔΑ͏ͳࣄଶ͸ආ͚ ͍ͨ
75. Osaka RubyKaigi 04 ந৅ߏจ໦Λ ௒͑ͯ
76. Osaka RubyKaigi 04 ந৅ߏจ໦͔Β ۩৅ߏจ໦΁ - ߏจ໦ͷઃܭ -
77. ۩৅ߏจ໦ͱ͸ • ۩৅ߏจ໦ (Concrete Syntax Tree: CST) • ׅހͳͲASTͰ͸ࣦΘΕͯ͠·͏৘ใ΋࢒ͨ͠ߏจ໦ • AST͕ҙຯ(Semantics)ʹ஫໨͍ͯ͠Δͷʹରͯ͠ɺCST͸ߏจ(Syntax)ʹ஫໨ ͍ͯ͠Δ • ಛ௃ 1.τʔΫϯΛද͢σʔλߏ଄Λಋೖ͢Δ 2.lexerͰམͱͯ͠͠·͏৘ใΛτʔΫϯʹඥ͚ͮΔ 3.ϊʔυ͕τʔΫϯΛ࣋ͭΑ͏ʹ͢Δ • ߏจ໦͔Β΋ͱͷίʔυΛ׬શʹ෮ݩͰ͖ΔΑ͏ʹ͢Δ
78. τʔΫϯ༻ͷσʔλߏ଄ • ߏจ໦͔ΒίʔυΛ෮ݩ͢ΔͨΊʹ͸τʔΫϯͷ৘ใ΋ඞཁ • τʔΫϯछผΛද͢`type`ͱτʔΫϯͷࣈ໘Λද͢`text`Λ΋ͬͨߏ଄ • ࠷దԽͷ༨஍͸͍Ζ͍Ζ͋Δ͕͜͜Ͱ͸γϯϓϧͳߏ଄ʹ͢Δ
79. Trivia • LexerͰམͱͯ͠͠·͏৘ใΛTriviaͱΑͿ • ۭന, վߦ, ίϝϯτͳͲ͕Trivia Trivia (comment) Trivia (spaces) Trivia (new line)
80. Trivia • Triviaͷࣈ໘Λද͢`text`Λ΋ͬͨߏ଄ • ͜Ε΋࠷దԽͷ༨஍͕͋Δ • Trivia͸τʔΫϯʹඥ͚ͮͯ؅ཧ͢Δ • τʔΫϯ͸ͦͷલޙʹTriviaΛ΋ͭ͜ͱ͕Ͱ͖Δ
81. ϊʔυͱτʔΫϯ • ϊʔυ͕τʔΫϯΛ࣋ͯΔΑ͏ʹ͢Δ • ͍ͭ͘τʔΫϯΛ͔࣋ͭ͸ϊʔυͷछྨʹΑΔ
82. Node, Token and Trivia NODE_IF IF cond action_a END Token NODE ຌྫ space (1) NL (1) + space (2) NL (1) Trivia • ϊʔυ͸ϊʔυ / τʔΫϯΛ΋ͭ • τʔΫϯ͸લޙʹTriviaΛ΋ͭ
83. ߏจ໦͔Βίʔυ΁ • ਂ͞༏ઌ୳ࡧͭͭ͠จࣈྻΛdump͢Δͱ΋ͱͷίʔυʹͳΔ Token NODE ຌྫ NODE_IF IF cond action_a END space (1) NL (1) + space (2) NL (1) Trivia
84. ߏจղੳͱͷ૬ੑ • Trivia (ۭന΍վߦ) ͸ੜ੒نଇͷ͋ΒΏΔͱ͜Ζʹग़ͯ͘Δ • TriviaΛτʔΫϯ (ऴ୺ه߸)ͱͯ͠ѻ͏ͷ͸൥ࡶ • ࠓճͷํ๏ͳΒߏจղੳ্͸طଘͷτʔΫϯͷଐੑͱͯ͠ѻ͑Δ TriviaΛτʔΫϯͱͯ͠ѻ͏৔߹ TriviaΛτʔΫϯͷଐੑͱͯ͠ѻ͏৔߹
85. Error Recovery • parser͕error recoveryΛ͢Δͱ͖ͷಈ࡞ • TokenΛແࢹ͢Δ • TokenΛ௥Ճ͢Δ • ௥Ճ͞ΕͨTokenΛ۩৅ߏจ໦ͰදݱͰ͖Δ NODE_IF IF true (a + b) END Token NODE ຌྫ Missing Token
86. Lexer, Parser and Trivia • ͍··Ͱ͸LexerͰམ͍ۭͪͯͨനͳͲͷ৘ใ͕Triviaͱͯ͠τʔΫϯʹඥ ͮ͘ • Parser͸tokenͱnodeΛ૊Έ߹Θͤͯߏจ໦Λͭ͘Δ Ruby Script Lexer Parser Token Trivia Syntax Tree Token Node
87. Triviaͷৄࡉ • “cond\n action_a”ΛΈͯΈΔͱվߦ(1)ͱۭന(2)͕͋Δ • ͜ΕΛcondͷtrailing_triviaʹ͢Δ͔ɺaction_aͷleading_triviaʹ͢Δ͔ if cond action_a end if cond action_a end if cond action_a end
88. Triviaͷϧʔϧ • ϧʔϧ1: τʔΫϯ͸ࣗ਎ͷ͋ͱʹଓ͘TriviaΛอ࣋͢Δɻ͜ͷͱ͖վߦจࣈ͸ؚ·ͳ ͍ • ϧʔϧ 2: 1Ͱॴଐͷܾ·Βͳ͍Trivia͸ͦͷ௚ޙͷτʔΫϯʹॴଐ͢Δ • Ifͷ͋ͱͷۭന͸ifʹଐ͢Δ (ϧʔϧ1) • action_a ͸௚લͷվߦͱ2ͭͷۭനΛ΋ͭ (ϧʔϧ1&2) • end͸௚લͷվߦΛ΋ͭ (ϧʔϧ1&2) if cond\n action_a\n end if cond\n action_a\n end if cond\n action_a\n end
89. ϝϦοτ1: ίϝϯτͷѻ͍ • ͜ͷϧʔϧʹै͏ͱίϝϯτ͕ظ଴ͨ͠τʔΫϯʹඥͮ͘ • ϝιουͷલͷϒϩοΫίϝϯτ͸defʹɺࣜͷ͋ͱͷίϝϯτ͸ࣜͷ຤ඌ ͷτʔΫϯʹඥͮ͘
90. ϝϦοτ2: ߦ಄ͷτʔΫϯ • leading_trivia͕վߦ͔Β࢝·Δ͔ΛνΣοΫ͢Δ͜ͱͰɺͦͷτʔΫϯ͕ ߦ಄ͷτʔΫϯ͔Ͳ͏͔൑ผͰ͖Δ • IndentΛܭࢉ͢Δͱ͖ʹߦ಄ͷτʔΫϯ͔Ͳ͏͔͕ॏཁ Leading (nl + sp + sp) Leading (nl)
91. ·ͱΊ: ۩৅ߏจ໦ • ۩৅ߏจ໦ (Concrete Syntax Tree: CST) • ۭനͳͲΛtriviaͱͯ͠τʔΫϯʹඥ͚ͮΔ • ϊʔυ͸ϊʔυ͚ͩͰͳ͘τʔΫϯ΋࣋ͭ
92. ·ͱΊ: ۩৅ߏจ໦ • ਌ϊʔυ͔Βࢠϊʔυɺࢠϊʔυ͔Β਌ϊʔυͷ྆ํ޲ʹḷΓ͍ͨ • τʔΫϯͷҐஔ৘ใΛऔಘ͍ͨ͠ • ίϝϯτͷ಺༰΍Ґஔ৘ใΛऔಘ͍ͨ͠ • ֤छϊʔυʹରͯ͠Syntax͔ΒΈͨͱ͖ͷϊʔυͷछྨͱɺSemantics͔ΒΈͨͱ͖ ͷϊʔυͷछྨͷ2ͭͷ৘ใΛ͍࣋ͨͤͨ • ϊʔυͷॻ͖׵͑Λߦ͏͜ͱͰɺίʔυͷॻ͖׵͑Λߦ͍͍ͨ • ϊʔυͷ͍࣋ͬͯΔ৘ใͷΈΛ༻͍ͯݩͷίʔυΛ׬શʹ෮ݩ͍ͨ͠ • ϊʔυͷॻ͖׵͑ʹΑͬͯ΄͔ͷϊʔυͷҐஔ৘ใΛߋ৽͢ΔΑ͏ͳࣄଶ͸ආ͚ ͍ͨ
93. Osaka RubyKaigi 04 Red Green Tree - ߏจ໦ͷ࣮૷ -
94. Red Green Treeͱ͸ • C# (Roslyn)ͷൃ໌ • Swift (SwiftSyntax)΍ rust-analyzer (LSP)Ͱ΋࢖ΘΕ͍ͯΔ • ߏจ໦ΛRed NodeͱGreen Nodeͱ͍͏2ͭͷσʔλߏ଄Ͱදݱ͢Δ • ͜ͷൃදͰ͸swift-syntaxͷ࣮૷Λओʹࢀߟʹ͍ͯ͠Δ • https://github.com/swiftlang/swift-syntax
95. ͳͥRedͱGreenͳͷ͔ • https://learn.microsoft.com/en-us/archive/blogs/ericlippert/persistence- facades-and-roslyns-red-green-trees Incidentally, these are called "red/green trees" because those were the whiteboard marker colours we used to draw the data structure in the design meeting. There's no other meaning to the colours. • σʔλߏ଄ʹ͍ͭͯٞ࿦͍ͯ͠Δ࣌ʹ࢖͍ͬͯͨϚʔΧʔ͕੺ͱ྘ͩͬ ͨɻͦΕҎ্ͷཧ༝͸ͳ͍ɻ
96. Green Node • ࢠϊʔυ(Green Node)͓ΑͼτʔΫϯ΁ͷࢀরΛ΋ͭ • ࣗ਎ͷ෯(width)Λ͍࣋ͬͯΔ • Offset(ϑΝΠϧઌ಄͔ΒͷҐஔ৘ใ)Λ࣋ͨͳ͍
97. Green Nodeͷߏ଄ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7
98. Green Nodeͷಛ௃(1) • ࢠϊʔυ(Green Node)͓ΑͼτʔΫϯ΁ͷࢀরΛ΋ͭ Token NODE ຌྫ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7
99. Green Nodeͷಛ௃(2) • ࣗ਎ͷ෯(width)Λ͍࣋ͬͯΔ • IF tokenͷ෯͕2͡Όͳ͍ͷ͸trivia͕͋Δ͔Β Green Node ຌྫ ௨ৗͷNode࣮૷ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 NODE_IF o ff set: 0 NODE_IF o ff set: 30 Token
100. OffsetΛ͍࣋ͬͯΔ৔߹ • มԽͷ͋ͬͨϊʔυ/τʔΫϯͷޙଓͷ͢΂ͯͷཁૉʹӨڹ͢Δ • action_bͷ຤ඌʹۭന͕ೖͬͨ࣌ ӨڹΛड͚Δཁૉ NODE_IF o ff set: 0 IF o ff set: 0 NODE_IF o ff set: 30 condition_a o ff set: 3 action_a o ff set: 14 NODE_ELSE o ff set: 25 END o ff set: 86 ELSE o ff set: 25 IF o ff set: 30 condition_b o ff set: 35 action_b o ff set: 46 -> 47 END o ff set: 79 NODE_ELSE o ff set: 59 action_c o ff set: 66 ELSE o ff set: 59
101. ෯Λ͍࣋ͬͯΔ৔߹ • มԽͷ͋ͬͨϊʔυ/τʔΫϯͷ਌ཁૉʹӨڹ͕ݶΒΕΔ • ໦ͷߴ͞(log n)ͰࡁΉ ӨڹΛड͚Δཁૉ NODE_IF width: 90 -> 91 IF width: 3 NODE_IF width: 56 -> 57 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 -> 62 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 -> 14 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7
102. Red Node • ࢠϊʔυ(Green Node / τʔΫϯ)ͱ਌ϊʔυ(Red Node)΁ͷࢀরΛ΋ͭ • ϑΝΠϧͷઌ಄͔ΒͷҐஔ৘ใ(offset)Λ΋ͭ • Red Nodeͷߏங͸ඞཁʹԠͯ͡lazyʹߦ͏
103. Red Nodeͷߏ଄ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 NODE_IF o ff set: 30
104. Red Nodeͷಛ௃(1) • ࢠϊʔυ(Green Node)ͱ਌ϊʔυ(Red Node)΁ͷࢀরΛ΋ͭ • ࢠ͔Β਌΁ͷΞΫηε͕؆୯ Token Green NODE ຌྫ Red NODE NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 NODE_IF o ff set: 30
105. Red Nodeͷಛ௃(2) • ϑΝΠϧͷઌ಄͔ΒͷҐஔ৘ใ(offset)Λ΋ͭ Token Green NODE ຌྫ Red NODE NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 NODE_IF o ff set: 30
106. offsetͷܭࢉํ๏ • offset͸਌Red NodeͷoffsetͱࢠͷGreen Nodeͷwidth͔ΒܭࢉͰ͖Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 (3 + 11 + 11) NODE_IF o ff set: 30 25 + (5)
107. Red Nodeͷಛ௃(3) • Red Nodeͷߏங͸ඞཁʹԠͯ͡lazyʹߦ͏ • Green Node͸ߏจղੳͷ݁ՌͳͷͰৗʹ࡞Δ͕ɺRed Node͸ඞཁʹͳΔ ·Ͱੜ੒Λ஗ԆͰ͖Δ
108. Red/Green Nodeͷ࢖͍෼͚ • Green Node • ϊʔυʹہॴԽ͞Εͨ(ࣗ਎ͱࢠཁૉ͚ͩͰܭࢉͰ͖Δ)৘ใΛ࣋ͨͤΔ • Red Node • ͦ͏Ͱͳ͍৘ใΛ࣋ͨͤΔ • ྫ: offsetͱߦ൪߸ͷmap͸Red Nodeʹ࣋ͨͤΔ
109. Red Node & Green Node • Red Node΋Green Node΋immutableͰpersistentͳσʔλߏ଄ͱ͢Δ • ߏจ໦ͷϢʔβʔʹ͸Red NodeͷΈΛެ։͠ɺGreen Node͸internalͳσ ʔλߏ଄ͱ͢Δ
110. Immutable • Ұ౓࡞ͬͨNodeͷଐੑΛޙ͔Βߋ৽͠ͳ͍/Ͱ͖ͳ͍ • มߋΛՃ͍͑ͨͱ͖͸ίϐʔΛͭͬͯ͘Ұ෦ͷଐੑΛॻ͖׵͑Δ • େ෦෼ͷNode΍Token͸ڞ༗Ͱ͖Δ IF width: 3 END width: 4 NODE_IF width: 90 NODE_IF width: 90 IF width: 3 END width: 4 NODE_IF width: 91 IF width: 4 If ͷ͋ͱʹۭനΛ௥Ճ
111. Ӭଓσʔλߏ଄ (Persistent) • ߏจ໦ͷมߋͷલޙɺͲͪΒͷόʔδϣϯʹ΋ΞΫηεͰ͖Δ NODE_IF (ver.1) width: 90 IF width: 3 END width: 4 NODE_IF (ver.1) width: 90 IF width: 3 END width: 4 NODE_IF (ver.2) width: 91 IF width: 4
112. ύʔε݁Ռ͸Red Node • ιʔείʔυΛύʔεͨ͠ͱ͖͸rootͷRed Node͕ฦͬͯ͘Δ parse ͜Ε͕ฦͬͯ͘Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 NODE_IF o ff set: 0
113. ࢠཁૉ΋Red Node • ࢠϊʔυɺτʔΫϯʹΞΫηεͨ͠ͱ͖͸Red Nodeʹwrap͞Εͨཁૉ͕ ฦΔ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 NODE_IF o ff set: 0 TOKEN_IF o ff set: 0 NODE_ELSE o ff set: 25 node#if_token node#else_node
114. ৽ن࡞੒࣌΋Red Node • NodeΛ৽͘͠࡞Δͱ͖͸Red NodeΛ࡞Δ • ৽نʹelsifͱelseͷτʔΫϯΛ࡞Δ͕ɺ͜Ε΋Red Node (Red Token) • ৽͘͠࡞ͬͨRed Node͸Ҿ਺ͷRed Node͔ΒGreen NodeΛऔΓग़ͯ͠ɺ ࣗ෼ͷGreen Nodeʹඥ͚ͮΔ # good if condition_a action_a elsif condition_b action_b else action_c end
115. Node࡞੒ͷཪଆ NODE_ELSE width: 61 NODE_IF width: 56 NODE_IF o ff set: 25 condition_b o ff set: 30 action_b o ff set: 41 action_c o ff set: 47 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7
116. Node࡞੒ͷཪଆ NODE_ELSE width: 61 NODE_IF width: 56 NODE_IF o ff set: 25 condition_b o ff set: 30 action_b o ff set: 41 action_c o ff set: 47 ELSIF width: 7 ELSE width: 5 ELSIF o ff set: 0 ELSE o ff set: 0 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 Tokenͷੜ੒
117. Node࡞੒ͷཪଆ NODE_ELSE width: 61 NODE_IF width: 56 NODE_IF o ff set: 25 condition_b o ff set: 30 action_b o ff set: 41 action_c o ff set: 47 ELSIF width: 7 ELSE width: 5 ELSIF o ff set: 0 ELSE o ff set: 0 ຌྫ NODE_ELSIFͷҾ਺ NODE_ELSIF o ff set: 0 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ৽͍͠Node
118. Node࡞੒ͷཪଆ NODE_ELSE width: 61 NODE_IF width: 56 NODE_IF o ff set: 25 condition_b o ff set: 30 action_b o ff set: 41 action_c o ff set: 47 NODE_ELSIF width: 49 ELSIF width: 7 ELSE width: 5 ELSIF o ff set: 0 ELSE o ff set: 0 ຌྫ NODE_ELSIFͷҾ਺ NODE_ELSIF o ff set: 0 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ৽͍͠Node width: 11 + 13 + 13 + 7 + 5 = 49
119. ߏจ໦ͷҰ෦Λॻ͖׵͑Δ • ֎ଆͷifͷelseΛ࡞ͬͨelsif nodeͰஔ͖׵͑Δ # bad if condition_a action_a else if condition_b action_b else action_c end end
120. ॻ͖׵͑ͷཪଆ NODE_IF width: 90 IF width: 3 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 • ॻ͖׵͑લͷঢ়ଶ
121. ॻ͖׵͑ͷཪଆ NODE_IF width: 90 IF width: 3 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 NODE_ELSIF width: 49 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 NODE_ELSIF o ff set: 0 • ઌ΄Ͳͭͬͨ͘NODE_ELSIFΛ΋ͬͯ͘Δ
122. ॻ͖׵͑ͷཪଆ NODE_IF width: 90 IF width: 3 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 NODE_ELSIF width: 49 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25 NODE_IF o ff set: 0 NODE_IF width: 78 NODE_ELSIF o ff set: 0 • ৽͘͠NODE_IF (Red Node & Green Node)Λ ͭ͘Δ
123. ਂ͍஍఺ͷॻ͖׵͑ • ྫ: action_c ͷtrivia͕มΘͬͨͱ͖ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 if condition_a action_a else if condition_b action_b else action_c ; end end ߋ৽
124. ਂ͍஍఺ͷॻ͖׵͑ • ߋ৽લͷߏจ໦ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ߋ৽ action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF
125. ਂ͍஍఺ͷॻ͖׵͑ • ৽͘͠action_cͷnodeΛͭ͘Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ߋ৽ action_c width: 15 action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF
126. ਂ͍஍఺ͷॻ͖׵͑ • action_cͷRed NodeΛͭ͘Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ߋ৽ action_c width: 15 action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF action_c
127. ਂ͍஍఺ͷॻ͖׵͑ • ࢠཁૉ΋෯΋มΘΔͷͰGreen NodeΛͭ͘Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ߋ৽ action_c width: 15 action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF NODE_ELSE width: 22 action_c
128. ਂ͍஍఺ͷॻ͖׵͑ • NODE_ELSEͷRed NodeΛͭ͘Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ߋ৽ action_c width: 15 action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF NODE_ELSE width: 22 action_c NODE_ELSE
129. ਂ͍஍఺ͷॻ͖׵͑ • ಉ༷ʹͯ͠root·ͰͷશͯͷRed NodeΛ࡞Γ௚͢ඞཁ͕͋Δ NODE_IF width: 90 IF width: 3 NODE_IF width: 56 condition_a width: 11 action_a width: 11 NODE_ELSE width: 61 END width: 4 ELSE width: 5 IF width: 5 condition_b width: 11 action_b width: 13 END width: 7 NODE_ELSE width: 20 action_c width: 13 ELSE width: 7 ߋ৽ action_c width: 15 action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF NODE_ELSE width: 22 NODE_IF width: 58 NODE_ELSE width: 63 NODE_IF width: 92 action_c NODE_ELSE NODE_IF NODE_ELSE NODE_IF
130. Rewriterͷඞཁੑ • ߏจ໦ʹରͯ͠਺ճ͔͠ॻ͖׵͑ΛߦΘͳ͍ͳΒ͜ͷίετ͸ແࢹͰ͖Δ • RuboCopͳͲͰ͸ߏจ໦ͷ͋ͪͪ͜Λॻ͖׵͑Δ͜ͱʹͳΔ • ౎౓root node·ͰͷRed NodeΛ࡞Δͷ͸ແବ • ޮ཰తͳॻ͖׵͑Λࢧԉ͢Δػೳ͕ඞཁ • RewriterΛ࣮૷͢Δ
131. Rewriterͷ࣮૷ • ֤ϊʔυʹରԠͨ͠ϝιουΛఆٛ͢Δ (#on_xxx) • ࢠཁૉΛϝιουʹ౉ͯ͠ɺҰͭͰ΋มԽ͍ͯͨ͠Βࣗ਎Λ࠶࡞੒͢Δ • ࡞Γ௚͢৔߹͸root nodeͱͯ͠࡞Γ௚͢
132. Rewriterͷܧঝ • ؔ৺ͷ͋ΔϊʔυʹରԠͨ͠ϝιουΛ্ॻ͖͢Δ • ඞཁʹԠͯ͡৽͍͠ϊʔυΛroot nodeͱͯ͠࡞ͬͯฦ͢
133. Rewriterͷ࢓૊Έ • ਂ͞༏ઌͰϊʔυΛḷͬͯɺมԽ͕͋Ε͹ؼΓ͕͚ʹϊʔυΛ࠶ߏங͢Δ NODE_CLASS ver. 1 NODE_DEF ver. 1 NODE_DEF ver. 1 NODE_IF ver. 1 NODE_CALL ver. 1 NODE_CASE ver. 1 ߋ৽ NODE_CLASS ver. 1 NODE_DEF ver. 2 NODE_DEF ver. 1 NODE_IF ver. 2 NODE_CALL ver. 1 NODE_CASE ver. 2 ߋ৽ ৽ن ৽ن ࡞Γ௚͢લʹӈͷDEF΁͍͘
134. Rewriterͷ࢓૊Έ • 1ϊʔυʹରͯ͠࠷େ1૊ͷϊʔυΛ࡞Δ͚ͩͰࡁΉ • ͦΕͧΕ͸root nodeͱͯ͠ϊʔυΛ࡞ΔͷͰ਌ͷߋ৽Λ͠ͳͯ͘ࡁΉ NODE_CLASS ver. 1 NODE_DEF ver. 2 NODE_DEF ver. 1 NODE_IF ver. 2 NODE_CALL ver. 1 NODE_CASE ver. 2 ߋ৽ NODE_CLASS ver. 1 NODE_DEF ver. 2 NODE_DEF ver. 2 NODE_IF ver. 2 NODE_CALL ver. 2 NODE_CASE ver. 2 NODE_CLASS ver. 2 NODE_DEF ver. 2 NODE_DEF ver. 2 NODE_IF ver. 2 NODE_CALL ver. 2 NODE_CASE ver. 2 ৽ن ৽ن
135. Rewriter ଞݴޠͰͷ࣮૷ • Rewriter͸൚༻తͳ࣮૷ͳͷͰίΞʹ͍ۙͱ͜ΖͰఏڙ͢Δͷ͕Α͍ͷͰ ͸ͳ͍͔ • Roslyn: CSharpSyntaxRewriter • https://github.com/dotnet/roslyn/blob/v4.2.0/src/Compilers/CSharp/ Portable/Syntax/CSharpSyntaxRewriter.cs • SwiftSyntax: SyntaxRewriter • https://github.com/swiftlang/swift-syntax/blob/600.0.0- prerelease-2024-07-30/Sources/SwiftSyntax/generated/ SyntaxRewriter.swift
136. Ӭଓσʔλߏ଄ͷརศੑ • ݩͷ໦ɺॻ͖׵͑ޙͷ෦෼໦ɺॻ͖׵͑ޙͷશମͷ໦͕ͦΕͧΕ҆શʹख ʹೖΔ
137. Ӭଓσʔλߏ଄ͷརศੑ • Code FormatterͷRule (Cop)͸ߏจ໦Λड͚औͬͯɺ৽͍͠෦෼໦Λੜ੒ ͯ͠ɺߏจ໦શମΛߋ৽͢Δ • Rule͸ߋ৽લޙͷ໦Λ΋ͯΔ • มߋલޙͷίʔυΛදࣔͰ͖Δ IfInsideElse if condition_a action_a else if condition_b action_b else action_c end end if condition_a action_a elsif condition_b action_b else action_c end NODE_IF width: 78 NODE_ELSE width: 49 NODE_IF o ff set: 0 NODE_ELSIF o ff set: 0 NODE_IF width: 90 NODE_ELSE width: 61 NODE_IF o ff set: 0 NODE_ELSE o ff set: 25
138. ·ͱΊ: Red Green Tree • ߏจ໦ΛRed Tree/NodeͱGreen Tree/Nodeͱ͍͏2ͭͷσʔλߏ଄Ͱද͢ • Green Node͸ࣗ਎ͱࢠཁૉʹดͨ͡৘ใΛ΋ͭ • Red Node͸ͦΕҎ֎ͷ৘ใΛ΋ͭ • Red Node΋Green Node΋immutableͰpersistentͳσʔλߏ଄ • ߏจ໦ͷϢʔβʔʹ͸Red NodeͷΈΛެ։͠ɺGreen Node͸internalͳσ ʔλߏ଄ͱ͢Δ
139. ·ͱΊ: Red Green Tree • ਌ϊʔυ͔Βࢠϊʔυɺࢠϊʔυ͔Β਌ϊʔυͷ྆ํ޲ʹḷΓ͍ͨ • τʔΫϯͷҐஔ৘ใΛऔಘ͍ͨ͠ • ίϝϯτͷ಺༰΍Ґஔ৘ใΛऔಘ͍ͨ͠ • ֤छϊʔυʹରͯ͠Syntax͔ΒΈͨͱ͖ͷϊʔυͷछྨͱɺSemantics͔ΒΈͨͱ͖ ͷϊʔυͷछྨͷ2ͭͷ৘ใΛ͍࣋ͨͤͨ • ϊʔυͷॻ͖׵͑Λߦ͏͜ͱͰɺίʔυͷॻ͖׵͑Λߦ͍͍ͨ • ϊʔυͷ͍࣋ͬͯΔ৘ใͷΈΛ༻͍ͯݩͷίʔυΛ׬શʹ෮ݩ͍ͨ͠ • ϊʔυͷॻ͖׵͑ʹΑͬͯ΄͔ͷϊʔυͷҐஔ৘ใΛߋ৽͢ΔΑ͏ͳࣄଶ͸ආ͚ͨ ͍
140. Osaka RubyKaigi 04 ۩৅ߏจ໦ͱ ந৅ߏจ໦ͷཱ྆
141. ϊʔυʹ྆ํͷ৘ใΛ΋ͨͤΔ • Green NodeʹSyntaxͱSemanticsͷ2ͭͷtypeΛ࣋ͨͤΔ • Semanticsʹ஫໨ͯ͠ΞΫηε͍ͨ͠ͱ͖ͷͨΊʹhelperؔ਺Λఆٛ͢Δ
142. ϊʔυͷߏ଄ • enum node_syntax_type • ϊʔυ͝ͱʹҟͳΔ • enum node_semantics_type • semantics͕ಉ͡ͳΒಉ͡ (ྫ: NODE_SEMANTICS_IF)
143. semantics nodeͱͯ͠ΞΫηε͢Δ • if nodeͷछผʹΑΒͣ৚݅۟ (condition) ͕΄͍͠ • node_syntax_typeͰ෼ذͨ͠ίʔυΛॻ͍ͯɺؔ਺ͱͯ͠ఏڙ͢Δ
144. ·ͱΊ: ͞·͟·ͳϢʔεέʔε • ਌ϊʔυ͔Βࢠϊʔυɺࢠϊʔυ͔Β਌ϊʔυͷ྆ํ޲ʹḷΓ͍ͨ • τʔΫϯͷҐஔ৘ใΛऔಘ͍ͨ͠ • ίϝϯτͷ಺༰΍Ґஔ৘ใΛऔಘ͍ͨ͠ • ֤छϊʔυʹରͯ͠Syntax͔ΒΈͨͱ͖ͷϊʔυͷछྨͱɺSemantics͔ΒΈͨͱ͖ ͷϊʔυͷछྨͷ2ͭͷ৘ใΛ͍࣋ͨͤͨ • ϊʔυͷॻ͖׵͑Λߦ͏͜ͱͰɺίʔυͷॻ͖׵͑Λߦ͍͍ͨ • ϊʔυͷ͍࣋ͬͯΔ৘ใͷΈΛ༻͍ͯݩͷίʔυΛ׬શʹ෮ݩ͍ͨ͠ • ϊʔυͷॻ͖׵͑ʹΑͬͯ΄͔ͷϊʔυͷҐஔ৘ใΛߋ৽͢ΔΑ͏ͳࣄଶ͸ආ͚ ͍ͨ
145. Osaka RubyKaigi 04 Rubyʹ࣮૷͢Δʹ͋ͨͬͯ Open Problems
146. Կ͕trivia͔ • Rubyʹ͓͍ͯ͸վߦ͸τʔΫϯͱͯ͠ѻΘΕΔέʔεͱlexerͰམͱ͞ΕΔ έʔε (tIGNORED_NL) ͕͋Δ • ํ๏1: τʔΫϯʹͳͬͨվߦ͸τʔΫϯɺͦ͏Ͱͳ͍΋ͷ͸triviaʹ͢Δ • ํ๏2: શͯͷվߦΛtriviaʹ͢Δ • ηϛίϩϯ͸trivialʹͨ͠ํ͕ѻ͍΍͍͢ͷͰ͸ͳ͍͔ʁ • ()͸ϊʔυͷ΄͏͕͍͍ͷ͔ʁ
147. ώΞυΩϡϝϯτ • ώΞυΩϡϝϯτ͸1ߦʹෳ਺ݸॻ͘͜ͱ͕Ͱ͖Δ • ͜ͷΑ͏ͳέʔεͰ΋ݩͷίʔυ͕෮ݩͰ͖ΔΑ͏ͳϊʔυʹ͠ͳ͍ͱ͍ ͚ͳ͍ • Green Tree͸۪௚ʹͭͬͯ͘ɺRed Treeͷoffset΍ςΩετʹ໭͢ͱ͖ʹ޻ ෉͢Δͷ͕Α͍ͷͰ͸ͳ͍͔ puts <<~STR1 + <<~STR2 111 STR1 aaa STR2 # => # 111 # aaa NODE_HEREDOC width: 90 <<~ width: 3 111 width: 4 STR1 width: 4 STR1 width: 5
148. Lexer interface • ͍··Ͱ͸τʔΫϯΛฦ͢ΠϯλʔϑΣΠεͱϊʔυΛฦ͢ΠϯλʔϑΣ Πε͕ผͩͬͨ • ۩৅ߏจ໦ʹͳͬͯϊʔυ͕τʔΫϯΛؚΉΑ͏ʹͳͬͨ͋ͱͰ΋͜ͷೋ ͭͷΠϯλʔϑΣΠε͸ผͷ΄͏͕͍͍ͷ͔ʁ • irbʹ͓͚Δ৭෇͚΍minifierͷ࣮૷ʹ͓͍ͯ͸ͲͪΒͷํ͕ศརͳͷ͔ʁ • ϊʔυͷ΋ͭߏ଄ʹؔ͢Δ৘ใ͕ඞཁͳͷ͔ʁ
149. Osaka RubyKaigi 04 ·ͱΊ
150. ߏจ໦ઃܭͷݪཧݪଇΛٻΊͯ • ߏจ໦͸ͦͷ౎౓ΞυϗοΫʹ֦ு͞Ε͖ͯͨΑ͏ʹݟ͑Δ • ͦͷઃܭʹݪཧݪଇ͸ͳ͍ͷͩΖ͏͔ʁ • ·ͣϢʔεέʔεΛௐ΂Δ • ίʔυΛղੳ͠Α͏ͱ͢Δͱίʔυͷ࣮ߦʹൺ΂ͯΑΓଟ͘ͷ৘ใ͕ඞ ཁʹͳΔ • ίʔυͷॻ͖׵͑ΛςΩετϕʔεͰߦ͏ͷ͸೉͍͠ • طଘͷߏจ໦࣮૷Ͱ͸ߏจ໦ͷॻ͖׵͕͑େม
151. ߏจ໦΁ͷཁ๬ • ਌ϊʔυ͔Βࢠϊʔυɺࢠϊʔυ͔Β਌ϊʔυͷ྆ํ޲ʹḷΓ͍ͨ • τʔΫϯͷҐஔ৘ใΛऔಘ͍ͨ͠ • ίϝϯτͷ಺༰΍Ґஔ৘ใΛऔಘ͍ͨ͠ • ֤छϊʔυʹରͯ͠Syntax͔ΒΈͨͱ͖ͷϊʔυͷछྨͱɺSemantics͔ΒΈͨͱ͖ ͷϊʔυͷछྨͷ2ͭͷ৘ใΛ͍࣋ͨͤͨ • ϊʔυͷॻ͖׵͑Λߦ͏͜ͱͰɺίʔυͷॻ͖׵͑Λߦ͍͍ͨ • ϊʔυͷ͍࣋ͬͯΔ৘ใͷΈΛ༻͍ͯݩͷίʔυΛ׬શʹ෮ݩ͍ͨ͠ • ϊʔυͷॻ͖׵͑ʹΑͬͯ΄͔ͷϊʔυͷҐஔ৘ใΛߋ৽͢ΔΑ͏ͳࣄଶ͸ආ͚ ͍ͨ
152. ߏจ໦ͷઃܭͱ࣮૷ • ߏจ໦ͷઃܭ: • ۩৅ߏจ໦ • ߏจ໦ͷ࣮૷: • Red Green Tree • ந৅ߏจͷ৘ใΛ΋ͬͨ۩৅ߏจ໦ • ߏจ໦΁ͷཁ๬Λୡ੒Ͱ͖ͨʂ • ࠓޙ΍Δ͜ͱ • Rubyݻ༗ͷ໰୊Λղ͘ • ࣮૷Λ͢Δ
153. Osaka RubyKaigi 04 Thank You !!!