Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Syntax Tree (RubyConf Mini)

Kevin Newton
November 16, 2022
55

Syntax Tree (RubyConf Mini)

Syntax Tree is a new toolkit for interacting with the Ruby parse tree. It can be used to analyze, inspect, debug, and format your Ruby code. In this talk we'll walk through how it works, how to use it in your own applications, and the exciting future possibilities enabled by Syntax Tree.

Kevin Newton

November 16, 2022
Tweet

Transcript

  1. Syntax Tree

  2. Kevin Newton @kddnewton

  3. Syntax Tree

  4. What is Syntax Tree?

  5. Syntax Tree concepts

  6. Parser Syntax Tree concepts

  7. Parser Syntax tree Syntax Tree concepts

  8. Parser Syntax tree Visitor Syntax Tree concepts

  9. Parser · Syntax tree · Visitor

  10. Parser · Syntax tree · Visitor Parser

  11. Matz is nice so we are nice. Parser · Syntax

    tree · Visitor
  12. Lexical analysis Parser · Syntax tree · Visitor

  13. Matz is nice so we are nice. Parser · Syntax

    tree · Visitor
  14. Matz is nice so we are nice . Parser ·

    Syntax tree · Visitor
  15. is nice so we are nice . Matz Parser ·

    Syntax tree · Visitor
  16. is nice so we are nice . Matz Parser ·

    Syntax tree · Visitor
  17. Matz is nice we are nice . so Parser ·

    Syntax tree · Visitor
  18. Matz is nice so we are nice . Parser ·

    Syntax tree · Visitor
  19. Matz is nice so we are nice . Parser ·

    Syntax tree · Visitor
  20. Matz is nice so we are . nice Parser ·

    Syntax tree · Visitor
  21. Matz is nice so we are nice . Parser ·

    Syntax tree · Visitor
  22. Matz is nice so we are nice . Parser ·

    Syntax tree · Visitor
  23. Semantic analysis Parser · Syntax tree · Visitor

  24. Parser · Syntax tree · Visitor is nice are nice

  25. Grammar Parser · Syntax tree · Visitor

  26. Grammar Parser · Syntax tree · Visitor verb-phrase : VERB

    ADJECTIVE
  27. Parser · Syntax tree · Visitor is nice are nice

  28. Parser · Syntax tree · Visitor is nice verb phrase

    verb phrase are nice
  29. Parser · Syntax tree · Visitor Matz is nice verb

    phrase we verb phrase are nice
  30. Grammar Parser · Syntax tree · Visitor verb-phrase : VERB

    ADJECTIVE
  31. Grammar Parser · Syntax tree · Visitor verb-phrase : VERB

    ADJECTIVE 
 subject-phrase : NOUN verb-phrase
  32. Parser · Syntax tree · Visitor Matz is nice verb

    phrase we verb phrase are nice
  33. Parser · Syntax tree · Visitor Matz is nice subject

    phrase subject phrase verb phrase we verb phrase are nice
  34. Parser · Syntax tree · Visitor Matz is nice so

    subject phrase subject phrase verb phrase we verb phrase are nice
  35. Grammar Parser · Syntax tree · Visitor verb-phrase : VERB

    ADJECTIVE 
 subject-phrase : NOUN verb-phrase
  36. Grammar Parser · Syntax tree · Visitor verb-phrase : VERB

    ADJECTIVE 
 subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase
  37. Parser · Syntax tree · Visitor Matz is nice so

    subject phrase subject phrase verb phrase we verb phrase are nice
  38. Parser · Syntax tree · Visitor Matz is nice so

    subord. conjunction subject phrase subject phrase verb phrase we verb phrase are nice
  39. Parser · Syntax tree · Visitor Matz is nice so

    . subord. conjunction subject phrase subject phrase verb phrase we verb phrase are nice
  40. verb-phrase : VERB ADJECTIVE 
 subject-phrase : NOUN verb-phrase 


    subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase Grammar Parser · Syntax tree · Visitor
  41. verb-phrase : VERB ADJECTIVE 
 subject-phrase : NOUN verb-phrase 


    subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase 
 sentence : subordinating-conjunction PERIOD Grammar Parser · Syntax tree · Visitor
  42. Parser · Syntax tree · Visitor Matz is nice so

    . subord. conjunction subject phrase subject phrase verb phrase we verb phrase are nice
  43. Parser · Syntax tree · Visitor Matz is nice so

    . sentence subord. conjunction subject phrase subject phrase verb phrase we verb phrase are nice
  44. verb-phrase : VERB ADJECTIVE 
 subject-phrase : NOUN verb-phrase 


    subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase 
 sentence : subordinating-conjunction PERIOD Grammar Parser · Syntax tree · Visitor
  45. Parser · Syntax tree · Visitor Syntax tree

  46. verb-phrase : VERB ADJECTIVE 
 subject-phrase : NOUN verb-phrase 


    subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase 
 sentence : subordinating-conjunction PERIOD Grammar Parser · Syntax tree · Visitor
  47. Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE

    subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD
  48. Parser · Syntax tree · Visitor class VerbPhrase attr_reader :verb,

    :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD
  49. Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE

    subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end
  50. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end
  51. Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE

    subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end
  52. Parser · Syntax tree · Visitor verb-phrase : VERB ADJECTIVE

    subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end
  53. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end
  54. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end
  55. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end
  56. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end
  57. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end
  58. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end
  59. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end
  60. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end
  61. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end
  62. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end class Conjunction attr_reader :value def initialize(value:) @value = value end end
  63. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end class Conjunction attr_reader :value def initialize(value:) @value = value end end
  64. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end class Conjunction attr_reader :value def initialize(value:) @value = value end end class Period attr_reader :value def initialize(value:) @value = value end end
  65. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end verb-phrase : VERB ADJECTIVE subject-phrase : NOUN verb-phrase subordinating-conjunction : 
 subject-phrase CONJUNCTION subject-phrase sentence : subordinating-conjunction PERIOD class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end class Conjunction attr_reader :value def initialize(value:) @value = value end end class Period attr_reader :value def initialize(value:) @value = value end end
  66. Parser · Syntax tree · Visitor class SubjectPhrase attr_reader :noun,

    :verb_phrase def initialize(noun:, verb_phrase:) @noun = noun @verb_phrase = verb_phrase end end class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end class Conjunction attr_reader :value def initialize(value:) @value = value end end class Period attr_reader :value def initialize(value:) @value = value end end
  67. class SubjectPhrase attr_reader :noun, :verb_phrase def initialize(noun:, verb_phrase:) @noun =

    noun @verb_phrase = verb_phrase end end class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end end class SubordinatingConjunction attr_reader :left, :conjunction, :right def initialize(left:, conjunction:, right:) @left = left @conjunction = conjunction @right = right end end class Sentence attr_reader :phrase, :punctuation def initialize(phrase:, punctuation:) @phrase = phrase @punctuation = punctuation end end class Verb attr_reader :value def initialize(value:) @value = value end end class Adjective attr_reader :value def initialize(value:) @value = value end end class Noun attr_reader :value def initialize(value:) @value = value end end class Conjunction attr_reader :value def initialize(value:) @value = value end end class Period attr_reader :value def initialize(value:) @value = value end end Parser · Syntax tree · Visitor
  68. class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb =

    verb @adjective = adjective end end Parser · Syntax tree · Visitor
  69. Parser · Syntax tree · Visitor class VerbPhrase attr_reader :verb,

    :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end def deconstruct_keys(keys) { verb: verb, adjective: adjective } end def ==(other) other in VerbPhrase[ verb: ^(verb), adjective: ^(adjective) ] end end
  70. class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb =

    verb @adjective = adjective end end Parser · Syntax tree · Visitor
  71. Parser · Syntax tree · Visitor class VerbPhrase attr_reader :verb,

    :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end def copy(verb: self.verb, adjective: self.adjective) VerbPhrase.new(verb: verb, adjective: adjective) end end
  72. class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb =

    verb @adjective = adjective end end Parser · Syntax tree · Visitor
  73. Parser · Syntax tree · Visitor class VerbPhrase attr_reader :verb,

    :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end def accept(visitor) visitor.visit_verb_phrase(self) end def child_nodes [verb, adjective] end end
  74. Parser · Syntax tree · Visitor class VerbPhrase attr_reader :verb,

    :adjective def initialize(verb:, adjective:) @verb = verb @adjective = adjective end def accept(visitor) visitor.visit_verb_phrase(self) end def child_nodes [verb, adjective] end end
  75. Visitor Parser · Syntax tree · Visitor

  76. class VerbPhrase attr_reader :verb, :adjective def initialize(verb:, adjective:) @verb =

    verb @adjective = adjective end def accept(visitor) visitor.visit_verb_phrase(self) end def child_nodes [verb, adjective] end end Parser · Syntax tree · Visitor
  77. class VerbPhrase def accept(visitor) visitor.visit_verb_phrase(self) end end Parser · Syntax

    tree · Visitor
  78. class VerbPhrase def accept(visitor) visitor.visit_verb_phrase(self) end end 
 class Visitor

    def visit_verb_phrase(node) # ... end end node.accept(Visitor.new) Parser · Syntax tree · Visitor
  79. class VerbPhrase def accept(visitor) visitor.visit_verb_phrase(self) end end 
 class Visitor

    def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end def visit_verb_phrase(node) # ... end end node.accept(Visitor.new) Parser · Syntax tree · Visitor
  80. class Visitor def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node)

    end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end def visit_verb_phrase(node) # ... end end node.accept(Visitor.new) Parser · Syntax tree · Visitor
  81. class Visitor def visit(node) node.accept(self) end def visit_adjective(node) end def

    visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end def visit_verb_phrase(node) # ... end end node.accept(Visitor.new) Parser · Syntax tree · Visitor
  82. Parser · Syntax tree · Visitor class Visitor def visit(node)

    node.accept(self) end def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end def visit_verb_phrase(node) node.child_nodes.each { |node| visit(node) } end end node.accept(Visitor.new)
  83. Parser · Syntax tree · Visitor class Visitor def visit(node)

    node.accept(self) end def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end def visit_verb_phrase(node) node.child_nodes.each { |node| visit(node) } end end
  84. Parser · Syntax tree · Visitor class Visitor def visit(node)

    node.accept(self) end def visit_child_nodes(node) node.child_nodes.each { |node| visit(node) } end def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end def visit_verb_phrase(node) visit_child_nodes(node) end end
  85. Parser · Syntax tree · Visitor class Visitor def visit(node)

    node.accept(self) end def visit_child_nodes(node) node.child_nodes.each { |node| visit(node) } end def visit_adjective(node) end def visit_conjunction(node) end def visit_noun(node) end def visit_verb(node) end def visit_punctuation(node) end def visit_sentence(node) end def visit_subordinating_conjunction(node) end def visit_subject_phrase(node) end alias visit_verb_phrase visit_child_nodes end
  86. Parser · Syntax tree · Visitor class Visitor def visit(node)

    node.accept(self) end def visit_child_nodes(node) node.child_nodes.each { |node| visit(node) } end alias visit_adjective visit_child_nodes alias visit_conjunction visit_child_nodes alias visit_noun visit_child_nodes alias visit_verb visit_child_nodes alias visit_punctuation visit_child_nodes alias visit_sentence visit_child_nodes alias visit_subordinating_conjunction 
 visit_child_nodes alias visit_subject_phrase visit_child_nodes alias visit_verb_phrase visit_child_nodes end
  87. Parser · Syntax tree · Visitor

  88. Parser · Syntax tree · Visitor class SentenceCountVisitor < Visitor

    attr_reader :count def initialize @count = 0 end def visit_sentence(node) @count += 1 super end end visitor = SentenceCountVisitor.new node.accept(visitor) puts visitor.count
  89. Parser · Syntax tree · Visitor class PrettyPrintVisitor < Visitor

    attr_reader :q def initialize @q = PP.new(+"") end def visit_sentence(node) q.group(2, "sentence(", ")") do visit(node.phrase) visit(node.punctuation) end end end visitor = PrettyPrintVisitor.new node.accept(visitor) visitor.q.flush puts visitor.q.output
  90. Parser · Syntax tree · Visitor class FormatterVisitor < Visitor

    attr_reader :q def initialize @q = PP.new(+"") end def visit_sentence(node) q.group do q.format(node.phrase) q.format(node.punctuation) end end end visitor = FormatterVisitor.new node.accept(visitor) visitor.q.flush puts visitor.q.output
  91. What is Syntax Tree?

  92. Syntax Tree

  93. Object layer representing the result of parsing Ruby Syntax Tree

  94. Object layer representing the result of parsing Ruby Tools to

    interact with and manipulate that layer Syntax Tree
  95. Build a syntax tree Syntax Tree

  96. Build a syntax tree Format the syntax tree Syntax Tree

  97. Build a syntax tree Format the syntax tree Create a

    CLI Syntax Tree
  98. Build a syntax tree Format the syntax tree Create a

    CLI Create a language server (LSP) Syntax Tree
  99. Build a syntax tree Format the syntax tree Create a

    CLI Create a language server (LSP) Translate the syntax tree Syntax Tree
  100. Build · Format · CLI · LSP · Translate Syntax

    Tree
  101. Build a syntax tree Build · Format · CLI ·

    LSP · Translate
  102. Define each node Build · Format · CLI · LSP

    · Translate
  103. Build · Format · CLI · LSP · Translate class

    VCall < Node attr_reader :value, :location, :comments def initialize(value:, location:, comments: []) @value = value @location = location @comments = comments end def accept(visitor) visitor.visit_vcall(self) end def child_nodes [value] end alias deconstruct child_nodes def deconstruct_keys(keys) { value: value, location: location, comments: comments } end end
  104. Build · Format · CLI · LSP · Translate class

    VCall < Node attr_reader :value, :location, :comments def initialize(value:, location:, comments: []) @value = value @location = location @comments = comments end def accept(visitor) visitor.visit_vcall(self) end def child_nodes [value] end alias deconstruct child_nodes def deconstruct_keys(keys) { value: value, location: location, comments: comments } end end Named fields
  105. Build · Format · CLI · LSP · Translate class

    VCall < Node attr_reader :value, :location, :comments def initialize(value:, location:, comments: []) @value = value @location = location @comments = comments end def accept(visitor) visitor.visit_vcall(self) end def child_nodes [value] end alias deconstruct child_nodes def deconstruct_keys(keys) { value: value, location: location, comments: comments } end end Named fields #location/#comments
  106. Build · Format · CLI · LSP · Translate class

    VCall < Node attr_reader :value, :location, :comments def initialize(value:, location:, comments: []) @value = value @location = location @comments = comments end def accept(visitor) visitor.visit_vcall(self) end def child_nodes [value] end alias deconstruct child_nodes def deconstruct_keys(keys) { value: value, location: location, comments: comments } end end Named fields #location/#comments #accept/#child_nodes
  107. Build · Format · CLI · LSP · Translate class

    VCall < Node attr_reader :value, :location, :comments def initialize(value:, location:, comments: []) @value = value @location = location @comments = comments end def accept(visitor) visitor.visit_vcall(self) end def child_nodes [value] end alias deconstruct child_nodes def deconstruct_keys(keys) { value: value, location: location, comments: comments } end end Named fields #location/#comments #accept/#child_nodes #deconstruct/ 
 #deconstruct_keys
  108. Build · Format · CLI · LSP · Translate class

    VCall < Node attr_reader :value, :location, :comments def initialize(value:, location:, comments: []) @value = value @location = location @comments = comments end def accept(visitor) visitor.visit_vcall(self) end def child_nodes [value] end alias deconstruct child_nodes def deconstruct_keys(keys) { value: value, location: location, comments: comments } end end Named fields #location/#comments #accept/#child_nodes #deconstruct/ 
 #deconstruct_keys Immutable
  109. Define each node Build · Format · CLI · LSP

    · Translate
  110. Ripper::EVENTS.count # => 190 Define each node Build · Format

    · CLI · LSP · Translate
  111. Ripper::EVENTS.count # => 190 SyntaxTree::Node.descendants.count 
 # => 162 Define

    each node Build · Format · CLI · LSP · Translate
  112. Ripper::EVENTS.count # => 190 SyntaxTree::Node.descendants.count 
 # => 162 qwords_new/qwords_add

    -> SyntaxTree::QWords Define each node Build · Format · CLI · LSP · Translate
  113. How we use ripper Build · Format · CLI ·

    LSP · Translate
  114. var_ref : user_variable { /*%%%*/ if (!($$ = gettable(p, $1,

    &@$))) $$ = NEW_BEGIN(0, &@$); /*% if (id_is_var(p, get_id($1))) { $$ = dispatch1(var_ref, $1); } else { $$ = dispatch1(vcall, $1); } %*/ } | keyword_variable { /*%%%*/ if (!($$ = gettable(p, $1, &@$))) $$ = NEW_BEGIN(0, &@$); /*% %*/ /*% ripper: var_ref!($1) %*/ } ; Build · Format · CLI · LSP · Translate
  115. var_ref : user_variable { /*%%%*/ if (!($$ = gettable(p, $1,

    &@$))) $$ = NEW_BEGIN(0, &@$); /*% if (id_is_var(p, get_id($1))) { $$ = dispatch1(var_ref, $1); } else { $$ = dispatch1(vcall, $1); } %*/ } | keyword_variable { /*%%%*/ if (!($$ = gettable(p, $1, &@$))) $$ = NEW_BEGIN(0, &@$); /*% %*/ /*% ripper: var_ref!($1) %*/ } ; Build · Format · CLI · LSP · Translate
  116. class Foo def bar baz end end Build · Format

    · CLI · LSP · Translate
  117. class Foo def bar baz end end Build · Format

    · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
  118. class Foo def bar baz end end Build · Format

    · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
  119. class Foo def bar baz end end Build · Format

    · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
  120. class Foo def bar baz end end Build · Format

    · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
  121. class Foo def bar baz end end Build · Format

    · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
  122. class Foo def bar baz end end Build · Format

    · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
  123. class Foo def bar baz end end Build · Format

    · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
  124. class Foo def bar baz end end Build · Format

    · CLI · LSP · Translate class Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
  125. Build · Format · CLI · LSP · Translate class

    Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) [:class, [const, superclass, bodystmt]] end def on_def(name, params, bodystmt) [:def, [name, params, bodystmt]] end end
  126. Build · Format · CLI · LSP · Translate class

    Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_kw(value) = @stack << [:@kw, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) kw = @stack.rfind { |token| token in [:@kw, "class"] } [:class, [@stack.delete(kw), const, superclass, bodystmt]] end def on_def(name, params, bodystmt) kw = @stack.rfind { |token| token in [:@kw, "def"] } [:def, [@stack.delete(kw), name, params, bodystmt]] end end
  127. Build · Format · CLI · LSP · Translate class

    Parser < Ripper def on_const(value) = [:@const, value] def on_ident(value) = [:@ident, value] def on_kw(value) = @stack << [:@kw, value] def on_const_ref(const) = [:const_ref, const] def on_vcall(ident) = [:vcall, ident] def on_class(const, superclass, bodystmt) kw = @stack.rfind { |token| token in [:@kw, "class"] } [:class, [@stack.delete(kw), const, superclass, bodystmt]] end def on_def(name, params, bodystmt) kw = @stack.rfind { |token| token in [:@kw, "def"] } [:def, [@stack.delete(kw), name, params, bodystmt]] end end
  128. Build · Format · CLI · LSP · Translate ->

    (positional; local1, local2) {}
  129. Build · Format · CLI · LSP · Translate ->

    (positional; local1, local2) {} range = (location.start_char + 1)...location.end_char locals = lambda_locals(source[range]) Paren.new( lparen: params.lparen, contents: LambdaVar.new( params: params.contents, locals: locals, location: location ), location: params.location, comments: params.comments )
  130. class Foo def bar baz end end Build · Format

    · CLI · LSP · Translate
  131. Build · Format · CLI · LSP · Translate class

    Foo # comment def bar baz end end
  132. Build · Format · CLI · LSP · Translate class

    Foo def bar # comment baz end end
  133. Build · Format · CLI · LSP · Translate class

    Foo def bar baz # comment end end
  134. Build · Format · CLI · LSP · Translate #

    comment class Foo def bar baz end end
  135. Build · Format · CLI · LSP · Translate class

    Foo # comment def bar baz end end
  136. Build · Format · CLI · LSP · Translate #

    comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end
  137. Build · Format · CLI · LSP · Translate #

    comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end [ ]
  138. Build · Format · CLI · LSP · Translate #

    comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end [ # comment1, ]
  139. Build · Format · CLI · LSP · Translate #

    comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end [ # comment1, # comment2, ]
  140. Build · Format · CLI · LSP · Translate #

    comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end [ # comment1, # comment2, # comment3, ]
  141. Build · Format · CLI · LSP · Translate #

    comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end [ # comment1, # comment2, # comment3, # comment4, ]
  142. Build · Format · CLI · LSP · Translate #

    comment1 class Foo # comment2 # comment3 def bar # comment4 baz # comment5 end end [ # comment1, # comment2, # comment3, # comment4, # comment5 ]
  143. Build · Format · CLI · LSP · Translate [

    # comment1, # comment2, # comment3, # comment4, # comment5 ] (program (statements ((class (const_ref (const "Foo")) (bodystmt (statements ((void_stmt), (def (ident "bar") (params) (bodystmt (statements ((vcall (ident "baz")))))))))))))
  144. Build · Format · CLI · LSP · Translate (program

    (statements ((comment "# comment1"), (class (const_ref (const "Foo") ((comment "# comment2"))) (bodystmt (statements ((void_stmt), (comment "# comment3"), (def (ident "bar") (params ((comment "# comment4"))) (bodystmt (statements ((vcall (ident "baz") ((comment "# comment5"))))))))))))))
  145. Walking the tree Build · Format · CLI · LSP

    · Translate
  146. Build · Format · CLI · LSP · Translate class

    Visitor def visit(node) node&.accept(self) end def visit_child_nodes(node) node.child_nodes.map do |node| visit(node) end end end
  147. Build · Format · CLI · LSP · Translate class

    VCall < Node def accept(visitor) visitor.visit_vcall(self) end end class Visitor def visit(node) node&.accept(self) end def visit_child_nodes(node) node.child_nodes.map do |node| visit(node) end end end
  148. Build · Format · CLI · LSP · Translate class

    VCall < Node def accept(visitor) visitor.visit_vcall(self) end end # Visit an ARef node. alias visit_aref visit_child_nodes # Visit an ARefField node. alias visit_aref_field visit_child_nodes # Visit an Alias node. alias visit_alias visit_child_nodes # Visit an ArgBlock node. alias visit_arg_block visit_child_nodes # Visit an ArgParen node. alias visit_arg_paren visit_child_nodes # Visit an ArgStar node. alias visit_arg_star visit_child_nodes # Visit an Args node. alias visit_args visit_child_nodes # Visit an ArgsForward node. alias visit_args_forward visit_child_nodes # Visit an ArrayLiteral node. alias visit_array visit_child_nodes # Visit an AryPtn node. alias visit_aryptn visit_child_nodes # Visit an Assign node. alias visit_assign visit_child_nodes # Visit an Assoc node. alias visit_assoc visit_child_nodes # Visit an AssocSplat node. alias visit_assoc_splat visit_child_nodes # Visit a Backref node. alias visit_backref visit_child_nodes class Visitor def visit(node) node&.accept(self) end def visit_child_nodes(node) node.child_nodes.map do |node| visit(node) end end end
  149. Build · Format · CLI · LSP · Translate class

    VCall < Node def accept(visitor) visitor.visit_vcall(self) end end alias visit_vcall visit_child_nodes # Visit a VoidStmt node. alias visit_void_stmt visit_child_nodes # Visit a When node. alias visit_when visit_child_nodes # Visit a While node. alias visit_while visit_child_nodes # Visit a WhileMod node. alias visit_while_mod visit_child_nodes # Visit a Word node. alias visit_word visit_child_nodes # Visit a Words node. alias visit_words visit_child_nodes # Visit a WordsBeg node. alias visit_words_beg visit_child_nodes # Visit a XString node. alias visit_xstring visit_child_nodes # Visit a XStringLiteral node. alias visit_xstring_literal visit_child_nodes # Visit a Yield node. alias visit_yield visit_child_nodes # Visit a Yield0 node. alias visit_yield0 visit_child_nodes # Visit a ZSuper node. alias visit_zsuper visit_child_nodes # Visit an EndContent node. alias visit___end__ visit_child_nodes class Visitor def visit(node) node&.accept(self) end def visit_child_nodes(node) node.child_nodes.map do |node| visit(node) end end end
  150. Build · Format · CLI · LSP · Translate class

    RegexpVisitor < Visitor def visit_regexp_literal(node) # do something with regexp literal node here node.options # call super to continue walking the tree super end end
  151. Build · Format · CLI · LSP · Translate class

    JSONVisitor < Visitor def visit_aref(node) node(node, "aref") do field("collection", node.collection) field("index", node.index) comments(node) end end def visit_aref_field(node) node(node, "aref_field") do field("collection", node.collection) field("index", node.index) comments(node) end end def visit_alias(node) node(node, "alias") do field("left", node.left) field("right", node.right) comments(node) end end
  152. Format the syntax tree Build · Format · CLI ·

    LSP · Translate
  153. Algorithm Build · Format · CLI · LSP · Translate

  154. prettyprint Algorithm Build · Format · CLI · LSP ·

    Translate
  155. prettyprint Build group, text, breakable to layout content Algorithm Build

    · Format · CLI · LSP · Translate
  156. prettyprint Build group, text, breakable to layout content Break outermost

    groups to fit to print width Algorithm Build · Format · CLI · LSP · Translate
  157. Build · Format · CLI · LSP · Translate [foo1,

    foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2]]
  158. Build · Format · CLI · LSP · Translate [foo1,

    foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2]]
  159. Build · Format · CLI · LSP · Translate [foo1,

    foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2]]
  160. Build · Format · CLI · LSP · Translate [

    foo1, foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2] ]
  161. Build · Format · CLI · LSP · Translate [

    foo1, foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2] ]
  162. Build · Format · CLI · LSP · Translate [

    foo1, foo2, [ bar1, bar2, bar3, bar4 ], foo3, foo4, [baz1, baz2] ]
  163. breakGroup([group([text(["["]), indent([breakable, text(["foo1", ","]), breakable, text(["foo2", ","]), breakable, group([text(["["]), indent([breakable,

    text(["bar1", ","]), breakable, text(["bar2", ","]), breakable, text(["bar3", ","]), breakable, text(["bar4"])]), breakable, text(["]"])]), text([","]), breakable, text(["foo3", ","]), breakable, text(["foo4", ","]), breakable, ... Build · Format · CLI · LSP · Translate
  164. breakGroup([group([text(["["]), indent([breakable, text(["foo1", ","]), breakable, text(["foo2", ","]), breakable, group([text(["["]), indent([breakable,

    text(["bar1", ","]), breakable, text(["bar2", ","]), breakable, text(["bar3", ","]), breakable, text(["bar4"])]), breakable, text(["]"])]), text([","]), breakable, text(["foo3", ","]), breakable, text(["foo4", ","]), breakable, ... Build · Format · CLI · LSP · Translate
  165. breakGroup([group([text(["["]), indent([breakable, text(["foo1", ","]), breakable, text(["foo2", ","]), breakable, group([text(["["]), indent([breakable,

    text(["bar1", ","]), breakable, text(["bar2", ","]), breakable, text(["bar3", ","]), breakable, text(["bar4"])]), breakable, text(["]"])]), text([","]), breakable, text(["foo3", ","]), breakable, text(["foo4", ","]), breakable, ... Build · Format · CLI · LSP · Translate
  166. breakGroup([group([text(["["]), indent([breakable, text(["foo1", ","]), breakable, text(["foo2", ","]), breakable, group([text(["["]), indent([breakable,

    text(["bar1", ","]), breakable, text(["bar2", ","]), breakable, text(["bar3", ","]), breakable, text(["bar4"])]), breakable, text(["]"])]), text([","]), breakable, text(["foo3", ","]), breakable, text(["foo4", ","]), breakable, ... Build · Format · CLI · LSP · Translate
  167. breakGroup([group([text(["["]), indent([breakable, text(["foo1", ","]), breakable, text(["foo2", ","]), breakable, group([text(["["]), indent([breakable,

    text(["bar1", ","]), breakable, text(["bar2", ","]), breakable, text(["bar3", ","]), breakable, text(["bar4"])]), breakable, text(["]"])]), text([","]), breakable, text(["foo3", ","]), breakable, text(["foo4", ","]), breakable, ... Build · Format · CLI · LSP · Translate
  168. I made too many additions to prettyprint Build · Format

    · CLI · LSP · Translate 😅😅😅
  169. prettyprint Build · Format · CLI · LSP · Translate

  170. prettier_print Build · Format · CLI · LSP · Translate

  171. Build · Format · CLI · LSP · Translate if

    foo bar += baz end
  172. Build · Format · CLI · LSP · Translate if

    foo bar += baz end
  173. Build · Format · CLI · LSP · Translate bar

    += baz if foo
  174. Build · Format · CLI · LSP · Translate if

    foo bar else baz end
  175. Build · Format · CLI · LSP · Translate if

    foo bar else baz end
  176. Build · Format · CLI · LSP · Translate foo

    ? bar : baz
  177. Build · Format · CLI · LSP · Translate here_is_a_method_call(1,

    2, <<~HEREDOC this is the content of the heredoc HEREDOC )
  178. Build · Format · CLI · LSP · Translate here_is_a_method_call(1,

    2, <<~HEREDOC this is the content of the heredoc HEREDOC )
  179. Build · Format · CLI · LSP · Translate here_is_a_method_call(1,

    2, <<~HEREDOC) this is the content of the heredoc HEREDOC
  180. Build · Format · CLI · LSP · Translate def

    foo return [1, 2, 3].map { |element| element * 2 } # comment end
  181. Build · Format · CLI · LSP · Translate def

    foo return [1, 2, 3].map { |element| element * 2 } # comment end
  182. Build · Format · CLI · LSP · Translate def

    foo return( [1, 2, 3].map do |element| element * 2 end ) # comment end
  183. Build · Format · CLI · LSP · Translate class

    If < Node def format(q) if node.consequent || node.statements.empty? q.group { format_break(q, force: true) } else q.group do q .if_break { format_break(q, force: false) } .if_flat do Parentheses.flat(q) do q.format(node.statements) q.text(" if ") q.format(node.predicate) end end end end end end
  184. Build · Format · CLI · LSP · Translate class

    If < Node def format(q) if node.consequent || node.statements.empty? q.group { format_break(q, force: true) } else q.group do q .if_break { format_break(q, force: false) } .if_flat do Parentheses.flat(q) do q.format(node.statements) q.text(" if ") q.format(node.predicate) end end end end end end
  185. Create a CLI Build · Format · CLI · LSP

    · Translate
  186. Build · Format · CLI · LSP · Translate stree

    ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree check [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Check that the given files are formatted as syntax tree would format them stree debug [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Check that the given files can be formatted idempotently stree doc [--plugins=...] [-e SCRIPT] FILE Print out the doc tree that would be used to format the given files stree expr [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the first expression of the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree json [--plugins=...] [-e SCRIPT] FILE Print out the JSON representation of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree help Display this help message stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files stree version Output the current version of syntax tree stree write [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Read, format, and write back the source of the given files --plugins=... A comma-separated list of plugins to load. --print-width=NUMBER The maximum line width to use when formatting. -e SCRIPT Parse an inline Ruby string.
  187. Build · Format · CLI · LSP · Translate stree

    ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
  188. Build · Format · CLI · LSP · Translate stree

    ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
  189. Build · Format · CLI · LSP · Translate $

    stree ast -e '1+2'
  190. Build · Format · CLI · LSP · Translate $

    stree ast -e '1+2' (program (statements ((binary (int "1") + (int "2")))))
  191. Build · Format · CLI · LSP · Translate stree

    ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
  192. Build · Format · CLI · LSP · Translate stree

    ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
  193. Build · Format · CLI · LSP · Translate $

    stree format -e '1+2'
  194. Build · Format · CLI · LSP · Translate $

    stree format -e '1+2' 1 + 2
  195. Build · Format · CLI · LSP · Translate stree

    ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
  196. Build · Format · CLI · LSP · Translate stree

    ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
  197. Build · Format · CLI · LSP · Translate $

    stree match -e '1+2'
  198. Build · Format · CLI · LSP · Translate $

    stree match -e '1+2' SyntaxTree::Program[ statements: SyntaxTree::Statements[ body: [ SyntaxTree::Binary[ left: SyntaxTree::Int[value: "1"], operator: :+, right: SyntaxTree::Int[value: "2"] ] ] ] ]
  199. Build · Format · CLI · LSP · Translate stree

    ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
  200. Build · Format · CLI · LSP · Translate stree

    ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files
  201. Build · Format · CLI · LSP · Translate $

    stree search Int -e '1+2'
  202. Build · Format · CLI · LSP · Translate $

    stree search Int -e '1+2' script:1:0: 1+2 script:1:2: 1+2
  203. Build · Format · CLI · LSP · Translate $

    stree search Int -e '1+2'
  204. Build · Format · CLI · LSP · Translate $

    stree search 'Int[value: "2"]' -e '1+2'
  205. Build · Format · CLI · LSP · Translate $

    stree search 'Int[value: "2"]' -e '1+2' 
 
 script:1:2: 1+2
  206. Build · Format · CLI · LSP · Translate $

    stree search <(stree match -e '1+2') \ 
 -e '1+2'
  207. Build · Format · CLI · LSP · Translate $

    stree search <(stree match -e '1+2') \ 
 -e '1+2' 
 
 script:1:0: 1+2
  208. Build · Format · CLI · LSP · Translate stree

    ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files stree write [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Read, format, and write back the source of the given files
  209. Build · Format · CLI · LSP · Translate stree

    ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the AST corresponding to the given files stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Print out the formatted version of the given files stree match [--plugins=...] [-e SCRIPT] FILE Print out a pattern-matching Ruby expression that would match the given files stree lsp [--plugins=...] [--print-width=NUMBER] Run syntax tree in language server mode stree search PATTERN [-e SCRIPT] FILE Search for the given pattern in the given files stree write [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE Read, format, and write back the source of the given files
  210. Create a language server Build · Format · CLI ·

    LSP · Translate
  211. Language server protocol Build · Format · CLI · LSP

    · Translate
  212. Language server protocol Runs a JSON RPC server behind an

    editor Build · Format · CLI · LSP · Translate
  213. Language server protocol Runs a JSON RPC server behind an

    editor textDocument/didChange textDocument/didOpen textDocument/didClose Build · Format · CLI · LSP · Translate
  214. Language server protocol Runs a JSON RPC server behind an

    editor textDocument/didChange textDocument/didOpen textDocument/didClose textDocument/formatting Build · Format · CLI · LSP · Translate
  215. Language server protocol Runs a JSON RPC server behind an

    editor textDocument/didChange textDocument/didOpen textDocument/didClose textDocument/formatting textDocument/inlayHint Build · Format · CLI · LSP · Translate
  216. Build · Format · CLI · LSP · Translate

  217. Shopify/ruby-lsp Build · Format · CLI · LSP · Translate

  218. Shopify/ruby-lsp Document highlight Build · Format · CLI · LSP

    · Translate
  219. Shopify/ruby-lsp Document highlight Document symbols Build · Format · CLI

    · LSP · Translate
  220. Shopify/ruby-lsp Document highlight Document symbols Folding ranges Build · Format

    · CLI · LSP · Translate
  221. Shopify/ruby-lsp Document highlight Document symbols Folding ranges Formatting Build ·

    Format · CLI · LSP · Translate
  222. Shopify/ruby-lsp Document highlight Document symbols Folding ranges Formatting Selection ranges

    Build · Format · CLI · LSP · Translate
  223. Shopify/ruby-lsp Document highlight Document symbols Folding ranges Formatting Selection ranges

    Semantic highlighting Build · Format · CLI · LSP · Translate
  224. Translate the syntax tree Build · Format · CLI ·

    LSP · Translate
  225. Translate the syntax tree Build · Format · CLI ·

    LSP · Translate
  226. Translate the syntax tree seattlerb/ruby_parser Build · Format · CLI

    · LSP · Translate
  227. Build · Format · CLI · LSP · Translate class

    RubyParserVisitor < Visitor def visit_binary(node) left = visit(node.left) right = visit(node.right) case node in { operator: :and } s(:and, left, right) in { operator: :!~ } s(:not, s(:call, left, :=~, right)) else s(:call, left, node.operator, right) end end # ... end
  228. Translate the syntax tree seattlerb/ruby_parser Build · Format · CLI

    · LSP · Translate
  229. Translate the syntax tree seattlerb/ruby_parser whitequark/parser Build · Format ·

    CLI · LSP · Translate
  230. Build · Format · CLI · LSP · Translate class

    ParserVisitor < Visitor def visit_binary(node) left = visit(node.left) right = visit(node.right) case node in { operator: :and } s(:and, [left, right]) in { operator: :or } s(:or, [left, right]) else s(:send, [left, node.operator, right]) end end # ... end
  231. Translate the syntax tree seattlerb/ruby_parser whitequark/parser Build · Format ·

    CLI · LSP · Translate
  232. Translate the syntax tree seattlerb/ruby_parser whitequark/parser YARV Build · Format

    · CLI · LSP · Translate
  233. Build · Format · CLI · LSP · Translate class

    YARVVisitor < Visitor def visit_binary(node) case node.operator when :"&&" # ... when :"||" # ... else visit(node.left) visit(node.right) builder.send( node.operator, 1, VM_CALL_ARGS_SIMPLE ) end end # ... end
  234. Translate the syntax tree seattlerb/ruby_parser whitequark/parser YARV Build · Format

    · CLI · LSP · Translate
  235. Translate the syntax tree seattlerb/ruby_parser whitequark/parser YARV Ruby Build ·

    Format · CLI · LSP · Translate
  236. Build · Format · CLI · LSP · Translate class

    MutationVisitor < Visitor def visit_binary(node) node.copy( left: visit(node.left), right: visit(node.right) ) end end
  237. Build · Format · CLI · LSP · Translate mutator

    = SyntaxTree.mutation do |mutator| query = <<~RUBY IfNode[predicate: Assign | OpAssign] | UnlessNode[predicate: Assign | OpAssign] RUBY mutator.mutate(query) do |node| node.copy(predicate: SyntaxTree::Paren.new( lparen: SyntaxTree::LParen.default, contents: node.predicate, location: node.predicate.location )) end end mutated = node.accept(mutator)
  238. Translate the syntax tree seattlerb/ruby_parser whitequark/parser YARV Ruby Build ·

    Format · CLI · LSP · Translate
  239. Syntax Tree

  240. Parser Syntax Tree

  241. Parser Syntax tree Syntax Tree

  242. Parser Syntax tree Visitor Syntax Tree

  243. Syntax Tree

  244. Syntax Tree Build a syntax tree

  245. Syntax Tree Build a syntax tree Format the syntax tree

  246. Syntax Tree Build a syntax tree Format the syntax tree

    Create a CLI
  247. Syntax Tree Build a syntax tree Format the syntax tree

    Create a CLI Create a language server (LSP)
  248. Syntax Tree Build a syntax tree Format the syntax tree

    Create a CLI Create a language server (LSP) Translate the syntax tree
  249. Thank you!

  250. Syntax Tree github.com/ruby-syntax-tree Kevin Newton @kddnewton