Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Syntax Tree (RubyConf Mini)

Kevin Newton
November 16, 2022
79

Syntax Tree (RubyConf Mini)

Syntax Tree is a new toolkit for interacting with the Ruby parse tree. It can be used to analyze, inspect, debug, and format your Ruby code. In this talk we'll walk through how it works, how to use it in your own applications, and the exciting future possibilities enabled by Syntax Tree.

Kevin Newton

November 16, 2022
Tweet

Transcript

  1. Kevin Newton
    @kddnewton

    View full-size slide

  2. What is Syntax Tree?

    View full-size slide

  3. Syntax Tree concepts

    View full-size slide

  4. Parser
    Syntax Tree concepts

    View full-size slide

  5. Parser


    Syntax tree
    Syntax Tree concepts

    View full-size slide

  6. Parser


    Syntax tree


    Visitor
    Syntax Tree concepts

    View full-size slide

  7. Parser · Syntax tree · Visitor

    View full-size slide

  8. Parser · Syntax tree · Visitor
    Parser

    View full-size slide

  9. Matz is nice so we are nice.
    Parser · Syntax tree · Visitor

    View full-size slide

  10. Lexical analysis
    Parser · Syntax tree · Visitor

    View full-size slide

  11. Matz is nice so we are nice.
    Parser · Syntax tree · Visitor

    View full-size slide

  12. Matz is nice so
    we are nice .
    Parser · Syntax tree · Visitor

    View full-size slide

  13. is nice so
    we are nice .
    Matz
    Parser · Syntax tree · Visitor

    View full-size slide

  14. is nice so
    we are nice .
    Matz
    Parser · Syntax tree · Visitor

    View full-size slide

  15. Matz is nice
    we are nice .
    so
    Parser · Syntax tree · Visitor

    View full-size slide

  16. Matz is nice so
    we are nice .
    Parser · Syntax tree · Visitor

    View full-size slide

  17. Matz is nice so
    we are nice .
    Parser · Syntax tree · Visitor

    View full-size slide

  18. Matz is nice so
    we are .
    nice
    Parser · Syntax tree · Visitor

    View full-size slide

  19. Matz is nice so
    we are nice .
    Parser · Syntax tree · Visitor

    View full-size slide

  20. Matz is nice so
    we are nice .
    Parser · Syntax tree · Visitor

    View full-size slide

  21. Semantic analysis
    Parser · Syntax tree · Visitor

    View full-size slide

  22. Parser · Syntax tree · Visitor
    is nice are nice

    View full-size slide

  23. Grammar
    Parser · Syntax tree · Visitor

    View full-size slide

  24. Grammar
    Parser · Syntax tree · Visitor
    verb-phrase : VERB ADJECTIVE

    View full-size slide

  25. Parser · Syntax tree · Visitor
    is nice are nice

    View full-size slide

  26. Parser · Syntax tree · Visitor
    is nice
    verb


    phrase
    verb


    phrase
    are nice

    View full-size slide

  27. Parser · Syntax tree · Visitor
    Matz
    is nice
    verb


    phrase
    we verb


    phrase
    are nice

    View full-size slide

  28. Grammar
    Parser · Syntax tree · Visitor
    verb-phrase : VERB ADJECTIVE

    View full-size slide

  29. Grammar
    Parser · Syntax tree · Visitor
    verb-phrase : VERB ADJECTIVE

    subject-phrase : NOUN verb-phrase

    View full-size slide

  30. Parser · Syntax tree · Visitor
    Matz
    is nice
    verb


    phrase
    we verb


    phrase
    are nice

    View full-size slide

  31. Parser · Syntax tree · Visitor
    Matz
    is nice
    subject


    phrase
    subject


    phrase
    verb


    phrase
    we verb


    phrase
    are nice

    View full-size slide

  32. Parser · Syntax tree · Visitor
    Matz
    is nice
    so
    subject


    phrase
    subject


    phrase
    verb


    phrase
    we verb


    phrase
    are nice

    View full-size slide

  33. Grammar
    Parser · Syntax tree · Visitor
    verb-phrase : VERB ADJECTIVE

    subject-phrase : NOUN verb-phrase

    View full-size slide

  34. Grammar
    Parser · Syntax tree · Visitor
    verb-phrase : VERB ADJECTIVE

    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase

    View full-size slide

  35. Parser · Syntax tree · Visitor
    Matz
    is nice
    so
    subject


    phrase
    subject


    phrase
    verb


    phrase
    we verb


    phrase
    are nice

    View full-size slide

  36. Parser · Syntax tree · Visitor
    Matz
    is nice
    so
    subord.


    conjunction
    subject


    phrase
    subject


    phrase
    verb


    phrase
    we verb


    phrase
    are nice

    View full-size slide

  37. Parser · Syntax tree · Visitor
    Matz
    is nice
    so
    .
    subord.


    conjunction
    subject


    phrase
    subject


    phrase
    verb


    phrase
    we verb


    phrase
    are nice

    View full-size slide

  38. verb-phrase : VERB ADJECTIVE

    subject-phrase : NOUN verb-phrase

    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase
    Grammar
    Parser · Syntax tree · Visitor

    View full-size slide

  39. verb-phrase : VERB ADJECTIVE

    subject-phrase : NOUN verb-phrase

    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase

    sentence : subordinating-conjunction PERIOD
    Grammar
    Parser · Syntax tree · Visitor

    View full-size slide

  40. Parser · Syntax tree · Visitor
    Matz
    is nice
    so
    .
    subord.


    conjunction
    subject


    phrase
    subject


    phrase
    verb


    phrase
    we verb


    phrase
    are nice

    View full-size slide

  41. Parser · Syntax tree · Visitor
    Matz
    is nice
    so
    .
    sentence
    subord.


    conjunction
    subject


    phrase
    subject


    phrase
    verb


    phrase
    we verb


    phrase
    are nice

    View full-size slide

  42. verb-phrase : VERB ADJECTIVE

    subject-phrase : NOUN verb-phrase

    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase

    sentence : subordinating-conjunction PERIOD
    Grammar
    Parser · Syntax tree · Visitor

    View full-size slide

  43. Parser · Syntax tree · Visitor
    Syntax tree

    View full-size slide

  44. verb-phrase : VERB ADJECTIVE

    subject-phrase : NOUN verb-phrase

    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase

    sentence : subordinating-conjunction PERIOD
    Grammar
    Parser · Syntax tree · Visitor

    View full-size slide

  45. Parser · Syntax tree · Visitor
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD

    View full-size slide

  46. Parser · Syntax tree · Visitor
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD

    View full-size slide

  47. Parser · Syntax tree · Visitor
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    View full-size slide

  48. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    View full-size slide

  49. Parser · Syntax tree · Visitor
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    View full-size slide

  50. Parser · Syntax tree · Visitor
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    View full-size slide

  51. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end

    View full-size slide

  52. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end

    View full-size slide

  53. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end

    View full-size slide

  54. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end
    class Verb


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end

    View full-size slide

  55. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end
    class Verb


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end

    View full-size slide

  56. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end
    class Verb


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Adjective


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end

    View full-size slide

  57. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end
    class Verb


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Adjective


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end

    View full-size slide

  58. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end
    class Verb


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Adjective


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Noun


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end

    View full-size slide

  59. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end
    class Verb


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Adjective


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Noun


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end

    View full-size slide

  60. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end
    class Verb


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Adjective


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Noun


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Conjunction


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end


    View full-size slide

  61. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end
    class Verb


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Adjective


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Noun


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Conjunction


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end


    View full-size slide

  62. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end
    class Verb


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Adjective


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Noun


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Conjunction


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end


    class Period


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end


    View full-size slide

  63. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    verb-phrase : VERB ADJECTIVE


    subject-phrase : NOUN verb-phrase


    subordinating-conjunction :

    subject-phrase CONJUNCTION subject-phrase


    sentence : subordinating-conjunction PERIOD
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end
    class Verb


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Adjective


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Noun


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Conjunction


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end


    class Period


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end


    View full-size slide

  64. Parser · Syntax tree · Visitor
    class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end
    class Verb


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Adjective


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Noun


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Conjunction


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end


    class Period


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end


    View full-size slide

  65. class SubjectPhrase


    attr_reader :noun, :verb_phrase


    def initialize(noun:, verb_phrase:)


    @noun = noun


    @verb_phrase = verb_phrase


    end


    end
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    class SubordinatingConjunction


    attr_reader :left, :conjunction, :right


    def initialize(left:, conjunction:, right:)


    @left = left


    @conjunction = conjunction


    @right = right


    end


    end
    class Sentence


    attr_reader :phrase, :punctuation


    def initialize(phrase:, punctuation:)


    @phrase = phrase


    @punctuation = punctuation


    end


    end
    class Verb


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Adjective


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Noun


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end
    class Conjunction


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end


    class Period


    attr_reader :value


    def initialize(value:)


    @value = value


    end


    end


    Parser · Syntax tree · Visitor

    View full-size slide

  66. class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    Parser · Syntax tree · Visitor

    View full-size slide

  67. Parser · Syntax tree · Visitor
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    def deconstruct_keys(keys)


    { verb: verb, adjective: adjective }


    end


    def ==(other)


    other in VerbPhrase[


    verb: ^(verb),


    adjective: ^(adjective)


    ]


    end


    end


    View full-size slide

  68. class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    Parser · Syntax tree · Visitor

    View full-size slide

  69. Parser · Syntax tree · Visitor
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    def copy(verb: self.verb, adjective: self.adjective)


    VerbPhrase.new(verb: verb, adjective: adjective)


    end


    end


    View full-size slide

  70. class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    end


    Parser · Syntax tree · Visitor

    View full-size slide

  71. Parser · Syntax tree · Visitor
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    def accept(visitor)


    visitor.visit_verb_phrase(self)


    end


    def child_nodes


    [verb, adjective]


    end


    end


    View full-size slide

  72. Parser · Syntax tree · Visitor
    class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    def accept(visitor)


    visitor.visit_verb_phrase(self)


    end


    def child_nodes


    [verb, adjective]


    end


    end


    View full-size slide

  73. Visitor
    Parser · Syntax tree · Visitor

    View full-size slide

  74. class VerbPhrase


    attr_reader :verb, :adjective


    def initialize(verb:, adjective:)


    @verb = verb


    @adjective = adjective


    end


    def accept(visitor)


    visitor.visit_verb_phrase(self)


    end


    def child_nodes


    [verb, adjective]


    end


    end


    Parser · Syntax tree · Visitor

    View full-size slide

  75. class VerbPhrase


    def accept(visitor)


    visitor.visit_verb_phrase(self)


    end


    end


    Parser · Syntax tree · Visitor

    View full-size slide

  76. class VerbPhrase


    def accept(visitor)


    visitor.visit_verb_phrase(self)


    end


    end



    class Visitor


    def visit_verb_phrase(node)


    # ...


    end


    end


    node.accept(Visitor.new)


    Parser · Syntax tree · Visitor

    View full-size slide

  77. class VerbPhrase


    def accept(visitor)


    visitor.visit_verb_phrase(self)


    end


    end



    class Visitor


    def visit_adjective(node) end


    def visit_conjunction(node) end


    def visit_noun(node) end


    def visit_verb(node) end


    def visit_punctuation(node) end


    def visit_sentence(node) end


    def visit_subordinating_conjunction(node) end


    def visit_subject_phrase(node) end


    def visit_verb_phrase(node)


    # ...


    end


    end


    node.accept(Visitor.new)


    Parser · Syntax tree · Visitor

    View full-size slide

  78. class Visitor


    def visit_adjective(node) end


    def visit_conjunction(node) end


    def visit_noun(node) end


    def visit_verb(node) end


    def visit_punctuation(node) end


    def visit_sentence(node) end


    def visit_subordinating_conjunction(node) end


    def visit_subject_phrase(node) end


    def visit_verb_phrase(node)


    # ...


    end


    end


    node.accept(Visitor.new)


    Parser · Syntax tree · Visitor

    View full-size slide

  79. class Visitor


    def visit(node)


    node.accept(self)


    end


    def visit_adjective(node) end


    def visit_conjunction(node) end


    def visit_noun(node) end


    def visit_verb(node) end


    def visit_punctuation(node) end


    def visit_sentence(node) end


    def visit_subordinating_conjunction(node) end


    def visit_subject_phrase(node) end


    def visit_verb_phrase(node)


    # ...


    end


    end


    node.accept(Visitor.new)
    Parser · Syntax tree · Visitor

    View full-size slide

  80. Parser · Syntax tree · Visitor
    class Visitor


    def visit(node)


    node.accept(self)


    end


    def visit_adjective(node) end


    def visit_conjunction(node) end


    def visit_noun(node) end


    def visit_verb(node) end


    def visit_punctuation(node) end


    def visit_sentence(node) end


    def visit_subordinating_conjunction(node) end


    def visit_subject_phrase(node) end


    def visit_verb_phrase(node)


    node.child_nodes.each { |node| visit(node) }


    end


    end


    node.accept(Visitor.new)


    View full-size slide

  81. Parser · Syntax tree · Visitor
    class Visitor


    def visit(node)


    node.accept(self)


    end


    def visit_adjective(node) end


    def visit_conjunction(node) end


    def visit_noun(node) end


    def visit_verb(node) end


    def visit_punctuation(node) end


    def visit_sentence(node) end


    def visit_subordinating_conjunction(node) end


    def visit_subject_phrase(node) end


    def visit_verb_phrase(node)


    node.child_nodes.each { |node| visit(node) }


    end


    end

    View full-size slide

  82. Parser · Syntax tree · Visitor
    class Visitor


    def visit(node)


    node.accept(self)


    end


    def visit_child_nodes(node)


    node.child_nodes.each { |node| visit(node) }


    end


    def visit_adjective(node) end


    def visit_conjunction(node) end


    def visit_noun(node) end


    def visit_verb(node) end


    def visit_punctuation(node) end


    def visit_sentence(node) end


    def visit_subordinating_conjunction(node) end


    def visit_subject_phrase(node) end


    def visit_verb_phrase(node)


    visit_child_nodes(node)


    end


    end


    View full-size slide

  83. Parser · Syntax tree · Visitor
    class Visitor


    def visit(node)


    node.accept(self)


    end


    def visit_child_nodes(node)


    node.child_nodes.each { |node| visit(node) }


    end


    def visit_adjective(node) end


    def visit_conjunction(node) end


    def visit_noun(node) end


    def visit_verb(node) end


    def visit_punctuation(node) end


    def visit_sentence(node) end


    def visit_subordinating_conjunction(node) end


    def visit_subject_phrase(node) end


    alias visit_verb_phrase visit_child_nodes


    end


    View full-size slide

  84. Parser · Syntax tree · Visitor
    class Visitor


    def visit(node)


    node.accept(self)


    end


    def visit_child_nodes(node)


    node.child_nodes.each { |node| visit(node) }


    end


    alias visit_adjective visit_child_nodes


    alias visit_conjunction visit_child_nodes


    alias visit_noun visit_child_nodes


    alias visit_verb visit_child_nodes


    alias visit_punctuation visit_child_nodes


    alias visit_sentence visit_child_nodes


    alias visit_subordinating_conjunction

    visit_child_nodes


    alias visit_subject_phrase visit_child_nodes


    alias visit_verb_phrase visit_child_nodes


    end


    View full-size slide

  85. Parser · Syntax tree · Visitor

    View full-size slide

  86. Parser · Syntax tree · Visitor
    class SentenceCountVisitor < Visitor


    attr_reader :count


    def initialize


    @count = 0


    end


    def visit_sentence(node)


    @count += 1


    super


    end


    end


    visitor = SentenceCountVisitor.new


    node.accept(visitor)


    puts visitor.count


    View full-size slide

  87. Parser · Syntax tree · Visitor
    class PrettyPrintVisitor < Visitor


    attr_reader :q


    def initialize


    @q = PP.new(+"")


    end


    def visit_sentence(node)


    q.group(2, "sentence(", ")") do


    visit(node.phrase)


    visit(node.punctuation)


    end


    end


    end


    visitor = PrettyPrintVisitor.new


    node.accept(visitor)


    visitor.q.flush


    puts visitor.q.output


    View full-size slide

  88. Parser · Syntax tree · Visitor
    class FormatterVisitor < Visitor


    attr_reader :q


    def initialize


    @q = PP.new(+"")


    end


    def visit_sentence(node)


    q.group do


    q.format(node.phrase)


    q.format(node.punctuation)


    end


    end


    end


    visitor = FormatterVisitor.new


    node.accept(visitor)


    visitor.q.flush


    puts visitor.q.output


    View full-size slide

  89. What is Syntax Tree?

    View full-size slide

  90. Object layer representing the result of parsing Ruby
    Syntax Tree

    View full-size slide

  91. Object layer representing the result of parsing Ruby


    Tools to interact with and manipulate that layer
    Syntax Tree

    View full-size slide

  92. Build a syntax tree
    Syntax Tree

    View full-size slide

  93. Build a syntax tree


    Format the syntax tree
    Syntax Tree

    View full-size slide

  94. Build a syntax tree


    Format the syntax tree


    Create a CLI
    Syntax Tree

    View full-size slide

  95. Build a syntax tree


    Format the syntax tree


    Create a CLI


    Create a language server (LSP)
    Syntax Tree

    View full-size slide

  96. Build a syntax tree


    Format the syntax tree


    Create a CLI


    Create a language server (LSP)


    Translate the syntax tree
    Syntax Tree

    View full-size slide

  97. Build · Format · CLI · LSP · Translate
    Syntax Tree

    View full-size slide

  98. Build a syntax tree
    Build · Format · CLI · LSP · Translate

    View full-size slide

  99. Define each node
    Build · Format · CLI · LSP · Translate

    View full-size slide

  100. Build · Format · CLI · LSP · Translate
    class VCall < Node


    attr_reader :value, :location, :comments


    def initialize(value:, location:, comments: [])


    @value = value


    @location = location


    @comments = comments


    end


    def accept(visitor)


    visitor.visit_vcall(self)


    end


    def child_nodes


    [value]


    end


    alias deconstruct child_nodes


    def deconstruct_keys(keys)


    { value: value, location: location, comments: comments }


    end


    end


    View full-size slide

  101. Build · Format · CLI · LSP · Translate
    class VCall < Node


    attr_reader :value, :location, :comments


    def initialize(value:, location:, comments: [])


    @value = value


    @location = location


    @comments = comments


    end


    def accept(visitor)


    visitor.visit_vcall(self)


    end


    def child_nodes


    [value]


    end


    alias deconstruct child_nodes


    def deconstruct_keys(keys)


    { value: value, location: location, comments: comments }


    end


    end


    Named fields

    View full-size slide

  102. Build · Format · CLI · LSP · Translate
    class VCall < Node


    attr_reader :value, :location, :comments


    def initialize(value:, location:, comments: [])


    @value = value


    @location = location


    @comments = comments


    end


    def accept(visitor)


    visitor.visit_vcall(self)


    end


    def child_nodes


    [value]


    end


    alias deconstruct child_nodes


    def deconstruct_keys(keys)


    { value: value, location: location, comments: comments }


    end


    end


    Named fields


    #location/#comments

    View full-size slide

  103. Build · Format · CLI · LSP · Translate
    class VCall < Node


    attr_reader :value, :location, :comments


    def initialize(value:, location:, comments: [])


    @value = value


    @location = location


    @comments = comments


    end


    def accept(visitor)


    visitor.visit_vcall(self)


    end


    def child_nodes


    [value]


    end


    alias deconstruct child_nodes


    def deconstruct_keys(keys)


    { value: value, location: location, comments: comments }


    end


    end


    Named fields


    #location/#comments


    #accept/#child_nodes

    View full-size slide

  104. Build · Format · CLI · LSP · Translate
    class VCall < Node


    attr_reader :value, :location, :comments


    def initialize(value:, location:, comments: [])


    @value = value


    @location = location


    @comments = comments


    end


    def accept(visitor)


    visitor.visit_vcall(self)


    end


    def child_nodes


    [value]


    end


    alias deconstruct child_nodes


    def deconstruct_keys(keys)


    { value: value, location: location, comments: comments }


    end


    end


    Named fields


    #location/#comments


    #accept/#child_nodes


    #deconstruct/

    #deconstruct_keys

    View full-size slide

  105. Build · Format · CLI · LSP · Translate
    class VCall < Node


    attr_reader :value, :location, :comments


    def initialize(value:, location:, comments: [])


    @value = value


    @location = location


    @comments = comments


    end


    def accept(visitor)


    visitor.visit_vcall(self)


    end


    def child_nodes


    [value]


    end


    alias deconstruct child_nodes


    def deconstruct_keys(keys)


    { value: value, location: location, comments: comments }


    end


    end


    Named fields


    #location/#comments


    #accept/#child_nodes


    #deconstruct/

    #deconstruct_keys


    Immutable

    View full-size slide

  106. Define each node
    Build · Format · CLI · LSP · Translate

    View full-size slide

  107. Ripper::EVENTS.count # => 190
    Define each node
    Build · Format · CLI · LSP · Translate

    View full-size slide

  108. Ripper::EVENTS.count # => 190


    SyntaxTree::Node.descendants.count

    # => 162
    Define each node
    Build · Format · CLI · LSP · Translate

    View full-size slide

  109. Ripper::EVENTS.count # => 190


    SyntaxTree::Node.descendants.count

    # => 162


    qwords_new/qwords_add ->
    SyntaxTree::QWords
    Define each node
    Build · Format · CLI · LSP · Translate

    View full-size slide

  110. How we use ripper
    Build · Format · CLI · LSP · Translate

    View full-size slide

  111. var_ref : user_variable


    {


    /*%%%*/


    if (!($$ = gettable(p, $1, &@$))) $$ = NEW_BEGIN(0, &@$);


    /*%


    if (id_is_var(p, get_id($1))) {


    $$ = dispatch1(var_ref, $1);


    }


    else {


    $$ = dispatch1(vcall, $1);


    }


    %*/


    }


    | keyword_variable


    {


    /*%%%*/


    if (!($$ = gettable(p, $1, &@$))) $$ = NEW_BEGIN(0, &@$);


    /*% %*/


    /*% ripper: var_ref!($1) %*/


    }


    ;


    Build · Format · CLI · LSP · Translate

    View full-size slide

  112. var_ref : user_variable


    {


    /*%%%*/


    if (!($$ = gettable(p, $1, &@$))) $$ = NEW_BEGIN(0, &@$);


    /*%


    if (id_is_var(p, get_id($1))) {


    $$ = dispatch1(var_ref, $1);


    }


    else {


    $$ = dispatch1(vcall, $1);


    }


    %*/


    }


    | keyword_variable


    {


    /*%%%*/


    if (!($$ = gettable(p, $1, &@$))) $$ = NEW_BEGIN(0, &@$);


    /*% %*/


    /*% ripper: var_ref!($1) %*/


    }


    ;


    Build · Format · CLI · LSP · Translate

    View full-size slide

  113. class Foo


    def bar


    baz


    end


    end


    Build · Format · CLI · LSP · Translate

    View full-size slide

  114. class Foo


    def bar


    baz


    end


    end


    Build · Format · CLI · LSP · Translate
    class Parser < Ripper


    def on_const(value) = [:@const, value]


    def on_ident(value) = [:@ident, value]


    def on_const_ref(const) = [:const_ref, const]


    def on_vcall(ident) = [:vcall, ident]


    def on_class(const, superclass, bodystmt)


    [:class, [const, superclass, bodystmt]]


    end


    def on_def(name, params, bodystmt)


    [:def, [name, params, bodystmt]]


    end


    end


    View full-size slide

  115. class Foo


    def bar


    baz


    end


    end


    Build · Format · CLI · LSP · Translate
    class Parser < Ripper


    def on_const(value) = [:@const, value]


    def on_ident(value) = [:@ident, value]


    def on_const_ref(const) = [:const_ref, const]


    def on_vcall(ident) = [:vcall, ident]


    def on_class(const, superclass, bodystmt)


    [:class, [const, superclass, bodystmt]]


    end


    def on_def(name, params, bodystmt)


    [:def, [name, params, bodystmt]]


    end


    end


    View full-size slide

  116. class Foo


    def bar


    baz


    end


    end


    Build · Format · CLI · LSP · Translate
    class Parser < Ripper


    def on_const(value) = [:@const, value]


    def on_ident(value) = [:@ident, value]


    def on_const_ref(const) = [:const_ref, const]


    def on_vcall(ident) = [:vcall, ident]


    def on_class(const, superclass, bodystmt)


    [:class, [const, superclass, bodystmt]]


    end


    def on_def(name, params, bodystmt)


    [:def, [name, params, bodystmt]]


    end


    end


    View full-size slide

  117. class Foo


    def bar


    baz


    end


    end


    Build · Format · CLI · LSP · Translate
    class Parser < Ripper


    def on_const(value) = [:@const, value]


    def on_ident(value) = [:@ident, value]


    def on_const_ref(const) = [:const_ref, const]


    def on_vcall(ident) = [:vcall, ident]


    def on_class(const, superclass, bodystmt)


    [:class, [const, superclass, bodystmt]]


    end


    def on_def(name, params, bodystmt)


    [:def, [name, params, bodystmt]]


    end


    end


    View full-size slide

  118. class Foo


    def bar


    baz


    end


    end


    Build · Format · CLI · LSP · Translate
    class Parser < Ripper


    def on_const(value) = [:@const, value]


    def on_ident(value) = [:@ident, value]


    def on_const_ref(const) = [:const_ref, const]


    def on_vcall(ident) = [:vcall, ident]


    def on_class(const, superclass, bodystmt)


    [:class, [const, superclass, bodystmt]]


    end


    def on_def(name, params, bodystmt)


    [:def, [name, params, bodystmt]]


    end


    end


    View full-size slide

  119. class Foo


    def bar


    baz


    end


    end


    Build · Format · CLI · LSP · Translate
    class Parser < Ripper


    def on_const(value) = [:@const, value]


    def on_ident(value) = [:@ident, value]


    def on_const_ref(const) = [:const_ref, const]


    def on_vcall(ident) = [:vcall, ident]


    def on_class(const, superclass, bodystmt)


    [:class, [const, superclass, bodystmt]]


    end


    def on_def(name, params, bodystmt)


    [:def, [name, params, bodystmt]]


    end


    end


    View full-size slide

  120. class Foo


    def bar


    baz


    end


    end


    Build · Format · CLI · LSP · Translate
    class Parser < Ripper


    def on_const(value) = [:@const, value]


    def on_ident(value) = [:@ident, value]


    def on_const_ref(const) = [:const_ref, const]


    def on_vcall(ident) = [:vcall, ident]


    def on_class(const, superclass, bodystmt)


    [:class, [const, superclass, bodystmt]]


    end


    def on_def(name, params, bodystmt)


    [:def, [name, params, bodystmt]]


    end


    end


    View full-size slide

  121. class Foo


    def bar


    baz


    end


    end


    Build · Format · CLI · LSP · Translate
    class Parser < Ripper


    def on_const(value) = [:@const, value]


    def on_ident(value) = [:@ident, value]


    def on_const_ref(const) = [:const_ref, const]


    def on_vcall(ident) = [:vcall, ident]


    def on_class(const, superclass, bodystmt)


    [:class, [const, superclass, bodystmt]]


    end


    def on_def(name, params, bodystmt)


    [:def, [name, params, bodystmt]]


    end


    end


    View full-size slide

  122. Build · Format · CLI · LSP · Translate
    class Parser < Ripper


    def on_const(value) = [:@const, value]


    def on_ident(value) = [:@ident, value]


    def on_const_ref(const) = [:const_ref, const]


    def on_vcall(ident) = [:vcall, ident]


    def on_class(const, superclass, bodystmt)


    [:class, [const, superclass, bodystmt]]


    end


    def on_def(name, params, bodystmt)


    [:def, [name, params, bodystmt]]


    end


    end


    View full-size slide

  123. Build · Format · CLI · LSP · Translate
    class Parser < Ripper


    def on_const(value) = [:@const, value]


    def on_ident(value) = [:@ident, value]


    def on_kw(value) = @stack << [:@kw, value]


    def on_const_ref(const) = [:const_ref, const]


    def on_vcall(ident) = [:vcall, ident]


    def on_class(const, superclass, bodystmt)


    kw = @stack.rfind { |token| token in [:@kw, "class"] }


    [:class, [@stack.delete(kw), const, superclass, bodystmt]]


    end


    def on_def(name, params, bodystmt)


    kw = @stack.rfind { |token| token in [:@kw, "def"] }


    [:def, [@stack.delete(kw), name, params, bodystmt]]


    end


    end


    View full-size slide

  124. Build · Format · CLI · LSP · Translate
    class Parser < Ripper


    def on_const(value) = [:@const, value]


    def on_ident(value) = [:@ident, value]


    def on_kw(value) = @stack << [:@kw, value]


    def on_const_ref(const) = [:const_ref, const]


    def on_vcall(ident) = [:vcall, ident]


    def on_class(const, superclass, bodystmt)


    kw = @stack.rfind { |token| token in [:@kw, "class"] }


    [:class, [@stack.delete(kw), const, superclass, bodystmt]]


    end


    def on_def(name, params, bodystmt)


    kw = @stack.rfind { |token| token in [:@kw, "def"] }


    [:def, [@stack.delete(kw), name, params, bodystmt]]


    end


    end


    View full-size slide

  125. Build · Format · CLI · LSP · Translate
    -> (positional; local1, local2) {}


    View full-size slide

  126. Build · Format · CLI · LSP · Translate
    -> (positional; local1, local2) {}


    range = (location.start_char + 1)...location.end_char


    locals = lambda_locals(source[range])


    Paren.new(


    lparen: params.lparen,


    contents:


    LambdaVar.new(


    params: params.contents,


    locals: locals,


    location: location


    ),


    location: params.location,


    comments: params.comments


    )


    View full-size slide

  127. class Foo


    def bar


    baz


    end


    end


    Build · Format · CLI · LSP · Translate

    View full-size slide

  128. Build · Format · CLI · LSP · Translate
    class Foo # comment


    def bar


    baz


    end


    end


    View full-size slide

  129. Build · Format · CLI · LSP · Translate
    class Foo


    def bar # comment


    baz


    end


    end


    View full-size slide

  130. Build · Format · CLI · LSP · Translate
    class Foo


    def bar


    baz # comment


    end


    end


    View full-size slide

  131. Build · Format · CLI · LSP · Translate
    # comment


    class Foo


    def bar


    baz


    end


    end


    View full-size slide

  132. Build · Format · CLI · LSP · Translate
    class Foo


    # comment


    def bar


    baz


    end


    end


    View full-size slide

  133. Build · Format · CLI · LSP · Translate
    # comment1


    class Foo # comment2


    # comment3


    def bar # comment4


    baz # comment5


    end


    end


    View full-size slide

  134. Build · Format · CLI · LSP · Translate
    # comment1


    class Foo # comment2


    # comment3


    def bar # comment4


    baz # comment5


    end


    end


    [


    ]

    View full-size slide

  135. Build · Format · CLI · LSP · Translate
    # comment1


    class Foo # comment2


    # comment3


    def bar # comment4


    baz # comment5


    end


    end


    [


    # comment1,


    ]

    View full-size slide

  136. Build · Format · CLI · LSP · Translate
    # comment1


    class Foo # comment2


    # comment3


    def bar # comment4


    baz # comment5


    end


    end


    [


    # comment1,


    # comment2,


    ]

    View full-size slide

  137. Build · Format · CLI · LSP · Translate
    # comment1


    class Foo # comment2


    # comment3


    def bar # comment4


    baz # comment5


    end


    end


    [


    # comment1,


    # comment2,


    # comment3,


    ]

    View full-size slide

  138. Build · Format · CLI · LSP · Translate
    # comment1


    class Foo # comment2


    # comment3


    def bar # comment4


    baz # comment5


    end


    end


    [


    # comment1,


    # comment2,


    # comment3,


    # comment4,


    ]

    View full-size slide

  139. Build · Format · CLI · LSP · Translate
    # comment1


    class Foo # comment2


    # comment3


    def bar # comment4


    baz # comment5


    end


    end


    [


    # comment1,


    # comment2,


    # comment3,


    # comment4,


    # comment5


    ]

    View full-size slide

  140. Build · Format · CLI · LSP · Translate
    [


    # comment1,


    # comment2,


    # comment3,


    # comment4,


    # comment5


    ]
    (program


    (statements


    ((class


    (const_ref (const "Foo"))


    (bodystmt


    (statements


    ((void_stmt),


    (def


    (ident "bar")


    (params)


    (bodystmt


    (statements


    ((vcall


    (ident "baz")))))))))))))


    View full-size slide

  141. Build · Format · CLI · LSP · Translate
    (program


    (statements


    ((comment "# comment1"),


    (class


    (const_ref (const "Foo") ((comment "# comment2")))


    (bodystmt


    (statements


    ((void_stmt),


    (comment "# comment3"),


    (def


    (ident "bar")


    (params ((comment "# comment4")))


    (bodystmt


    (statements


    ((vcall


    (ident "baz")


    ((comment "# comment5"))))))))))))))


    View full-size slide

  142. Walking the tree
    Build · Format · CLI · LSP · Translate

    View full-size slide

  143. Build · Format · CLI · LSP · Translate
    class Visitor


    def visit(node)


    node&.accept(self)


    end


    def visit_child_nodes(node)


    node.child_nodes.map do |node|


    visit(node)


    end


    end


    end


    View full-size slide

  144. Build · Format · CLI · LSP · Translate
    class VCall < Node


    def accept(visitor)


    visitor.visit_vcall(self)


    end


    end


    class Visitor


    def visit(node)


    node&.accept(self)


    end


    def visit_child_nodes(node)


    node.child_nodes.map do |node|


    visit(node)


    end


    end


    end


    View full-size slide

  145. Build · Format · CLI · LSP · Translate
    class VCall < Node


    def accept(visitor)


    visitor.visit_vcall(self)


    end


    end


    # Visit an ARef node.


    alias visit_aref visit_child_nodes


    # Visit an ARefField node.


    alias visit_aref_field visit_child_nodes


    # Visit an Alias node.


    alias visit_alias visit_child_nodes


    # Visit an ArgBlock node.


    alias visit_arg_block visit_child_nodes


    # Visit an ArgParen node.


    alias visit_arg_paren visit_child_nodes


    # Visit an ArgStar node.


    alias visit_arg_star visit_child_nodes


    # Visit an Args node.


    alias visit_args visit_child_nodes


    # Visit an ArgsForward node.


    alias visit_args_forward visit_child_nodes


    # Visit an ArrayLiteral node.


    alias visit_array visit_child_nodes


    # Visit an AryPtn node.


    alias visit_aryptn visit_child_nodes


    # Visit an Assign node.


    alias visit_assign visit_child_nodes


    # Visit an Assoc node.


    alias visit_assoc visit_child_nodes


    # Visit an AssocSplat node.


    alias visit_assoc_splat visit_child_nodes


    # Visit a Backref node.


    alias visit_backref visit_child_nodes


    class Visitor


    def visit(node)


    node&.accept(self)


    end


    def visit_child_nodes(node)


    node.child_nodes.map do |node|


    visit(node)


    end


    end


    end


    View full-size slide

  146. Build · Format · CLI · LSP · Translate
    class VCall < Node


    def accept(visitor)


    visitor.visit_vcall(self)


    end


    end



    alias visit_vcall visit_child_nodes


    # Visit a VoidStmt node.


    alias visit_void_stmt visit_child_nodes


    # Visit a When node.


    alias visit_when visit_child_nodes


    # Visit a While node.


    alias visit_while visit_child_nodes


    # Visit a WhileMod node.


    alias visit_while_mod visit_child_nodes


    # Visit a Word node.


    alias visit_word visit_child_nodes


    # Visit a Words node.


    alias visit_words visit_child_nodes


    # Visit a WordsBeg node.


    alias visit_words_beg visit_child_nodes


    # Visit a XString node.


    alias visit_xstring visit_child_nodes


    # Visit a XStringLiteral node.


    alias visit_xstring_literal visit_child_nodes


    # Visit a Yield node.


    alias visit_yield visit_child_nodes


    # Visit a Yield0 node.


    alias visit_yield0 visit_child_nodes


    # Visit a ZSuper node.


    alias visit_zsuper visit_child_nodes


    # Visit an EndContent node.


    alias visit___end__ visit_child_nodes


    class Visitor


    def visit(node)


    node&.accept(self)


    end


    def visit_child_nodes(node)


    node.child_nodes.map do |node|


    visit(node)


    end


    end


    end


    View full-size slide

  147. Build · Format · CLI · LSP · Translate
    class RegexpVisitor < Visitor


    def visit_regexp_literal(node)


    # do something with regexp literal node here


    node.options


    # call super to continue walking the tree


    super


    end


    end


    View full-size slide

  148. Build · Format · CLI · LSP · Translate
    class JSONVisitor < Visitor


    def visit_aref(node)


    node(node, "aref") do


    field("collection", node.collection)


    field("index", node.index)


    comments(node)


    end


    end


    def visit_aref_field(node)


    node(node, "aref_field") do


    field("collection", node.collection)


    field("index", node.index)


    comments(node)


    end


    end


    def visit_alias(node)


    node(node, "alias") do


    field("left", node.left)


    field("right", node.right)


    comments(node)


    end


    end

    View full-size slide

  149. Format the syntax tree
    Build · Format · CLI · LSP · Translate

    View full-size slide

  150. Algorithm
    Build · Format · CLI · LSP · Translate

    View full-size slide

  151. prettyprint
    Algorithm
    Build · Format · CLI · LSP · Translate

    View full-size slide

  152. prettyprint


    Build group, text, breakable to layout content
    Algorithm
    Build · Format · CLI · LSP · Translate

    View full-size slide

  153. prettyprint


    Build group, text, breakable to layout content


    Break outermost groups to fit to print width
    Algorithm
    Build · Format · CLI · LSP · Translate

    View full-size slide

  154. Build · Format · CLI · LSP · Translate
    [foo1, foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2]]


    View full-size slide

  155. Build · Format · CLI · LSP · Translate
    [foo1, foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2]]


    View full-size slide

  156. Build · Format · CLI · LSP · Translate
    [foo1, foo2, [bar1, bar2, bar3, bar4], foo3, foo4, [baz1, baz2]]


    View full-size slide

  157. Build · Format · CLI · LSP · Translate
    [


    foo1,


    foo2,


    [bar1, bar2, bar3, bar4],


    foo3,


    foo4,


    [baz1, baz2]


    ]


    View full-size slide

  158. Build · Format · CLI · LSP · Translate
    [


    foo1,


    foo2,


    [bar1, bar2, bar3, bar4],


    foo3,


    foo4,


    [baz1, baz2]


    ]


    View full-size slide

  159. Build · Format · CLI · LSP · Translate
    [


    foo1,


    foo2,


    [


    bar1,


    bar2,


    bar3,


    bar4


    ],


    foo3,


    foo4,


    [baz1, baz2]


    ]


    View full-size slide

  160. breakGroup([group([text(["["]),


    indent([breakable,


    text(["foo1", ","]),


    breakable,


    text(["foo2", ","]),


    breakable,


    group([text(["["]),


    indent([breakable,


    text(["bar1", ","]),


    breakable,


    text(["bar2", ","]),


    breakable,


    text(["bar3", ","]),


    breakable,


    text(["bar4"])]),


    breakable,


    text(["]"])]),


    text([","]),


    breakable,


    text(["foo3", ","]),


    breakable,


    text(["foo4", ","]),


    breakable,


    ...
    Build · Format · CLI · LSP · Translate

    View full-size slide

  161. breakGroup([group([text(["["]),


    indent([breakable,


    text(["foo1", ","]),


    breakable,


    text(["foo2", ","]),


    breakable,


    group([text(["["]),


    indent([breakable,


    text(["bar1", ","]),


    breakable,


    text(["bar2", ","]),


    breakable,


    text(["bar3", ","]),


    breakable,


    text(["bar4"])]),


    breakable,


    text(["]"])]),


    text([","]),


    breakable,


    text(["foo3", ","]),


    breakable,


    text(["foo4", ","]),


    breakable,


    ...
    Build · Format · CLI · LSP · Translate

    View full-size slide

  162. breakGroup([group([text(["["]),


    indent([breakable,


    text(["foo1", ","]),


    breakable,


    text(["foo2", ","]),


    breakable,


    group([text(["["]),


    indent([breakable,


    text(["bar1", ","]),


    breakable,


    text(["bar2", ","]),


    breakable,


    text(["bar3", ","]),


    breakable,


    text(["bar4"])]),


    breakable,


    text(["]"])]),


    text([","]),


    breakable,


    text(["foo3", ","]),


    breakable,


    text(["foo4", ","]),


    breakable,


    ...
    Build · Format · CLI · LSP · Translate

    View full-size slide

  163. breakGroup([group([text(["["]),


    indent([breakable,


    text(["foo1", ","]),


    breakable,


    text(["foo2", ","]),


    breakable,


    group([text(["["]),


    indent([breakable,


    text(["bar1", ","]),


    breakable,


    text(["bar2", ","]),


    breakable,


    text(["bar3", ","]),


    breakable,


    text(["bar4"])]),


    breakable,


    text(["]"])]),


    text([","]),


    breakable,


    text(["foo3", ","]),


    breakable,


    text(["foo4", ","]),


    breakable,


    ...
    Build · Format · CLI · LSP · Translate

    View full-size slide

  164. breakGroup([group([text(["["]),


    indent([breakable,


    text(["foo1", ","]),


    breakable,


    text(["foo2", ","]),


    breakable,


    group([text(["["]),


    indent([breakable,


    text(["bar1", ","]),


    breakable,


    text(["bar2", ","]),


    breakable,


    text(["bar3", ","]),


    breakable,


    text(["bar4"])]),


    breakable,


    text(["]"])]),


    text([","]),


    breakable,


    text(["foo3", ","]),


    breakable,


    text(["foo4", ","]),


    breakable,


    ...
    Build · Format · CLI · LSP · Translate

    View full-size slide

  165. I made too many additions
    to prettyprint
    Build · Format · CLI · LSP · Translate
    😅😅😅

    View full-size slide

  166. prettyprint
    Build · Format · CLI · LSP · Translate

    View full-size slide

  167. prettier_print
    Build · Format · CLI · LSP · Translate

    View full-size slide

  168. Build · Format · CLI · LSP · Translate
    if foo


    bar += baz


    end


    View full-size slide

  169. Build · Format · CLI · LSP · Translate
    if foo


    bar += baz


    end


    View full-size slide

  170. Build · Format · CLI · LSP · Translate
    bar += baz if foo


    View full-size slide

  171. Build · Format · CLI · LSP · Translate
    if foo


    bar


    else


    baz


    end


    View full-size slide

  172. Build · Format · CLI · LSP · Translate
    if foo


    bar


    else


    baz


    end


    View full-size slide

  173. Build · Format · CLI · LSP · Translate
    foo ? bar : baz


    View full-size slide

  174. Build · Format · CLI · LSP · Translate
    here_is_a_method_call(1, 2,


    <<~HEREDOC


    this is the content of the heredoc


    HEREDOC


    )


    View full-size slide

  175. Build · Format · CLI · LSP · Translate
    here_is_a_method_call(1, 2,


    <<~HEREDOC


    this is the content of the heredoc


    HEREDOC


    )


    View full-size slide

  176. Build · Format · CLI · LSP · Translate
    here_is_a_method_call(1, 2, <<~HEREDOC)


    this is the content of the heredoc


    HEREDOC


    View full-size slide

  177. Build · Format · CLI · LSP · Translate
    def foo


    return [1, 2, 3].map { |element| element * 2 } # comment


    end


    View full-size slide

  178. Build · Format · CLI · LSP · Translate
    def foo


    return [1, 2, 3].map { |element| element * 2 } # comment


    end


    View full-size slide

  179. Build · Format · CLI · LSP · Translate
    def foo


    return(


    [1, 2, 3].map do |element|


    element * 2


    end


    ) # comment


    end


    View full-size slide

  180. Build · Format · CLI · LSP · Translate
    class If < Node


    def format(q)


    if node.consequent || node.statements.empty?


    q.group { format_break(q, force: true) }


    else


    q.group do


    q


    .if_break { format_break(q, force: false) }


    .if_flat do


    Parentheses.flat(q) do


    q.format(node.statements)


    q.text(" if ")


    q.format(node.predicate)


    end


    end


    end


    end


    end


    end


    View full-size slide

  181. Build · Format · CLI · LSP · Translate
    class If < Node


    def format(q)


    if node.consequent || node.statements.empty?


    q.group { format_break(q, force: true) }


    else


    q.group do


    q


    .if_break { format_break(q, force: false) }


    .if_flat do


    Parentheses.flat(q) do


    q.format(node.statements)


    q.text(" if ")


    q.format(node.predicate)


    end


    end


    end


    end


    end


    end


    View full-size slide

  182. Create a CLI
    Build · Format · CLI · LSP · Translate

    View full-size slide

  183. Build · Format · CLI · LSP · Translate
    stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the AST corresponding to the given files


    stree check [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Check that the given files are formatted as syntax tree would format them


    stree debug [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Check that the given files can be formatted idempotently


    stree doc [--plugins=...] [-e SCRIPT] FILE


    Print out the doc tree that would be used to format the given files


    stree expr [-e SCRIPT] FILE


    Print out a pattern-matching Ruby expression that would match the first


    expression of the given files


    stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the formatted version of the given files


    stree json [--plugins=...] [-e SCRIPT] FILE


    Print out the JSON representation of the given files


    stree match [--plugins=...] [-e SCRIPT] FILE


    Print out a pattern-matching Ruby expression that would match the given files


    stree help


    Display this help message


    stree lsp [--plugins=...] [--print-width=NUMBER]


    Run syntax tree in language server mode


    stree search PATTERN [-e SCRIPT] FILE


    Search for the given pattern in the given files


    stree version


    Output the current version of syntax tree


    stree write [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Read, format, and write back the source of the given files


    --plugins=...


    A comma-separated list of plugins to load.


    --print-width=NUMBER


    The maximum line width to use when formatting.


    -e SCRIPT


    Parse an inline Ruby string.

    View full-size slide

  184. Build · Format · CLI · LSP · Translate
    stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the AST corresponding to the given files


    stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the formatted version of the given files


    stree match [--plugins=...] [-e SCRIPT] FILE


    Print out a pattern-matching Ruby expression that would match the given files


    stree lsp [--plugins=...] [--print-width=NUMBER]


    Run syntax tree in language server mode


    stree search PATTERN [-e SCRIPT] FILE


    Search for the given pattern in the given files

    View full-size slide

  185. Build · Format · CLI · LSP · Translate
    stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the AST corresponding to the given files


    stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the formatted version of the given files


    stree match [--plugins=...] [-e SCRIPT] FILE


    Print out a pattern-matching Ruby expression that would match the given files


    stree lsp [--plugins=...] [--print-width=NUMBER]


    Run syntax tree in language server mode


    stree search PATTERN [-e SCRIPT] FILE


    Search for the given pattern in the given files

    View full-size slide

  186. Build · Format · CLI · LSP · Translate
    $ stree ast -e '1+2'


    View full-size slide

  187. Build · Format · CLI · LSP · Translate
    $ stree ast -e '1+2'


    (program


    (statements


    ((binary (int "1") + (int "2")))))


    View full-size slide

  188. Build · Format · CLI · LSP · Translate
    stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the AST corresponding to the given files


    stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the formatted version of the given files


    stree match [--plugins=...] [-e SCRIPT] FILE


    Print out a pattern-matching Ruby expression that would match the given files


    stree lsp [--plugins=...] [--print-width=NUMBER]


    Run syntax tree in language server mode


    stree search PATTERN [-e SCRIPT] FILE


    Search for the given pattern in the given files

    View full-size slide

  189. Build · Format · CLI · LSP · Translate
    stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the AST corresponding to the given files


    stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the formatted version of the given files


    stree match [--plugins=...] [-e SCRIPT] FILE


    Print out a pattern-matching Ruby expression that would match the given files


    stree lsp [--plugins=...] [--print-width=NUMBER]


    Run syntax tree in language server mode


    stree search PATTERN [-e SCRIPT] FILE


    Search for the given pattern in the given files

    View full-size slide

  190. Build · Format · CLI · LSP · Translate
    $ stree format -e '1+2'

    View full-size slide

  191. Build · Format · CLI · LSP · Translate
    $ stree format -e '1+2'


    1 + 2


    View full-size slide

  192. Build · Format · CLI · LSP · Translate
    stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the AST corresponding to the given files


    stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the formatted version of the given files


    stree match [--plugins=...] [-e SCRIPT] FILE


    Print out a pattern-matching Ruby expression that would match the given files


    stree lsp [--plugins=...] [--print-width=NUMBER]


    Run syntax tree in language server mode


    stree search PATTERN [-e SCRIPT] FILE


    Search for the given pattern in the given files

    View full-size slide

  193. Build · Format · CLI · LSP · Translate
    stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the AST corresponding to the given files


    stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the formatted version of the given files


    stree match [--plugins=...] [-e SCRIPT] FILE


    Print out a pattern-matching Ruby expression that would match the given files


    stree lsp [--plugins=...] [--print-width=NUMBER]


    Run syntax tree in language server mode


    stree search PATTERN [-e SCRIPT] FILE


    Search for the given pattern in the given files

    View full-size slide

  194. Build · Format · CLI · LSP · Translate
    $ stree match -e '1+2'

    View full-size slide

  195. Build · Format · CLI · LSP · Translate
    $ stree match -e '1+2'
    SyntaxTree::Program[


    statements: SyntaxTree::Statements[


    body: [


    SyntaxTree::Binary[


    left: SyntaxTree::Int[value: "1"],


    operator: :+,


    right: SyntaxTree::Int[value: "2"]


    ]


    ]


    ]


    ]


    View full-size slide

  196. Build · Format · CLI · LSP · Translate
    stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the AST corresponding to the given files


    stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the formatted version of the given files


    stree match [--plugins=...] [-e SCRIPT] FILE


    Print out a pattern-matching Ruby expression that would match the given files


    stree lsp [--plugins=...] [--print-width=NUMBER]


    Run syntax tree in language server mode


    stree search PATTERN [-e SCRIPT] FILE


    Search for the given pattern in the given files

    View full-size slide

  197. Build · Format · CLI · LSP · Translate
    stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the AST corresponding to the given files


    stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the formatted version of the given files


    stree match [--plugins=...] [-e SCRIPT] FILE


    Print out a pattern-matching Ruby expression that would match the given files


    stree lsp [--plugins=...] [--print-width=NUMBER]


    Run syntax tree in language server mode


    stree search PATTERN [-e SCRIPT] FILE


    Search for the given pattern in the given files

    View full-size slide

  198. Build · Format · CLI · LSP · Translate
    $ stree search Int -e '1+2'

    View full-size slide

  199. Build · Format · CLI · LSP · Translate
    $ stree search Int -e '1+2'
    script:1:0: 1+2


    script:1:2: 1+2

    View full-size slide

  200. Build · Format · CLI · LSP · Translate
    $ stree search Int -e '1+2'

    View full-size slide

  201. Build · Format · CLI · LSP · Translate
    $ stree search 'Int[value: "2"]' -e '1+2'

    View full-size slide

  202. Build · Format · CLI · LSP · Translate
    $ stree search 'Int[value: "2"]' -e '1+2'


    script:1:2: 1+2

    View full-size slide

  203. Build · Format · CLI · LSP · Translate
    $ stree search <(stree match -e '1+2') \

    -e '1+2'

    View full-size slide

  204. Build · Format · CLI · LSP · Translate
    $ stree search <(stree match -e '1+2') \

    -e '1+2'


    script:1:0: 1+2

    View full-size slide

  205. Build · Format · CLI · LSP · Translate
    stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the AST corresponding to the given files


    stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the formatted version of the given files


    stree match [--plugins=...] [-e SCRIPT] FILE


    Print out a pattern-matching Ruby expression that would match the given files


    stree lsp [--plugins=...] [--print-width=NUMBER]


    Run syntax tree in language server mode


    stree search PATTERN [-e SCRIPT] FILE


    Search for the given pattern in the given files


    stree write [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Read, format, and write back the source of the given files

    View full-size slide

  206. Build · Format · CLI · LSP · Translate
    stree ast [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the AST corresponding to the given files


    stree format [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Print out the formatted version of the given files


    stree match [--plugins=...] [-e SCRIPT] FILE


    Print out a pattern-matching Ruby expression that would match the given files


    stree lsp [--plugins=...] [--print-width=NUMBER]


    Run syntax tree in language server mode


    stree search PATTERN [-e SCRIPT] FILE


    Search for the given pattern in the given files


    stree write [--plugins=...] [--print-width=NUMBER] [-e SCRIPT] FILE


    Read, format, and write back the source of the given files

    View full-size slide

  207. Create a


    language server
    Build · Format · CLI · LSP · Translate

    View full-size slide

  208. Language server protocol
    Build · Format · CLI · LSP · Translate

    View full-size slide

  209. Language server protocol
    Runs a JSON RPC server behind an editor
    Build · Format · CLI · LSP · Translate

    View full-size slide

  210. Language server protocol
    Runs a JSON RPC server behind an editor


    textDocument/didChange


    textDocument/didOpen


    textDocument/didClose
    Build · Format · CLI · LSP · Translate

    View full-size slide

  211. Language server protocol
    Runs a JSON RPC server behind an editor


    textDocument/didChange


    textDocument/didOpen


    textDocument/didClose


    textDocument/formatting
    Build · Format · CLI · LSP · Translate

    View full-size slide

  212. Language server protocol
    Runs a JSON RPC server behind an editor


    textDocument/didChange


    textDocument/didOpen


    textDocument/didClose


    textDocument/formatting


    textDocument/inlayHint
    Build · Format · CLI · LSP · Translate

    View full-size slide

  213. Build · Format · CLI · LSP · Translate

    View full-size slide

  214. Shopify/ruby-lsp
    Build · Format · CLI · LSP · Translate

    View full-size slide

  215. Shopify/ruby-lsp
    Document highlight
    Build · Format · CLI · LSP · Translate

    View full-size slide

  216. Shopify/ruby-lsp
    Document highlight


    Document symbols
    Build · Format · CLI · LSP · Translate

    View full-size slide

  217. Shopify/ruby-lsp
    Document highlight


    Document symbols


    Folding ranges
    Build · Format · CLI · LSP · Translate

    View full-size slide

  218. Shopify/ruby-lsp
    Document highlight


    Document symbols


    Folding ranges


    Formatting
    Build · Format · CLI · LSP · Translate

    View full-size slide

  219. Shopify/ruby-lsp
    Document highlight


    Document symbols


    Folding ranges


    Formatting


    Selection ranges
    Build · Format · CLI · LSP · Translate

    View full-size slide

  220. Shopify/ruby-lsp
    Document highlight


    Document symbols


    Folding ranges


    Formatting


    Selection ranges


    Semantic highlighting
    Build · Format · CLI · LSP · Translate

    View full-size slide

  221. Translate the


    syntax tree
    Build · Format · CLI · LSP · Translate

    View full-size slide

  222. Translate the syntax tree
    Build · Format · CLI · LSP · Translate

    View full-size slide

  223. Translate the syntax tree
    seattlerb/ruby_parser
    Build · Format · CLI · LSP · Translate

    View full-size slide

  224. Build · Format · CLI · LSP · Translate
    class RubyParserVisitor < Visitor


    def visit_binary(node)


    left = visit(node.left)


    right = visit(node.right)


    case node


    in { operator: :and }


    s(:and, left, right)


    in { operator: :!~ }


    s(:not, s(:call, left, :=~, right))


    else


    s(:call, left, node.operator, right)


    end


    end


    # ...


    end


    View full-size slide

  225. Translate the syntax tree
    seattlerb/ruby_parser
    Build · Format · CLI · LSP · Translate

    View full-size slide

  226. Translate the syntax tree
    seattlerb/ruby_parser


    whitequark/parser
    Build · Format · CLI · LSP · Translate

    View full-size slide

  227. Build · Format · CLI · LSP · Translate
    class ParserVisitor < Visitor


    def visit_binary(node)


    left = visit(node.left)


    right = visit(node.right)


    case node


    in { operator: :and }


    s(:and, [left, right])


    in { operator: :or }


    s(:or, [left, right])


    else


    s(:send, [left, node.operator, right])


    end


    end


    # ...


    end


    View full-size slide

  228. Translate the syntax tree
    seattlerb/ruby_parser


    whitequark/parser
    Build · Format · CLI · LSP · Translate

    View full-size slide

  229. Translate the syntax tree
    seattlerb/ruby_parser


    whitequark/parser


    YARV
    Build · Format · CLI · LSP · Translate

    View full-size slide

  230. Build · Format · CLI · LSP · Translate
    class YARVVisitor < Visitor


    def visit_binary(node)


    case node.operator


    when :"&&"


    # ...


    when :"||"


    # ...


    else


    visit(node.left)


    visit(node.right)


    builder.send(


    node.operator,


    1,


    VM_CALL_ARGS_SIMPLE


    )


    end


    end


    # ...


    end


    View full-size slide

  231. Translate the syntax tree
    seattlerb/ruby_parser


    whitequark/parser


    YARV
    Build · Format · CLI · LSP · Translate

    View full-size slide

  232. Translate the syntax tree
    seattlerb/ruby_parser


    whitequark/parser


    YARV


    Ruby
    Build · Format · CLI · LSP · Translate

    View full-size slide

  233. Build · Format · CLI · LSP · Translate
    class MutationVisitor < Visitor


    def visit_binary(node)


    node.copy(


    left: visit(node.left),


    right: visit(node.right)


    )


    end


    end

    View full-size slide

  234. Build · Format · CLI · LSP · Translate
    mutator =


    SyntaxTree.mutation do |mutator|


    query = <<~RUBY


    IfNode[predicate: Assign | OpAssign] |


    UnlessNode[predicate: Assign | OpAssign]


    RUBY


    mutator.mutate(query) do |node|


    node.copy(predicate: SyntaxTree::Paren.new(


    lparen: SyntaxTree::LParen.default,


    contents: node.predicate,


    location: node.predicate.location


    ))


    end


    end


    mutated = node.accept(mutator)


    View full-size slide

  235. Translate the syntax tree
    seattlerb/ruby_parser


    whitequark/parser


    YARV


    Ruby
    Build · Format · CLI · LSP · Translate

    View full-size slide

  236. Parser
    Syntax Tree

    View full-size slide

  237. Parser


    Syntax tree
    Syntax Tree

    View full-size slide

  238. Parser


    Syntax tree


    Visitor
    Syntax Tree

    View full-size slide

  239. Syntax Tree
    Build a syntax tree


    View full-size slide

  240. Syntax Tree
    Build a syntax tree


    Format the syntax tree


    View full-size slide

  241. Syntax Tree
    Build a syntax tree


    Format the syntax tree


    Create a CLI


    View full-size slide

  242. Syntax Tree
    Build a syntax tree


    Format the syntax tree


    Create a CLI


    Create a language server (LSP)


    View full-size slide

  243. Syntax Tree
    Build a syntax tree


    Format the syntax tree


    Create a CLI


    Create a language server (LSP)


    Translate the syntax tree

    View full-size slide

  244. Syntax Tree


    github.com/ruby-syntax-tree
    Kevin Newton
    @kddnewton

    View full-size slide