Slide 1

Slide 1 text

玩转 AST 构建自己的代码分析和代码重写工具 黄志敏 @flyerhzm

Slide 2

Slide 2 text

什么是 AST ? 抽象语法树(Abstract Syntax Tree )

Slide 3

Slide 3 text

class RubyChinaConf def location 'Shanghai' end def year 2023 end end (class (const nil :RubyChinaConf) nil (begin (def :location (args) (str "Shanghai")) (def :year (args) (int 2023))))

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

AST 的用途 编译器 解释器 代码静态分析工具

Slide 6

Slide 6 text

哪些 RUBY 工具使用 AST ?

Slide 7

Slide 7 text

FORMATTER 如何工作 ?

Slide 8

Slide 8 text

class RubyChinaConf # body end

Slide 9

Slide 9 text

class RubyChinaConf def location 'Shanghai' end end

Slide 10

Slide 10 text

class RubyChinaConf def location 'Shanghai' end def year 2023 end end

Slide 11

Slide 11 text

PRETTIER [ group([ group(["class ", "RubyChinaConf"]), indent([ hardline, group([ group([ group(["def ", "location", group([])]), indent([hardline, group(["'", "Shanghai", "'"])]), hardline, "end", ]), hardline, hardline, group([ group(["def ", "year", group([])]), indent([hardline, group("2023")]), hardline, "end", ]), ]), ]), hardline, "end", ]), hardline, ]

Slide 12

Slide 12 text

foobar = { foo: foo, bar: bar } group([ "foobar", " = ", group([ "{", indent([ line, group(["foo:", indent([line, "foo"])]), ",", line, group(["bar:", indent([line, "bar"])]), ]), line, "}", ]), ])

Slide 13

Slide 13 text

LINTER 如何工作 ?

Slide 14

Slide 14 text

案例 rails_best_practices synvert

Slide 15

Slide 15 text

RAILS_BEST_PRACTICES

Slide 16

Slide 16 text

记录 MODEL 和 ASSOCIATION # app/models/*.rb add_callback :start_class do |node| @klass = node.class_name.to_s end add_callback :start_command do |node| if node.message.to_s == 'belongs_to' association_name = node.arguments.all.first.to_s add_association(@klass, association_name) end end

Slide 17

Slide 17 text

记录 TABLE 和 INDEX # db/schema.rb add_callback :start_command do |node| if node.message.to_s == 'add_index' table_name = node.arguments.all.first.to_s index_column = node.arguments.all[1].to_object @index_columns[table_name] << index_column end end

Slide 18

Slide 18 text

检查遗漏的 INDEX add_callback :after_check do associations.each do |klass, association_name| unless @index_columns[klass.table_name].include?(association_name.column_name) add_error "always add db index (#{klass.table_name} => #{association_name.column_name})", table_node.file, table_node.line_number end end end

Slide 19

Slide 19 text

SYNVERT Synvert::Rewriter.new 'factory_bot', 'use_short_syntax' do within_files '**/*.rb' do find_node '.send[receiver=FactoryBot][message in (create build build_stubbed attributes_for)]' do delete :receiver, :dot end end end

Slide 20

Slide 20 text

源代码转换成 AST ------------ | AST Node | ------------ /|\ | --------------- | Source Code | ---------------

Slide 21

Slide 21 text

AST NODE 包含 node type children location (start, end, line, column) source

Slide 22

Slide 22 text

增加 ATTRIBUTE ripper + syntax_tree + syntax_tree_ext parser + parser_node_ext code = 'FactoryBot.create(:user)' node = Parser::CurrentRuby.parse(code) node.children[0] node.receiver node.children[1] node.message

Slide 23

Slide 23 text

查找 AST NODE https://github.com/xinminlabs/node-query-ruby -------------- | Node Query | -------------- /|\ | ------------ | AST Node | ------------ /|\ | --------------- | Source Code | ---------------

Slide 24

Slide 24 text

CSS /* HTML */
/* CSS */ .item[data-type='image']

Slide 25

Slide 25 text

NQL (NODE QUERY LANGUAGE) # Ruby Code FactoryBot.create(:user) # AST Node (send (const nil :FactoryBot) :create (sym :user)) # NQL .send[receiver=FactoryBot][message=create]

Slide 26

Slide 26 text

REX & RACC (LEX & YACC)

Slide 27

Slide 27 text

解析 NODE TYPE .send

Slide 28

Slide 28 text

class NodeQueryLexer macros NODE_TYPE /\.[a-zA-Z]+/ rules # [:state] pattern [actions] /\s+/ /#{NODE_TYPE}/ { [:tNODE_TYPE, text[1..]] } end

Slide 29

Slide 29 text

class NodeQueryParser token tNODE_TYPE rule basic_selector : tNODE_TYPE { Compiler::BasicSelector.new(node_type: val[0]) } end end

Slide 30

Slide 30 text

module Compiler class BasicSelector def initialize(node_type:) @node_type = node_type end def query_nodes(node) # iterate all child nodes, # check if child node matches node_type end def match?(node) @node_type.to_sym == NodeQuery.adapter.get_node_type(node) end end end selector = NodeQueryParser.new.parse(nql) selector.query_nodes(node) # matching_nodes

Slide 31

Slide 31 text

解析 ATTRIBUTE .send[reciever=FactoryBot]

Slide 32

Slide 32 text

OPEN_ATTRIBUTE /\[/ CLOSE_ATTRIBUTE /\]/ IDENTIFIER /[@\*\-\.\w]*\w/ IDENTIFIER_VALUE /[@\.\w!&:\?<>=]+/ rules /#{OPEN_ATTRIBUTE}/ { @state = :KEY; [:tOPEN_ATTRIBUTE, text] } :KEY /\s+/ :KEY /=/ { @state = :VALUE; [:tOPERATOR, '='] } :KEY /#{IDENTIFIER}/ { [:tKEY, text] } :VALUE /\s+/ :VALUE /#{CLOSE_ATTRIBUTE}/ { @state = nil; [:tCLOSE_ATTRIBUTE, text] } :VALUE /#{IDENTIFIER_VALUE}/ { [:tIDENTIFIER_VALUE, text] } end class NodeQueryLexer 1 macros 2 NODE_TYPE /\.[a-zA-Z]+/ 3 4 5 6 7 8 # [:state] pattern [actions] 9 /\s+/ 10 /#{NODE_TYPE}/ { [:tNODE_TYPE, text[1..]] } 11 12 13 14 15 16 17 18 19

Slide 33

Slide 33 text

token tNODE_TYPE tKEY tIDENTIFIER_VALUE tOPEN_ATTRIBUTE tCLOSE_ATTRIBUTE tOPERATOR | tNODE_TYPE attribute { Compiler::BasicSelector.new(node_type: val[0], attribute: val[1]) } attribute : tOPEN_ATTRIBUTE tKEY tOPERATOR value tCLOSE_ATTRIBUTE { Compiler::Attribute.new(key: val[1], value: val[3], opera value : tIDENTIFIER_VALUE { Compiler::Identifier.new(value: val[0]) } end class NodeQueryParser 1 2 rule 3 basic_selector 4 : tNODE_TYPE { Compiler::BasicSelector.new(node_type: val[0]) } 5 6 7 8 9 10 11 12 13 end 14

Slide 34

Slide 34 text

module Compiler class BasicSelector def initialize(node_type:, attribute: nil) @node_type = node_type @attribute = attribute end def query_nodes(node) # iterate all child nodes, # check if child node matches node_type and attribute end def match?(node) @node_type.to_sym == NodeQuery.adapter.get_node_type(node) && (!@attribute || @attribute.match?(node)) end end class Attribute def initialize(key:, value:, operator: '=') @key = key @value = value @operator = operator end def match?(node) @value.match?(NodeQuery::Helper.get_target_node(node, @key), node, @operator) end end end selector = NodeQueryParser.new.parse(nql) selector.query_nodes(node) # matching_nodes

Slide 35

Slide 35 text

解析多个 ATTRIBUTE .send[reciever=FactoryBot][message=create]

Slide 36

Slide 36 text

| tNODE_TYPE attribute_list { Compiler::BasicSelector.new(node_type: val[0], attribute_list: val[1]) } attribute_list : attribute attribute_list { Compiler::AttributeList.new(attribute: val[0], rest: val[1]) } | attribute { Compiler::AttributeList.new(attribute: val[0]) } class NodeQueryParser 1 token tNODE_TYPE tKEY tIDENTIFIER_VALUE tOPEN_ATTRIBUTE tCLOSE_ATTRIBUTE tOPERATOR 2 rule 3 basic_selector 4 : tNODE_TYPE { Compiler::BasicSelector.new(node_type: val[0]) } 5 6 7 8 9 10 11 attribute 12 : tOPEN_ATTRIBUTE tKEY tOPERATOR value tCLOSE_ATTRIBUTE { Compiler::Attribute.new(key: val[1], value: val[3], opera 13 14 value 15 : tIDENTIFIER_VALUE { Compiler::Identifier.new(value: val[0]) } 16 end 17 end 18

Slide 37

Slide 37 text

解析 VALUE 类型 .send[arguments.first=:user][arguments.last=true]

Slide 38

Slide 38 text

FALSE /false/ FLOAT /\-?\d+\.\d+/ INTEGER /\-?\d+/ NIL /nil/ SYMBOL /:[\w!\?<>=]+/ TRUE /true/ SINGLE_QUOTE_STRING /'.*?'/ DOUBLE_QUOTE_STRING /".*?"/ rules :VALUE /#{NIL}/ { [:tNIL, nil] } :VALUE /#{TRUE}/ { [:tBOOLEAN, true] } :VALUE /#{FALSE}/ { [:tBOOLEAN, false] } :VALUE /#{SYMBOL}/ { [:tSYMBOL, text[1..-1].to_sym] } :VALUE /#{FLOAT}/ { [:tFLOAT, text.to_f] } :VALUE /#{INTEGER}/ { [:tINTEGER, text.to_i] } :VALUE /#{DOUBLE_QUOTE_STRING}/ { [:tSTRING, text[1...-1]] } :VALUE /#{SINGLE_QUOTE_STRING}/ { [:tSTRING, text[1...-1]] } class NodeQueryLexer 1 macros 2 NODE_TYPE /\.[a-zA-Z]+/ 3 OPEN_ATTRIBUTE /\[/ 4 CLOSE_ATTRIBUTE /\]/ 5 IDENTIFIER /[@\*\-\.\w]*\w/ 6 IDENTIFIER_VALUE /[@\.\w!&:\?<>=]+/ 7 8 9 10 11 12 13 14 15 16 # [:state] pattern [actions] 17 /\s+/ 18 /#{NODE_TYPE}/ { [:tNODE_TYPE, text[1..]] } 19 /#{OPEN_ATTRIBUTE}/ { @state = :KEY; [:tOPEN_ATTRIBUTE, text] } 20 :KEY /\s+/ 21 :KEY /=/ { @state = :VALUE; [:tOPERATOR, '='] } 22 :KEY /#{IDENTIFIER}/ { [:tKEY, text] } 23 :VALUE /\s+/ 24 :VALUE /#{CLOSE_ATTRIBUTE}/ { @state = nil; [:tCLOSE_ATTRIBUTE, text] } 25 26 27 28 29 30 31 32 33 :VALUE /#{IDENTIFIER_VALUE}/ { [:tIDENTIFIER_VALUE, text] } 34

Slide 39

Slide 39 text

end 35

Slide 40

Slide 40 text

tBOOLEAN tFLOAT tINTEGER tNIL tSTRING tSYMBOL | tBOOLEAN { Compiler::Boolean.new(value: val[0]) } | tFLOAT { Compiler::Float.new(value: val[0]) } | tINTEGER { Compiler::Integer.new(value: val[0])} | tNIL { Compiler::Nil.new(value: val[0]) } | tSTRING { Compiler::String.new(value: val[0]) } | tSYMBOL { Compiler::Symbol.new(value: val[0]) } end class NodeQueryParser 1 token tNODE_TYPE tKEY tIDENTIFIER_VALUE tOPEN_ATTRIBUTE tCLOSE_ATTRIBUTE tOPERATOR 2 3 rule 4 basic_selector 5 : tNODE_TYPE { Compiler::BasicSelector.new(node_type: val[0]) } 6 | tNODE_TYPE attribute_list { Compiler::BasicSelector.new(node_type: val[0], attribute_list: val[1]) } 7 8 attribute_list 9 : attribute attribute_list { Compiler::AttributeList.new(attribute: val[0], rest: val[1]) } 10 | attribute { Compiler::AttributeList.new(attribute: val[0]) } 11 12 attribute 13 : tOPEN_ATTRIBUTE tKEY tOPERATOR value tCLOSE_ATTRIBUTE { Compiler::Attribute.new(key: val[1], value: val[3], opera 14 15 value 16 : tIDENTIFIER_VALUE { Compiler::Identifier.new(value: val[0]) } 17 18 19 20 21 22 23 24 end 25

Slide 41

Slide 41 text

解析操作符 .send[arguments.size>0]

Slide 42

Slide 42 text

class NodeQueryLexer macros OPERATOR /(\^=|\$=|\*=|!=|=~|!~|>=|<=|>|<|=|not includes|includes|not in|in)/i rules # [:state] pattern [actions] # :KEY /=/ { @state = :VALUE; [:tOPERATOR, '='] } :KEY /#{OPERATOR}/ { @state = :VALUE; [:tOPERATOR, text] } end

Slide 43

Slide 43 text

查找 DEBUG 代码 puts "hello world" p current_user # NQL '.send[receiver=nil][message IN (puts p)]'

Slide 44

Slide 44 text

查找 HASH ROCKET 的 KEY { :foo => 'bar' } # NQL %q{.pair[key=.sym][key=~/\A:([^'"]+)\z/]}

Slide 45

Slide 45 text

查找 HASH 的键值对是相同的 a: a, c: c, some_method( 1 2 b: b + 2, 3 4 d: d + 4 5 ) 6 # NQL '.pair[key="{{value}}"]'

Slide 46

Slide 46 text

查找 MINITEST SETUP 方法中没有调用 SUPER def setup do_something end class TestMeme < Minitest::Test 1 2 3 4 end 5 # NQL '.class[parent_class=Minitest::Test] .def[name=setup]:not_has(> .super)'

Slide 47

Slide 47 text

查找 HASH 的值 render nothing: true, status: :created render nothing: true 1 2 # NQL '.hash[status_value!=nil]'

Slide 48

Slide 48 text

替换源代码 https://github.com/xinminlabs/node-mutation-ruby -------------- ----------------- | Node Query | | Node Mutation | -------------- ----------------- |\¯¯ ¯¯/| \ / ------------ | AST Node | ------------ /|\ | --------------- | Source Code | ---------------

Slide 49

Slide 49 text

REPLACE_WITH errors[:base] = "author not present" 1 2 errors.add(:base, "author not present") 3 replace_with node, '{{receiver}}.add({{arguments.0}}, {{arguments.1}})'

Slide 50

Slide 50 text

REPLACE_WITH errors[:base] = "author not present" 1 2 errors.add(:base, "author not present") 3 errors.add(:base, "author not present") errors[:base] = "author not present" 1 2 3 replace_with node, '{{receiver}}.add({{arguments.0}}, {{arguments.1}})'

Slide 51

Slide 51 text

REPLACE_WITH errors[:base] = "author not present" 1 2 errors.add(:base, "author not present") 3 errors.add(:base, "author not present") errors[:base] = "author not present" 1 2 3 replace_with node, '{{receiver}}.add({{arguments.0}}, {{arguments.1}})' [{ start: 0, end: 'errors[:base] = "author not present"'.length, new_code: '{{receiver}}.add({{arguments.0}}, {{arguments.1}})' }]

Slide 52

Slide 52 text

REPLACE class Post < ActiveRecord::Base end 1 2 3 class Post < ApplicationRecord 4 end 5 replace node, :parent_class, with: 'ApplicationRecord'

Slide 53

Slide 53 text

REPLACE class Post < ActiveRecord::Base end 1 2 3 class Post < ApplicationRecord 4 end 5 class Post < ApplicationRecord end class Post < ActiveRecord::Base 1 end 2 3 4 5 replace node, :parent_class, with: 'ApplicationRecord'

Slide 54

Slide 54 text

REPLACE class Post < ActiveRecord::Base end 1 2 3 class Post < ApplicationRecord 4 end 5 class Post < ApplicationRecord end class Post < ActiveRecord::Base 1 end 2 3 4 5 replace node, :parent_class, with: 'ApplicationRecord' [{ start: 'class Post < '.length, end: 'class Post < ActiveRecord::Base'.length, new_code: 'ApplicationRecord' }]

Slide 55

Slide 55 text

INSERT open('http://test.com') 1 2 URI.open('http://test.com') 3 insert node, 'URI.', at: 'beginning'

Slide 56

Slide 56 text

INSERT open('http://test.com') 1 2 URI.open('http://test.com') 3 URI.open('http://test.com') open('http://test.com') 1 2 3 insert node, 'URI.', at: 'beginning'

Slide 57

Slide 57 text

INSERT open('http://test.com') 1 2 URI.open('http://test.com') 3 URI.open('http://test.com') open('http://test.com') 1 2 3 insert node, 'URI.', at: 'beginning' [{ start: 0, end: 0, new_code: 'URI.' }]

Slide 58

Slide 58 text

PREPEND def setup do_something end class TestMeme < Minitest::Test 1 2 3 4 end 5 6 class TestMeme < Minitest::Test 7 def setup 8 super 9 do_something 10 end 11 end 12 prepend node, 'super'

Slide 59

Slide 59 text

PREPEND def setup do_something end class TestMeme < Minitest::Test 1 2 3 4 end 5 6 class TestMeme < Minitest::Test 7 def setup 8 super 9 do_something 10 end 11 end 12 def setup super do_something end class TestMeme < Minitest::Test 1 def setup 2 do_something 3 end 4 end 5 6 class TestMeme < Minitest::Test 7 8 9 10 11 end 12 prepend node, 'super'

Slide 60

Slide 60 text

PREPEND def setup do_something end class TestMeme < Minitest::Test 1 2 3 4 end 5 6 class TestMeme < Minitest::Test 7 def setup 8 super 9 do_something 10 end 11 end 12 def setup super do_something end class TestMeme < Minitest::Test 1 def setup 2 do_something 3 end 4 end 5 6 class TestMeme < Minitest::Test 7 8 9 10 11 end 12 prepend node, 'super' [{ start: "class TestMeme < Minitest::Test\n def setup\n ".length, end: "class TestMeme < Minitest::Test\n def setup\n ".length, new_code: "super\n " }]

Slide 61

Slide 61 text

APPEND def teardown clean_something end class TestMeme < Minitest::Test 1 2 3 4 end 5 6 class TestMeme < Minitest::Test 7 def teardown 8 clean_something 9 super 10 end 11 end 12 append node, 'super'

Slide 62

Slide 62 text

APPEND def teardown clean_something end class TestMeme < Minitest::Test 1 2 3 4 end 5 6 class TestMeme < Minitest::Test 7 def teardown 8 clean_something 9 super 10 end 11 end 12 def teardown clean_something super end class TestMeme < Minitest::Test 1 def teardown 2 clean_something 3 end 4 end 5 6 class TestMeme < Minitest::Test 7 8 9 10 11 end 12 append node, 'super'

Slide 63

Slide 63 text

APPEND def teardown clean_something end class TestMeme < Minitest::Test 1 2 3 4 end 5 6 class TestMeme < Minitest::Test 7 def teardown 8 clean_something 9 super 10 end 11 end 12 def teardown clean_something super end class TestMeme < Minitest::Test 1 def teardown 2 clean_something 3 end 4 end 5 6 class TestMeme < Minitest::Test 7 8 9 10 11 end 12 append node, 'super' [{ start: "class TestMeme < Minitest::Test\n def teardown\n clean_someting".length, end: "class TestMeme < Minitest::Test\n def teardown\n clean_someting".length, new_code: "\n super" }]

Slide 64

Slide 64 text

DELETE FactoryBot.create(:user) 1 2 create(:user) 3 delete node, :receiver, :dot

Slide 65

Slide 65 text

DELETE FactoryBot.create(:user) 1 2 create(:user) 3 create(:user) FactoryBot.create(:user) 1 2 3 delete node, :receiver, :dot

Slide 66

Slide 66 text

DELETE FactoryBot.create(:user) 1 2 create(:user) 3 create(:user) FactoryBot.create(:user) 1 2 3 delete node, :receiver, :dot [{ start: 0, end: 'FactoryBot.'.length, new_code: '' }]

Slide 67

Slide 67 text

REMOVE puts( 'hello'\ 'world' ) 1 2 3 4 5 # nothing 6 remove node

Slide 68

Slide 68 text

REMOVE puts( 'hello'\ 'world' ) 1 2 3 4 5 # nothing 6 # nothing puts( 1 'hello'\ 2 'world' 3 ) 4 5 6 remove node

Slide 69

Slide 69 text

REMOVE puts( 'hello'\ 'world' ) 1 2 3 4 5 # nothing 6 # nothing puts( 1 'hello'\ 2 'world' 3 ) 4 5 6 remove node [{ start: 0, end: "puts(\n 'hello'\\\n 'world'\n)".length, new_code: '' }]

Slide 70

Slide 70 text

HASH NODE after_commit :add_to_index_later, on: :create 1 2 after_create_commit :add_to_index_later 3 replace node, :message, with: 'after_{{arguments.-1.on_value.to_value}}_commit' delete node, 'arguments.-1.on_pair', and_comma: true

Slide 71

Slide 71 text

HASH NODE after_commit :add_to_index_later, on: :create 1 2 after_create_commit :add_to_index_later 3 after_create_commit :add_to_index_later after_commit :add_to_index_later, on: :create 1 2 3 replace node, :message, with: 'after_{{arguments.-1.on_value.to_value}}_commit' delete node, 'arguments.-1.on_pair', and_comma: true

Slide 72

Slide 72 text

HASH NODE after_commit :add_to_index_later, on: :create 1 2 after_create_commit :add_to_index_later 3 after_create_commit :add_to_index_later after_commit :add_to_index_later, on: :create 1 2 3 replace node, :message, with: 'after_{{arguments.-1.on_value.to_value}}_commit' delete node, 'arguments.-1.on_pair', and_comma: true [{ start: 0, end: 'after_commit'.length, new_code: 'after_{{arguments.-1.on_value.to_value}}_commit' }, { start: 'after_commit :add_to_index_later'.length, end: 'after_commit :add_to_index_later, :on: :create'.length, new_code: '' }]

Slide 73

Slide 73 text

查找 + 替换 require 'parser/current' require 'parser_node_ext' require 'node_query' require 'node_mutation' 1 2 3 4 5 source = "foo\nputs foo\n" 6 node = Parser::CurrentRuby.parse(source) 7 query = NodeQuery.new('.send[receiver=nil][message IN (p puts)]') 8 mutation = NodeMutation.new(source) 9 query.query_nodes(node).each do |matching_node| 10 mutation.remove(matching_node) 11 end 12 result = mutation.process 13 result.new_source # foo\n 14

Slide 74

Slide 74 text

查找 + 替换 require 'parser/current' require 'parser_node_ext' require 'node_query' require 'node_mutation' 1 2 3 4 5 source = "foo\nputs foo\n" 6 node = Parser::CurrentRuby.parse(source) 7 query = NodeQuery.new('.send[receiver=nil][message IN (p puts)]') 8 mutation = NodeMutation.new(source) 9 query.query_nodes(node).each do |matching_node| 10 mutation.remove(matching_node) 11 end 12 result = mutation.process 13 result.new_source # foo\n 14 source = "foo\nputs foo\n" node = Parser::CurrentRuby.parse(source) query = NodeQuery.new('.send[receiver=nil][message IN (p puts)]') query.query_nodes(node).each do |matching_node| end require 'parser/current' 1 require 'parser_node_ext' 2 require 'node_query' 3 require 'node_mutation' 4 5 6 7 8 mutation = NodeMutation.new(source) 9 10 mutation.remove(matching_node) 11 12 result = mutation.process 13 result.new_source # foo\n 14

Slide 75

Slide 75 text

查找 + 替换 require 'parser/current' require 'parser_node_ext' require 'node_query' require 'node_mutation' 1 2 3 4 5 source = "foo\nputs foo\n" 6 node = Parser::CurrentRuby.parse(source) 7 query = NodeQuery.new('.send[receiver=nil][message IN (p puts)]') 8 mutation = NodeMutation.new(source) 9 query.query_nodes(node).each do |matching_node| 10 mutation.remove(matching_node) 11 end 12 result = mutation.process 13 result.new_source # foo\n 14 source = "foo\nputs foo\n" node = Parser::CurrentRuby.parse(source) query = NodeQuery.new('.send[receiver=nil][message IN (p puts)]') query.query_nodes(node).each do |matching_node| end require 'parser/current' 1 require 'parser_node_ext' 2 require 'node_query' 3 require 'node_mutation' 4 5 6 7 8 mutation = NodeMutation.new(source) 9 10 mutation.remove(matching_node) 11 12 result = mutation.process 13 result.new_source # foo\n 14 source = "foo\nputs foo\n" mutation = NodeMutation.new(source) mutation.remove(matching_node) result = mutation.process result.new_source # foo\n require 'parser/current' 1 require 'parser_node_ext' 2 require 'node_query' 3 require 'node_mutation' 4 5 6 node = Parser::CurrentRuby.parse(source) 7 query = NodeQuery.new('.send[receiver=nil][message IN (p puts)]') 8 9 query.query_nodes(node).each do |matching_node| 10 11 end 12 13 14

Slide 76

Slide 76 text

SYNVERT 代码升级工具(比如 rails ) 代码自动重写工具 基于 AST 的查找替换工具 https://synvert.net/

Slide 77

Slide 77 text

-------------------- --------------- . | Synvert Snippets | --- | Synvert CLI | . -------------------- --------------- . /|\ | ---------------- | Synvert Core | ---------------- ¯¯/| |\¯¯ / \ -------------- ----------------- | Node Query | | Node Mutation | -------------- ----------------- |\¯¯ ¯¯/| \ / ------------ | AST Node | ------------ /|\ | --------------- | Source Code | ---------------

Slide 78

Slide 78 text

Synvert::Rewriter.new 'ruby', 'remove_debug_code' do within_files Synvert::ALL_RUBY_FILES do find_node '.send[receiver=nil][message IN (puts p)]' do remove end end end

Slide 79

Slide 79 text

Synvert::Rewriter.new 'rails', 'convert_dynamic_finders' do configure(parser: Synvert::PARSER_PARSER) helper_method :dynamic_finder_to_hash do |prefix| # convert 'email_and_active(email, true)' to 'email: email, active: true' end if_gem 'rails', '>= 3.0' within_files Synvert::ALL_RUBY_FILES + Synvert::ALL_RAKE_FILES do # find_all_by_email_and_active(email, true) => where(email: email, active: true) find_node '.send[message=~/^find_all_by_/]' do hash_params = dynamic_finder_to_hash('find_all_by_') if hash_params replace :message, with: 'where' replace :arguments, with: hash_params end end

Slide 80

Slide 80 text

# find_last_by_email_and_active(email, true) => where(email: email, active: true).last find_node '.send[message=~/^find_last_by_/]' do hash_params = dynamic_finder_to_hash('find_last_by_') if hash_params replace :message, with: 'where' replace :arguments, with: hash_params insert '.last', at: 'end' end end # find_by_email_and_active(email, true) => find_by(email: email, active: true) find_node '.send[message=~/^find_by_/]' do if :find_by_id == node.message replace :message, with: 'find_by' replace :arguments, with: 'id: {{arguments}}' elsif :find_by_sql != node.message hash_params = dynamic_finder_to_hash('find_by_') if hash_params replace :message, with: 'find_by' replace :arguments, with: hash_params end end end end end

Slide 81

Slide 81 text

文件处理 add_file 'app/models/application_record.rb', <<~EOS class ApplicationRecord < ActiveRecord::Base self.abstract_class = true end EOS remove_file 'config/initializers/secret_token.rb'

Slide 82

Slide 82 text

SNIPPET 组合 add_snippet 'minitest', 'assert_empty' add_snippet 'minitest/assert_instance_of' add_snippet '/Users/flyerhzm/.synvert-ruby/lib/minitest/assert_match.rb' add_snippet 'https://github.com/xinminlabs/synvert-snippets-ruby/blob/main/lib/minitest/assert_silent.rb'

Slide 83

Slide 83 text

SYNVERT 命令行 synvert-ruby --run rspec/use_new_syntax synvert-ruby --run ~/.synvert-ruby/lib/rspec/use_new_syntax.rb synvert-ruby --run https://raw.githubusercontent.com/xinminlabs/synvert-snippets-ruby/master/lib/rspec/use_new_syntax.rb

Slide 84

Slide 84 text

-------------------------------------------------------------- | ---------------------- --------------- --------------- | | | Synvert VSCode Ext | | Synvert GUI | | Synvert Web | | | ---------------------- --------------- --------------- | -------------------------------------------------------------- /|\ /|\ | | -------------------- --------------- --------------- | Synvert Snippets | --- | Synvert CLI | | Synvert API | -------------------- --------------- --------------- /|\ | ---------------- | Synvert Core | ---------------- ¯¯/| |\¯¯ / \ -------------- ----------------- | Node Query | | Node Mutation | -------------- ----------------- |\¯¯ ¯¯/| \ / --------------- | AST | --------------- /|\ | --------------- | Source Code | ---------------

Slide 85

Slide 85 text

SYNVERT GUI

Slide 86

Slide 86 text

运行 SNIPPET

Slide 87

Slide 87 text

自动生成 SNIPPET

Slide 88

Slide 88 text

显示 SNIPPETS

Slide 89

Slide 89 text

No content

Slide 90

Slide 90 text

演示

Slide 91

Slide 91 text

No content

Slide 92

Slide 92 text

SYNVERT WEB (PLAYGROUND)

Slide 93

Slide 93 text

SYNVERT JAVASCRIPT new Synvert.Rewriter("typescript", "array-type", () => { configure({ parser: Synvert.Parser.TYPESCRIPT }); // const z: Array = ['a', 'b']; // => // const z: (string|number)[] = ['a', 'b']; withinFiles(Synvert.ALL_TS_FILES, function () { findNode( `.TypeReference [typeName.escapedText=Array] [typeArguments.0=.UnionType]`, () => { replaceWith("({{typeArguments}})[]"); } ); }); });

Slide 94

Slide 94 text

CSS/LESS/SASS/SCSS new Synvert.Rewriter("sass", "convert-to-scss", () => { configure({ parser: Synvert.Parser.GONZALES_PE }); withinFiles(Synvert.ALL_SASS_FILES, function () { findNode(".atrule .string", () => { insert(";", { at: "end" }); }); findNode(".declaration, .include", () => { insert(";", { at: "end", conflictPosition: -99 }); }); findNode(".mixin .operator", () => { replaceWith("@mixin "); }); findNode(".include .operator", () => { replaceWith("@include "); }); findNode(".mixin", () => { const conflictPosition = -this.currentNode.start.column; insert(" {", { at: "end", to: "arguments" }); insertAfter("}", { to: "block", newLinePosition: "after", conflictPosition, }); }); findNode(".ruleset", () => { const conflictPosition = -this.currentNode.start.column; insert(" {", { at: "end", to: "selector" }); insertAfter("}", { to: "block", newLinePosition: "before", conflictPosition, }); });

Slide 95

Slide 95 text

}); renameFile(Synvert.ALL_SASS_FILES, (filePath) => filePath.replace(/\.sass$/, ".scss")); });

Slide 96

Slide 96 text

联系我们 https://synvert.net/contact_us

Slide 97

Slide 97 text

谢谢 提问?

Slide 98

Slide 98 text

No content