Writing Lint for Ruby

RubyKaigi 2017
http://rubykaigi.org/2017/presentations/p_ck_.html

Transcript

1. © 2017 Actcat, Inc. Masataka Kuwabara / Actcat, Inc. RubyKaigi 2017 Sep. 20, 2017 Writing Lint for Ruby
2. © 2017 Actcat, Inc. Masataka Kuwabara @pocke Actcat, Inc. / SideCI RuboCop’s core developer Reek Collaborator
3. © 2017 Actcat, Inc. https://sideci.com SideCI reviews your Pull-Request automatically. 1. Open a Pull-Request on GitHub. 2. SideCI reviews the Pull-Request. 3. Fix reviewed code, or ignore the review. Actcat, Inc / SideCI
4. © 2017 Actcat, Inc. How does SideCI review code? ▪ SideCI uses static analyzers such as Lint. • RuboCop • Reek • Brakeman • Querly • …
5. © 2017 Actcat, Inc. Goal ▪ You will be able to write: • Rules of Lint • Lint tools We can prevent known bugs by writing Lint.
6. © 2017 Actcat, Inc. Agenda ▪ What’s Lint? ▪ How does Lint work? ▪ What’s can / cannot Lint do? ▪ How can I write Lint?
7. © 2017 Actcat, Inc. What’s Lint?
8. © 2017 Actcat, Inc. What’s Lint? ▪ Original Lint is a static analyzer for C proglam. ▪ In this talk, Lint means a bug detector for any languages. For example: • JavaScript: ESLint • Python: Pylint • Ruby: RuboCop
9. © 2017 Actcat, Inc. Example of Ruby(RuboCop) # Use the && operator to # compare multiple values. if 10 < x < 20 do_something end Syntax is valid, but it’s an incorrect usage of `<` operator.
10. © 2017 Actcat, Inc. Example of RuboCop # foo(bar) { body } # foo(bar { body }) foo bar { body } # x * y # x(*y) x *y
11. © 2017 Actcat, Inc. Lint is static bug detector ▪ Lint detect code that can be a bug. • Invalid usage. • Ambiguous code. • They are valid about Syntax.
12. © 2017 Actcat, Inc. How does Lint work?
13. © 2017 Actcat, Inc. if 1 p 'Hello' end Abstract Syntax Tree(AST) s(:if, s(:int, 1), s(:send, nil, :p, s(:str, "Hello")), nil) Ruby Code AST(parser gem)
14. © 2017 Actcat, Inc. Parser Gem ▪ Parser gem parse ruby code to AST. ▪ Many Lints use this gem. • RuboCop, Reek, Querly
15. © 2017 Actcat, Inc. Example of Parser gem require 'parser/ruby24' node = Parser::Ruby24.parse('if 1; p "hello"; end') # => s(:if, s(:int, 1), ...) node.type # => :if node.children # => [s(:int, 1), s(:send, nil, :p, ...), nil]
16. © 2017 Actcat, Inc. Metadata of Parser gem node.loc.line # => 1 node.loc.column # => 0 node.loc.expression.source # => 'if 1; p "Hello"; end' node.loc.end # => #<P::S::Range 17...20>
17. © 2017 Actcat, Inc. Other parsers for Ruby ▪ Ripper • A standard library. • Runtime Ruby version == Parsed Ruby version. ▪ ruby_parser • https://github.com/seattlerb/ruby_parser
18. © 2017 Actcat, Inc. Traverser def traverse(node, visitor) visitor.__send__(:"on_#{node.type}", node) node.children.each do |child| traverse(child, visitor) if child.is_a?(Parser::AST::Node) end end ▪ Depth-first search. ▪ Call “on_#{node.type}”(e.g. on_send) each node.
19. © 2017 Actcat, Inc. Traverser example s(:if, s(:int, 1), s(:send, nil, :p, s(:str, "Hello")), nil)
20. © 2017 Actcat, Inc. Visitor Pattern class IntInCondVisitor def on_if(node) cond = node.children.first if cond.type == :int warn "Do not use an int literal in condition!!!" \ " (#{cond.loc.line}:#{cond.loc.column})" end end # TODO def method_missing(*); end end
21. © 2017 Actcat, Inc. DEMO https://github.com/pocke/rubykaigi2017
22. © 2017 Actcat, Inc. What’s can / cannot Lint for Ruby do?
23. © 2017 Actcat, Inc. A local variable is just a variable # RuboCop warns about the code if 1 end # RuboCop does not warns the code. num = 1 if num end
24. © 2017 Actcat, Inc. A local variable is just a variable ▪ Many Lint doesn’t trace local variables. • In `var = 1; puts var`. • “var” is just a variable, not an integer. • It is not impossible, but complexity. • For example: Brakeman can trace lvar.
25. © 2017 Actcat, Inc. Lint cannot know method / class / constant definition accurately ▪ In `foo(BAR, baz)` • Lint cannot know the foo method definition. • Lint cannot know the BAR class definition.
26. © 2017 Actcat, Inc. Example: invalid sprintf() usage # RuboCop says # “number of arguments does not match” sprintf('%s, %s', str) # but maybe the `sprintf` is redefined. def sprintf(t, s) puts t + s end
27. © 2017 Actcat, Inc. Lint does not execute your code ▪ Lint doesn’t know: • Loaded library • Monkey patch • eval • User input value ▪ Testing may be more appropriate in some cases.
28. © 2017 Actcat, Inc. Lint can... ▪ Understand AST. • Even if complex. ▪ Analyze code without execution. • Faster than execution.
29. © 2017 Actcat, Inc. How can I write Lint?
30. © 2017 Actcat, Inc. Add a cop(rule) to RuboCop ▪ For general cases. • `if 1 ; end` • Like `ruby -cw` ▪ Easy to write. • RuboCop has many helper methods. • AST matcher, extended AST node. • RuboCop provides visitor, config file, etc.
31. © 2017 Actcat, Inc. How to write a cop ▪ Run `rake new_cop[Lint/NAME]` in RuboCop project. • DEMO
32. © 2017 Actcat, Inc. RuboCop Plugin ▪ For a specific framework / library. • For example: backus/rubocop-rspec ▪ You can use RuboCop’s helpers.
33. © 2017 Actcat, Inc. New Lint Tool ▪ For: • Lint + X; e.g. Lint + Git Diff. • Ruby + X; e.g. Ruby + YAML. ▪ It is out of scope of RuboCop. • You should create a new Lint tool. • Or use tool except RuboCop.
34. © 2017 Actcat, Inc. Flowchart to chose how to implement Lint ▪ If Ruby only: • if general cases: • Add a new rule to RuboCop. • else: • Create RuboCop plugin. ▪ else: • Create a new Lint tool.
35. © 2017 Actcat, Inc. Conclusion ▪ Lint is static bug detector. ▪ Lint traverses AST. ▪ Lint does not execute your code. ▪ There are several choices for writing Lint.