Slide 1
Slide 1 text
© 2017 Actcat, Inc.
Masataka Kuwabara / Actcat, Inc.
RubyKaigi 2017
Sep. 20, 2017
Writing Lint for Ruby
Slide 2
Slide 2 text
© 2017 Actcat, Inc.
Masataka Kuwabara @pocke
Actcat, Inc. / SideCI
RuboCop’s core developer
Reek Collaborator
Slide 3
Slide 3 text
© 2017 Actcat, Inc.
https://sideci.com
SideCI reviews your Pull-Request automatically.
1. Open a Pull-Request on GitHub.
2. SideCI reviews the Pull-Request.
3. Fix reviewed code, or ignore the review.
Actcat, Inc / SideCI
Slide 4
Slide 4 text
© 2017 Actcat, Inc.
How does SideCI review code?
■ SideCI uses static analyzers such as Lint.
● RuboCop
● Reek
● Brakeman
● Querly
● …
Slide 5
Slide 5 text
© 2017 Actcat, Inc.
Goal
■ You will be able to write:
● Rules of Lint
● Lint tools
We can prevent known bugs by writing Lint.
Slide 6
Slide 6 text
© 2017 Actcat, Inc.
Agenda
■ What’s Lint?
■ How does Lint work?
■ What’s can / cannot Lint do?
■ How can I write Lint?
Slide 7
Slide 7 text
© 2017 Actcat, Inc.
What’s Lint?
Slide 8
Slide 8 text
© 2017 Actcat, Inc.
What’s Lint?
■ Original Lint is a static analyzer for C proglam.
■ In this talk, Lint means a bug detector for any
languages. For example:
● JavaScript: ESLint
● Python: Pylint
● Ruby: RuboCop
Slide 9
Slide 9 text
© 2017 Actcat, Inc.
Example of Ruby(RuboCop)
# Use the && operator to
# compare multiple values.
if 10 < x < 20
do_something
end
Syntax is valid, but it’s an incorrect usage of
`<` operator.
Slide 10
Slide 10 text
© 2017 Actcat, Inc.
Example of RuboCop
# foo(bar) { body }
# foo(bar { body })
foo bar { body }
# x * y
# x(*y)
x *y
Slide 11
Slide 11 text
© 2017 Actcat, Inc.
Lint is static bug detector
■ Lint detect code that can be a bug.
● Invalid usage.
● Ambiguous code.
● They are valid about Syntax.
Slide 12
Slide 12 text
© 2017 Actcat, Inc.
How does Lint work?
Slide 13
Slide 13 text
© 2017 Actcat, Inc.
if 1
p 'Hello'
end
Abstract Syntax Tree(AST)
s(:if,
s(:int, 1),
s(:send, nil, :p,
s(:str, "Hello")),
nil)
Ruby Code AST(parser gem)
Slide 14
Slide 14 text
© 2017 Actcat, Inc.
Parser Gem
■ Parser gem parse ruby code to AST.
■ Many Lints use this gem.
● RuboCop, Reek, Querly
Slide 15
Slide 15 text
© 2017 Actcat, Inc.
Example of Parser gem
require 'parser/ruby24'
node = Parser::Ruby24.parse('if 1; p "hello";
end')
# => s(:if, s(:int, 1), ...)
node.type
# => :if
node.children
# => [s(:int, 1), s(:send, nil, :p, ...), nil]
Slide 16
Slide 16 text
© 2017 Actcat, Inc.
Metadata of Parser gem
node.loc.line # => 1
node.loc.column # => 0
node.loc.expression.source
# => 'if 1; p "Hello"; end'
node.loc.end
# => #
Slide 17
Slide 17 text
© 2017 Actcat, Inc.
Other parsers for Ruby
■ Ripper
● A standard library.
● Runtime Ruby version == Parsed Ruby version.
■ ruby_parser
● https://github.com/seattlerb/ruby_parser
Slide 18
Slide 18 text
© 2017 Actcat, Inc.
Traverser
def traverse(node, visitor)
visitor.__send__(:"on_#{node.type}", node)
node.children.each do |child|
traverse(child, visitor) if child.is_a?(Parser::AST::Node)
end
end
■ Depth-first search.
■ Call “on_#{node.type}”(e.g. on_send) each node.
Slide 19
Slide 19 text
© 2017 Actcat, Inc.
Traverser example
s(:if,
s(:int, 1),
s(:send,
nil, :p, s(:str, "Hello")),
nil)
Slide 20
Slide 20 text
© 2017 Actcat, Inc.
Visitor Pattern
class IntInCondVisitor
def on_if(node)
cond = node.children.first
if cond.type == :int
warn "Do not use an int literal in condition!!!" \
" (#{cond.loc.line}:#{cond.loc.column})"
end
end
# TODO
def method_missing(*); end
end
Slide 21
Slide 21 text
© 2017 Actcat, Inc.
DEMO
https://github.com/pocke/rubykaigi2017
Slide 22
Slide 22 text
© 2017 Actcat, Inc.
What’s can / cannot Lint for Ruby do?
Slide 23
Slide 23 text
© 2017 Actcat, Inc.
A local variable is just a variable
# RuboCop warns about the code
if 1
end
# RuboCop does not warns the code.
num = 1
if num
end
Slide 24
Slide 24 text
© 2017 Actcat, Inc.
A local variable is just a variable
■ Many Lint doesn’t trace local variables.
● In `var = 1; puts var`.
● “var” is just a variable, not an integer.
● It is not impossible, but complexity.
● For example: Brakeman can trace lvar.
Slide 25
Slide 25 text
© 2017 Actcat, Inc.
Lint cannot know method / class /
constant definition accurately
■ In `foo(BAR, baz)`
● Lint cannot know the foo method definition.
● Lint cannot know the BAR class definition.
Slide 26
Slide 26 text
© 2017 Actcat, Inc.
Example: invalid sprintf() usage
# RuboCop says
# “number of arguments does not match”
sprintf('%s, %s', str)
# but maybe the `sprintf` is redefined.
def sprintf(t, s)
puts t + s
end
Slide 27
Slide 27 text
© 2017 Actcat, Inc.
Lint does not execute your code
■ Lint doesn’t know:
● Loaded library
● Monkey patch
● eval
● User input value
■ Testing may be more appropriate in some cases.
Slide 28
Slide 28 text
© 2017 Actcat, Inc.
Lint can...
■ Understand AST.
● Even if complex.
■ Analyze code without execution.
● Faster than execution.
Slide 29
Slide 29 text
© 2017 Actcat, Inc.
How can I write Lint?
Slide 30
Slide 30 text
© 2017 Actcat, Inc.
Add a cop(rule) to RuboCop
■ For general cases.
● `if 1 ; end`
● Like `ruby -cw`
■ Easy to write.
● RuboCop has many helper methods.
● AST matcher, extended AST node.
● RuboCop provides visitor, config file, etc.
Slide 31
Slide 31 text
© 2017 Actcat, Inc.
How to write a cop
■ Run `rake new_cop[Lint/NAME]` in RuboCop
project.
● DEMO
Slide 32
Slide 32 text
© 2017 Actcat, Inc.
RuboCop Plugin
■ For a specific framework / library.
● For example: backus/rubocop-rspec
■ You can use RuboCop’s helpers.
Slide 33
Slide 33 text
© 2017 Actcat, Inc.
New Lint Tool
■ For:
● Lint + X; e.g. Lint + Git Diff.
● Ruby + X; e.g. Ruby + YAML.
■ It is out of scope of RuboCop.
● You should create a new Lint tool.
● Or use tool except RuboCop.
Slide 34
Slide 34 text
© 2017 Actcat, Inc.
Flowchart to chose how to
implement Lint
■ If Ruby only:
● if general cases:
● Add a new rule to RuboCop.
● else:
● Create RuboCop plugin.
■ else:
● Create a new Lint tool.
Slide 35
Slide 35 text
© 2017 Actcat, Inc.
Conclusion
■ Lint is static bug detector.
■ Lint traverses AST.
■ Lint does not execute your code.
■ There are several choices for writing Lint.