Building your own Code metrics tool for Ruby

there are three parts of the talk

first we will cover how to use the parser gem to parse ruby code
this we will use to analyse ruby code and build a simple custom
metric of our own. this a static analysis tool

in the second part of the talk we will see at some
simple low-fidelity visualizations that we can do

In the third and last part we will see how to get
metrics out of production code

Transcript

1. Building your own Code metrics tool for Ruby Metrics and Visualization by @deepak_kannan http://codemancers.com/ Monday, 24 March 14 Hello friends meat of the talk is about code metrics we will touch upon visualization and metrics in production
2. WHY TOOLS Monday, 24 March 14
3. LEVERAGE Monday, 24 March 14 humans are expert tool makers and creators eg. levers. lifting a stone is hard. a lever makes it easy tools as the same way. they give us leverage
4. COOL :-) Monday, 24 March 14 Create Your own Lightsaber The coolness :-)
5. WHY ARE METRICS IMPORTANT Monday, 24 March 14
6. MEASURE Monday, 24 March 14 because intuition can fail
7. EXPLORE Monday, 24 March 14 create a map of the unknown. good for consultants jumping in an unknown codebase
8. WHAT IS A METRIC eg. cyclomatic complexity Monday, 24 March 14
9. A NUMBER Which represents a fact Monday, 24 March 14 cyclomatic complexity. we have taken this essential fact and boiled it down to a number
10. TOOLS Our choices Monday, 24 March 14 for doing static analysis
11. SYBOLIC EXPRESSIONS Monday, 24 March 14
12. (class (const nil :Foo) nil (begin (def :initialize (args) (ivasgn :@i (int 1))) (def :process (args) (begin (lvasgn :test (send (float 2.5) :+ (float 2.5))) (lvasgn :b (int 1)) (ivasgn :@a (send (lvar :test) :+ (lvar :b))))))) Monday, 24 March 14 Output It is a sexp. quite messy. but un-ambiquous
13. LISP Monday, 24 March 14 sexp is a short form of symbolic expression John McCarthy here. creator of LISP
14. LISP Monday, 24 March 14 enough about LISP could not resist the sidetrack once we encountered sexp's
15. (class (const nil :Foo) nil (begin (def :initialize (args) (ivasgn :@i (int 1))) (def :process (args) (begin (lvasgn :test (send (float 2.5) :+ (float 2.5))) (lvasgn :b (int 1)) (ivasgn :@a (send (lvar :test) :+ (lvar :b))))))) Monday, 24 March 14 Output. It is a sexp. quite messy bunch of rules which go into parsing ruby code. precedence, def foo; end; f = foo #no parens un-ambiquous representation of that
16. RIPPER ruby stdlib require 'ripper' Monday, 24 March 14 Advantage is that it is ruby stdlib
17. require 'ripper' #=> [:program, [:vcall, [:@ident, "to", [1, 10]]]] pp Ripper.sexp 'Hey, like to#$%$#$!PARSE RUBY' #~ syntax error, unexpected tIDENTIFIER, expecting keyword_do or '{' or '(' Hey, like to#$%$#$!PARSE RUBY DOES NOT HANDLE BAD SYNTAX Monday, 24 March 14 Disadvantage is that it does not handle bad syntax
18. RUBY_PARSER https://github.com/seattlerb/ruby_parser Monday, 24 March 14 good option but i prefer the gem parser
19. simple example require 'ruby_parser' pp RubyParser.new.parse('1+1') s(:call, s(:lit, 1), :+, s(:lit, 1)) Monday, 24 March 14
20. POPULAR Monday, 24 March 14 lot of code analysis tools like flay (similar code) flog (cyclomatic complexity) reek (code smells) use ruby_parser
21. PARSER https://github.com/whitequark/parser Monday, 24 March 14 like its output
22. https://github.com/whitequark/parser require 'parser/current' pp Parser::CurrentRuby.parse("1 + 1") (send (int 1) :+ (int 1)) Monday, 24 March 14 like the output format. AST output is full documented as well
23. Output is a just a little bit nicer require 'parser/current' pp Parser::CurrentRuby.parse("1 + 1.2") (send (int 1) :+ (float 1.2)) Monday, 24 March 14 Output is just a little bit nicer tells me that one is int and one is float
24. ruby_parser identifies everything as lit (literal) require 'ruby_parser' pp RubyParser.new.parse('1 + 1.2') s(:call, s(:lit, 1), :+, s(:lit, 1.2)) Monday, 24 March 14
25. PARSING OUTPUT parsing output of parser which is a sexp (symbolic expression) Monday, 24 March 14 once we get a sexp we have to read and process it. in short this is the process we will take a look at two short examples
26. SIMPLE EXAMPLE Monday, 24 March 14
27. A CUSTOM METRIC Monday, 24 March 14
28. SINGLE NAME VARIABLES Monday, 24 March 14 single name variables are bad eg. a = 1 + 2
29. OPERATION ON FLOATS Monday, 24 March 14 operations on floats is bad eg. a = 1 + 2.5
30. HAVING BOTH IS WORSE eg: a = 1 + 2.5 Monday, 24 March 14 eg. a = 1 + 2.5
31. SETUP SCRIPT Monday, 24 March 14
32. INPUT class Foo attr_accessor :i def initialize @i = 1 end def process test = 2.5 + 2.5 b = 1 @a = test + b end end Monday, 24 March 14
33. (class (const nil :Foo) nil (begin (def :initialize (args) (ivasgn :@i (int 1))) (def :process (args) (begin (lvasgn :test (send (float 2.5) :+ (float 2.5))) (lvasgn :b (int 1)) (ivasgn :@a (send (lvar :test) :+ (lvar :b))))))) Monday, 24 March 14 Output It is a sexp. quite messy
34. INVOCATION parser = Parser::CurrentRuby.parse("a = 1 + 2") Monday, 24 March 14 We will do it slightly differently
35. code = File.read(path) builder = SexpLisper::Builder.new parser = Parser::CurrentRuby.new(builder) parser.diagnostics.all_errors_are_fatal = true parser.diagnostics.ignore_warnings = true ast, comments = parser.parse_with_comments(source_buffer) custom builder Diagnostics Monday, 24 March 14 we also need to handle parse errors i return nil as ast in those cases and handle nil
36. grepping Floats class FloatCheck < Parser::AST::Processor def on_float(node) log(node, :float) end end dispatch on type Monday, 24 March 14 we are dispatching on type of the node
37. {"input1.rb"=> [ { line: 9, value: 2.5, type: :float }, { line: 9, value: 2.5, type: :float } ] } OUTPUT Monday, 24 March 14
38. Single name variables Monday, 24 March 14 many types here. this is because of ruby's grammer
39. AST node types for variables ivasgn To instance variable eg: @foo = bar lvasgn To local variable eg: foo = bar lvar local variable eg: foo = bar; 10 > foo Monday, 24 March 14 many types here. this is because of ruby's grammer see parser gems AST docs. one more reason to prefer that gem others also class variables (cvar), global variable (gvar)
40. github.com:whitequark/parser/blob/master/doc/ AST_FORMAT.md Monday, 24 March 14 can see the documentation there
41. class SingleVariableCheck < Analysis def on_ivasgn(node) # eg: @i log(node, :single_var) if single_letter_variable? end def on_lvar(node) log(node, :single_var) if single_letter_variable? end end Monday, 24 March 14 lot of types here
42. OUTPUT {"input1.rb"=> [ { line: 10, value: :b, type: :single_var }, { line: 11, value: :@a, type: :single_var }, { line: 11, value: :b, type: :single_var } ] } Monday, 24 March 14 Then we aggregate and get a final score
43. SIMILAR TO PARSING XML Monday, 24 March 14 or maybe XML is similar to sexps :-) event parser
44. CHEAT use ruby-lint to parse Monday, 24 March 14
45. require 'parser/current' require 'ruby-lint' class Analysis < RubyLint::Iterator # track current_scope mainly end metric = DumbMetric.new code = File.read(path) ast, comments = RubyLint::Parser.new.parse(code, path) vm = RubyLint::VirtualMachine.new(:comments => comments) vm.run(ast) metric.process vm, ast use ruby-lint Monday, 24 March 14 RubyLint::VirtualMachine know about the AST defines callbacks like current_scope, on_ivar eg. when we encounter an instance variable
46. Monday, 24 March 14 Sadly No. barfs on some inputs. but it is neat RuboCop using AST::Processor and RubyLint has its own recursive function called Iterator your mileage may vary. check it out
47. EXAMPLE https://gist.github.com/deepak/9703757 Monday, 24 March 14 example is at the above gist
48. GLORIFIED GREP Monday, 24 March 14 it is like we have built our own grep with pattern matching (regex for source code) node.loc.expression.source node.loc.expression.source.lines.to_a
49. DEPENDENCY ANALYSIS Monday, 24 March 14
50. class ConstAnalyzer < Analysis def on_const node log node if has_dependency? node end end class ClassAnalyzer < Analysis def on_class node current_class = const_name(node) def_check = ConstAnalyzer.new(current_class, report) def_check.process(node) end def const_name node node.children[0].children[1] end end Monday, 24 March 14 dependency analysis we manually keep track of scope call ConstAnalyzer manually. having another on_const callback does not work
51. class Person attr_accessor :conference def initialize @conference = RubyConf.new end end class RubyConf SCHEDULE = {"morning 10" => "keynote", "evening 6" => "drinks" end class Feedback attr_accessor :conference def initialize @conference = RubyConf.new end end Monday, 24 March 14 input
52. { :Person => [{:line=>5, :value=>:RubyConf, :type=>:dependency}], :Feedback => [ {:line=>17, :value=>:RubyConf, :type=>:dependency} ] } output Monday, 24 March 14 input
53. VISUALIZATION using graphviz Monday, 24 March 14
54. Monday, 24 March 14
55. EXAMPLE https://gist.github.com/deepak/9717938 Monday, 24 March 14 example is at the above gist
56. VISUALIZATION Monday, 24 March 14
57. LINEGRAPH track changes over time using git Monday, 24 March 14
58. USE GIT Monday, 24 March 14 use git to see old revisions and run a metric for each revision
59. #!/bin/bash # usage: # ./report_for_each_sha.sh 'ruby ../dumb_metric.rb fibonacci.rb' set -e test_command=$1 main() { revs=`git log --oneline | cut -d ' ' -f 1` for rev in $revs; do echo $rev git checkout --quiet $rev score=$(eval "$test_command") echo "$rev, $score" >> output.csv done git checkout master } main Monday, 24 March 14 checkout revision and run script of that version. use a small bash script
60. USE HIGHCHARTS for linecharts Monday, 24 March 14 use git to see old revisions and run a metric for each revision use jsfiddle for quick prototypes
61. Monday, 24 March 14
62. TREEMAPS for metrics on a project Monday, 24 March 14
63. require 'rake' require 'treemap' require 'treemap/image_output' treemap = Treemap::Node.new(label: 'dumb metric') dir_score.each do |dir, score| child = Treemap::Node.new(size: score, label: dir) treemap.add_child(child) end output = Treemap::HtmlOutput.new do |o| o.width = 800 o.height = 600 o.center_labels_at_depth = 1 end html = output.to_html(treemap) File.open('dumb_metric_treemap.html', 'w') { |out| out << html } Monday, 24 March 14 treemap library is a bit old
64. Monday, 24 March 14 useful for showing structured data (eg. a single number as metric) in a limited area. can show many data points. ran it on active_support
65. PRODUCTION METRICS Monday, 24 March 14 now for the last part, we will talk about production metrics
66. HELP TO FACT CHECK against production Monday, 24 March 14 why they are important. giving example of cyclomatic complexity of a function production metrics help to fact check against production
67. GEM METRIKS https://github.com/eric/metriks Monday, 24 March 14
68. COUNTERS Monday, 24 March 14 can increment and decrement it
69. counter = Metriks.counter('duplicate_offline_user_detected') counter.increment Monday, 24 March 14 use it to track expensive computations. track if an obscure if condition is hit
70. gauge = Metriks.gauge('pending_payments', Proc.new { payments.count }) Monday, 24 March 14 used to track a changing value
71. THANKS TO https://github.com/whitequark/parser https://github.com/seattlerb/ruby_parser Aja Hammerly's talk on Visualization at GoGaRuCo 2013 Monday, 24 March 14
72. FIN. @deepak_kannan http://codemancers.com Monday, 24 March 14