The Metric Talks or Not

Slide 1

Slide 1 text

The Metric Talks or Not RubyKaigi 2013 @irohiroki

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

Topics Background Various Metrics Real World Survey

Slide 4

Slide 4 text

Background

Slide 5

Slide 5 text

Do you code?

Slide 6

Slide 6 text

What kind of software do you code?

Slide 7

Slide 7 text

Does it evolve?

Slide 8

Slide 8 text

3 types of system S-Type P-Type E-Type Calculator Ches player Most of systems

Slide 9

Slide 9 text

As an E-type system evolves its complexity increases unless work is done to maintain or reduce it. Lehman’s law

Slide 10

Slide 10 text

Too complex to evolve → Software decay

Slide 11

Slide 11 text

What makes system evolve New features Environment changes Technical findings → Press to evolve → Press to decay

Slide 12

Slide 12 text

Visualized Pressure sample

Slide 13

Slide 13 text

Technical Debt... ? sample

Slide 14

Slide 14 text

What makes complexity decrease sample ?

Slide 15

Slide 15 text

Paranoids? djb? What makes complexity decrease

Slide 16

Slide 16 text

Is he objective? or structured? Can he persuade your boss?

Slide 17

Slide 17 text

No djb in your team... ?

Slide 18

Slide 18 text

Metrics

Slide 19

Slide 19 text

Metrics are objective and structured

Slide 20

Slide 20 text

Metrics can be visualized

Slide 21

Slide 21 text

Metrics for Ruby

Slide 22

Slide 22 text

Category Static analysis Coverage analysis Statistical analysis

Slide 23

Slide 23 text

Curious Ninja Master Maid

Slide 24

Slide 24 text

What is static analysis? Analysis which evaluates a snapshot of source code without running it. Static analysis includes: - Cyclomatic complexity, - ABC metric, - Duplicate code detection, - many kind of checkers, - and very original scoring tool, Flog.

Slide 25

Slide 25 text

Cyclomatic complexity? What is that? Can you draw control flow graphs of any of your methods? A graph consists of nodes and edges, right? Cyclomatic complexty is calculated as e - n + 2 where e and n are the number of edges and nodes. so, the cyclomatic complexity of this is “2”.

Slide 26

Slide 26 text

OK. So, what is it good for? It suggests the number of tests the method ought to have. And of course, it represents how hard the method is to understand.

Slide 27

Slide 27 text

I see. Which tool do I use to get that cyclomatic complexity? There is Saikuro. The fork of metricfu is maintained on Github. Roodi, a source code checker, has a couple of check items based on cyclomatic complexity.

Slide 28

Slide 28 text

Huh. Is it really of any help? Cyclomatic complexity provides one point of view on how to examine code. The more views you consider, the more findings you get.

Slide 29

Slide 29 text

OK. What view does ABC metric provide? ABC stands for “Assign”, “Branch”, and “Condition”. These are the basic elements of imperative programming languages like Ruby. ABC metric indicates magnitude of code with the following formula: where a, b, and c are the number of assigns, branches, and conditions respectively.

Slide 30

Slide 30 text

Magnitude of code? Aren’t you happy with LoC? LoC, Lines of Code, depends on coding style too much. ABC metric is better than LoC.

Slide 31

Slide 31 text

OK. How do you use these magnitude values? Imagine a bloated method which fills your entire editor window. Bloated methods like that signal the beginning of software decay.

Slide 32

Slide 32 text

What is bloated? At what value should we fix the code? It’s up to development teams. Metrics provide a sign of issues, and only that. If you want a precise criteria, base it on past records. If you set a target too high, your team will get exhausted.

Slide 33

Slide 33 text

I see. Do we have any gem for ABC metric? Yes. The Metric_ABC gem calculates the magnitude of the ABC metric for each method in your code.

Slide 34

Slide 34 text

I’ve heard of Flog. Isn’t it better than ABC? Good point. Metric_ABC and Flog are similar. And Flog provides another point of view. It emphasizes how hard code is to test. e.g., it gives relatively high points to `eval’. It’s not a generic algorithm, but specific to Ruby.

Slide 35

Slide 35 text

Cyclomatic complexity, ABC metric, and Flog. Which one should I use? Let’s say you are hiring, how many perspectives do you need at job interviews? Like perspectives, a single tool can be easily cheated. Would you like to know more?

Slide 36

Slide 36 text

Sure. I guess there are many source code checkers out there. True. One of the most important checkers is duplicate code detector. Ruby has Flay for that.

Slide 37

Slide 37 text

So it detects copy-pastes to DRY up the code? Yes, it compares portions of code grammatically. That is, differences of variable names, literals, and other non-significant words are ignored when matching.

Slide 38

Slide 38 text

Sounds nice. What about other checkers? Reek, Cane, and Roodi are all similar tools. They checks things like long methods, assignments in conditionals, documentation, and many more.

Slide 39

Slide 39 text

So many tools! How can I make all this work!?

Slide 40

Slide 40 text

What about Metric_fu?

Slide 41

Slide 41 text

Visualization by Metric_fu

Slide 42

Slide 42 text

OK, enough for static analysis. But I think coverage tools are more popular than that. Maybe. SimpleCov is the de facto standard for Ruby 1.9+.

Slide 43

Slide 43 text

Is 100% coverage possible? It depends on the layer. For business logic, you can go for 100%. But what’s important is to be realistic. That is the same for both coverage and static metrics.

Slide 44

Slide 44 text

Last question: What’s statistical analysis? Statistical analysis predicts software defects based on the modification history. It is known that frequently changed code blocks are more likely to have bugs. Do you use any version control system like Git? Then you are ready!

Slide 45

Slide 45 text

Sweet. The name of the statistical analysis tool, please? There are a few of them. Churn provides method level analysis. Bugspots is an implementation of the bug prediction heuristic at Google. It uses only “fixing” commits, and works at the file level.

Slide 46

Slide 46 text

How should we use these predictions? Does your team perform code review? How long does it take? Is it efficient enough? What if you know hotspots prior to the review?

Slide 47

Slide 47 text

Thank you, Master... Maid. You are welcome.

Slide 48

Slide 48 text

Real World Survey

Slide 49

Slide 49 text

Samples Github 10 stars minimum 1,311 Repositories Cyclomatic complexity, ABC, Flog

Slide 50

Slide 50 text

Histogram of Cyclomatic Complexity

Slide 51

Slide 51 text

Histogram of ABC Metric

Slide 52

Slide 52 text

Histogram of Flog

Slide 53

Slide 53 text

Cyclomatic Complexity Worst 3 1. 118: FooBarWidget/rubyenterpriseedition RDoc::Fortran95parser#parse_program_or_module 2. 92: rubinius/rubinius REXML::XPathParser#expr 2. 92: jsn/rbot Irc::Client#process

Slide 54

Slide 54 text

What does that mean? Parser / Protocol handler Pile of conditions Big case block

Slide 55

Slide 55 text

ABC Metric Worst 3 1. 301: FooBarWidget/rubyenterpriseedition RDoc::Fortran95parser#parse_program_or_module 2. 291: jsn/rbot Irc::Bot#initialize 3. 277: edavis10/redmine Redmine::Export::PDF#issue_to_pdf

Slide 56

Slide 56 text

What does that mean? Unique Sequential Static

Slide 57

Slide 57 text

Flog Worst 3 1. 4103.6: evanfarrar/opensprints main#none 2. 3858.4: jamtur01/puppet_old main#none 3. 3813.7: lak/puppet-old main#none

Slide 58

Slide 58 text

What does that mean? Flog can measure top level scripts setup.rb Same as ABC metric