Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Metric Talks or Not

The Metric Talks or Not

at Ruby Kaigi 2013, day 3.

Hiroki Yoshioka

June 01, 2013
Tweet

More Decks by Hiroki Yoshioka

Other Decks in Programming

Transcript

  1. The Metric Talks or Not
    RubyKaigi 2013
    @irohiroki

    View Slide

  2. View Slide

  3. Topics
    Background
    Various Metrics
    Real World Survey

    View Slide

  4. Background

    View Slide

  5. Do you code?

    View Slide

  6. What kind of software
    do you code?

    View Slide

  7. Does it evolve?

    View Slide

  8. 3 types of system
    S-Type
    P-Type
    E-Type
    Calculator
    Ches player
    Most of systems

    View Slide

  9. As an E-type system evolves
    its complexity increases
    unless work is done to maintain or reduce it.
    Lehman’s law

    View Slide

  10. Too complex to evolve
    → Software decay

    View Slide

  11. What makes system evolve
    New features
    Environment changes
    Technical findings
    → Press to evolve
    → Press to decay

    View Slide

  12. Visualized Pressure
    sample

    View Slide

  13. Technical Debt... ?
    sample

    View Slide

  14. What makes complexity decrease
    sample
    ?

    View Slide

  15. Paranoids?
    djb?
    What makes complexity decrease

    View Slide

  16. Is he objective? or structured?
    Can he persuade your boss?

    View Slide

  17. No djb in your team... ?

    View Slide

  18. Metrics

    View Slide

  19. Metrics are objective and structured

    View Slide

  20. Metrics can be visualized

    View Slide

  21. Metrics for Ruby

    View Slide

  22. Category
    Static analysis
    Coverage analysis
    Statistical analysis

    View Slide

  23. Curious Ninja
    Master Maid

    View Slide

  24. What is static analysis?
    Analysis which evaluates a snapshot of source
    code without running it.
    Static analysis includes:
    - Cyclomatic complexity,
    - ABC metric,
    - Duplicate code detection,
    - many kind of checkers,
    - and very original scoring tool, Flog.

    View Slide

  25. Cyclomatic complexity? What is that?
    Can you draw control flow graphs of any of your
    methods? A graph consists of nodes and edges,
    right? Cyclomatic complexty is calculated as
    e - n + 2
    where e and n are the number of edges and
    nodes.
    so, the cyclomatic complexity
    of this is “2”.

    View Slide

  26. OK. So, what is it good for?
    It suggests the number of tests the
    method ought to have.
    And of course, it represents how hard the
    method is to understand.

    View Slide

  27. I see. Which tool do I use to get that cyclomatic
    complexity?
    There is Saikuro. The fork of metricfu is
    maintained on Github.
    Roodi, a source code checker, has a couple of
    check items based on cyclomatic complexity.

    View Slide

  28. Huh. Is it really of any help?
    Cyclomatic complexity provides one
    point of view on how to examine code.
    The more views you consider, the more
    findings you get.

    View Slide

  29. OK. What view does ABC metric provide?
    ABC stands for “Assign”, “Branch”, and
    “Condition”. These are the basic elements of
    imperative programming languages like Ruby.
    ABC metric indicates magnitude of code with the
    following formula:
    where a, b, and c are the number of assigns,
    branches, and conditions respectively.

    View Slide

  30. Magnitude of code? Aren’t you happy with LoC?
    LoC, Lines of Code, depends on coding
    style too much.
    ABC metric is better than LoC.

    View Slide

  31. OK. How do you use these magnitude values?
    Imagine a bloated method which fills
    your entire editor window. Bloated
    methods like that signal the beginning
    of software decay.

    View Slide

  32. What is bloated? At what value should we fix the
    code?
    It’s up to development teams. Metrics
    provide a sign of issues, and only that.
    If you want a precise criteria, base it on
    past records. If you set a target too high,
    your team will get exhausted.

    View Slide

  33. I see. Do we have any gem for ABC metric?
    Yes. The Metric_ABC gem calculates the
    magnitude of the ABC metric for each
    method in your code.

    View Slide

  34. I’ve heard of Flog. Isn’t it better than ABC?
    Good point. Metric_ABC and Flog are
    similar. And Flog provides another point of
    view.
    It emphasizes how hard code is to test. e.g.,
    it gives relatively high points to `eval’.
    It’s not a generic algorithm, but specific to
    Ruby.

    View Slide

  35. Cyclomatic complexity, ABC metric, and Flog.
    Which one should I use?
    Let’s say you are hiring, how many
    perspectives do you need at job
    interviews?
    Like perspectives, a single tool can be
    easily cheated.
    Would you like to know more?

    View Slide

  36. Sure. I guess there are many source code
    checkers out there.
    True. One of the most important
    checkers is duplicate code detector.
    Ruby has Flay for that.

    View Slide

  37. So it detects copy-pastes to DRY up the code?
    Yes, it compares portions of code
    grammatically.
    That is, differences of variable names,
    literals, and other non-significant
    words are ignored when matching.

    View Slide

  38. Sounds nice. What about other checkers?
    Reek, Cane, and Roodi are all similar
    tools.
    They checks things like long methods,
    assignments in conditionals,
    documentation, and many more.

    View Slide

  39. So many tools!
    How can I make all this work!?

    View Slide

  40. What about Metric_fu?

    View Slide

  41. Visualization by Metric_fu

    View Slide

  42. OK, enough for static analysis. But I think
    coverage tools are more popular than that.
    Maybe. SimpleCov is the de facto
    standard for Ruby 1.9+.

    View Slide

  43. Is 100% coverage possible?
    It depends on the layer. For business
    logic, you can go for 100%.
    But what’s important is to be realistic.
    That is the same for both coverage and
    static metrics.

    View Slide

  44. Last question: What’s statistical analysis?
    Statistical analysis predicts software
    defects based on the modification
    history. It is known that frequently
    changed code blocks are more likely to
    have bugs.
    Do you use any version control system
    like Git? Then you are ready!

    View Slide

  45. Sweet. The name of the statistical analysis tool,
    please?
    There are a few of them. Churn provides
    method level analysis.
    Bugspots is an implementation of the
    bug prediction heuristic at Google. It
    uses only “fixing” commits, and works at
    the file level.

    View Slide

  46. How should we use these predictions?
    Does your team perform code review?
    How long does it take? Is it efficient
    enough?
    What if you know hotspots prior to the
    review?

    View Slide

  47. Thank you, Master... Maid.
    You are welcome.

    View Slide

  48. Real World Survey

    View Slide

  49. Samples
    Github
    10 stars minimum
    1,311 Repositories
    Cyclomatic complexity, ABC, Flog

    View Slide

  50. Histogram of Cyclomatic Complexity

    View Slide

  51. Histogram of ABC Metric

    View Slide

  52. Histogram of Flog

    View Slide

  53. Cyclomatic Complexity Worst 3
    1. 118: FooBarWidget/rubyenterpriseedition
    RDoc::Fortran95parser#parse_program_or_module
    2. 92: rubinius/rubinius
    REXML::XPathParser#expr
    2. 92: jsn/rbot
    Irc::Client#process

    View Slide

  54. What does that mean?
    Parser / Protocol handler
    Pile of conditions
    Big case block

    View Slide

  55. ABC Metric Worst 3
    1. 301: FooBarWidget/rubyenterpriseedition
    RDoc::Fortran95parser#parse_program_or_module
    2. 291: jsn/rbot
    Irc::Bot#initialize
    3. 277: edavis10/redmine
    Redmine::Export::PDF#issue_to_pdf

    View Slide

  56. What does that mean?
    Unique
    Sequential
    Static

    View Slide

  57. Flog Worst 3
    1. 4103.6: evanfarrar/opensprints
    main#none
    2. 3858.4: jamtur01/puppet_old
    main#none
    3. 3813.7: lak/puppet-old
    main#none

    View Slide

  58. What does that mean?
    Flog can measure top level scripts
    setup.rb
    Same as ABC metric

    View Slide

  59. Conclusion
    Objective and structured
    Visualizable
    Worst score can be justified

    View Slide

  60. Thank You.
    Presented by @irohiroki

    View Slide