Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Here Be Dragons: The Hidden Gems of Technical Debt

Here Be Dragons: The Hidden Gems of Technical Debt

How do you find the most unmaintainable code in your codebase? What will you prioritize in your next technical debt spike, and why?

In this talk you will learn how you can use RubyCritic, SimpleCov, Flog, Reek, and Skunk to slay dragons in your next refactoring adventure! Find out how the relationship between churn, complexity, and code coverage can give you a common language to talk about code quality and increase trust in your code.

Presented at RubyConf Mini 2022.

Ernesto Tagwerker

November 16, 2022
Tweet

More Decks by Ernesto Tagwerker

Other Decks in Programming

Transcript

  1. Here Be Dragons:
    The Hidden Gems
    of Tech Debt
    Ernesto Tagwerker (@etagwerker)


    RubyConf Mini, November 2022

    View Slide

  2. 🤓 My pronouns are he/him

    View Slide

  3. 🇦🇷 Hi, I’m from Argentina

    View Slide

  4. View Slide

  5. 🦅 I live in Philadelphia

    View Slide

  6. View Slide

  7. 👨💻 I love Ruby & Open Source!

    View Slide

  8. View Slide

  9. 👪 Thanks to my sponsors!

    View Slide

  10. Teo, Claire, and Luli

    View Slide

  11. Founder &


    CTO


    @OmbuLabs &


    @FastRubyIO

    View Slide

  12. Open Source


    Contributor/Maintainer


    rubycritic; skunk;


    bundler-leak; next_rails

    View Slide

  13. Here Be Dragons
    (Story Time)

    View Slide

  14. “hic sunt dracones”

    View Slide

  15. “here be dragons”

    View Slide

  16. By Abraham Ortelius - The Library of Congress, Public Domain

    View Slide

  17. By Abraham Ortelius - The Library of Congress, Public Domain

    View Slide

  18. “Be careful with this part of
    the world, nobody has been
    there, there may be
    monsters.”

    View Slide

  19. View Slide

  20. The Hunt-Lenox
    Globe

    View Slide

  21. Credit: Rare Book Division, The New York Public Library

    View Slide

  22. Credit: University of Rochester

    View Slide

  23. What if there was a
    map to your
    codebase?

    View Slide

  24. “Be careful with this


    part of the world,


    nobody has been there,


    there may be monsters.”

    View Slide

  25. “Be careful with this


    part of the codebase,


    nobody wants to touch it,


    there is a lot of tech debt.”

    View Slide

  26. How do you
    fi
    nd these
    dragons in your
    codebase?

    View Slide

  27. How do you
    fi
    nd


    tech debt


    in your codebase?

    View Slide

  28. View Slide

  29. Tech Debt &


    Dragons

    View Slide

  30. 1. The Complexity Dragon

    View Slide

  31. 1. The Complexity Dragon


    2. The Smelly Dragon

    View Slide

  32. 1. The Complexity Dragon


    2. The Smelly Dragon


    3. The Shapeshifting Dragon

    View Slide

  33. 1. The Complexity Dragon


    2. The Smelly Dragon


    3. The Shapeshifting Dragon


    4. The Untested Dragon

    View Slide

  34. The
    Complexity


    Dragon

    View Slide

  35. View Slide

  36. Complexity


    1 class Foo
    2 def yay
    3 a = eval "1+1"
    4 if a == 2
    5 puts "yay"
    6 end
    7 end
    8 end

    View Slide

  37. Complexity


    flog

    View Slide

  38. Complexity


    “flog reports the
    most tortured
    code”

    View Slide

  39. Complexity


    $ flog foo.rb


    10.6: flog total

    View Slide

  40. Complexity


    1 class Foo
    2 def yay
    3 a = eval "1+1" # 1.2 + 6.0
    4 if a == 2 # + 1.2
    5 puts “yay" # + 1.2
    6 end. # =
    7 end. # 10.6
    8 end

    View Slide

  41. 55 ## flog.rb
    56 # Eval forms
    57
    58 SCORES.merge!(:define_method => 5,
    59 :eval => 5,
    60 :module_eval => 5,
    61 :class_eval => 5,
    62 :instance_eval => 5)
    63

    View Slide

  42. 62 ## flog.rb
    63 # Various "magic" usually used for "clever code"
    64
    65 SCORES.merge!(:alias_method => 2,
    66 :extend => 2,
    67 :include => 2,
    68 :instance_method => 2,
    69 :instance_methods => 2,
    70 :method_added => 2,
    71 :method_defined? => 2,
    72 :method_removed => 2,
    73 :method_undefined => 2,
    74 :private_class_method => 2,
    75 :private_instance_methods => 2,
    76 :private_method_defined? => 2,
    77 :protected_instance_methods => 2,
    78 :protected_method_defined? => 2,

    View Slide

  43. 37 ## flog.rb
    38 # Various non-call constructs
    39
    40 OTHER_SCORES = {
    41 :alias => 2,
    42 :assignment => 1,
    43 :block => 1,
    44 :block_pass => 1,
    45 :block_call => 1,
    46 :branch => 1,
    47 :lit_fixnum => 0.25,
    48 :sclass => 5,
    49 :super => 1,
    50 :to_proc_icky! => 10,
    51 :to_proc_lasgn => 15,
    52 :yield => 1,
    53 }
    54

    View Slide

  44. Flog won’t report:


    - Known code smells


    - Duplicated code


    - Code that constantly
    changes

    View Slide

  45. The


    Smelly
    Dragon

    View Slide

  46. View Slide

  47. Code Smells


    “Smells are indicators of
    where your code might be
    hard to read, maintain or
    evolve.”

    View Slide

  48. Code Smells


    They are not necessarily
    wrong.

    View Slide

  49. Code Smells


    For example: Duplicated
    code is a code smell.

    View Slide

  50. Code Smells


    “Duplication is far cheaper
    than the wrong
    abstraction.” Sandi Metz

    View Slide

  51. Code Smells


    Reek


    (github.com/troessner/reek)

    View Slide

  52. View Slide

  53. View Slide

  54. View Slide

  55. 1 module Reek
    2 module SmellDetectors
    3 class UncommunicativeVariableName < BaseDetector
    4 #
    5 # Checks the given +context+ for uncommunicative names.
    6 #
    7 # @return [Array]
    8 #
    9 def sniff
    10 variable_names.select do |name, _lines|
    11 uncommunicative_variable_name?(name)
    12 end.map do |name, lines|
    13 smell_warning(
    14 lines: lines,
    15 message: "has the variable name '#{name}'",
    16 parameters: { name: name.to_s })
    17 end
    18 end

    View Slide

  56. Code Smells


    1 class Foo
    2 def yay
    3 a = eval "1+1"
    4 if a == 2
    5 puts "yay"
    6 end
    7 end
    8 end

    View Slide

  57. View Slide

  58. Reek


    $ gem install reek


    View Slide

  59. Reek


    $ reek foo.rb


    View Slide

  60. Code Smells


    $ reek foo.rb


    foo.rb -- 2 warnings:


    [1]:IrresponsibleModule: Foo has no
    descriptive comment


    [3]:UncommunicativeVariableName:
    Foo#yay has the variable name 'a'

    View Slide

  61. Code Smells


    1 class Foo
    2 def yay
    3 a = eval "1+1"
    4 if a == 2
    5 puts "yay"
    6 end
    7 end
    8 end

    View Slide

  62. Code Smells


    1 class Foo
    2 def yay
    3 eval_result = eval "1+1"
    4 if eval_result == 2
    5 puts "yay"
    6 end
    7 end
    8 end

    View Slide

  63. Code Smells


    $ reek foo.rb


    foo.rb -- 1 warning:


    [1]:IrresponsibleModule: Foo has no
    descriptive comment

    View Slide

  64. View Slide

  65. Code Smells


    $ reek foo.rb


    foo.rb -- 1 warning:


    [1]:IrresponsibleModule: Foo has no
    descriptive comment

    View Slide

  66. Code Smells


    1 class Foo
    2 def yay
    3 eval_result = eval "1+1"
    4 if eval_result == 2
    5 puts "yay"
    6 end
    7 end
    8 end

    View Slide

  67. Code Smells


    1 ---
    2
    3 ### .reek.yml (Generic smell configuration)
    4
    5 detectors:
    6 IrresponsibleModule:
    7 enabled: false

    View Slide

  68. Code Smells


    $ reek foo.rb


    Inspecting 1
    fi
    le(s):


    .

    View Slide

  69. View Slide

  70. Reek won’t report:


    - Is this file
    constantly changing?


    - Is this file super
    complex?


    View Slide

  71. The


    Shapeshifting
    Dragon

    View Slide

  72. View Slide

  73. Churn

    View Slide

  74. Churn


    git

    View Slide

  75. Churn


    $ git log --follow --format=%h foo.rb | cat


    573c8f5


    2ccc5b8


    b447bc3

    View Slide

  76. Churn


    $ git log --follow --format=%h foo.rb | cat


    573c8f5


    2ccc5b8


    b447bc3


    # Churn Count = 3

    View Slide

  77. Churn


    (github.com/danmayer/churn)

    View Slide

  78. Churn


    - Mercurial


    - Git


    - SVN

    View Slide

  79. Churn


    minitest $ churn --start_date "12 years ago" -e rb


    View Slide

  80. Churn


    - By itself it adds
    little value


    - Useful when combined
    with other metrics

    View Slide

  81. How can we bring
    it all together?


    - Complexity


    - Code Smells


    - Churn

    View Slide

  82. RubyCritic


    View Slide

  83. "Cartography" for
    Rubyists


    RubyCritic


    (github.com/whitesmith/rubycritic)

    View Slide

  84. RubyCritic


    $ gem install rubycritic


    View Slide

  85. RubyCritic


    $ rubycritic


    View Slide

  86. View Slide

  87. View Slide

  88. GPA Pie Graph


    GPA Grade (Cost)

    View Slide

  89. GPA Pie Graph


    Costly Files:


    - Hard to understand


    View Slide

  90. GPA Pie Graph


    Costly Files:


    - Hard to understand


    - Hard to maintain


    View Slide

  91. RubyCritic’s Cost


    - Code Smells (reek)


    - Complexity (flog)


    View Slide

  92. RubyCritic’s Cost


    module RubyCritic


    class AnalysedModule


    def cost


    smells.map(&:cost).inject(0.0, :+) + # From Reek


    (complexity / COMPLEXITY_FACTOR) # From Flog


    end


    end


    end

    View Slide

  93. Churn vs.


    Complexity


    Churn (SCM)


    Complexity (flog)

    View Slide

  94. Churn
    Complexity
    50
    0%
    1 100_000

    View Slide

  95. Churn
    Complexity
    100
    0%
    1 100_000
    user.rb (changed: 2 times; complexity: 100_000 FPs )

    View Slide

  96. Churn
    Complexity
    100
    0%
    1 100_000
    account.rb
    account.rb (changed: 27 times; complexity: 100 FPs)

    View Slide

  97. 100%
    0%
    1 100_000
    Complexity
    Churn

    View Slide

  98. 100%
    0%
    1 100_000
    Complexity
    Churn

    View Slide

  99. 100%
    0%
    1 100_000
    “No dragons here.” ❤
    Complexity
    Churn

    View Slide

  100. 100%
    0%
    1 100_000
    “No one understands these
    fi
    les but they work. So
    don’t change them.”
    Complexity
    Churn

    View Slide

  101. 100%
    0%
    1 100_000
    “[…], if the code never
    changes, it's not costing us
    money.” Sandi Metz
    Complexity
    Churn

    View Slide

  102. 100%
    0%
    1 100_000
    “These
    fi
    les are easy to
    understand but you need
    to change them often
    because of reasons…”
    Complexity
    Churn

    View Slide

  103. 100%
    0%
    1 100_000
    Complexity
    Churn
    “These
    fi
    les are complex and
    change a lot…”

    View Slide

  104. 100%
    0%
    1 100_000
    Complexity
    Churn
    “Sometimes a class becomes so
    complex that refactoring seems too
    dif
    fi
    cult.” Michael Feathers

    View Slide

  105. View Slide

  106. 100%
    0%
    1 100_000
    “Here be Dragons.”
    Complexity
    Churn
    “Here be Dragons.”
    “Here be Dragons.” “Here be Dragons.”

    View Slide

  107. View Slide

  108. View Slide

  109. The Untested
    Dragon

    View Slide

  110. View Slide

  111. Code Coverage
    SimpleCov


    (github.com/simplecov-ruby/simplecov)

    View Slide

  112. Code Coverage


    # Gemfile


    group :test do


    gem "simplecov", require: false


    end


    View Slide

  113. Code Coverage


    # spec/spec_helper.rb


    if ENV["COVERAGE"] == "true"


    require 'simplecov'


    SimpleCov.start do


    add_group "Libraries", "lib"


    add_filter "/spec/"


    track_files "**/*.rb"


    end


    end


    View Slide

  114. Code Coverage


    $ COVERAGE=true rspec


    View Slide

  115. View Slide

  116. View Slide

  117. Recap


    1. The Complexity Dragon (
    fl
    og)


    2. The Smelly Dragon (reek)


    3. The Shapeshifting Dragon (churn)


    4. The Untested Dragon (simplecov)

    View Slide

  118. Finding Dragons


    Example: User vs. Account Class

    View Slide

  119. user.rb


    churn: 10


    complexity: 10


    smells: 10


    Tech debt score: 1,000
    (10*10*10)


    View Slide

  120. account.rb


    churn: 10


    complexity: 10


    smells: 10


    Tech debt score: 1,000
    (10*10*10)


    View Slide

  121. user.rb


    churn: 10


    complexity: 10


    smells: 10


    Tech debt score: 1,000
    (10*10*10)


    account.rb


    churn: 10


    complexity: 10


    smells: 10


    Tech debt score: 1,000
    (10*10*10)


    View Slide

  122. User and Account
    have the same
    amount of tech debt

    View Slide

  123. “Do they?”

    View Slide

  124. user.rb


    churn: 10


    complexity: 10


    smells: 10


    code_coverage: 0%


    Tech debt score:
    100,000


    (10*10*10) * 100


    View Slide

  125. account.rb


    churn: 10


    complexity: 10


    smells: 10


    code_coverage: 90%


    Tech debt score:


    10,000


    (10*10*10) * 10

    View Slide

  126. user.rb


    churn: 10


    complexity: 10


    smells: 10


    code_coverage: 0%


    Tech debt score:
    100,000


    account.rb


    churn: 10


    complexity: 10


    smells: 10


    code_coverage: 90%


    Tech debt score:


    10,000


    View Slide

  127. User has more tech
    debt than Account

    View Slide

  128. Code Smells vs.
    Complexity vs.


    Code Coverage

    View Slide

  129. View Slide

  130. Cost

    View Slide

  131. Cost


    - Code Smells (reek)


    - Complexity (flog)


    View Slide

  132. Cost
    Code Coverage

    View Slide

  133. What if we could
    calculate a new signal
    that combines both of
    them?

    View Slide

  134. Cost
    Code Coverage
    SkunkScore

    View Slide

  135. SkunkScore =


    f(cost, code_coverage)

    View Slide

  136. Why Code Coverage?


    Files which lack tests
    are harder to maintain

    View Slide

  137. Skunk


    (github.com/fastruby/skunk)

    View Slide

  138. Skunk


    $ gem install skunk

    View Slide

  139. Skunk


    $ skunk

    View Slide

  140. View Slide

  141. View Slide

  142. Skunk


    def skunk_score
    return cost.round(2) if coverage == PERFECT_COVERAGE
    (cost * penalty_factor).round(2)
    end
    def penalty_factor
    PERFECT_COVERAGE - coverage.to_i
    end

    View Slide

  143. Now we can
    fi
    nally draw
    our map!


    View Slide

  144. Here be dragons
    Here be dragons
    Here be dragons Here be dragons
    Here be dragons

    View Slide

  145. 100%
    0%
    1 100_000
    Complexity
    Churn
    “Here be dragons.”
    “Here be dragons.”
    “Here be dragons.”

    View Slide

  146. View Slide

  147. How to Slay
    Dragons

    View Slide

  148. View Slide

  149. Dragon Slaying 101:


    - Define what’s a dragon


    View Slide

  150. What’s a dragon?


    - Team discussion on
    “tech debt”


    - Toolkit configuration


    View Slide

  151. Dragon Slaying 101:


    - Define what’s a dragon


    - Draw the map and place
    “here be dragons”


    View Slide

  152. Dragon Slaying 101:


    - Define what’s a dragon


    - Draw the map and place
    “here be dragons”


    - Pick your battles
    based on metrics

    View Slide

  153. Slaying the untested dragon


    View Slide

  154. Slaying the untested dragon


    View Slide

  155. Slaying the untested dragon


    View Slide

  156. Slaying the smelly dragon


    View Slide

  157. $ reek foo.rb


    foo.rb -- 2 warnings:


    [1]:IrresponsibleModule: Foo has no descriptive
    comment


    [3]:UncommunicativeVariableName: Foo#yay
    has the variable name 'a'
    Slaying the smelly dragon


    View Slide

  158. $ reek foo.rb


    foo.rb -- 2 warnings:


    [1]:IrresponsibleModule: Foo has no
    descriptive comment


    [3]:UncommunicativeVariableName: Foo#yay
    has the variable name 'a'
    Slaying the smelly dragon


    View Slide

  159. Slaying the smelly dragon


    View Slide

  160. Slaying the complexity dragon


    View Slide

  161. Slaying the complexity dragon


    View Slide

  162. 1. Flog


    2. Reek


    3. Churn


    4. SimpleCov


    5. RubyCritic


    6. Skunk


    View Slide

  163. 8. Attractor


    9. MetricFu


    10. CodeClimate


    11. CodeScene


    12. Codacy


    13. CodeCov


    14. SonarQube


    15. Flay


    16. Sandi Meter


    17. Coverband


    18. Etc..
    1. Flog


    2. Reek


    3. Churn


    4. SimpleCov


    5. RubyCritic


    6. Skunk


    7. Rubocop

    View Slide

  164. 1. Pick your dragons


    View Slide

  165. 1. Pick your dragons


    2. Pick your toolkit

    View Slide

  166. 1. Pick your dragons


    2. Pick your toolkit


    3. Slay your dragons

    View Slide

  167. Thank you!


    @etagwerker - https://speakerdeck.com/etagwerker

    View Slide

  168. Resources
    1. https://github.com/whitesmith/rubycritic


    2.https://github.com/colszowka/simplecov


    3.https://github.com/metricfu/metric_fu


    4.https://github.com/julianrubisch/attractor


    5.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-
    gems.html


    6.https://www.reddit.com/r/ruby/comments/2bq092/rubycritic/


    7.http://jakescruggs.blogspot.com/2008/08/whats-good-
    fl
    og-score.html


    8.http://ruby.sadi.st/Flog.html


    9.https://github.com/makaroni4/sandi_meter

    View Slide

  169. Resources
    1.https://codeclimate.com/blog/deciphering-ruby-code-metrics/


    2.https://www.stickyminds.com/article/getting-empirical-about-refactoring


    3.https://www.sandimetz.com/blog/2017/9/13/breaking-up-the-behemoth


    4.https://github.com/troessner/reek


    5.https://github.com/seattlerb/
    fl
    ay


    6.https://github.com/seattlerb/
    fl
    og


    7.http://www.sqa.net/iso9126.html


    8.https://www.slideshare.net/mscottford/important-metrics-for-measuring-code-health


    9.https://www.theatlantic.com/technology/archive/2013/12/no-old-maps-actually-
    say-here-be-dragons/282267/


    10.https://education.nationalgeographic.org/resource/here-be-dragons


    11.https://resurrect3d.lib.rochester.edu/models/5bdc5e604270f527332282dd

    View Slide