Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Escaping The Tar Pit at RubyConf 2019

Escaping The Tar Pit at RubyConf 2019

Presentation about avoiding and escaping the tar pit. The case for a new metric: The StinkScore.

The StinkScore is a combination of smells; complexity; and code coverage. You can use it as a compass to get out of the tar pit and gradually (and measurably) pay off technical debt. 🤓❤️

Presented at RubyConf in Nashville (Nov 19th, 2019)

F77032adcbe77d2777bb0e0c30873159?s=128

Ernesto Tagwerker

November 19, 2019
Tweet

Transcript

  1. Escaping The Tar Pit Ernesto Tagwerker (@etagwerker) RubyConf, November 2019

  2. ! Hi, I’m from Argentina I live in Philadelphia #

    I ❤ Open Source
  3. Founder & Software Engineer @OmbuLabs

  4. Founder & Software Engineer @fastrubyio

  5. Open Source Maintainer database_cleaner; bundler-leak; next_rails

  6. Inspiration

  7. Chapter 1: The Tar Pit

  8. “The fiercer the struggle, the more entangling the tar, and

    no beast is so strong or so skillful but that they ultimately sink.” Fred Brooks
  9. “Large-system programming has over the past decade been such a

    tar pit, and many great and powerful beasts have thrashed violently in it.” Fred Brooks
  10. We are in the tar pit or trying to avoid

    it
  11. None
  12. Tar Pit Symptoms > Projects running over budget

  13. Tar Pit Symptoms > Projects running over budget > Taking

    forever to ship small changes
  14. Tar Pit Symptoms > Projects running over budget > Taking

    forever to ship small changes > Sacrificing quality, increasing tech debt
  15. None
  16. Part 1: How to avoid the tar pit

  17. “THAT’D BE GREAT” BOSS: “IF YOU COULD COME IN ON

    THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
  18. “THAT’D BE GREAT” CLIENT: “IF YOU COULD COME IN ON

    THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
  19. CLIENT: “HOW LONG IS IT GOING TO TAKE?”

  20. “BECAUSE I NEED IT BY END OF DAY TODAY” CLIENT:

    “HOW LONG IS IT GOING TO TAKE?”
  21. How can we quickly assess code quality?

  22. Part 2: How to get out of the tar pit

  23. How can we gradually pay off technical debt?

  24. Part 1: Avoiding the tar pit

  25. None
  26. “It can’t be that hard. It’s a Ruby project.”

  27. How can we quickly assess code quality?

  28. Assessing Quality ✅ Paid Services ✅ Open Source Gems

  29. Assessing Quality ✅ Paid Services ➡ Open Source Gems

  30. Open Source Gems ✅ Static code analysis

  31. Open Source Gems ✅ Static code analysis ✅ Code coverage

  32. Open Source Gems ✅ Static code analysis ✅ Code coverage

    ✅ Code smells
  33. What is software quality anyway?

  34. There are hundreds of definitions

  35. “The degree to which a system, component, or process meets

    implicit and explicit requirements.” IEEE
  36. “It works as expected.”

  37. “It works as expected and it is not a PITA

    to maintain it.”
  38. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅

    Efficiency ✅ Maintainability ✅ Portability
  39. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅

    Efficiency ➡ Maintainability ✅ Portability
  40. Maintainability

  41. Maintainability 1. Code Coverage

  42. Maintainability 1. Code Coverage 2. Code Quality

  43. Code Coverage SimpleCov (github.com/colszowka/simplecov)

  44. Code Coverage # Gemfile group :test do gem "simplecov", require:

    false end
  45. Code Coverage # spec/spec_helper.rb if ENV["COVERAGE"] == "true" require 'simplecov'

    SimpleCov.start do add_group "Models", "models" add_filter "/spec/" track_files "**/*.rb" end end
  46. Code Coverage $ COVERAGE=true rspec

  47. Code Coverage

  48. Code Quality n options

  49. Code Quality n options (flog; flay; reek; churn; RubyCritic; MetricFu;

    attractor; rubocop; fukuzatsu; turbulence; …)
  50. Code Quality RubyCritic (github.com/whitesmith/rubycritic)

  51. Churn

  52. Churn git

  53. Churn $ git log --follow --format=%h default.rb | cat 573c8f5

    2ccc5b8 b447bc3
  54. Churn $ git log --follow --format=%h default.rb | cat 573c8f5

    2ccc5b8 b447bc3 # Churn Count = 3
  55. Complexity

  56. Complexity flog

  57. Complexity $ flog foo.rb => 10.8: flog total

  58. Complexity class Foo def yay # 10.8 = a =

    eval "1+1" # 1.2 + 6.0 + if a == 2 # 1.2 + 1.2 + puts "yay" # 1.2 end end end
  59. Complexity $ flog foo.rb => 10.8: flog total

  60. Churn vs. Complexity

  61. Churn Complexity 50 0 1 100_000

  62. Churn Complexity 100 0 1 100_000 user.rb (changed: 2 times;

    complexity: 100_000 )
  63. Churn Complexity 100 0 1 100_000 user.rb (changed: 27 times;

    complexity: 100)
  64. Code Quality $ gem install rubycritic

  65. Code Quality $ rubycritic

  66. None
  67. GPA Pie Graph Complexity (flog) Code Smells (reek)

  68. Churn vs. Complexity Churn (SCM) Complexity (flog)

  69. 100 0 1 100_000 Complexity Churn

  70. 100 0 1 100_000 Complexity Churn

  71. 100 0 1 100_000 “Welcome to the good place.” ❤

    Complexity Churn
  72. 100 0 1 100_000 No one understands these files but

    they work. So don’t change them. Complexity Churn
  73. 100 0 1 100_000 “[…], if the code never changes,

    it's not costing us money.” Sandi Metz Complexity Churn
  74. 100 0 1 100_000 Everybody understands these files but you

    need to change them often Complexity Churn
  75. 100 0 1 100_000 Complexity Churn These modules are complex

    and change a lot… '
  76. 100 0 1 100_000 Complexity Churn “Sometimes a class becomes

    so complex that refactoring seems too difficult.” Michael Feathers
  77. Are you getting into a tar pit, is it a

    dumpster fire, or have you found a project which is easy to maintain?
  78. 100 0 1 100_000 Welcome to the tar pit. Complexity

    Churn
  79. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic)

  80. Signal #1 Code Coverage

  81. Signal #2 Complexity

  82. Coverage & Complexity

  83. But what if we could calculate a new signal that

    combines both?
  84. Signal #3 StinkScore

  85. StinkScore = f(code_quality, code_coverage)

  86. StinkScore = f(code_smells, code_complexity, code_coverage)

  87. Files which lack tests should be penalized

  88. foo.rb complexity: 10 smells: 10 smell points: 100 (10*10)

  89. bar.rb complexity: 10 smells: 10 smell points: 100 (10*10)

  90. foo.rb complexity: 10 smells: 10 smell points: 100 (10*10) bar.rb

    complexity: 10 smells: 10 smell points: 100 (10*10)
  91. foo.rb and bar.rb are equally stinky

  92. foo.rb complexity: 10 smells: 10 code_coverage: 0 stink_score: 10,000 (10*10)*100

  93. bar.rb complexity: 10 smells: 10 code_coverage: 100 stink_score: 100 (10*10)*1

  94. foo.rb complexity: 10 smells: 10 code_coverage: 0 stink_score: 10,000 (10*10)*100

    bar.rb complexity: 10 smells: 10 code_coverage: 100 stink_score: 100 (10*10)*1
  95. foo.rb is considerably stinkier than bar.rb

  96. How can we automate this analysis?

  97. Skunk A Stink Score Calculator (github.com/fastruby/skunk)

  98. Skunk $ gem install skunk

  99. Skunk $ skunk

  100. None
  101. None
  102. Skunk $ skunk ... StinkScore Total: 13231.069999999996 Modules Analysed: 71

    StinkScore Average: 186.3530985915493 Worst StinkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
  103. Skunk $ skunk ... StinkScore Total: 13231.069999999996 Modules Analysed: 71

    StinkScore Average: 186.3530985915493 Worst StinkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
  104. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) +

    (complexity / COMPLEXITY_FACTOR) end end end
  105. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) +

    # From Reek (complexity / COMPLEXITY_FACTOR) end end end
  106. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) +

    # From Reek (complexity / COMPLEXITY_FACTOR) # From Flog end end end
  107. Skunk module RubyCritic class AnalysedModule def stink_score return cost if

    perfect_coverage? cost * (PERFECT_COVERAGE - coverage.to_i) end end end
  108. Skunk module RubyCritic class AnalysedModule def stink_score return cost if

    perfect_coverage? cost * (PERFECT_COVERAGE - coverage.to_i) # Penalty Factor = 100% - 20% => 80 end end end
  109. Skunk Warning > skunk-v0.3.1

  110. Skunk Warning > skunk-v0.3.1 > COVERAGE=TRUE rake test

  111. Skunk Warning > skunk-v0.3.1 > COVERAGE=TRUE rake test > feedback

    wanted
  112. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic) 3.

    StinkScore (Skunk)
  113. You are here:

  114. Part 2: Getting out of the tar pit

  115. So, where do we begin?

  116. removing files will decrease our stink_score

  117. Find Dead Code Coverband (github.com/danmayer/coverband)

  118. None
  119. refactoring complex files will decrease our stink_score

  120. 100 0 1 100_000 Complexity Churn “Here be dragons. These

    modules are complex and change a lot…” '
  121. 100 0% 1 100_000 Complexity Churn “Great candidates for refactoring

    (paying technical debt!)” #
  122. Churn vs. Complexity is not enough to prioritize

  123. Stink Score Table

  124. Refactor These Files:

  125. Refactor These Files:

  126. 100 0 1 100_000 Complexity Churn git.rb

  127. 100 0 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb refactor! git.rb

  128. 100 0 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb

  129. Skunk $ skunk -b master Base branch (master) average stink

    score: 79.78 Feature branch (refactor/git) average stink score: 71.79
  130. Skunk $ skunk -b master Base branch (master) average stink

    score: 79.78 Feature branch (refactor/git) average stink score: 71.79 10% better ( ( (
  131. writing tests for our app will decrease our stink_score

  132. Stink Score Table

  133. Write Tests Here:

  134. Skunk* $ skunk -b master Base branch (master) average stink

    score: 79.78 Feature branch (refactor/git) average stink score: 71.79
  135. Time StinkScore Average 100 (days) 0 1 100

  136. stink_score Code Coverage; Complexity; and Smells

  137. stink_score Your compass to get out of the tar pit

  138. Thank you! @etagwerker 138 Thank you! ‣ github.com/fastruby/skunk ‣ @OmbuLabs

    ‣ @fastrubyio ‣ @etagwerker
  139. Resources 1. https://github.com/whitesmith/rubycritic 2.https://github.com/colszowka/simplecov 3.https://github.com/metricfu/metric_fu 4.https://github.com/julianrubisch/attractor 5.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 6.https://www.reddit.com/r/ruby/comments/2bq092/rubycritic/ 7.http://jakescruggs.blogspot.com/2008/08/whats-good-flog-score.html 8.http://ruby.sadi.st/Flog.html

    9.http://ruby.sadi.st/Flay.html 10.https://github.com/makaroni4/sandi_meter 11.https://www.fastruby.io/blog/code-quality/intruducing-skunk-stink-score- calculator.html
  140. Resources 1.https://codeclimate.com/blog/deciphering-ruby-code-metrics/ 2.https://www.stickyminds.com/article/getting-empirical-about-refactoring 3.https://www.sandimetz.com/blog/2017/9/13/breaking-up-the-behemoth 4.https://github.com/troessner/reek 5.https://github.com/seattlerb/flay 6.https://github.com/seattlerb/flog 7.http://www.sqa.net/iso9126.html 8.https://www.slideshare.net/mscottford/important-metrics-for-measuring-code-health 9.https://dilbert.com/strip/2006-12-08

    10.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 11.https://www.fastruby.io/blog/code-quality/code-coverage/rubycritic-4-2-0- simplecov-support.html