Escaping The Tar Pit at SoCraTes

Escaping The Tar Pit at SoCraTes

Nobody wants to inherit a project that reeks but here we are: Stuck in the tar pit. How can we get out? Could we have avoided it in the first place?

In this talk you will learn how to use a few, great libraries that will guide you out of that sticky tar you are in. On top of that, you will learn a repeatable way to gradually pay off technical debt.

F77032adcbe77d2777bb0e0c30873159?s=128

Ernesto Tagwerker

April 01, 2020
Tweet

Transcript

  1. Escaping The Tar Pit Ernesto Tagwerker (@etagwerker) SoCraTes, April 1,

    2020
  2. Hi, I’m from Argentina I live in Philadelphia I ❤

    Open Source
  3. Open Source Maintainer

  4. Founder & Software Engineer @OmbuLabs

  5. Founder & Software Engineer @fastrubyio

  6. Inspiration

  7. Chapter 1: The Tar Pit

  8. “Large-system programming has over the past decade been such a

    tar pit, and many great and powerful beasts have thrashed violently in it.” Fred Brooks
  9. “The fiercer the struggle, the more entangling the tar, and

    no beast is so strong or so skillful but that they ultimately sink.” Fred Brooks
  10. We are in the tar pit or trying to avoid

    it
  11. None
  12. Tar Pit Symptoms > Projects running over budget

  13. Tar Pit Symptoms > Projects running over budget > Taking

    forever to ship small changes
  14. Tar Pit Symptoms > Projects running over budget > Taking

    forever to ship small changes > Sacrificing quality, increasing tech debt
  15. None
  16. Part 1: How to avoid the tar pit

  17. Part 2: How to get out of the tar pit

  18. Part 1: Avoiding the tar pit

  19. “THAT’D BE GREAT” BOSS: “IF YOU COULD COME IN ON

    THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
  20. “THAT’D BE GREAT” CLIENT: “IF YOU COULD COME IN ON

    THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
  21. CLIENT: “HOW LONG IS IT GOING TO TAKE?”

  22. “BECAUSE I NEED IT BY END OF DAY TODAY” CLIENT:

    “HOW LONG IS IT GOING TO TAKE?”
  23. How can we quickly assess code quality?

  24. Assessing Quality ✅ Paid Services ✅ Open Source Gems

  25. Assessing Quality ✅ Paid Services ➡ Open Source Gems

  26. Open Source Gems ✅ Static code analysis

  27. Open Source Gems ✅ Static code analysis ✅ Code coverage

  28. Open Source Gems ✅ Static code analysis ✅ Code coverage

    ✅ Code smells
  29. What is software quality anyway?

  30. There are hundreds of definitions

  31. <rant>

  32. <rant>

  33. You can define your own baseline of “software quality”

  34. </rant>

  35. “The degree to which a system, component, or process meets

    implicit and explicit requirements.” IEEE
  36. “It works as expected.”

  37. “It works as expected and it is not a PITA

    to maintain it.”
  38. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅

    Efficiency ✅ Maintainability ✅ Portability
  39. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅

    Efficiency ➡ Maintainability ✅ Portability
  40. Maintainability

  41. Maintainability 1. Code Coverage

  42. Maintainability 1. Code Coverage 2. Code Quality

  43. Code Coverage SimpleCov (github.com/colszowka/simplecov)

  44. Code Coverage # Gemfile group :test do gem "simplecov", require:

    false end
  45. Code Coverage $ COVERAGE=true rake test

  46. Code Coverage

  47. Code Quality n options

  48. Code Quality n options (flog; flay; reek; churn; RubyCritic; MetricFu;

    attractor; rubocop; fukuzatsu; turbulence; …)
  49. Code Quality RubyCritic (github.com/whitesmith/rubycritic)

  50. Churn

  51. Churn git

  52. Churn $ git log --follow --format=%h default.rb | cat 573c8f5

    2ccc5b8 b447bc3
  53. Churn $ git log --follow --format=%h default.rb | cat 573c8f5

    2ccc5b8 b447bc3 # Churn Count = 3
  54. Complexity

  55. Complexity flog

  56. Complexity $ flog default.rb => 10.8: flog total

  57. Complexity class Default def yay # 10.8 = a =

    eval "1+1" # 1.2 + 6.0 + if a == 2 # 1.2 + 1.2 + puts "yay" # 1.2 end end end
  58. Complexity $ flog user.rb => 10.8: flog total

  59. Churn vs. Complexity

  60. Churn Complexity 100 0 1 100_000 user.rb (changed: 2 times;

    complexity: 100_000 )
  61. Churn Complexity 100 0 1 100_000 user.rb (changed: 27 times;

    complexity: 100)
  62. Code Quality $ gem install rubycritic

  63. Code Quality $ rubycritic

  64. None
  65. None
  66. GPA Pie Graph Complexity (flog) Code Smells (reek)

  67. Churn vs. Complexity Churn (SCM) Complexity (flog)

  68. 100 0 1 100_000 Complexity Churn

  69. 100 0 1 100_000 Complexity Churn

  70. 100 0 1 100_000 Complexity Churn

  71. 100 0 1 100_000 “Welcome to the good place.” ❤

    Complexity Churn
  72. 100 0 1 100_000 No one understands these files but

    they work. So don’t change them. Complexity Churn
  73. 100 0 1 100_000 “[…], if the code never changes,

    it's not costing us money.” Sandi Metz Complexity Churn
  74. 100 0 1 100_000 Everybody understands these files but you

    need to change them often Complexity Churn
  75. 100 0 1 100_000 Complexity Churn These modules are complex

    and are constantly changing
  76. 100 0 1 100_000 Complexity Churn “Sometimes a class becomes

    so complex that refactoring seems too difficult.” Michael Feathers
  77. Are you getting into a tar pit, is it a

    dumpster fire, or have you found a project which is easy to maintain?
  78. 100 0 1 100_000 Welcome to the tar pit. Complexity

    Churn
  79. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic)

  80. Signal #1 Code Coverage

  81. Signal #2 Complexity

  82. Coverage & Complexity

  83. But what if we could calculate a new signal that

    combines both?
  84. Signal #3 SkunkScore

  85. SkunkScore = f(code_quality, code_coverage)

  86. Complex files which lack tests should be our top priority

  87. user.rb complexity: 10 smells: 10 smell points: 100 (10*10)

  88. account.rb complexity: 10 smells: 10 smell points: 100 (10*10)

  89. user.rb complexity: 10 smells: 10 smell points: 100 (10*10) account.rb

    complexity: 10 smells: 10 smell points: 100 (10*10)
  90. user.rb and account.rb are equally stinky*

  91. Are user.rb and account.rb equally stinky?

  92. user.rb complexity: 10 smells: 10 code_coverage: 0 skunk_score: 10,000 (10*10)*100

  93. account.rb complexity: 10 smells: 10 code_coverage: 100 skunk_score: 100 (10*10)*1

  94. user.rb complexity: 10 smells: 10 code_coverage: 0 skunk_score: 10,000 (10*10)*100

    account.rb complexity: 10 smells: 10 code_coverage: 100 skunk_score: 100 (10*10)*1
  95. user.rb is considerably stinkier than account.rb

  96. user.rb complexity: 10 smells: 10 code_coverage: 0 skunk_score: 10,000 (10*10)*100

    You should write some tests for user.rb account.rb complexity: 10 smells: 10 code_coverage: 100 skunk_score: 100 (10*10)*1
  97. How can we automate this analysis?

  98. A SkunkScore Calculator (github.com/fastruby/skunk)

  99. Skunk $ gem install skunk

  100. Skunk $ skunk

  101. None
  102. Skunk $ skunk ... SkunkScore Total: 13231.069999999996 Modules Analysed: 71

    SkunkScore Average: 186.3530985915493 Worst SkunkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
  103. Skunk $ skunk ... SkunkScore Total: 13231.069999999996 Modules Analysed: 71

    SkunkScore Average: 186.3530985915493 Worst SkunkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
  104. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) +

    (complexity / COMPLEXITY_FACTOR) end end end
  105. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) +

    # From Reek (complexity / COMPLEXITY_FACTOR) end end end
  106. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) +

    # From Reek (complexity / COMPLEXITY_FACTOR) # From Flog end end end
  107. Skunk module RubyCritic class AnalysedModule def skunk_score return cost if

    perfect_coverage? cost * (PERFECT_COVERAGE - coverage.to_i) end end end
  108. Skunk module RubyCritic class AnalysedModule def skunk_score return cost if

    perfect_coverage? cost * (PERFECT_COVERAGE - coverage.to_i) # Penalty Factor = 100% - 20% => 80 end end end
  109. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic) 3.

    SkunkScore (Skunk)
  110. You are here:

  111. Part 2: Getting out of the tar pit

  112. How can we gradually pay off technical debt?

  113. 1. Removing files 2. Refactoring files 3. Increasing coverage

  114. 1. Removing files

  115. Find Dead Code Coverband (github.com/danmayer/coverband)

  116. None
  117. 2.Refactoring files

  118. 100 0 1 100_000 Complexity Churn

  119. 100 0 1 100_000 Complexity Churn “Great candidates for refactoring

    (paying technical debt!)”
  120. Churn vs. Complexity is not enough to prioritize

  121. SkunkScore Table

  122. Refactor These Files:

  123. Refactor These Files:

  124. 100 0 1 100_000 Complexity Churn git.rb

  125. 100 0 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb refactor! git.rb

  126. Skunk $ skunk -b master Base branch (master) average skunk

    score: 79.78 Feature branch (refactor/git) average skunk score: 71.79
  127. Skunk $ skunk -b master Base branch (master) average skunk

    score: 79.78 Feature branch (refactor/git) average skunk score: 71.79 10% better
  128. 2. Asdf 3. Increasing coverage

  129. SkunkScore Table

  130. Write Tests Here:

  131. Maintainability Code Coverage; Complexity; and Smells

  132. SkunkScore A new way to communicate about tech debt

  133. “I paid off some tech debt with this PR.”

  134. “I paid off 5% of our tech debt with this

    PR.”
  135. “This project is in pretty good shape.”

  136. “This project has a SkunkScore average of 120.”

  137. SkunkScore -Open Source -Configurable

  138. SkunkScore Your compass to get out of the tar pit

  139. Time SkunkScore Average 100 (days) 0 1 100

  140. Thank you! @etagwerker 140 ‣ github.com/fastruby/skunk ‣ @OmbuLabs ‣ @fastrubyio

    ‣ @etagwerker
  141. None
  142. Thank you! @etagwerker 142 Thank you!

  143. Resources 1. https://github.com/whitesmith/rubycritic 2.https://github.com/colszowka/simplecov 3.https://github.com/metricfu/metric_fu 4.https://github.com/julianrubisch/attractor 5.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 6.https://www.reddit.com/r/ruby/comments/2bq092/rubycritic/ 7.http://jakescruggs.blogspot.com/2008/08/whats-good-flog-score.html 8.http://ruby.sadi.st/Flog.html

    9.http://ruby.sadi.st/Flay.html 10.https://github.com/makaroni4/sandi_meter 11.https://www.fastruby.io/blog/code-quality/intruducing-skunk-stink-score- calculator.html
  144. Resources 1.https://codeclimate.com/blog/deciphering-ruby-code-metrics/ 2.https://www.stickyminds.com/article/getting-empirical-about-refactoring 3.https://www.sandimetz.com/blog/2017/9/13/breaking-up-the-behemoth 4.https://github.com/troessner/reek 5.https://github.com/seattlerb/flay 6.https://github.com/seattlerb/flog 7.http://www.sqa.net/iso9126.html 8.https://www.slideshare.net/mscottford/important-metrics-for-measuring-code-health 9.https://dilbert.com/strip/2006-12-08

    10.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 11.https://www.fastruby.io/blog/code-quality/code-coverage/rubycritic-4-2-0- simplecov-support.html