Escaping The Tar Pit at RubyConf AU 2020

Escaping The Tar Pit at RubyConf AU 2020

Nobody wants to inherit a project that reeks but here we are: Stuck in the tar pit. How can we get out? Could we have avoided it in the first place? In this talk you will learn how to use a few, great Ruby gems that will guide you out of that sticky tar you are in.

Thanks to RubyConf AU and https://fastruby.io for sponsoring my trip to Melbourne! ❤️

Also, in case you missed my talk and want to watch it online. You can find it over here: https://youtu.be/ORb2UQ0_8zg

F77032adcbe77d2777bb0e0c30873159?s=128

Ernesto Tagwerker

February 20, 2020
Tweet

Transcript

  1. Escaping The Tar Pit Ernesto Tagwerker (@etagwerker) RubyConf Australia, February

    20th, 2020
  2. Hi, I’m from Argentina I live in Philadelphia I ❤

    Open Source
  3. Open Source Maintainer database_cleaner; bundler-leak; next_rails

  4. Founder & Software Engineer @OmbuLabs

  5. Founder & Software Engineer @fastrubyio

  6. Inspiration

  7. Chapter 1: The Tar Pit

  8. “Large-system programming has over the past decade been such a

    tar pit, and many great and powerful beasts have thrashed violently in it.” Fred Brooks
  9. “The fiercer the struggle, the more entangling the tar, and

    no beast is so strong or so skillful but that they ultimately sink.” Fred Brooks
  10. We are in the tar pit or trying to avoid

    it
  11. None
  12. Tar Pit Symptoms > Projects running over budget

  13. Tar Pit Symptoms > Projects running over budget > Taking

    forever to ship small changes
  14. Tar Pit Symptoms > Projects running over budget > Taking

    forever to ship small changes > Sacrificing quality, increasing tech debt
  15. None
  16. Part 1: How to avoid the tar pit

  17. Part 2: How to get out of the tar pit

  18. Part 1: Avoiding the tar pit

  19. “THAT’D BE GREAT” BOSS: “IF YOU COULD COME IN ON

    THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
  20. “THAT’D BE GREAT” CLIENT: “IF YOU COULD COME IN ON

    THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
  21. CLIENT: “HOW LONG IS IT GOING TO TAKE?”

  22. “BECAUSE I NEED IT BY END OF DAY TODAY” CLIENT:

    “HOW LONG IS IT GOING TO TAKE?”
  23. How can we quickly assess code quality?

  24. Assessing Quality ✅ Paid Services ✅ Open Source Gems

  25. Assessing Quality ✅ Paid Services ➡ Open Source Gems

  26. Open Source Gems ✅ Static code analysis

  27. Open Source Gems ✅ Static code analysis ✅ Code coverage

  28. Open Source Gems ✅ Static code analysis ✅ Code coverage

    ✅ Code smells
  29. What is software quality anyway?

  30. There are hundreds of definitions

  31. <rant>

  32. <rant>

  33. You can define your own baseline of “software quality”

  34. </rant>

  35. “The degree to which a system, component, or process meets

    implicit and explicit requirements.” IEEE
  36. “It works as expected.”

  37. “It works as expected and it is not a PITA

    to maintain it.”
  38. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅

    Efficiency ✅ Maintainability ✅ Portability
  39. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅

    Efficiency ➡ Maintainability ✅ Portability
  40. Maintainability

  41. Maintainability 1. Code Coverage

  42. Maintainability 1. Code Coverage 2. Code Quality

  43. Code Coverage SimpleCov (github.com/colszowka/simplecov)

  44. Code Coverage # Gemfile group :test do gem "simplecov", require:

    false end
  45. Code Coverage # spec/spec_helper.rb if ENV["COVERAGE"] == "true" require 'simplecov'

    SimpleCov.start do add_group "Models", "models" add_filter "/spec/" track_files "**/*.rb" end end
  46. Code Coverage $ COVERAGE=true rake test

  47. Code Coverage

  48. Code Quality n options

  49. Code Quality n options (flog; flay; reek; churn; RubyCritic; MetricFu;

    attractor; rubocop; fukuzatsu; turbulence; …)
  50. Code Quality RubyCritic (github.com/whitesmith/rubycritic)

  51. Churn

  52. Churn git

  53. Churn $ git log --follow --format=%h default.rb | cat 573c8f5

    2ccc5b8 b447bc3
  54. Churn $ git log --follow --format=%h default.rb | cat 573c8f5

    2ccc5b8 b447bc3 # Churn Count = 3
  55. Complexity

  56. Complexity flog

  57. Complexity $ flog default.rb => 10.8: flog total

  58. Complexity class Default def yay # 10.8 = a =

    eval "1+1" # 1.2 + 6.0 + if a == 2 # 1.2 + 1.2 + puts "yay" # 1.2 end end end
  59. Complexity $ flog foo.rb => 10.8: flog total

  60. Churn vs. Complexity

  61. Churn Complexity 100 0 1 100_000 user.rb (changed: 2 times;

    complexity: 100_000 )
  62. Churn Complexity 100 0 1 100_000 user.rb (changed: 27 times;

    complexity: 100)
  63. Code Quality $ gem install rubycritic

  64. Code Quality $ rubycritic

  65. None
  66. None
  67. GPA Pie Graph Complexity (flog) Code Smells (reek)

  68. Churn vs. Complexity Churn (SCM) Complexity (flog)

  69. 100 0 1 100_000 Complexity Churn

  70. 100 0 1 100_000 Complexity Churn

  71. 100 0 1 100_000 Complexity Churn

  72. 100 0 1 100_000 “Welcome to the good place.” ❤

    Complexity Churn
  73. 100 0 1 100_000 No one understands these files but

    they work. So don’t change them. Complexity Churn
  74. 100 0 1 100_000 “[…], if the code never changes,

    it's not costing us money.” Sandi Metz Complexity Churn
  75. 100 0 1 100_000 Everybody understands these files but you

    need to change them often Complexity Churn
  76. 100 0 1 100_000 Complexity Churn These modules are complex

    and are constantly changing
  77. 100 0 1 100_000 Complexity Churn “Sometimes a class becomes

    so complex that refactoring seems too difficult.” Michael Feathers
  78. Are you getting into a tar pit, is it a

    dumpster fire, or have you found a project which is easy to maintain?
  79. 100 0 1 100_000 Welcome to the tar pit. Complexity

    Churn
  80. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic)

  81. Signal #1 Code Coverage

  82. Signal #2 Complexity

  83. Coverage & Complexity

  84. But what if we could calculate a new signal that

    combines both?
  85. Signal #3 StinkScore

  86. StinkScore = f(code_quality, code_coverage)

  87. Complex files which lack tests should be our top priority

  88. foo.rb complexity: 10 smells: 10 smell points: 100 (10*10)

  89. bar.rb complexity: 10 smells: 10 smell points: 100 (10*10)

  90. foo.rb complexity: 10 smells: 10 smell points: 100 (10*10) bar.rb

    complexity: 10 smells: 10 smell points: 100 (10*10)
  91. foo.rb and bar.rb are equally stinky*

  92. foo.rb complexity: 10 smells: 10 code_coverage: 0 stink_score: 10,000 (10*10)*100

  93. bar.rb complexity: 10 smells: 10 code_coverage: 100 stink_score: 100 (10*10)*1

  94. foo.rb complexity: 10 smells: 10 code_coverage: 0 stink_score: 10,000 (10*10)*100

    bar.rb complexity: 10 smells: 10 code_coverage: 100 stink_score: 100 (10*10)*1
  95. foo.rb is considerably stinkier than bar.rb

  96. foo.rb complexity: 10 smells: 10 code_coverage: 0 stink_score: 10,000 (10*10)*100

    You should write some tests for foo.rb bar.rb complexity: 10 smells: 10 code_coverage: 100 stink_score: 100 (10*10)*1
  97. How can we automate this analysis?

  98. A Stink Score Calculator (github.com/fastruby/skunk)

  99. Skunk $ gem install skunk

  100. Skunk $ skunk

  101. None
  102. Skunk $ skunk ... StinkScore Total: 13231.069999999996 Modules Analysed: 71

    StinkScore Average: 186.3530985915493 Worst StinkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
  103. Skunk $ skunk ... StinkScore Total: 13231.069999999996 Modules Analysed: 71

    StinkScore Average: 186.3530985915493 Worst StinkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
  104. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) +

    (complexity / COMPLEXITY_FACTOR) end end end
  105. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) +

    # From Reek (complexity / COMPLEXITY_FACTOR) end end end
  106. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) +

    # From Reek (complexity / COMPLEXITY_FACTOR) # From Flog end end end
  107. Skunk module RubyCritic class AnalysedModule def stink_score return cost if

    perfect_coverage? cost * (PERFECT_COVERAGE - coverage.to_i) end end end
  108. Skunk module RubyCritic class AnalysedModule def stink_score return cost if

    perfect_coverage? cost * (PERFECT_COVERAGE - coverage.to_i) # Penalty Factor = 100% - 20% => 80 end end end
  109. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic) 3.

    StinkScore (Skunk)
  110. You are here:

  111. Part 2: Getting out of the tar pit

  112. How can we gradually pay off technical debt?

  113. 1. Removing files 2. Refactoring files 3. Increasing coverage

  114. 1. Removing files

  115. Find Dead Code Coverband (github.com/danmayer/coverband)

  116. None
  117. 2.Refactoring files

  118. 100 0 1 100_000 Complexity Churn

  119. 100 0 1 100_000 Complexity Churn “Great candidates for refactoring

    (paying technical debt!)”
  120. Churn vs. Complexity is not enough to prioritize

  121. Stink Score Table

  122. Refactor These Files:

  123. Refactor These Files:

  124. 100 0 1 100_000 Complexity Churn git.rb

  125. 100 0 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb refactor! git.rb

  126. Skunk $ skunk -b master Base branch (master) average stink

    score: 79.78 Feature branch (refactor/git) average stink score: 71.79
  127. Skunk $ skunk -b master Base branch (master) average stink

    score: 79.78 Feature branch (refactor/git) average stink score: 71.79 10% better
  128. 2. Asdf 3. Increasing coverage

  129. Stink Score Table

  130. Write Tests Here:

  131. Maintainability Code Coverage; Complexity; and Smells

  132. Stink Score A new way to communicate about tech debt

  133. “I paid off some tech debt with this PR.”

  134. “I paid off 5% of our tech debt with this

    PR.”
  135. “This project is in pretty good shape.”

  136. “This project has a stink score average of 120.”

  137. Stink Score -Open Source -Configurable

  138. Stink Score Your compass to get out of the tar

    pit
  139. Time StinkScore Average 100 (days) 0 1 100

  140. Thank you! @etagwerker 140 ‣ github.com/fastruby/skunk ‣ @OmbuLabs ‣ @fastrubyio

    ‣ @etagwerker
  141. None
  142. Thank you! @etagwerker 142 Thank you!

  143. Resources 1. https://github.com/whitesmith/rubycritic 2.https://github.com/colszowka/simplecov 3.https://github.com/metricfu/metric_fu 4.https://github.com/julianrubisch/attractor 5.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 6.https://www.reddit.com/r/ruby/comments/2bq092/rubycritic/ 7.http://jakescruggs.blogspot.com/2008/08/whats-good-flog-score.html 8.http://ruby.sadi.st/Flog.html

    9.http://ruby.sadi.st/Flay.html 10.https://github.com/makaroni4/sandi_meter 11.https://www.fastruby.io/blog/code-quality/intruducing-skunk-stink-score- calculator.html
  144. Resources 1.https://codeclimate.com/blog/deciphering-ruby-code-metrics/ 2.https://www.stickyminds.com/article/getting-empirical-about-refactoring 3.https://www.sandimetz.com/blog/2017/9/13/breaking-up-the-behemoth 4.https://github.com/troessner/reek 5.https://github.com/seattlerb/flay 6.https://github.com/seattlerb/flog 7.http://www.sqa.net/iso9126.html 8.https://www.slideshare.net/mscottford/important-metrics-for-measuring-code-health 9.https://dilbert.com/strip/2006-12-08

    10.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 11.https://www.fastruby.io/blog/code-quality/code-coverage/rubycritic-4-2-0- simplecov-support.html