Escaping The Tar Pit at NYC.rb

Nobody wants to inherit a project that reeks but here we are: Stuck in the tar pit. How can we get out? Could we have avoided it in the first place?

In this talk you will learn how to use a few, great Ruby gems that will guide you out of that sticky tar you are in. On top of that, you will learn a repeatable way to gradually pay off technical debt.

Transcript

1. Escaping The Tar Pit Ernesto Tagwerker (@etagwerker) NYC.rb, May 13th, 2020
2. Hi, I’m from Argentina I live in Philadelphia I ❤ Open Source
3. Open Source Maintainer database_cleaner; bundler-leak; next_rails
4. Founder & Software Engineer OmbuLabs
5. Founder & Software Engineer FastRuby.io
6. Inspiration
7. Chapter 1: The Tar Pit
8. “Large-system programming has over the past decade been such a tar pit, and many great and powerful beasts have thrashed violently in it.” Fred Brooks
9. “The fiercer the struggle, the more entangling the tar, and no beast is so strong or so skillful but that they ultimately sink.” Fred Brooks
10. We are in the tar pit or trying to avoid it
12. Tar Pit Symptoms > Projects running over budget
13. Tar Pit Symptoms > Projects running over budget > Taking forever to ship small changes
14. Tar Pit Symptoms > Projects running over budget > Taking forever to ship small changes > Sacrificing quality, increasing tech debt
16. Part 1: How to avoid the tar pit
17. Part 2: How to get out of the tar pit
18. Part 1: Avoiding the tar pit
19. “THAT’D BE GREAT” BOSS: “IF YOU COULD COME IN ON THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
20. “THAT’D BE GREAT” CLIENT: “IF YOU COULD COME IN ON THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
21. CLIENT: “HOW LONG IS IT GOING TO TAKE?”
22. “BECAUSE I NEED IT BY END OF DAY TODAY” CLIENT: “HOW LONG IS IT GOING TO TAKE?”
23. How can we quickly assess code quality?
24. Assessing Quality ✅ Paid Services ✅ Open Source Gems
25. Assessing Quality ✅ Paid Services ➡ Open Source Gems
26. Open Source Gems ✅ Static code analysis
27. Open Source Gems ✅ Static code analysis ✅ Code coverage
28. Open Source Gems ✅ Static code analysis ✅ Code coverage ✅ Code smells
29. What is software quality anyway?
30. There are hundreds of definitions
31. <rant>
32. <rant>
33. You can define your own baseline of “software quality”
34. </rant>
35. “The degree to which a system, component, or process meets implicit and explicit requirements.” IEEE
36. “It works as expected.”
37. “It works as expected and it is not a PITA to maintain it.”
38. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅ Efficiency ✅ Maintainability ✅ Portability
39. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅ Efficiency ➡ Maintainability ✅ Portability
40. Maintainability
41. Maintainability 1. Code Coverage
42. Maintainability 1. Code Coverage 2. Code Quality
43. Code Coverage SimpleCov (github.com/colszowka/simplecov)
44. Code Coverage # Gemfile group :test do gem "simplecov", require: false end
45. Code Coverage # spec/spec_helper.rb if ENV["COVERAGE"] == "true" require 'simplecov' SimpleCov.start do add_group "Models", "models" add_filter "/spec/" track_files "**/*.rb" end end
46. Code Coverage $ COVERAGE=true rake test
47. Code Coverage
48. Code Quality n options
49. Code Quality n options (flog; flay; reek; churn; RubyCritic; MetricFu; attractor; rubocop; fukuzatsu; turbulence; …)
50. Code Quality RubyCritic (github.com/whitesmith/rubycritic)
51. Churn
52. Churn git
53. Churn $ git log --follow --format=%h default.rb | cat 573c8f5 2ccc5b8 b447bc3
54. Churn $ git log --follow --format=%h default.rb | cat 573c8f5 2ccc5b8 b447bc3 # Churn Count = 3
55. Complexity
56. Complexity flog
57. Complexity $ flog default.rb => 10.8: flog total
58. Complexity class Default def yay # 10.8 = a = eval "1+1" # 1.2 + 6.0 + if a == 2 # 1.2 + 1.2 + puts "yay" # 1.2 end end end
59. Complexity $ flog user.rb => 10.8: flog total
60. Churn vs. Complexity
61. Churn Complexity 100 0 1 100_000 user.rb (changed: 2 times; complexity: 100_000 )
62. Churn Complexity 100 0 1 100_000 user.rb (changed: 27 times; complexity: 100)
63. Code Quality $ gem install rubycritic
64. Code Quality $ rubycritic
67. GPA Pie Graph Complexity (flog) Code Smells (reek)
68. Churn vs. Complexity Churn (SCM) Complexity (flog)
69. 100 0 1 100_000 Complexity Churn
70. 100 0 1 100_000 Complexity Churn
71. 100 0 1 100_000 Complexity Churn
72. 100 0 1 100_000 “Welcome to the good place.” ❤ Complexity Churn
73. 100 0 1 100_000 No one understands these files but they work. So don’t change them. Complexity Churn
74. 100 0 1 100_000 “[…], if the code never changes, it's not costing us money.” Sandi Metz Complexity Churn
75. 100 0 1 100_000 Everybody understands these files but you need to change them often Complexity Churn
76. 100 0 1 100_000 Complexity Churn These modules are complex and are constantly changing
77. 100 0 1 100_000 Complexity Churn “Sometimes a class becomes so complex that refactoring seems too difficult.” Michael Feathers
78. Are you getting into a tar pit, is it a dumpster fire, or have you found a project which is easy to maintain?
79. 100 0 1 100_000 Welcome to the tar pit. Complexity Churn
80. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic)
81. Signal #1 Code Coverage
82. Signal #2 Complexity
83. Coverage & Complexity
84. But what if we could calculate a new signal that combines both?
85. Signal #3 SkunkScore
86. SkunkScore = f(code_quality, code_coverage)
87. Complex files which lack tests should be our top priority
88. user.rb complexity: 10 smells: 10 smell points: 100 (10*10)
89. shop.rb complexity: 10 smells: 10 smell points: 100 (10*10)
90. user.rb complexity: 10 smells: 10 smell points: 100 (10*10) shop.rb complexity: 10 smells: 10 smell points: 100 (10*10)
91. user.rb and shop.rb are equally stinky*
92. user.rb complexity: 10 smells: 10 code_coverage: 0 skunk_score: 10,000 (10*10)*100
93. shop.rb complexity: 10 smells: 10 code_coverage: 100 skunk_score: 100 (10*10)*1
94. user.rb complexity: 10 smells: 10 code_coverage: 0 skunk_score: 10,000 (10*10)*100 shop.rb complexity: 10 smells: 10 code_coverage: 100 skunk_score: 100 (10*10)*1
95. shop.rb smells considerably better than user.rb
96. user.rb complexity: 10 smells: 10 code_coverage: 0 skunk_score: 10,000 (10*10)*100 We should write some tests for user.rb shop.rb complexity: 10 smells: 10 code_coverage: 100 skunk_score: 100 (10*10)*1
97. How can we automate this analysis?
98. A Skunk Score Calculator (github.com/fastruby/skunk)
99. Skunk $ gem install skunk
100. Skunk $ skunk
102. Skunk $ skunk ... SkunkScore Total: 13231.069999999996 Modules Analysed: 71 SkunkScore Average: 186.3530985915493 Worst SkunkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
103. Skunk $ skunk ... SkunkScore Total: 13231.069999999996 Modules Analysed: 71 SkunkScore Average: 186.3530985915493 Worst SkunkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
104. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) + (complexity / COMPLEXITY_FACTOR) end end end
105. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) + # From Reek (complexity / COMPLEXITY_FACTOR) end end end
106. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) + # From Reek (complexity / COMPLEXITY_FACTOR) # From Flog end end end
107. Skunk module RubyCritic class AnalysedModule def skunk_score return cost if perfect_coverage? cost * (PERFECT_COVERAGE - coverage.to_i) end end end
108. Skunk module RubyCritic class AnalysedModule def skunk_score return cost if perfect_coverage? cost * (PERFECT_COVERAGE - coverage.to_i) # Penalty Factor = 100% - 20% => 80 end end end
109. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic) 3. SkunkScore (Skunk)
110. You are here:
111. Part 2: Getting out of the tar pit
112. How can we gradually pay off technical debt?
113. 1. Removing files 2. Refactoring files 3. Increasing coverage
114. 1. Removing files
115. Find Dead Code Coverband (github.com/danmayer/coverband)
117. 2.Refactoring files
118. 100 0 1 100_000 Complexity Churn
119. 100 0 1 100_000 Complexity Churn “Great candidates for refactoring (paying technical debt!)”
120. Churn vs. Complexity is not enough to prioritize
121. Skunk Score Table
122. Refactor These Files:
123. Refactor These Files:
124. 100 0 1 100_000 Complexity Churn git.rb
125. 100 0 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb refactor! git.rb
126. Skunk $ skunk -b master Base branch (master) average skunk score: 79.78 Feature branch (refactor/git) average skunk score: 71.79
127. Skunk $ skunk -b master Base branch (master) average skunk score: 79.78 Feature branch (refactor/git) average skunk score: 71.79 10% better
128. 2. Asdf 3. Increasing coverage
129. Skunk Score Table
130. Write Tests Here:
131. Maintainability Code Coverage; Complexity; and Smells
132. SkunkScore A new way to communicate about tech debt
133. “I paid off some tech debt with this PR.”
134. “I paid off 5% of our tech debt with this PR.”
135. “This project is in pretty good shape.”
136. “This project has a SkunkScore average of 120.”
137. Skunk Score - Open Source - Configurable
138. Skunk Score Your compass to get out of the tar pit
139. Time SkunkScore Average 100 (days) 0 1 100
140. Thank you! @etagwerker 140 ‣ github.com/fastruby/skunk ‣ @OmbuLabs ‣ @fastrubyio ‣ @etagwerker
142. Thank you! @etagwerker 142 Thank you!
143. Resources 1. https://github.com/whitesmith/rubycritic 2.https://github.com/colszowka/simplecov 3.https://github.com/metricfu/metric_fu 4.https://github.com/julianrubisch/attractor 5.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 6.https://www.reddit.com/r/ruby/comments/2bq092/rubycritic/ 7.http://jakescruggs.blogspot.com/2008/08/whats-good-flog-score.html 8.http://ruby.sadi.st/Flog.html 9.http://ruby.sadi.st/Flay.html 10.https://github.com/makaroni4/sandi_meter 11.https://www.fastruby.io/blog/code-quality/intruducing-skunk-stink-score-calculator.html 12.https://github.com/fastruby/skunk
144. Resources 1.https://codeclimate.com/blog/deciphering-ruby-code-metrics/ 2.https://www.stickyminds.com/article/getting-empirical-about-refactoring 3.https://www.sandimetz.com/blog/2017/9/13/breaking-up-the-behemoth 4.https://github.com/troessner/reek 5.https://github.com/seattlerb/flay 6.https://github.com/seattlerb/flog 7.http://www.sqa.net/iso9126.html 8.https://www.slideshare.net/mscottford/important-metrics-for-measuring-code-health 9.https://dilbert.com/strip/2006-12-08 10.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 11.https://www.fastruby.io/blog/code-quality/code-coverage/rubycritic-4-2-0-simplecov- support.html 12.https://www.fastruby.io/blog/code-quality/churn-vs-complexity-vs-coverage.html