Escaping The Tar Pit at SoCraTes

Nobody wants to inherit a project that reeks but here we are: Stuck in the tar pit. How can we get out? Could we have avoided it in the first place?

In this talk you will learn how to use a few, great libraries that will guide you out of that sticky tar you are in. On top of that, you will learn a repeatable way to gradually pay off technical debt.

Transcript

1. Escaping The Tar Pit Ernesto Tagwerker (@etagwerker) SoCraTes, April 1, 2020
2. Hi, I’m from Argentina I live in Philadelphia I ❤ Open Source
3. Open Source Maintainer
4. Founder & Software Engineer @OmbuLabs
5. Founder & Software Engineer @fastrubyio
6. Inspiration
7. Chapter 1: The Tar Pit
8. “Large-system programming has over the past decade been such a tar pit, and many great and powerful beasts have thrashed violently in it.” Fred Brooks
9. “The fiercer the struggle, the more entangling the tar, and no beast is so strong or so skillful but that they ultimately sink.” Fred Brooks
10. We are in the tar pit or trying to avoid it
12. Tar Pit Symptoms > Projects running over budget
13. Tar Pit Symptoms > Projects running over budget > Taking forever to ship small changes
14. Tar Pit Symptoms > Projects running over budget > Taking forever to ship small changes > Sacrificing quality, increasing tech debt
16. Part 1: How to avoid the tar pit
17. Part 2: How to get out of the tar pit
18. Part 1: Avoiding the tar pit
19. “THAT’D BE GREAT” BOSS: “IF YOU COULD COME IN ON THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
20. “THAT’D BE GREAT” CLIENT: “IF YOU COULD COME IN ON THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
21. CLIENT: “HOW LONG IS IT GOING TO TAKE?”
22. “BECAUSE I NEED IT BY END OF DAY TODAY” CLIENT: “HOW LONG IS IT GOING TO TAKE?”
23. How can we quickly assess code quality?
24. Assessing Quality ✅ Paid Services ✅ Open Source Gems
25. Assessing Quality ✅ Paid Services ➡ Open Source Gems
26. Open Source Gems ✅ Static code analysis
27. Open Source Gems ✅ Static code analysis ✅ Code coverage
28. Open Source Gems ✅ Static code analysis ✅ Code coverage ✅ Code smells
29. What is software quality anyway?
30. There are hundreds of definitions
31. <rant>
32. <rant>
33. You can define your own baseline of “software quality”
34. </rant>
35. “The degree to which a system, component, or process meets implicit and explicit requirements.” IEEE
36. “It works as expected.”
37. “It works as expected and it is not a PITA to maintain it.”
38. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅ Efficiency ✅ Maintainability ✅ Portability
39. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅ Efficiency ➡ Maintainability ✅ Portability
40. Maintainability
41. Maintainability 1. Code Coverage
42. Maintainability 1. Code Coverage 2. Code Quality
43. Code Coverage SimpleCov (github.com/colszowka/simplecov)
44. Code Coverage # Gemfile group :test do gem "simplecov", require: false end
45. Code Coverage $ COVERAGE=true rake test
46. Code Coverage
47. Code Quality n options
48. Code Quality n options (flog; flay; reek; churn; RubyCritic; MetricFu; attractor; rubocop; fukuzatsu; turbulence; …)
49. Code Quality RubyCritic (github.com/whitesmith/rubycritic)
50. Churn
51. Churn git
52. Churn $ git log --follow --format=%h default.rb | cat 573c8f5 2ccc5b8 b447bc3
53. Churn $ git log --follow --format=%h default.rb | cat 573c8f5 2ccc5b8 b447bc3 # Churn Count = 3
54. Complexity
55. Complexity flog
56. Complexity $ flog default.rb => 10.8: flog total
57. Complexity class Default def yay # 10.8 = a = eval "1+1" # 1.2 + 6.0 + if a == 2 # 1.2 + 1.2 + puts "yay" # 1.2 end end end
58. Complexity $ flog user.rb => 10.8: flog total
59. Churn vs. Complexity
60. Churn Complexity 100 0 1 100_000 user.rb (changed: 2 times; complexity: 100_000 )
61. Churn Complexity 100 0 1 100_000 user.rb (changed: 27 times; complexity: 100)
62. Code Quality $ gem install rubycritic
63. Code Quality $ rubycritic
66. GPA Pie Graph Complexity (flog) Code Smells (reek)
67. Churn vs. Complexity Churn (SCM) Complexity (flog)
68. 100 0 1 100_000 Complexity Churn
69. 100 0 1 100_000 Complexity Churn
70. 100 0 1 100_000 Complexity Churn
71. 100 0 1 100_000 “Welcome to the good place.” ❤ Complexity Churn
72. 100 0 1 100_000 No one understands these files but they work. So don’t change them. Complexity Churn
73. 100 0 1 100_000 “[…], if the code never changes, it's not costing us money.” Sandi Metz Complexity Churn
74. 100 0 1 100_000 Everybody understands these files but you need to change them often Complexity Churn
75. 100 0 1 100_000 Complexity Churn These modules are complex and are constantly changing
76. 100 0 1 100_000 Complexity Churn “Sometimes a class becomes so complex that refactoring seems too difficult.” Michael Feathers
77. Are you getting into a tar pit, is it a dumpster fire, or have you found a project which is easy to maintain?
78. 100 0 1 100_000 Welcome to the tar pit. Complexity Churn
79. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic)
80. Signal #1 Code Coverage
81. Signal #2 Complexity
82. Coverage & Complexity
83. But what if we could calculate a new signal that combines both?
84. Signal #3 SkunkScore
85. SkunkScore = f(code_quality, code_coverage)
86. Complex files which lack tests should be our top priority
87. user.rb complexity: 10 smells: 10 smell points: 100 (10*10)
88. account.rb complexity: 10 smells: 10 smell points: 100 (10*10)
89. user.rb complexity: 10 smells: 10 smell points: 100 (10*10) account.rb complexity: 10 smells: 10 smell points: 100 (10*10)
90. user.rb and account.rb are equally stinky*
91. Are user.rb and account.rb equally stinky?
92. user.rb complexity: 10 smells: 10 code_coverage: 0 skunk_score: 10,000 (10*10)*100
93. account.rb complexity: 10 smells: 10 code_coverage: 100 skunk_score: 100 (10*10)*1
94. user.rb complexity: 10 smells: 10 code_coverage: 0 skunk_score: 10,000 (10*10)*100 account.rb complexity: 10 smells: 10 code_coverage: 100 skunk_score: 100 (10*10)*1
95. user.rb is considerably stinkier than account.rb
96. user.rb complexity: 10 smells: 10 code_coverage: 0 skunk_score: 10,000 (10*10)*100 You should write some tests for user.rb account.rb complexity: 10 smells: 10 code_coverage: 100 skunk_score: 100 (10*10)*1
97. How can we automate this analysis?
98. A SkunkScore Calculator (github.com/fastruby/skunk)
99. Skunk $ gem install skunk
100. Skunk $ skunk
102. Skunk $ skunk ... SkunkScore Total: 13231.069999999996 Modules Analysed: 71 SkunkScore Average: 186.3530985915493 Worst SkunkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
103. Skunk $ skunk ... SkunkScore Total: 13231.069999999996 Modules Analysed: 71 SkunkScore Average: 186.3530985915493 Worst SkunkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
104. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) + (complexity / COMPLEXITY_FACTOR) end end end
105. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) + # From Reek (complexity / COMPLEXITY_FACTOR) end end end
106. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) + # From Reek (complexity / COMPLEXITY_FACTOR) # From Flog end end end
107. Skunk module RubyCritic class AnalysedModule def skunk_score return cost if perfect_coverage? cost * (PERFECT_COVERAGE - coverage.to_i) end end end
108. Skunk module RubyCritic class AnalysedModule def skunk_score return cost if perfect_coverage? cost * (PERFECT_COVERAGE - coverage.to_i) # Penalty Factor = 100% - 20% => 80 end end end
109. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic) 3. SkunkScore (Skunk)
110. You are here:
111. Part 2: Getting out of the tar pit
112. How can we gradually pay off technical debt?
113. 1. Removing files 2. Refactoring files 3. Increasing coverage
114. 1. Removing files
115. Find Dead Code Coverband (github.com/danmayer/coverband)
117. 2.Refactoring files
118. 100 0 1 100_000 Complexity Churn
119. 100 0 1 100_000 Complexity Churn “Great candidates for refactoring (paying technical debt!)”
120. Churn vs. Complexity is not enough to prioritize
121. SkunkScore Table
122. Refactor These Files:
123. Refactor These Files:
124. 100 0 1 100_000 Complexity Churn git.rb
125. 100 0 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb refactor! git.rb
126. Skunk $ skunk -b master Base branch (master) average skunk score: 79.78 Feature branch (refactor/git) average skunk score: 71.79
127. Skunk $ skunk -b master Base branch (master) average skunk score: 79.78 Feature branch (refactor/git) average skunk score: 71.79 10% better
128. 2. Asdf 3. Increasing coverage
129. SkunkScore Table
130. Write Tests Here:
131. Maintainability Code Coverage; Complexity; and Smells
132. SkunkScore A new way to communicate about tech debt
133. “I paid off some tech debt with this PR.”
134. “I paid off 5% of our tech debt with this PR.”
135. “This project is in pretty good shape.”
136. “This project has a SkunkScore average of 120.”
137. SkunkScore -Open Source -Configurable
138. SkunkScore Your compass to get out of the tar pit
139. Time SkunkScore Average 100 (days) 0 1 100
140. Thank you! @etagwerker 140 ‣ github.com/fastruby/skunk ‣ @OmbuLabs ‣ @fastrubyio ‣ @etagwerker
142. Thank you! @etagwerker 142 Thank you!
143. Resources 1. https://github.com/whitesmith/rubycritic 2.https://github.com/colszowka/simplecov 3.https://github.com/metricfu/metric_fu 4.https://github.com/julianrubisch/attractor 5.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 6.https://www.reddit.com/r/ruby/comments/2bq092/rubycritic/ 7.http://jakescruggs.blogspot.com/2008/08/whats-good-flog-score.html 8.http://ruby.sadi.st/Flog.html 9.http://ruby.sadi.st/Flay.html 10.https://github.com/makaroni4/sandi_meter 11.https://www.fastruby.io/blog/code-quality/intruducing-skunk-stink-score- calculator.html
144. Resources 1.https://codeclimate.com/blog/deciphering-ruby-code-metrics/ 2.https://www.stickyminds.com/article/getting-empirical-about-refactoring 3.https://www.sandimetz.com/blog/2017/9/13/breaking-up-the-behemoth 4.https://github.com/troessner/reek 5.https://github.com/seattlerb/flay 6.https://github.com/seattlerb/flog 7.http://www.sqa.net/iso9126.html 8.https://www.slideshare.net/mscottford/important-metrics-for-measuring-code-health 9.https://dilbert.com/strip/2006-12-08 10.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 11.https://www.fastruby.io/blog/code-quality/code-coverage/rubycritic-4-2-0- simplecov-support.html