Stuck in the Tar Pit at Sin City Ruby '24

Nobody wants to inherit a project that reeks but here we are: Stuck in the tar pit. How did we get here? In this talk you will learn how to use a few, great Ruby gems that will guide you out of that sticky tar you are in.

Transcript

1. Stuck in The Tar Pit Ernesto Tagwerker (@etagwerker) Sin City Ruby, March, 2024
2. 🇦🇷 Hi, I’m from Argentina
4. 🇦🇷 Hi, I’m from Argentina 🦅 I live in Philadelphia
6. 🇦🇷 Hi, I’m from Argentina 🦅 I live in Philadelphia 👨💻 I ❤ Open Source
7. Founder & CTO @OmbuLabs
8. Founder & CTO @FastRubyIO
9. 🚀 Rails Performance Optimization
10. 🚀 Rails Performance Optimization 🔥 Ruby/Rails Upgrades
11. 🚀 Rails Performance Optimization 🔥 Ruby/Rails Upgrades 👨💻 Fixed-cost Monthly Maintenance
12. Chapter 1: The Tar Pit
13. “Large-system programming has over the past decade been such a tar pit, and many great and powerful beasts have thrashed violently in it.” Fred Brooks
14. “The fi ercer the struggle, the more entangling the tar, and no beast is so strong or so skillful but that they ultimately sink.” Fred Brooks
16. Nobody gives a shit about tech debt*
17. Nobody gives a shit about tech debt (until it becomes obvious that there is too much)
19. Why?
20. Why? Because development velocity is sloooooow…
21. Why? Because that project took 3x what you said it would…
22. Why? Because production was down 3 hours due to a security hole…
23. We’ve all been there. There is too much “tech debt.”
25. This is NOT fi ne
26. Why is there too much tech debt? 1. Tooling 2. Culture
27. 1. Tooling
28. Assessing Tech Debt ✅ Paid Services ✅ Open Source Gems
29. Assessing Tech Debt ✅ Paid Services ➡ Open Source Gems
30. Open Source Gems ✅ Static code analysis
31. Open Source Gems ✅ Static code analysis ✅ Code coverage
32. Open Source Gems ✅ Static code analysis ✅ Code coverage ✅ Code smells
33. What is tech debt anyway?
35. Tech Debt Categories at Google 💸 (1) Migration is needed 💸 (2) Outdated documentation 💸 (3) Insuf fi cient testing 💸 (4) Code quality 💸 (5) Dead/Abandoned code
36. Tech Debt Categories at Google 💸 (6) Code degradation 💸 (7) Team lacks necessary expertise 💸 (8) Unstable Dependencies 💸 (9) Migration was poorly executed 💸 (10) Sub-optimal release process
37. Each team has its own perception of tech debt
38. <rant>
39. De fi ne your own baseline of tech debt
40. Maintainability
41. Maintainability 1. Up-to-date Dependencies
42. Maintainability 1. Up-to-date Dependencies 2. Code Coverage
43. Maintainability 1. Up-to-date Dependencies 2. Code Coverage 3. Code Quality
44. Outdated Dependencies Libyears (https://github.com/jaredbeck/libyear- bundler)
45. Dependencies $ gem install libyear-bundler $ libyear-bundler Gemfile
46. Dependencies $ libyear-bundler Gemfile System is 56.5 libyears behind Total releases behind: 209 Major, minor, patch versions behind: 4, 65, 15
47. Code Coverage SimpleCov (https://github.com/simplecov-ruby/ simplecov)
48. Code Coverage # Gemfile group :test do gem "simplecov", require: false end
49. Code Coverage # spec/spec_helper.rb if ENV["COVERAGE"] == "true" require 'simplecov' SimpleCov.start do add_group "Models", "models" add_filter "/spec/" track_files "**/*.rb" end end
50. Code Coverage $ COVERAGE=true rake test
51. Code Coverage
52. Code Coverage A signal about the application’s test suite and testing culture at the company.
53. Code Quality n options
54. Code Quality n options ( fl og; fl ay; reek; churn; RubyCritic; MetricFu; attractor; rubocop; fukuzatsu; turbulence; …)
55. Code Quality RubyCritic (github.com/whitesmith/rubycritic)
56. Complexity
57. Complexity flog
58. Complexity $ flog default.rb => 10.8: flog total
59. Complexity class Default def yay # 10.8 = a = eval "1+1" # 1.2 + 6.0 + if a == 2 # 1.2 + 1.2 + puts "yay" # 1.2 end end end
60. Complexity $ flog user.rb => 10.8: flog total
61. Complexity On its own it’s a pretty useless signal.
62. Churn vs. Complexity
63. Churn
64. Churn git
65. Churn $ git log --follow --format=%h default.rb | cat 573c8f5 2ccc5b8 b447bc3
66. Churn $ git log --follow --format=%h default.rb | cat 573c8f5 2ccc5b8 b447bc3 # Churn Count = 3
67. Code Quality $ gem install rubycritic
68. Code Quality $ rubycritic
69. Code Quality $ rubycritic app lib
72. GPA Pie Graph Complexity (flog) Code Smells (reek)
73. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) + (complexity / COMPLEXITY_FACTOR) end end end
74. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) + # From Reek (complexity / COMPLEXITY_FACTOR) end end end
75. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) + # From Reek (complexity / COMPLEXITY_FACTOR) # From Flog end end end
76. Churn vs. Complexity Churn (SCM) Complexity (flog)
77. 100 0 1 100_000 Complexity Churn
78. 100 0 1 100_000 “Welcome to the good place.” ❤ Complexity Churn
79. 100 0 1 100_000 No one understands these fi les but they work. So don’t change them. Complexity Churn
80. 100 0 1 100_000 “[…], if the code never changes, it's not costing us money.” Sandi Metz Complexity Churn
81. 100 0 1 100_000 Everybody understands these fi les but you need to change them often Complexity Churn
82. 100 0 1 100_000 Complexity Churn These modules are complex and are constantly changing 🤦 🤦 🤦
83. 100 0 1 100_000 Complexity Churn “Sometimes a class becomes so complex that refactoring seems too dif fi cult.” Michael Feathers
84. What does your project look like?
85. 100 0 1 100_000 Welcome to the tar pit. Complexity Churn
86. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic)
87. Signal #1 Code Coverage
88. Signal #2 Complexity
89. user.rb complexity: 10 smells: 10 churn: 10 tech debt points: 1,000 (10*10*10) account.rb complexity: 10 smells: 10 churn: 10 tech debt points: 1,000 (10*10*10)
90. user.rb and account.rb have the same amount of tech debt*
91. user.rb complexity: 10 smells: 10 churn: 10 code_coverage: 0 tech debt points: 100,000 (10*10*10*100) account.rb complexity: 10 smells: 10 churn: 10 code_coverage: 100 tech debt points: 1,000 (10*10*10*1)
92. user.rb has more tech debt than account.rb
93. user.rb complexity: 10 smells: 10 churn: 10 code_coverage: 0 tech debt points: 100,000 (10*10*10*100) More tests for user.rb? 😉 account.rb complexity: 10 smells: 10 churn: 10 code_coverage: 100 tech debt points: 1,000 (10*10*10*1)
94. user.rb complexity: 10 smells: 10 churn: 10 code_coverage: 0 tech debt points: 100,000 (10*10*10*100) account.rb complexity: 10 smells: 10 churn: 10 code_coverage: 100 tech debt points: 1,000 (10*10*10*1) Refactor account.rb? 😉
95. Complex files which lack tests likely have too much tech debt
96. Coverage & Complexity
97. Signal #3
98. Signal #3 Skunk Score 🦨
99. SkunkScore = f(complexity, code_coverage)
100. A Skunk Score Calculator (github.com/fastruby/skunk)
101. Skunk $ gem install skunk
102. Skunk $ skunk
104. Skunk $ skunk ... Skunk Score Total: 13231.069999999996 Modules Analysed: 71 Skunk Score Average: 186.3530985915493 Worst Skunk Score: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
105. Skunk module RubyCritic class AnalysedModule def skunk_score return cost if perfect_coverage? cost * (PERFECT_COVERAGE - coverage.to_i) end end end
106. Skunk module RubyCritic class AnalysedModule def skunk_score return cost if perfect_coverage? cost * (PERFECT_COVERAGE - coverage.to_i) # Penalty Factor = 100% - 20% => 80 end end end
107. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic) 3. Skunk Score (Skunk)
108. You are here:
109. Tech Debt
110. How can we gradually pay off technical debt?
111. 1. Removing files 2. Refactoring files 3. Increasing coverage
112. 1. Removing files
113. Find Dead Code
114. Coverband (github.com/danmayer/coverband)
116. 2.Refactoring files
117. 100 0 1 100_000 Complexity Churn “Great candidates for refactoring (paying technical debt!)” 👨💻
118. Churn vs. Complexity is not enough to prioritize
119. Skunk Score Table
120. Refactor These Files:
121. Refactor These Files:
122. 100 0 1 100_000 Complexity Churn git.rb
123. 100 0 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb refactor! git.rb
124. 2. Asdf 3. Increasing coverage
125. Skunk Score Table
126. Write Tests Here:
127. 2. Culture
132. Team Rebuild vs. Team Refactor
134. Technical Debt Maturity Model
136. Reactive # TODO: refactor me later # FIXME: won’t scale
137. Proactive tech_debt.md .codeclimate.yml
138. Strategic Assign tech debt champions Better planning (team- based priorities)
139. Structural Optimize tech debt management Organization-level standards
140. Structural Embed tech debt considerations into the developer workflow
141. How mature is your team?
142. “I paid off some tech debt with this PR.”
143. “I paid off 2% of our tech debt with this PR.”
144. “This project is in pretty good shape.”
145. “This project has a skunk score average of 120 and it’s 78.7 libyears behind.”
146. From Tech Debt to Tech Health
147. Tech Health It’s not a technical problem
149. Tech Debt It’s a company culture problem
151. Tech Health It’s a company- wide effort
152. From Tech Debt to Tech Health 1.Where are you? 2.Where do you want to be?
153. c 153 ‣ mastodon.social/@etagwerker ‣ mastodon.online/@OmbuLabs (AI/ML) ‣ ruby.social/@FastRuby (Tech Debt)
154. Thank you!
155. Questions?
156. Resources 1. https://github.com/whitesmith/rubycritic 2.https://github.com/colszowka/simplecov 3.https://github.com/metricfu/metric_fu 4.https://github.com/julianrubisch/attractor 5.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 6.https://www.reddit.com/r/ruby/comments/2bq092/rubycritic/ 7.http://jakescruggs.blogspot.com/2008/08/whats-good- fl og-score.html 8.http://ruby.sadi.st/Flog.html 9.http://ruby.sadi.st/Flay.html 10.https://github.com/makaroni4/sandi_meter 11.https://www.fastruby.io/blog/code-quality/intruducing-skunk-stink-score- calculator.html
157. Resources 1.https://codeclimate.com/blog/deciphering-ruby-code-metrics/ 2.https://www.stickyminds.com/article/getting-empirical-about-refactoring 3.https://www.sandimetz.com/blog/2017/9/13/breaking-up-the-behemoth 4.https://github.com/troessner/reek 5.https://github.com/seattlerb/ fl ay 6.https://github.com/seattlerb/ fl og 7.http://www.sqa.net/iso9126.html 8.https://www.slideshare.net/mscottford/important-metrics-for-measuring-code-health 9.https://dilbert.com/strip/2006-12-08 10.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 11.https://www.fastruby.io/blog/code-quality/code-coverage/rubycritic-4-2-0- simplecov-support.html
158. Resources 1. https://www.ombulabs.com/blog/tech-debt-maturity-model.html 2. https://www.computer.org/csdl/magazine/so/ 2023/03/10109339/1MESXKyAYNO 3. https://www.linkedin.com/pulse/how-google-measures-manages- tech-debt-abi-noda/ 4. https://www.forbes.com/sites/rogerdooley/2023/01/06/southwest- meltdown-could-cost-825-million/?sh=672485e8709f 5. https://www.ciodive.com/news/southwest-invests-billions-technology- cloud/705898/ 6. https://ieeexplore.ieee.org/document/10109339