Escaping The Tar Pit at Solidus Conf 2019

Presentation about avoiding and escaping the tar pit.

Introduction to the `skunk` gem. Talks a little bit about `flog`; `rubycritic`; `simplecov`; `reek`; and other tools you can use to get metrics related to the codebase.

Transcript

1. Escaping The Tar Pit Ernesto Tagwerker (@etagwerker) SolidusConf, October 2019
2. ! Hi, I’m from Argentina I live in Philadelphia # I once forked Spree
3. Founder & Software Engineer @OmbuLabs
4. Founder & Software Engineer @fastrubyio
5. Open Source Contributor & Maintainer database_cleaner; bundler-leak; next_rails
6. Inspiration
7. Chapter 1: The Tar Pit
8. “No scene from prehistory is quite so vivid as that of the mortal struggles of great beasts in the tar pits. […] The fiercer the struggle, the more entangling the tar, and no beast is so strong or so skillful but that they ultimately sink.” Fred Brooks
9. “Large-system programming has over the past decade been such a tar pit, and many great and powerful beasts have thrashed violently in it.” Fred Brooks
10. We are in the tar pit; trying to avoid it; or trying to get out.
12. Tar Pit Symptoms > Projects running over budget
13. Tar Pit Symptoms > Projects running over budget > Taking forever to ship small changes
14. Tar Pit Symptoms > Projects running over budget > Taking forever to ship small changes > Sacrificing quality, increasing tech debt
15. Part 1: How to avoid the tar pit
16. “THAT’D BE GREAT” BOSS: “IF YOU COULD COME IN ON THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
17. “THAT’D BE GREAT” CLIENT: “IF YOU COULD COME IN ON THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
18. “BECAUSE I NEED IT BY END OF DAY TODAY” CLIENT: “HOW LONG IS IT GOING TO TAKE?”
19. How can we quickly assess code quality?
20. Part 2: How to get out of the tar pit
21. How can we gradually pay off technical debt?
22. Part 1: Avoiding the tar pit
24. “It can’t be that hard. It’s a Rails project.”
25. How can we quickly assess code quality?
26. Assessing Quality ✅ Paid Services ✅ Open Source Gems
27. Assessing Quality ✅ Paid Services ➡ Open Source Gems
28. Open Source Gems ✅ Static code analysis
29. Open Source Gems ✅ Static code analysis ✅ Code coverage
30. Open Source Gems ✅ Static code analysis ✅ Code coverage ✅ Code smells
31. What is software quality anyway?
32. There are hundreds of definitions.
33. “The degree to which a system, component, or process meets implicit and explicit requirements.” IEEE
34. “It works as expected.”
35. “It works as expected and it is not a PITA to maintain it.”
36. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅ Efficiency ✅ Maintainability ✅ Portability
37. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅ Efficiency ➡ Maintainability ✅ Portability
38. Maintainability
39. Maintainability 1. Code Coverage
40. Maintainability 1. Code Coverage 2. Code Quality
41. Code Coverage SimpleCov (github.com/colszowka/simplecov)
42. Code Coverage # Gemfile group :test do gem "simplecov", require: false end
43. Code Coverage # spec/spec_helper.rb if ENV["COVERAGE"] == "true" require 'simplecov' SimpleCov.start do add_group "Models", "models" add_filter "/spec/" track_files "**/*.rb" end end
44. Code Coverage # spec/spec_helper.rb if ENV["COVERAGE"] == "true" require 'simplecov' SimpleCov.start "rails" end
45. Code Coverage $ COVERAGE=true rspec
46. Code Coverage
47. Code Quality n options
48. Code Quality n options (flog; flay; reek; churn; RubyCritic; MetricFu; attractor; rubocop; fukuzatsu; turbulence; …)
49. Code Quality RubyCritic (github.com/whitesmith/rubycritic)
50. Churn
51. Churn git
52. Churn $ git log --follow --format=%h default.rb | cat 573c8f5 2ccc5b8 b447bc3
53. Churn $ git log --follow --format=%h default.rb | cat 573c8f5 2ccc5b8 b447bc3 # Churn Count = 3
54. Complexity
55. Complexity flog
56. Complexity $ flog foo.rb 11.2: flog total
57. Complexity class Foo def yay # 11.2 = a = eval "1+1" # 1.2 + 6.0 + if a == 2 then # 1.2 + 1.2 + 0.4 + puts "yay" # 1.2 end end end
58. Churn Complexity 50 0% 1 100_000
59. Churn Complexity 100 0% 1 100_000 user.rb (changed: 2 times; complexity: 100_000 )
60. Churn Complexity 100 0% 1 100_000 user.rb user.rb (changed: 27 times; complexity: 100)
61. Code Quality $ gem install rubycritic
62. Code Quality $ rubycritic
64. GPA Pie Graph Complexity (flog) Code Smells (reek)
65. Churn vs. Complexity Churn (SCM) Complexity (flog)
66. 100% 0% 1 100_000 Complexity Churn
67. 100% 0% 1 100_000 Complexity Churn
68. 100% 0% 1 100_000 “Welcome to the good place.” ❤ Complexity Churn
69. solidus/core
70. 100% 0% 1 100_000 “No one understands these files but they work. So don’t change them.” Complexity Churn
71. 100% 0% 1 100_000 “[…], if the code never changes, it's not costing us money.” Sandi Metz Complexity Churn
72. solidus/core
73. 100% 0% 1 100_000 “Everybody understands these files but you need to change them often because of reasons…” Complexity Churn
74. solidus/core
75. 100% 0% 1 100_000 Complexity Churn “Here be dragons. These modules are complex and change a lot…” '
76. 100% 0% 1 100_000 Complexity Churn “Sometimes a class becomes so complex that refactoring seems too difficult.” Michael Feathers
77. solidus/core
78. Are you getting into a tar pit, is it a dumpster fire, or have you found a project which is easy to maintain?
79. 100% 0% 1 100_000 “Welcome to the tar pit.” Complexity Churn
81. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic)
82. Code Coverage Code Smells Complexity
83. Signal #1 Code Coverage
84. Signal #2 Complexity
85. Coverage & Complexity
86. But what if we could calculate a new signal that combines both?
87. Signal #3 StinkScore
88. StinkScore = f(code_quality, code_coverage)
89. StinkScore = f(code_smells, churn, code_complexity, code_coverage)
90. code_coverage Files which lack tests should be penalized
91. foo.rb churn: 10 complexity: 10 smells: 10 smell: 1,000 (10*10*10)
92. bar.rb churn: 10 complexity: 10 smells: 10 smell: 1,000 (10*10*10)
93. foo.rb churn: 10 complexity: 10 smells: 10 smell: 1,000 (10*10*10) bar.rb churn: 10 complexity: 10 smells: 10 smell: 1,000 (10*10*10)
94. foo.rb is not stinkier than bar.rb
95. foo.rb churn: 10 complexity: 10 smells: 10 code_coverage: 0 stink_score: 100,000 (10*10*10)*100
96. bar.rb churn: 10 complexity: 10 smells: 10 code_coverage: 100 stink_score: 1,000 (10*10*10)*1
97. foo.rb churn: 10 complexity: 10 smells: 10 code_coverage: 0 stink_score: 100,000 (10*10*10)*100 bar.rb churn: 10 complexity: 10 smells: 10 code_coverage: 100 stink_score: 1,000 (10*10*10)*1
98. foo.rb is considerably stinkier than bar.rb
99. Stink Score Calculator Skunk (github.com/fastruby/skunk)
100. Skunk $ gem install skunk
101. Skunk $ skunk
104. Skunk $ skunk ... StinkScore Total: 13220.859999999997 Modules Analysed: 71 StinkScore Average: 0.1862092957746 Worst StinkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
105. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) + # From Reek (complexity / COMPLEXITY_FACTOR) # From Flog end end end
106. Skunk module RubyCritic class AnalysedModule def stink_score return churn_times_cost if perfect_coverage? (churn_times_cost * (PERFECT_COVERAGE - coverage.to_i)) end def churn_times_cost safe_churn = churn > 0 ? churn : 1 safe_churn * cost end end end
107. Skunk module RubyCritic class AnalysedModule def stink_score return churn_times_cost if perfect_coverage? (churn_times_cost * (PERFECT_COVERAGE - coverage.to_i)) end def churn_times_cost safe_churn = churn > 0 ? churn : 1 safe_churn * cost end end end
108. Skunk Warning > skunk-v0.2.0 > feedback wanted
109. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic) 3. StinkScore (Skunk)
110. You are here:
111. Part 2: Getting out of the tar pit
112. So, where do we begin?
113. removing files will decrease our stink_score
114. Find Dead Code Coverband (github.com/danmayer/coverband)
116. refactoring complex files will decrease our stink_score
117. 100% 0% 1 100_000 Complexity Churn “Here be dragons. These modules are complex and change a lot…” '
118. 100% 0% 1 100_000 Complexity Churn “Great candidates for refactoring (paying technical debt!)” #
119. Churn vs. Complexity is not enough to prioritize
120. Stink Score Table
121. Refactor These Files:
122. Refactor These Files:
123. 100% 0% 1 100_000 Complexity Churn git.rb
124. 100% 0% 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb refactor! git.rb
125. 100% 0% 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb
126. Skunk $ skunk ... StinkScore Total: 13220.859999999997 11220 Modules Analysed: 71 StinkScore Average: 0.1862092957746 0.157
127. writing tests for our app will decrease our stink_score
128. Stink Score Table
129. Write Tests Here:
130. Skunk $ skunk ... StinkScore Total: 13220.859999999997 11220 Modules Analysed: 71 StinkScore Average: 0.1862092957746 0.157
131. stink_score your compass to get out of the tar pit
132. stink_score Code Coverage; Complexity; and Smells
133. Thank you! @etagwerker 133
134. Resources 1. https://github.com/whitesmith/rubycritic 2.https://github.com/colszowka/simplecov 3.https://github.com/metricfu/metric_fu 4.https://github.com/julianrubisch/attractor 5.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 6.https://www.reddit.com/r/ruby/comments/2bq092/rubycritic/ 7.http://jakescruggs.blogspot.com/2008/08/whats-good-flog-score.html 8.http://ruby.sadi.st/Flog.html 9.http://ruby.sadi.st/Flay.html 10.https://github.com/makaroni4/sandi_meter
135. Resources 1.https://codeclimate.com/blog/deciphering-ruby-code-metrics/ 2.https://www.stickyminds.com/article/getting-empirical-about-refactoring 3.https://www.sandimetz.com/blog/2017/9/13/breaking-up-the-behemoth 4.https://github.com/troessner/reek 5.https://github.com/seattlerb/flay 6.https://github.com/seattlerb/flog 7.http://www.sqa.net/iso9126.html 8.https://www.slideshare.net/mscottford/important-metrics-for-measuring- code-health 9.https://dilbert.com/strip/2006-12-08
136. Questions?