Escaping The Tar Pit at Philly.rb

Presentation about avoiding and escaping the tar pit.

Introduction to the `skunk` gem. Talks a little bit about `flog`; `rubycritic`; `simplecov`; `reek`; and other tools you can use to get metrics related to the codebase.

Transcript

1. Escaping The Tar Pit Ernesto Tagwerker (@etagwerker) Philly.rb, November 2019
2. ! Hi, I’m from Argentina I live in Philadelphia # I ❤ Open Source
3. Founder & Software Engineer @OmbuLabs
4. Founder & Software Engineer @fastrubyio
5. Open Source Contributor & Maintainer database_cleaner; bundler-leak; next_rails
6. Inspiration
7. Chapter 1: The Tar Pit
8. “No scene from prehistory is quite so vivid as that of the mortal struggles of great beasts in the tar pits. […] The fiercer the struggle, the more entangling the tar, and no beast is so strong or so skillful but that they ultimately sink.” Fred Brooks
9. “Large-system programming has over the past decade been such a tar pit, and many great and powerful beasts have thrashed violently in it.” Fred Brooks
10. We are in the tar pit; trying to avoid it; or trying to get out.
12. Tar Pit Symptoms > Projects running over budget
13. Tar Pit Symptoms > Projects running over budget > Taking forever to ship small changes
14. Tar Pit Symptoms > Projects running over budget > Taking forever to ship small changes > Sacrificing quality, increasing tech debt
16. Part 1: How to avoid the tar pit
17. “THAT’D BE GREAT” BOSS: “IF YOU COULD COME IN ON THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
18. “THAT’D BE GREAT” CLIENT: “IF YOU COULD COME IN ON THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”
19. CLIENT: “HOW LONG IS IT GOING TO TAKE?”
20. “BECAUSE I NEED IT BY END OF DAY TODAY” CLIENT: “HOW LONG IS IT GOING TO TAKE?”
21. How can we quickly assess code quality?
22. Part 2: How to get out of the tar pit
23. How can we gradually pay off technical debt?
24. Part 1: Avoiding the tar pit
26. “It can’t be that hard. It’s a Ruby project.”
27. How can we quickly assess code quality?
28. Assessing Quality ✅ Paid Services ✅ Open Source Gems
29. Assessing Quality ✅ Paid Services ➡ Open Source Gems
30. Open Source Gems ✅ Static code analysis
31. Open Source Gems ✅ Static code analysis ✅ Code coverage
32. Open Source Gems ✅ Static code analysis ✅ Code coverage ✅ Code smells
33. What is software quality anyway?
34. There are hundreds of definitions.
35. “The degree to which a system, component, or process meets implicit and explicit requirements.” IEEE
36. “It works as expected.”
37. “It works as expected and it is not a PITA to maintain it.”
38. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅ Efficiency ✅ Maintainability ✅ Portability
39. ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅ Efficiency ➡ Maintainability ✅ Portability
40. Maintainability
41. Maintainability 1. Code Coverage
42. Maintainability 1. Code Coverage 2. Code Quality
43. Code Coverage SimpleCov (github.com/colszowka/simplecov)
44. Code Coverage # Gemfile group :test do gem "simplecov", require: false end
45. Code Coverage # spec/spec_helper.rb if ENV["COVERAGE"] == "true" require 'simplecov' SimpleCov.start do add_group "Models", "models" add_filter "/spec/" track_files "**/*.rb" end end
46. Code Coverage # spec/spec_helper.rb if ENV["COVERAGE"] == "true" require 'simplecov' SimpleCov.start "rails" end
47. Code Coverage $ COVERAGE=true rspec
48. Code Coverage
49. Code Quality n options
50. Code Quality n options (flog; flay; reek; churn; RubyCritic; MetricFu; attractor; rubocop; fukuzatsu; turbulence; …)
51. Code Quality RubyCritic (github.com/whitesmith/rubycritic)
52. Churn
53. Churn git
54. Churn $ git log --follow --format=%h default.rb | cat 573c8f5 2ccc5b8 b447bc3
55. Churn $ git log --follow --format=%h default.rb | cat 573c8f5 2ccc5b8 b447bc3 # Churn Count = 3
56. Complexity
57. Complexity flog
58. Complexity $ flog foo.rb 11.2: flog total
59. Complexity class Foo def yay # 11.2 = a = eval "1+1" # 1.2 + 6.0 + if a == 2 # 1.2 + 1.2 + puts "yay" # 1.2 end end end
60. Complexity $ flog foo.rb 11.2: flog total
61. Churn vs. Complexity
62. Churn Complexity 50 0 1 100_000
63. Churn Complexity 100 0 1 100_000 user.rb (changed: 2 times; complexity: 100_000 )
64. Churn Complexity 100 0 1 100_000 user.rb (changed: 27 times; complexity: 100)
65. Code Quality $ gem install rubycritic
66. Code Quality $ rubycritic
68. GPA Pie Graph Complexity (flog) Code Smells (reek)
69. Churn vs. Complexity Churn (SCM) Complexity (flog)
70. 100 0 1 100_000 Complexity Churn
71. 100 0 1 100_000 Complexity Churn
72. 100 0 1 100_000 “Welcome to the good place.” ❤ Complexity Churn
73. 100 0 1 100_000 “No one understands these files but they work. So don’t change them.” Complexity Churn
74. 100 0 1 100_000 “[…], if the code never changes, it's not costing us money.” Sandi Metz Complexity Churn
75. 100 0 1 100_000 “Everybody understands these files but you need to change them often” Complexity Churn
76. 100 0 1 100_000 Complexity Churn “These modules are complex and change a lot…” '
77. 100 0 1 100_000 Complexity Churn “Sometimes a class becomes so complex that refactoring seems too difficult.” Michael Feathers
78. Are you getting into a tar pit, is it a dumpster fire, or have you found a project which is easy to maintain?
79. 100 0 1 100_000 “Welcome to the tar pit.” Complexity Churn
80. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic)
81. Signal #1 Code Coverage
82. Signal #2 Complexity
83. Coverage & Complexity
84. But what if we could calculate a new signal that combines both?
85. Signal #3 StinkScore
86. StinkScore = f(code_quality, code_coverage)
87. StinkScore = f(code_smells, churn, code_complexity, code_coverage)
88. code_coverage Files which lack tests should be penalized
89. foo.rb churn: 10 complexity: 10 smells: 10 smell: 1,000 (10*10*10)
90. bar.rb churn: 10 complexity: 10 smells: 10 smell: 1,000 (10*10*10)
91. foo.rb churn: 10 complexity: 10 smells: 10 smell: 1,000 (10*10*10) bar.rb churn: 10 complexity: 10 smells: 10 smell: 1,000 (10*10*10)
92. foo.rb is not stinkier than bar.rb
93. foo.rb churn: 10 complexity: 10 smells: 10 code_coverage: 0 stink_score: 100,000 (10*10*10)*100
94. bar.rb churn: 10 complexity: 10 smells: 10 code_coverage: 100 stink_score: 1,000 (10*10*10)*1
95. foo.rb churn: 10 complexity: 10 smells: 10 code_coverage: 0 stink_score: 100,000 (10*10*10)*100 bar.rb churn: 10 complexity: 10 smells: 10 code_coverage: 100 stink_score: 1,000 (10*10*10)*1
96. foo.rb is considerably stinkier than bar.rb
97. Stink Score Calculator Skunk (github.com/fastruby/skunk)
98. Skunk $ gem install skunk
99. Skunk $ skunk
102. Skunk $ skunk ... StinkScore Total: 13220.859999999997 Modules Analysed: 71 StinkScore Average: 0.1862092957746 Worst StinkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
103. RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) + # From Reek (complexity / COMPLEXITY_FACTOR) # From Flog end end end
104. Skunk module RubyCritic class AnalysedModule def stink_score return churn_times_cost if perfect_coverage? (churn_times_cost * (PERFECT_COVERAGE - coverage.to_i)) end def churn_times_cost safe_churn = churn > 0 ? churn : 1 safe_churn * cost end end end
105. Skunk module RubyCritic class AnalysedModule def stink_score return churn_times_cost if perfect_coverage? (churn_times_cost * (PERFECT_COVERAGE - coverage.to_i)) end def churn_times_cost safe_churn = churn > 0 ? churn : 1 safe_churn * cost end end end
106. Skunk Warning > skunk-v0.2.0 > feedback wanted
107. Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic) 3. StinkScore (Skunk)
108. You are here:
109. Part 2: Getting out of the tar pit
110. So, where do we begin?
111. removing files will decrease our stink_score
112. Find Dead Code Coverband (github.com/danmayer/coverband)
114. refactoring complex files will decrease our stink_score
115. 100 0 1 100_000 Complexity Churn “Here be dragons. These modules are complex and change a lot…” '
116. 100 0% 1 100_000 Complexity Churn “Great candidates for refactoring (paying technical debt!)” #
117. Churn vs. Complexity is not enough to prioritize
118. Stink Score Table
119. Refactor These Files:
120. Refactor These Files:
121. 100 0 1 100_000 Complexity Churn git.rb
122. 100 0 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb refactor! git.rb
123. 100 0 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb
124. Skunk $ skunk ... StinkScore Total: 13220.859999999997 11220 Modules Analysed: 71 StinkScore Average: 0.1862092957746 0.157
125. writing tests for our app will decrease our stink_score
126. Stink Score Table
127. Write Tests Here:
128. Skunk $ skunk ... StinkScore Total: 13220.859999999997 11220 Modules Analysed: 71 StinkScore Average: 0.1862092957746 0.157
129. Time StinkScore Average 100 (days) 0 1 100
130. stink_score your compass to get out of the tar pit
131. stink_score Code Coverage; Complexity; and Smells
132. Thank you! @etagwerker 132
133. Resources 1. https://github.com/whitesmith/rubycritic 2.https://github.com/colszowka/simplecov 3.https://github.com/metricfu/metric_fu 4.https://github.com/julianrubisch/attractor 5.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 6.https://www.reddit.com/r/ruby/comments/2bq092/rubycritic/ 7.http://jakescruggs.blogspot.com/2008/08/whats-good-flog-score.html 8.http://ruby.sadi.st/Flog.html 9.http://ruby.sadi.st/Flay.html 10.https://github.com/makaroni4/sandi_meter
134. Resources 1.https://codeclimate.com/blog/deciphering-ruby-code-metrics/ 2.https://www.stickyminds.com/article/getting-empirical-about-refactoring 3.https://www.sandimetz.com/blog/2017/9/13/breaking-up-the-behemoth 4.https://github.com/troessner/reek 5.https://github.com/seattlerb/flay 6.https://github.com/seattlerb/flog 7.http://www.sqa.net/iso9126.html 8.https://www.slideshare.net/mscottford/important-metrics-for-measuring- code-health 9.https://dilbert.com/strip/2006-12-08
135. Questions?