Slide 1
Slide 1 text
Stuck in
The Tar Pit
Ernesto Tagwerker (@etagwerker)
Sin City Ruby, March, 2024
Slide 2
Slide 2 text
🇦🇷 Hi, I’m from Argentina
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
🇦🇷 Hi, I’m from Argentina
🦅 I live in Philadelphia
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
🇦🇷 Hi, I’m from Argentina
🦅 I live in Philadelphia
👨💻 I ❤ Open Source
Slide 7
Slide 7 text
Founder & CTO
@OmbuLabs
Slide 8
Slide 8 text
Founder & CTO
@FastRubyIO
Slide 9
Slide 9 text
🚀 Rails Performance Optimization
Slide 10
Slide 10 text
🚀 Rails Performance Optimization
🔥 Ruby/Rails Upgrades
Slide 11
Slide 11 text
🚀 Rails Performance Optimization
🔥 Ruby/Rails Upgrades
👨💻 Fixed-cost Monthly Maintenance
Slide 12
Slide 12 text
Chapter 1:
The Tar Pit
Slide 13
Slide 13 text
“Large-system programming has
over the past decade been
such a tar pit, and many great
and powerful beasts have thrashed
violently in it.” Fred Brooks
Slide 14
Slide 14 text
“The
fi
ercer the struggle, the
more entangling the tar, and
no beast is so strong or so skillful
but that they ultimately sink.”
Fred Brooks
Slide 15
Slide 15 text
No content
Slide 16
Slide 16 text
Nobody gives a shit about
tech debt*
Slide 17
Slide 17 text
Nobody gives a shit about
tech debt (until it becomes
obvious that there is too much)
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
Why?
Slide 20
Slide 20 text
Why?
Because development velocity
is sloooooow…
Slide 21
Slide 21 text
Why?
Because that project took 3x
what you said it would…
Slide 22
Slide 22 text
Why?
Because production was down
3 hours due to a security hole…
Slide 23
Slide 23 text
We’ve all been there.
There is too much “tech debt.”
Slide 24
Slide 24 text
No content
Slide 25
Slide 25 text
This is NOT
fi
ne
Slide 26
Slide 26 text
Why is there
too much tech debt?
1. Tooling
2. Culture
Slide 27
Slide 27 text
1.
Tooling
Slide 28
Slide 28 text
Assessing Tech Debt
✅ Paid Services
✅ Open Source Gems
Slide 29
Slide 29 text
Assessing Tech Debt
✅ Paid Services
➡ Open Source Gems
Slide 30
Slide 30 text
Open Source Gems
✅ Static code analysis
Slide 31
Slide 31 text
Open Source Gems
✅ Static code analysis
✅ Code coverage
Slide 32
Slide 32 text
Open Source Gems
✅ Static code analysis
✅ Code coverage
✅ Code smells
Slide 33
Slide 33 text
What is tech debt
anyway?
Slide 34
Slide 34 text
No content
Slide 35
Slide 35 text
Tech Debt Categories at Google
💸 (1) Migration is needed
💸 (2) Outdated documentation
💸 (3) Insuf
fi
cient testing
💸 (4) Code quality
💸 (5) Dead/Abandoned code
Slide 36
Slide 36 text
Tech Debt Categories at Google
💸 (6) Code degradation
💸 (7) Team lacks necessary expertise
💸 (8) Unstable Dependencies
💸 (9) Migration was poorly executed
💸 (10) Sub-optimal release process
Slide 37
Slide 37 text
Each team has its own
perception of tech debt
Slide 38
Slide 38 text
Slide 39
Slide 39 text
De
fi
ne your own
baseline of tech debt
Slide 40
Slide 40 text
Maintainability
Slide 41
Slide 41 text
Maintainability
1. Up-to-date Dependencies
Slide 42
Slide 42 text
Maintainability
1. Up-to-date Dependencies
2. Code Coverage
Slide 43
Slide 43 text
Maintainability
1. Up-to-date Dependencies
2. Code Coverage
3. Code Quality
Slide 44
Slide 44 text
Outdated
Dependencies
Libyears
(https://github.com/jaredbeck/libyear-
bundler)
Slide 45
Slide 45 text
Dependencies
$ gem install libyear-bundler
$ libyear-bundler Gemfile
Slide 46
Slide 46 text
Dependencies
$ libyear-bundler Gemfile
System is 56.5 libyears behind
Total releases behind: 209
Major, minor, patch versions behind: 4, 65, 15
Slide 47
Slide 47 text
Code Coverage
SimpleCov
(https://github.com/simplecov-ruby/
simplecov)
Slide 48
Slide 48 text
Code Coverage
# Gemfile
group :test do
gem "simplecov", require: false
end
Slide 49
Slide 49 text
Code Coverage
# spec/spec_helper.rb
if ENV["COVERAGE"] == "true"
require 'simplecov'
SimpleCov.start do
add_group "Models", "models"
add_filter "/spec/"
track_files "**/*.rb"
end
end
Slide 50
Slide 50 text
Code Coverage
$ COVERAGE=true rake test
Slide 51
Slide 51 text
Code Coverage
Slide 52
Slide 52 text
Code Coverage
A signal about the application’s test suite
and testing culture at the company.
Slide 53
Slide 53 text
Code Quality
n options
Slide 54
Slide 54 text
Code Quality
n options (
fl
og;
fl
ay; reek;
churn; RubyCritic; MetricFu;
attractor; rubocop;
fukuzatsu; turbulence; …)
Slide 55
Slide 55 text
Code Quality
RubyCritic
(github.com/whitesmith/rubycritic)
Slide 56
Slide 56 text
Complexity
Slide 57
Slide 57 text
Complexity
flog
Slide 58
Slide 58 text
Complexity
$ flog default.rb
=> 10.8: flog total
Slide 59
Slide 59 text
Complexity
class Default
def yay # 10.8 =
a = eval "1+1" # 1.2 + 6.0 +
if a == 2 # 1.2 + 1.2 +
puts "yay" # 1.2
end
end
end
Slide 60
Slide 60 text
Complexity
$ flog user.rb
=> 10.8: flog total
Slide 61
Slide 61 text
Complexity
On its own it’s a pretty useless signal.
Slide 62
Slide 62 text
Churn
vs.
Complexity
Slide 63
Slide 63 text
Churn
Slide 64
Slide 64 text
Churn
git
Slide 65
Slide 65 text
Churn
$ git log --follow --format=%h default.rb | cat
573c8f5
2ccc5b8
b447bc3
Slide 66
Slide 66 text
Churn
$ git log --follow --format=%h default.rb | cat
573c8f5
2ccc5b8
b447bc3
# Churn Count = 3
Slide 67
Slide 67 text
Code Quality
$ gem install rubycritic
Slide 68
Slide 68 text
Code Quality
$ rubycritic
Slide 69
Slide 69 text
Code Quality
$ rubycritic app lib
Slide 70
Slide 70 text
No content
Slide 71
Slide 71 text
No content
Slide 72
Slide 72 text
GPA Pie Graph
Complexity (flog)
Code Smells (reek)
Slide 73
Slide 73 text
RubyCritic
module RubyCritic
class AnalysedModule
def cost
smells.map(&:cost).inject(0.0, :+) +
(complexity / COMPLEXITY_FACTOR)
end
end
end
Slide 74
Slide 74 text
RubyCritic
module RubyCritic
class AnalysedModule
def cost
smells.map(&:cost).inject(0.0, :+) + # From Reek
(complexity / COMPLEXITY_FACTOR)
end
end
end
Slide 75
Slide 75 text
RubyCritic
module RubyCritic
class AnalysedModule
def cost
smells.map(&:cost).inject(0.0, :+) + # From Reek
(complexity / COMPLEXITY_FACTOR) # From Flog
end
end
end
Slide 76
Slide 76 text
Churn vs.
Complexity
Churn (SCM)
Complexity (flog)
Slide 77
Slide 77 text
100
0
1 100_000
Complexity
Churn
Slide 78
Slide 78 text
100
0
1 100_000
“Welcome to
the good place.” ❤
Complexity
Churn
Slide 79
Slide 79 text
100
0
1 100_000
No one understands
these
fi
les but they
work. So don’t change
them.
Complexity
Churn
Slide 80
Slide 80 text
100
0
1 100_000
“[…], if the code
never changes,
it's not costing us
money.”
Sandi Metz
Complexity
Churn
Slide 81
Slide 81 text
100
0
1 100_000
Everybody
understands these
fi
les
but you need to
change them often
Complexity
Churn
Slide 82
Slide 82 text
100
0
1 100_000
Complexity
Churn
These modules are
complex and are
constantly changing
🤦 🤦 🤦
Slide 83
Slide 83 text
100
0
1 100_000
Complexity
Churn
“Sometimes a class
becomes so complex that
refactoring seems too
dif
fi
cult.”
Michael Feathers
Slide 84
Slide 84 text
What does your project
look like?
Slide 85
Slide 85 text
100
0
1 100_000
Welcome to the tar pit.
Complexity
Churn
Slide 86
Slide 86 text
Maintainability
1. Code Coverage (SimpleCov)
2. Code Quality (RubyCritic)
Slide 87
Slide 87 text
Signal #1
Code Coverage
Slide 88
Slide 88 text
Signal #2
Complexity
Slide 89
Slide 89 text
user.rb
complexity: 10
smells: 10
churn: 10
tech debt points:
1,000 (10*10*10)
account.rb
complexity: 10
smells: 10
churn: 10
tech debt points:
1,000 (10*10*10)
Slide 90
Slide 90 text
user.rb and account.rb
have the
same amount of
tech debt*
Slide 91
Slide 91 text
user.rb
complexity: 10
smells: 10
churn: 10
code_coverage: 0
tech debt points:
100,000 (10*10*10*100)
account.rb
complexity: 10
smells: 10
churn: 10
code_coverage: 100
tech debt points:
1,000 (10*10*10*1)
Slide 92
Slide 92 text
user.rb has
more tech debt
than account.rb
Slide 93
Slide 93 text
user.rb
complexity: 10
smells: 10
churn: 10
code_coverage: 0
tech debt points:
100,000 (10*10*10*100)
More tests for user.rb? 😉
account.rb
complexity: 10
smells: 10
churn: 10
code_coverage: 100
tech debt points:
1,000 (10*10*10*1)
Slide 94
Slide 94 text
user.rb
complexity: 10
smells: 10
churn: 10
code_coverage: 0
tech debt points:
100,000 (10*10*10*100)
account.rb
complexity: 10
smells: 10
churn: 10
code_coverage: 100
tech debt points:
1,000 (10*10*10*1)
Refactor account.rb? 😉
Slide 95
Slide 95 text
Complex files which
lack tests likely have
too much tech debt
Slide 96
Slide 96 text
Coverage & Complexity
Slide 97
Slide 97 text
Signal #3
Slide 98
Slide 98 text
Signal #3
Skunk Score 🦨
Slide 99
Slide 99 text
SkunkScore =
f(complexity, code_coverage)
Slide 100
Slide 100 text
A Skunk Score Calculator
(github.com/fastruby/skunk)
Slide 101
Slide 101 text
Skunk
$ gem install skunk
Slide 102
Slide 102 text
Skunk
$ skunk
Slide 103
Slide 103 text
No content
Slide 104
Slide 104 text
Skunk
$ skunk
...
Skunk Score Total: 13231.069999999996
Modules Analysed: 71
Skunk Score Average: 186.3530985915493
Worst Skunk Score: 2401.75 (lib/rubycritic/source_control_systems/git.rb)
Slide 105
Slide 105 text
Skunk
module RubyCritic
class AnalysedModule
def skunk_score
return cost if perfect_coverage?
cost * (PERFECT_COVERAGE - coverage.to_i)
end
end
end
Slide 106
Slide 106 text
Skunk
module RubyCritic
class AnalysedModule
def skunk_score
return cost if perfect_coverage?
cost * (PERFECT_COVERAGE - coverage.to_i)
# Penalty Factor = 100% - 20% => 80
end
end
end
Slide 107
Slide 107 text
Maintainability
1. Code Coverage (SimpleCov)
2. Code Quality (RubyCritic)
3. Skunk Score (Skunk)
Slide 108
Slide 108 text
You are here:
Slide 109
Slide 109 text
Tech Debt
Slide 110
Slide 110 text
How can we gradually
pay off technical debt?
Slide 111
Slide 111 text
1. Removing files
2. Refactoring files
3. Increasing coverage
Slide 112
Slide 112 text
1. Removing files
Slide 113
Slide 113 text
Find Dead Code
Slide 114
Slide 114 text
Coverband
(github.com/danmayer/coverband)
Slide 115
Slide 115 text
No content
Slide 116
Slide 116 text
2.Refactoring files
Slide 117
Slide 117 text
100
0
1 100_000
Complexity
Churn
“Great
candidates for
refactoring
(paying technical
debt!)” 👨💻
Slide 118
Slide 118 text
Churn vs. Complexity is
not enough to prioritize
Slide 119
Slide 119 text
Skunk Score Table
Slide 120
Slide 120 text
Refactor These Files:
Slide 121
Slide 121 text
Refactor These Files:
Slide 122
Slide 122 text
100
0
1 100_000
Complexity
Churn
git.rb
Slide 123
Slide 123 text
100
0
1 100_000
Complexity
Churn
git_calculator.rb
git_fetcher.rb
refactor!
git.rb
Slide 124
Slide 124 text
2. Asdf
3. Increasing
coverage
Slide 125
Slide 125 text
Skunk Score Table
Slide 126
Slide 126 text
Write Tests Here:
Slide 127
Slide 127 text
2.
Culture
Slide 128
Slide 128 text
No content
Slide 129
Slide 129 text
No content
Slide 130
Slide 130 text
No content
Slide 131
Slide 131 text
No content
Slide 132
Slide 132 text
Team Rebuild vs.
Team Refactor
Slide 133
Slide 133 text
No content
Slide 134
Slide 134 text
Technical Debt
Maturity Model
Slide 135
Slide 135 text
No content
Slide 136
Slide 136 text
Reactive
# TODO: refactor me
later
# FIXME: won’t scale
Slide 137
Slide 137 text
Proactive
tech_debt.md
.codeclimate.yml
Slide 138
Slide 138 text
Strategic
Assign tech debt
champions
Better planning (team-
based priorities)
Slide 139
Slide 139 text
Structural
Optimize tech debt
management
Organization-level
standards
Slide 140
Slide 140 text
Structural
Embed tech debt
considerations into the
developer workflow
Slide 141
Slide 141 text
How mature is
your team?
Slide 142
Slide 142 text
“I paid off some tech
debt with this PR.”
Slide 143
Slide 143 text
“I paid off 2% of our
tech debt with this PR.”
Slide 144
Slide 144 text
“This project is in pretty
good shape.”
Slide 145
Slide 145 text
“This project has a skunk
score average of 120
and it’s 78.7 libyears
behind.”
Slide 146
Slide 146 text
From Tech Debt to
Tech Health
Slide 147
Slide 147 text
Tech Health
It’s not a
technical
problem
Slide 148
Slide 148 text
No content
Slide 149
Slide 149 text
Tech Debt
It’s a company
culture
problem
Slide 150
Slide 150 text
No content
Slide 151
Slide 151 text
Tech Health
It’s a company-
wide effort
Slide 152
Slide 152 text
From Tech Debt to
Tech Health
1.Where are you?
2.Where do you want to
be?
Slide 153
Slide 153 text
c
153
‣ mastodon.social/@etagwerker
‣ mastodon.online/@OmbuLabs (AI/ML)
‣ ruby.social/@FastRuby (Tech Debt)
Slide 154
Slide 154 text
Thank you!
Slide 155
Slide 155 text
Questions?
Slide 156
Slide 156 text
Resources
1. https://github.com/whitesmith/rubycritic
2.https://github.com/colszowka/simplecov
3.https://github.com/metricfu/metric_fu
4.https://github.com/julianrubisch/attractor
5.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html
6.https://www.reddit.com/r/ruby/comments/2bq092/rubycritic/
7.http://jakescruggs.blogspot.com/2008/08/whats-good-
fl
og-score.html
8.http://ruby.sadi.st/Flog.html
9.http://ruby.sadi.st/Flay.html
10.https://github.com/makaroni4/sandi_meter
11.https://www.fastruby.io/blog/code-quality/intruducing-skunk-stink-score-
calculator.html
Slide 157
Slide 157 text
Resources
1.https://codeclimate.com/blog/deciphering-ruby-code-metrics/
2.https://www.stickyminds.com/article/getting-empirical-about-refactoring
3.https://www.sandimetz.com/blog/2017/9/13/breaking-up-the-behemoth
4.https://github.com/troessner/reek
5.https://github.com/seattlerb/
fl
ay
6.https://github.com/seattlerb/
fl
og
7.http://www.sqa.net/iso9126.html
8.https://www.slideshare.net/mscottford/important-metrics-for-measuring-code-health
9.https://dilbert.com/strip/2006-12-08
10.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html
11.https://www.fastruby.io/blog/code-quality/code-coverage/rubycritic-4-2-0-
simplecov-support.html
Slide 158
Slide 158 text
Resources
1. https://www.ombulabs.com/blog/tech-debt-maturity-model.html
2. https://www.computer.org/csdl/magazine/so/
2023/03/10109339/1MESXKyAYNO
3. https://www.linkedin.com/pulse/how-google-measures-manages-
tech-debt-abi-noda/
4. https://www.forbes.com/sites/rogerdooley/2023/01/06/southwest-
meltdown-could-cost-825-million/?sh=672485e8709f
5. https://www.ciodive.com/news/southwest-invests-billions-technology-
cloud/705898/
6. https://ieeexplore.ieee.org/document/10109339