Slide 1
Slide 1 text
Here Be Dragons:
The Hidden Gems
of Tech Debt
Ernesto Tagwerker (@etagwerker)
RubyConf Mini, November 2022
Slide 2
Slide 2 text
🤓 My pronouns are he/him
Slide 3
Slide 3 text
🇦🇷 Hi, I’m from Argentina
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
🦅 I live in Philadelphia
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
👨💻 I love Ruby & Open Source!
Slide 8
Slide 8 text
No content
Slide 9
Slide 9 text
đź‘Ş Thanks to my sponsors!
Slide 10
Slide 10 text
Teo, Claire, and Luli
Slide 11
Slide 11 text
Founder &
CTO
@OmbuLabs &
@FastRubyIO
Slide 12
Slide 12 text
Open Source
Contributor/Maintainer
rubycritic; skunk;
bundler-leak; next_rails
Slide 13
Slide 13 text
Here Be Dragons
(Story Time)
Slide 14
Slide 14 text
“hic sunt dracones”
Slide 15
Slide 15 text
“here be dragons”
Slide 16
Slide 16 text
By Abraham Ortelius - The Library of Congress, Public Domain
Slide 17
Slide 17 text
By Abraham Ortelius - The Library of Congress, Public Domain
Slide 18
Slide 18 text
“Be careful with this part of
the world, nobody has been
there, there may be
monsters.”
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
The Hunt-Lenox
Globe
Slide 21
Slide 21 text
Credit: Rare Book Division, The New York Public Library
Slide 22
Slide 22 text
Credit: University of Rochester
Slide 23
Slide 23 text
What if there was a
map to your
codebase?
Slide 24
Slide 24 text
“Be careful with this
part of the world,
nobody has been there,
there may be monsters.”
Slide 25
Slide 25 text
“Be careful with this
part of the codebase,
nobody wants to touch it,
there is a lot of tech debt.”
Slide 26
Slide 26 text
How do you
fi
nd these
dragons in your
codebase?
Slide 27
Slide 27 text
How do you
fi
nd
tech debt
in your codebase?
Slide 28
Slide 28 text
No content
Slide 29
Slide 29 text
Tech Debt &
Dragons
Slide 30
Slide 30 text
1. The Complexity Dragon
Slide 31
Slide 31 text
1. The Complexity Dragon
2. The Smelly Dragon
Slide 32
Slide 32 text
1. The Complexity Dragon
2. The Smelly Dragon
3. The Shapeshifting Dragon
Slide 33
Slide 33 text
1. The Complexity Dragon
2. The Smelly Dragon
3. The Shapeshifting Dragon
4. The Untested Dragon
Slide 34
Slide 34 text
The
Complexity
Dragon
Slide 35
Slide 35 text
No content
Slide 36
Slide 36 text
Complexity
1 class Foo
2 def yay
3 a = eval "1+1"
4 if a == 2
5 puts "yay"
6 end
7 end
8 end
Slide 37
Slide 37 text
Complexity
flog
Slide 38
Slide 38 text
Complexity
“flog reports the
most tortured
code”
Slide 39
Slide 39 text
Complexity
$ flog foo.rb
10.6: flog total
Slide 40
Slide 40 text
Complexity
1 class Foo
2 def yay
3 a = eval "1+1" # 1.2 + 6.0
4 if a == 2 # + 1.2
5 puts “yay" # + 1.2
6 end. # =
7 end. # 10.6
8 end
Slide 41
Slide 41 text
55 ## flog.rb
56 # Eval forms
57
58 SCORES.merge!(:define_method => 5,
59 :eval => 5,
60 :module_eval => 5,
61 :class_eval => 5,
62 :instance_eval => 5)
63
Slide 42
Slide 42 text
62 ## flog.rb
63 # Various "magic" usually used for "clever code"
64
65 SCORES.merge!(:alias_method => 2,
66 :extend => 2,
67 :include => 2,
68 :instance_method => 2,
69 :instance_methods => 2,
70 :method_added => 2,
71 :method_defined? => 2,
72 :method_removed => 2,
73 :method_undefined => 2,
74 :private_class_method => 2,
75 :private_instance_methods => 2,
76 :private_method_defined? => 2,
77 :protected_instance_methods => 2,
78 :protected_method_defined? => 2,
Slide 43
Slide 43 text
37 ## flog.rb
38 # Various non-call constructs
39
40 OTHER_SCORES = {
41 :alias => 2,
42 :assignment => 1,
43 :block => 1,
44 :block_pass => 1,
45 :block_call => 1,
46 :branch => 1,
47 :lit_fixnum => 0.25,
48 :sclass => 5,
49 :super => 1,
50 :to_proc_icky! => 10,
51 :to_proc_lasgn => 15,
52 :yield => 1,
53 }
54
Slide 44
Slide 44 text
Flog won’t report:
- Known code smells
- Duplicated code
- Code that constantly
changes
Slide 45
Slide 45 text
The
Smelly
Dragon
Slide 46
Slide 46 text
No content
Slide 47
Slide 47 text
Code Smells
“Smells are indicators of
where your code might be
hard to read, maintain or
evolve.”
Slide 48
Slide 48 text
Code Smells
They are not necessarily
wrong.
Slide 49
Slide 49 text
Code Smells
For example: Duplicated
code is a code smell.
Slide 50
Slide 50 text
Code Smells
“Duplication is far cheaper
than the wrong
abstraction.” Sandi Metz
Slide 51
Slide 51 text
Code Smells
Reek
(github.com/troessner/reek)
Slide 52
Slide 52 text
No content
Slide 53
Slide 53 text
No content
Slide 54
Slide 54 text
No content
Slide 55
Slide 55 text
1 module Reek
2 module SmellDetectors
3 class UncommunicativeVariableName < BaseDetector
4 #
5 # Checks the given +context+ for uncommunicative names.
6 #
7 # @return [Array]
8 #
9 def sniff
10 variable_names.select do |name, _lines|
11 uncommunicative_variable_name?(name)
12 end.map do |name, lines|
13 smell_warning(
14 lines: lines,
15 message: "has the variable name '#{name}'",
16 parameters: { name: name.to_s })
17 end
18 end
Slide 56
Slide 56 text
Code Smells
1 class Foo
2 def yay
3 a = eval "1+1"
4 if a == 2
5 puts "yay"
6 end
7 end
8 end
Slide 57
Slide 57 text
No content
Slide 58
Slide 58 text
Reek
$ gem install reek
Slide 59
Slide 59 text
Reek
$ reek foo.rb
Slide 60
Slide 60 text
Code Smells
$ reek foo.rb
foo.rb -- 2 warnings:
[1]:IrresponsibleModule: Foo has no
descriptive comment
[3]:UncommunicativeVariableName:
Foo#yay has the variable name 'a'
Slide 61
Slide 61 text
Code Smells
1 class Foo
2 def yay
3 a = eval "1+1"
4 if a == 2
5 puts "yay"
6 end
7 end
8 end
Slide 62
Slide 62 text
Code Smells
1 class Foo
2 def yay
3 eval_result = eval "1+1"
4 if eval_result == 2
5 puts "yay"
6 end
7 end
8 end
Slide 63
Slide 63 text
Code Smells
$ reek foo.rb
foo.rb -- 1 warning:
[1]:IrresponsibleModule: Foo has no
descriptive comment
Slide 64
Slide 64 text
No content
Slide 65
Slide 65 text
Code Smells
$ reek foo.rb
foo.rb -- 1 warning:
[1]:IrresponsibleModule: Foo has no
descriptive comment
Slide 66
Slide 66 text
Code Smells
1 class Foo
2 def yay
3 eval_result = eval "1+1"
4 if eval_result == 2
5 puts "yay"
6 end
7 end
8 end
Slide 67
Slide 67 text
Code Smells
1 ---
2
3 ### .reek.yml (Generic smell configuration)
4
5 detectors:
6 IrresponsibleModule:
7 enabled: false
Slide 68
Slide 68 text
Code Smells
$ reek foo.rb
Inspecting 1
fi
le(s):
.
Slide 69
Slide 69 text
No content
Slide 70
Slide 70 text
Reek won’t report:
- Is this file
constantly changing?
- Is this file super
complex?
Slide 71
Slide 71 text
The
Shapeshifting
Dragon
Slide 72
Slide 72 text
No content
Slide 73
Slide 73 text
Churn
Slide 74
Slide 74 text
Churn
git
Slide 75
Slide 75 text
Churn
$ git log --follow --format=%h foo.rb | cat
573c8f5
2ccc5b8
b447bc3
Slide 76
Slide 76 text
Churn
$ git log --follow --format=%h foo.rb | cat
573c8f5
2ccc5b8
b447bc3
# Churn Count = 3
Slide 77
Slide 77 text
Churn
(github.com/danmayer/churn)
Slide 78
Slide 78 text
Churn
- Mercurial
- Git
- SVN
Slide 79
Slide 79 text
Churn
minitest $ churn --start_date "12 years ago" -e rb
Slide 80
Slide 80 text
Churn
- By itself it adds
little value
- Useful when combined
with other metrics
Slide 81
Slide 81 text
How can we bring
it all together?
- Complexity
- Code Smells
- Churn
Slide 82
Slide 82 text
RubyCritic
Slide 83
Slide 83 text
"Cartography" for
Rubyists
RubyCritic
(github.com/whitesmith/rubycritic)
Slide 84
Slide 84 text
RubyCritic
$ gem install rubycritic
Slide 85
Slide 85 text
RubyCritic
$ rubycritic
Slide 86
Slide 86 text
No content
Slide 87
Slide 87 text
No content
Slide 88
Slide 88 text
GPA Pie Graph
GPA Grade (Cost)
Slide 89
Slide 89 text
GPA Pie Graph
Costly Files:
- Hard to understand
Slide 90
Slide 90 text
GPA Pie Graph
Costly Files:
- Hard to understand
- Hard to maintain
Slide 91
Slide 91 text
RubyCritic’s Cost
- Code Smells (reek)
- Complexity (flog)
Slide 92
Slide 92 text
RubyCritic’s Cost
module RubyCritic
class AnalysedModule
def cost
smells.map(&:cost).inject(0.0, :+) + # From Reek
(complexity / COMPLEXITY_FACTOR) # From Flog
end
end
end
Slide 93
Slide 93 text
Churn vs.
Complexity
Churn (SCM)
Complexity (flog)
Slide 94
Slide 94 text
Churn
Complexity
50
0%
1 100_000
Slide 95
Slide 95 text
Churn
Complexity
100
0%
1 100_000
user.rb (changed: 2 times; complexity: 100_000 FPs )
Slide 96
Slide 96 text
Churn
Complexity
100
0%
1 100_000
account.rb
account.rb (changed: 27 times; complexity: 100 FPs)
Slide 97
Slide 97 text
100%
0%
1 100_000
Complexity
Churn
Slide 98
Slide 98 text
100%
0%
1 100_000
Complexity
Churn
Slide 99
Slide 99 text
100%
0%
1 100_000
“No dragons here.” ❤
Complexity
Churn
Slide 100
Slide 100 text
100%
0%
1 100_000
“No one understands these
fi
les but they work. So
don’t change them.”
Complexity
Churn
Slide 101
Slide 101 text
100%
0%
1 100_000
“[…], if the code never
changes, it's not costing us
money.” Sandi Metz
Complexity
Churn
Slide 102
Slide 102 text
100%
0%
1 100_000
“These
fi
les are easy to
understand but you need
to change them often
because of reasons…”
Complexity
Churn
Slide 103
Slide 103 text
100%
0%
1 100_000
Complexity
Churn
“These
fi
les are complex and
change a lot…”
Slide 104
Slide 104 text
100%
0%
1 100_000
Complexity
Churn
“Sometimes a class becomes so
complex that refactoring seems too
dif
fi
cult.” Michael Feathers
Slide 105
Slide 105 text
No content
Slide 106
Slide 106 text
100%
0%
1 100_000
“Here be Dragons.”
Complexity
Churn
“Here be Dragons.”
“Here be Dragons.” “Here be Dragons.”
Slide 107
Slide 107 text
No content
Slide 108
Slide 108 text
No content
Slide 109
Slide 109 text
The Untested
Dragon
Slide 110
Slide 110 text
No content
Slide 111
Slide 111 text
Code Coverage
SimpleCov
(github.com/simplecov-ruby/simplecov)
Slide 112
Slide 112 text
Code Coverage
# Gemfile
group :test do
gem "simplecov", require: false
end
Slide 113
Slide 113 text
Code Coverage
# spec/spec_helper.rb
if ENV["COVERAGE"] == "true"
require 'simplecov'
SimpleCov.start do
add_group "Libraries", "lib"
add_filter "/spec/"
track_files "**/*.rb"
end
end
Slide 114
Slide 114 text
Code Coverage
$ COVERAGE=true rspec
Slide 115
Slide 115 text
No content
Slide 116
Slide 116 text
No content
Slide 117
Slide 117 text
Recap
1. The Complexity Dragon (
fl
og)
2. The Smelly Dragon (reek)
3. The Shapeshifting Dragon (churn)
4. The Untested Dragon (simplecov)
Slide 118
Slide 118 text
Finding Dragons
Example: User vs. Account Class
Slide 119
Slide 119 text
user.rb
churn: 10
complexity: 10
smells: 10
Tech debt score: 1,000
(10*10*10)
Slide 120
Slide 120 text
account.rb
churn: 10
complexity: 10
smells: 10
Tech debt score: 1,000
(10*10*10)
Slide 121
Slide 121 text
user.rb
churn: 10
complexity: 10
smells: 10
Tech debt score: 1,000
(10*10*10)
account.rb
churn: 10
complexity: 10
smells: 10
Tech debt score: 1,000
(10*10*10)
Slide 122
Slide 122 text
User and Account
have the same
amount of tech debt
Slide 123
Slide 123 text
“Do they?”
Slide 124
Slide 124 text
user.rb
churn: 10
complexity: 10
smells: 10
code_coverage: 0%
Tech debt score:
100,000
(10*10*10) * 100
Slide 125
Slide 125 text
account.rb
churn: 10
complexity: 10
smells: 10
code_coverage: 90%
Tech debt score:
10,000
(10*10*10) * 10
Slide 126
Slide 126 text
user.rb
churn: 10
complexity: 10
smells: 10
code_coverage: 0%
Tech debt score:
100,000
account.rb
churn: 10
complexity: 10
smells: 10
code_coverage: 90%
Tech debt score:
10,000
Slide 127
Slide 127 text
User has more tech
debt than Account
Slide 128
Slide 128 text
Code Smells vs.
Complexity vs.
Code Coverage
Slide 129
Slide 129 text
No content
Slide 130
Slide 130 text
Cost
Slide 131
Slide 131 text
Cost
- Code Smells (reek)
- Complexity (flog)
Slide 132
Slide 132 text
Cost
Code Coverage
Slide 133
Slide 133 text
What if we could
calculate a new signal
that combines both of
them?
Slide 134
Slide 134 text
Cost
Code Coverage
SkunkScore
Slide 135
Slide 135 text
SkunkScore =
f(cost, code_coverage)
Slide 136
Slide 136 text
Why Code Coverage?
Files which lack tests
are harder to maintain
Slide 137
Slide 137 text
Skunk
(github.com/fastruby/skunk)
Slide 138
Slide 138 text
Skunk
$ gem install skunk
Slide 139
Slide 139 text
Skunk
$ skunk
Slide 140
Slide 140 text
No content
Slide 141
Slide 141 text
No content
Slide 142
Slide 142 text
Skunk
def skunk_score
return cost.round(2) if coverage == PERFECT_COVERAGE
(cost * penalty_factor).round(2)
end
def penalty_factor
PERFECT_COVERAGE - coverage.to_i
end
Slide 143
Slide 143 text
Now we can
fi
nally draw
our map!
Slide 144
Slide 144 text
Here be dragons
Here be dragons
Here be dragons Here be dragons
Here be dragons
Slide 145
Slide 145 text
100%
0%
1 100_000
Complexity
Churn
“Here be dragons.”
“Here be dragons.”
“Here be dragons.”
Slide 146
Slide 146 text
No content
Slide 147
Slide 147 text
How to Slay
Dragons
Slide 148
Slide 148 text
No content
Slide 149
Slide 149 text
Dragon Slaying 101:
- Define what’s a dragon
Slide 150
Slide 150 text
What’s a dragon?
- Team discussion on
“tech debt”
- Toolkit configuration
Slide 151
Slide 151 text
Dragon Slaying 101:
- Define what’s a dragon
- Draw the map and place
“here be dragons”
Slide 152
Slide 152 text
Dragon Slaying 101:
- Define what’s a dragon
- Draw the map and place
“here be dragons”
- Pick your battles
based on metrics
Slide 153
Slide 153 text
Slaying the untested dragon
Slide 154
Slide 154 text
Slaying the untested dragon
Slide 155
Slide 155 text
Slaying the untested dragon
Slide 156
Slide 156 text
Slaying the smelly dragon
Slide 157
Slide 157 text
$ reek foo.rb
foo.rb -- 2 warnings:
[1]:IrresponsibleModule: Foo has no descriptive
comment
[3]:UncommunicativeVariableName: Foo#yay
has the variable name 'a'
Slaying the smelly dragon
Slide 158
Slide 158 text
$ reek foo.rb
foo.rb -- 2 warnings:
[1]:IrresponsibleModule: Foo has no
descriptive comment
[3]:UncommunicativeVariableName: Foo#yay
has the variable name 'a'
Slaying the smelly dragon
Slide 159
Slide 159 text
Slaying the smelly dragon
Slide 160
Slide 160 text
Slaying the complexity dragon
Slide 161
Slide 161 text
Slaying the complexity dragon
Slide 162
Slide 162 text
1. Flog
2. Reek
3. Churn
4. SimpleCov
5. RubyCritic
6. Skunk
Slide 163
Slide 163 text
8. Attractor
9. MetricFu
10. CodeClimate
11. CodeScene
12. Codacy
13. CodeCov
14. SonarQube
15. Flay
16. Sandi Meter
17. Coverband
18. Etc..
1. Flog
2. Reek
3. Churn
4. SimpleCov
5. RubyCritic
6. Skunk
7. Rubocop
Slide 164
Slide 164 text
1. Pick your dragons
Slide 165
Slide 165 text
1. Pick your dragons
2. Pick your toolkit
Slide 166
Slide 166 text
1. Pick your dragons
2. Pick your toolkit
3. Slay your dragons
Slide 167
Slide 167 text
Thank you!
@etagwerker - https://speakerdeck.com/etagwerker
Slide 168
Slide 168 text
Resources
1. https://github.com/whitesmith/rubycritic
2.https://github.com/colszowka/simplecov
3.https://github.com/metricfu/metric_fu
4.https://github.com/julianrubisch/attractor
5.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-
gems.html
6.https://www.reddit.com/r/ruby/comments/2bq092/rubycritic/
7.http://jakescruggs.blogspot.com/2008/08/whats-good-
fl
og-score.html
8.http://ruby.sadi.st/Flog.html
9.https://github.com/makaroni4/sandi_meter
Slide 169
Slide 169 text
Resources
1.https://codeclimate.com/blog/deciphering-ruby-code-metrics/
2.https://www.stickyminds.com/article/getting-empirical-about-refactoring
3.https://www.sandimetz.com/blog/2017/9/13/breaking-up-the-behemoth
4.https://github.com/troessner/reek
5.https://github.com/seattlerb/
fl
ay
6.https://github.com/seattlerb/
fl
og
7.http://www.sqa.net/iso9126.html
8.https://www.slideshare.net/mscottford/important-metrics-for-measuring-code-health
9.https://www.theatlantic.com/technology/archive/2013/12/no-old-maps-actually-
say-here-be-dragons/282267/
10.https://education.nationalgeographic.org/resource/here-be-dragons
11.https://resurrect3d.lib.rochester.edu/models/5bdc5e604270f527332282dd