Escaping The Tar Pit at Solidus Conf 2019

Slide 1

Slide 1 text

Escaping The Tar Pit Ernesto Tagwerker (@etagwerker) SolidusConf, October 2019

Slide 2

Slide 2 text

! Hi, I’m from Argentina I live in Philadelphia # I once forked Spree

Slide 3

Slide 3 text

Founder & Software Engineer @OmbuLabs

Slide 4

Slide 4 text

Founder & Software Engineer @fastrubyio

Slide 5

Slide 5 text

Open Source Contributor & Maintainer database_cleaner; bundler-leak; next_rails

Slide 6

Slide 6 text

Inspiration

Slide 7

Slide 7 text

Chapter 1: The Tar Pit

Slide 8

Slide 8 text

“No scene from prehistory is quite so vivid as that of the mortal struggles of great beasts in the tar pits. […] The ﬁercer the struggle, the more entangling the tar, and no beast is so strong or so skillful but that they ultimately sink.” Fred Brooks

Slide 9

Slide 9 text

“Large-system programming has over the past decade been such a tar pit, and many great and powerful beasts have thrashed violently in it.” Fred Brooks

Slide 10

Slide 10 text

We are in the tar pit; trying to avoid it; or trying to get out.

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

Tar Pit Symptoms > Projects running over budget

Slide 13

Slide 13 text

Tar Pit Symptoms > Projects running over budget > Taking forever to ship small changes

Slide 14

Slide 14 text

Tar Pit Symptoms > Projects running over budget > Taking forever to ship small changes > Sacriﬁcing quality, increasing tech debt

Slide 15

Slide 15 text

Part 1: How to avoid the tar pit

Slide 16

Slide 16 text

“THAT’D BE GREAT” BOSS: “IF YOU COULD COME IN ON THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”

Slide 17

Slide 17 text

“THAT’D BE GREAT” CLIENT: “IF YOU COULD COME IN ON THIS GREAT, LEGACY PROJECT AND MAINTAIN IT FROM NOW ON…”

Slide 18

Slide 18 text

“BECAUSE I NEED IT BY END OF DAY TODAY” CLIENT: “HOW LONG IS IT GOING TO TAKE?”

Slide 19

Slide 19 text

How can we quickly assess code quality?

Slide 20

Slide 20 text

Part 2: How to get out of the tar pit

Slide 21

Slide 21 text

How can we gradually pay off technical debt?

Slide 22

Slide 22 text

Part 1: Avoiding the tar pit

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

“It can’t be that hard. It’s a Rails project.”

Slide 25

Slide 25 text

How can we quickly assess code quality?

Slide 26

Slide 26 text

Assessing Quality ✅ Paid Services ✅ Open Source Gems

Slide 27

Slide 27 text

Assessing Quality ✅ Paid Services ➡ Open Source Gems

Slide 28

Slide 28 text

Open Source Gems ✅ Static code analysis

Slide 29

Slide 29 text

Open Source Gems ✅ Static code analysis ✅ Code coverage

Slide 30

Slide 30 text

Open Source Gems ✅ Static code analysis ✅ Code coverage ✅ Code smells

Slide 31

Slide 31 text

What is software quality anyway?

Slide 32

Slide 32 text

There are hundreds of deﬁnitions.

Slide 33

Slide 33 text

“The degree to which a system, component, or process meets implicit and explicit requirements.” IEEE

Slide 34

Slide 34 text

“It works as expected.”

Slide 35

Slide 35 text

“It works as expected and it is not a PITA to maintain it.”

Slide 36

Slide 36 text

ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅ Efﬁciency ✅ Maintainability ✅ Portability

Slide 37

Slide 37 text

ISO 9126-1 Software Quality Model ✅ Reliability ✅ Usability ✅ Efﬁciency ➡ Maintainability ✅ Portability

Slide 38

Slide 38 text

Maintainability

Slide 39

Slide 39 text

Maintainability 1. Code Coverage

Slide 40

Slide 40 text

Maintainability 1. Code Coverage 2. Code Quality

Slide 41

Slide 41 text

Code Coverage SimpleCov (github.com/colszowka/simplecov)

Slide 42

Slide 42 text

Code Coverage # Gemfile group :test do gem "simplecov", require: false end

Slide 43

Slide 43 text

Code Coverage # spec/spec_helper.rb if ENV["COVERAGE"] == "true" require 'simplecov' SimpleCov.start do add_group "Models", "models" add_filter "/spec/" track_files "**/*.rb" end end

Slide 44

Slide 44 text

Code Coverage # spec/spec_helper.rb if ENV["COVERAGE"] == "true" require 'simplecov' SimpleCov.start "rails" end

Slide 45

Slide 45 text

Code Coverage $ COVERAGE=true rspec

Slide 46

Slide 46 text

Code Coverage

Slide 47

Slide 47 text

Code Quality n options

Slide 48

Slide 48 text

Code Quality n options (ﬂog; ﬂay; reek; churn; RubyCritic; MetricFu; attractor; rubocop; fukuzatsu; turbulence; …)

Slide 49

Slide 49 text

Code Quality RubyCritic (github.com/whitesmith/rubycritic)

Slide 50

Slide 50 text

Churn

Slide 51

Slide 51 text

Churn git

Slide 52

Slide 52 text

Churn $ git log --follow --format=%h default.rb | cat 573c8f5 2ccc5b8 b447bc3

Slide 53

Slide 53 text

Churn $ git log --follow --format=%h default.rb | cat 573c8f5 2ccc5b8 b447bc3 # Churn Count = 3

Slide 54

Slide 54 text

Complexity

Slide 55

Slide 55 text

Complexity flog

Slide 56

Slide 56 text

Complexity $ flog foo.rb 11.2: flog total

Slide 57

Slide 57 text

Complexity class Foo def yay # 11.2 = a = eval "1+1" # 1.2 + 6.0 + if a == 2 then # 1.2 + 1.2 + 0.4 + puts "yay" # 1.2 end end end

Slide 58

Slide 58 text

Churn Complexity 50 0% 1 100_000

Slide 59

Slide 59 text

Churn Complexity 100 0% 1 100_000 user.rb (changed: 2 times; complexity: 100_000 )

Slide 60

Slide 60 text

Churn Complexity 100 0% 1 100_000 user.rb user.rb (changed: 27 times; complexity: 100)

Slide 61

Slide 61 text

Code Quality $ gem install rubycritic

Slide 62

Slide 62 text

Code Quality $ rubycritic

Slide 63

Slide 63 text

No content

Slide 64

Slide 64 text

GPA Pie Graph Complexity (flog) Code Smells (reek)

Slide 65

Slide 65 text

Churn vs. Complexity Churn (SCM) Complexity (flog)

Slide 66

Slide 66 text

100% 0% 1 100_000 Complexity Churn

Slide 67

Slide 67 text

100% 0% 1 100_000 Complexity Churn

Slide 68

Slide 68 text

100% 0% 1 100_000 “Welcome to the good place.” ❤ Complexity Churn

Slide 69

Slide 69 text

solidus/core

Slide 70

Slide 70 text

100% 0% 1 100_000 “No one understands these ﬁles but they work. So don’t change them.” Complexity Churn

Slide 71

Slide 71 text

100% 0% 1 100_000 “[…], if the code never changes, it's not costing us money.” Sandi Metz Complexity Churn

Slide 72

Slide 72 text

solidus/core

Slide 73

Slide 73 text

100% 0% 1 100_000 “Everybody understands these ﬁles but you need to change them often because of reasons…” Complexity Churn

Slide 74

Slide 74 text

solidus/core

Slide 75

Slide 75 text

100% 0% 1 100_000 Complexity Churn “Here be dragons. These modules are complex and change a lot…” '

Slide 76

Slide 76 text

100% 0% 1 100_000 Complexity Churn “Sometimes a class becomes so complex that refactoring seems too difﬁcult.” Michael Feathers

Slide 77

Slide 77 text

solidus/core

Slide 78

Slide 78 text

Are you getting into a tar pit, is it a dumpster ﬁre, or have you found a project which is easy to maintain?

Slide 79

Slide 79 text

100% 0% 1 100_000 “Welcome to the tar pit.” Complexity Churn

Slide 80

Slide 80 text

No content

Slide 81

Slide 81 text

Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic)

Slide 82

Slide 82 text

Code Coverage Code Smells Complexity

Slide 83

Slide 83 text

Signal #1 Code Coverage

Slide 84

Slide 84 text

Signal #2 Complexity

Slide 85

Slide 85 text

Coverage & Complexity

Slide 86

Slide 86 text

But what if we could calculate a new signal that combines both?

Slide 87

Slide 87 text

Signal #3 StinkScore

Slide 88

Slide 88 text

StinkScore = f(code_quality, code_coverage)

Slide 89

Slide 89 text

StinkScore = f(code_smells, churn, code_complexity, code_coverage)

Slide 90

Slide 90 text

code_coverage Files which lack tests should be penalized

Slide 91

Slide 91 text

foo.rb churn: 10 complexity: 10 smells: 10 smell: 1,000 (10*10*10)

Slide 92

Slide 92 text

bar.rb churn: 10 complexity: 10 smells: 10 smell: 1,000 (10*10*10)

Slide 93

Slide 93 text

foo.rb churn: 10 complexity: 10 smells: 10 smell: 1,000 (10*10*10) bar.rb churn: 10 complexity: 10 smells: 10 smell: 1,000 (10*10*10)

Slide 94

Slide 94 text

foo.rb is not stinkier than bar.rb

Slide 95

Slide 95 text

foo.rb churn: 10 complexity: 10 smells: 10 code_coverage: 0 stink_score: 100,000 (10*10*10)*100

Slide 96

Slide 96 text

bar.rb churn: 10 complexity: 10 smells: 10 code_coverage: 100 stink_score: 1,000 (10*10*10)*1

Slide 97

Slide 97 text

foo.rb churn: 10 complexity: 10 smells: 10 code_coverage: 0 stink_score: 100,000 (10*10*10)*100 bar.rb churn: 10 complexity: 10 smells: 10 code_coverage: 100 stink_score: 1,000 (10*10*10)*1

Slide 98

Slide 98 text

foo.rb is considerably stinkier than bar.rb

Slide 99

Slide 99 text

Stink Score Calculator Skunk (github.com/fastruby/skunk)

Slide 100

Slide 100 text

Skunk $ gem install skunk

Slide 101

Slide 101 text

Skunk $ skunk

Slide 102

Slide 102 text

No content

Slide 103

Slide 103 text

No content

Slide 104

Slide 104 text

Skunk $ skunk ... StinkScore Total: 13220.859999999997 Modules Analysed: 71 StinkScore Average: 0.1862092957746 Worst StinkScore: 2401.75 (lib/rubycritic/source_control_systems/git.rb)

Slide 105

Slide 105 text

RubyCritic module RubyCritic class AnalysedModule def cost smells.map(&:cost).inject(0.0, :+) + # From Reek (complexity / COMPLEXITY_FACTOR) # From Flog end end end

Slide 106

Slide 106 text

Skunk module RubyCritic class AnalysedModule def stink_score return churn_times_cost if perfect_coverage? (churn_times_cost * (PERFECT_COVERAGE - coverage.to_i)) end def churn_times_cost safe_churn = churn > 0 ? churn : 1 safe_churn * cost end end end

Slide 107

Slide 107 text

Slide 108

Slide 108 text

Skunk Warning > skunk-v0.2.0 > feedback wanted

Slide 109

Slide 109 text

Maintainability 1. Code Coverage (SimpleCov) 2. Code Quality (RubyCritic) 3. StinkScore (Skunk)

Slide 110

Slide 110 text

You are here:

Slide 111

Slide 111 text

Part 2: Getting out of the tar pit

Slide 112

Slide 112 text

So, where do we begin?

Slide 113

Slide 113 text

removing files will decrease our stink_score

Slide 114

Slide 114 text

Find Dead Code Coverband (github.com/danmayer/coverband)

Slide 115

Slide 115 text

No content

Slide 116

Slide 116 text

refactoring complex files will decrease our stink_score

Slide 117

Slide 117 text

100% 0% 1 100_000 Complexity Churn “Here be dragons. These modules are complex and change a lot…” '

Slide 118

Slide 118 text

100% 0% 1 100_000 Complexity Churn “Great candidates for refactoring (paying technical debt!)” #

Slide 119

Slide 119 text

Churn vs. Complexity is not enough to prioritize

Slide 120

Slide 120 text

Stink Score Table

Slide 121

Slide 121 text

Refactor These Files:

Slide 122

Slide 122 text

Refactor These Files:

Slide 123

Slide 123 text

100% 0% 1 100_000 Complexity Churn git.rb

Slide 124

Slide 124 text

100% 0% 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb refactor! git.rb

Slide 125

Slide 125 text

100% 0% 1 100_000 Complexity Churn git_calculator.rb git_fetcher.rb

Slide 126

Slide 126 text

Skunk $ skunk ... StinkScore Total: 13220.859999999997 11220 Modules Analysed: 71 StinkScore Average: 0.1862092957746 0.157

Slide 127

Slide 127 text

writing tests for our app will decrease our stink_score

Slide 128

Slide 128 text

Stink Score Table

Slide 129

Slide 129 text

Write Tests Here:

Slide 130

Slide 130 text

Skunk $ skunk ... StinkScore Total: 13220.859999999997 11220 Modules Analysed: 71 StinkScore Average: 0.1862092957746 0.157

Slide 131

Slide 131 text

stink_score your compass to get out of the tar pit

Slide 132

Slide 132 text

stink_score Code Coverage; Complexity; and Smells

Slide 133

Slide 133 text

Thank you! @etagwerker 133

Slide 134

Slide 134 text

Resources 1. https://github.com/whitesmith/rubycritic 2.https://github.com/colszowka/simplecov 3.https://github.com/metricfu/metric_fu 4.https://github.com/julianrubisch/attractor 5.https://www.fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html 6.https://www.reddit.com/r/ruby/comments/2bq092/rubycritic/ 7.http://jakescruggs.blogspot.com/2008/08/whats-good-ﬂog-score.html 8.http://ruby.sadi.st/Flog.html 9.http://ruby.sadi.st/Flay.html 10.https://github.com/makaroni4/sandi_meter

Slide 135

Slide 135 text

Resources 1.https://codeclimate.com/blog/deciphering-ruby-code-metrics/ 2.https://www.stickyminds.com/article/getting-empirical-about-refactoring 3.https://www.sandimetz.com/blog/2017/9/13/breaking-up-the-behemoth 4.https://github.com/troessner/reek 5.https://github.com/seattlerb/ﬂay 6.https://github.com/seattlerb/ﬂog 7.http://www.sqa.net/iso9126.html 8.https://www.slideshare.net/mscottford/important-metrics-for-measuring- code-health 9.https://dilbert.com/strip/2006-12-08

Slide 136

Slide 136 text

Questions?