Deletion Driven Development: Code to delete code! (RubyKaigi edition)

@chrisarcand
Deletion Driven
Development:
Code to delete code!

View Slide

Chris Arcand 
chrisarcand
chrisarcand
www.chrisarcand.com

View Slide

MINNESOTA

View Slide

MINNESOTA
Minneapolis
St. Paul

View Slide

View Slide

@chrisarcand chrisarcand.com
I
PROGRAMMING

View Slide

I
WRITING 
CODE

View Slide

However…

View Slide

I
DELETING 
CODE

View Slide

• Over-engineering 
Previous developer adding “potentially helpful” methods
• Poorly written code 
Super-speciﬁc, single edge case methods
• Refactoring, deprecations 
Callers get refactored over time and methods eventually
aren’t needed anymore
• etc, etc
Dead code? How?

View Slide

“Who cares?”

View Slide

“Deleting Code” (2002)
Ned Batchelder
http://nedbatchelder.com/text/deleting-code.html

View Slide

If you have a chunk of code you don't need
any more, there's one big reason to delete it
for real rather than leaving it in a disabled
state: To reduce noise and uncertainty.
Some of the worst enemies a developer has
are noise or uncertainty in his code,
because they prevent him from working with
it effectively in the future.
“

View Slide

What if we could programmatically
ﬁnd unused code to delete?

View Slide

Ruby code (.rb)
? ? ?

View Slide

Part I: Parsing the code

View Slide

Do you understand the following
sequences of characters?
How do you know?
Modiﬁed examples and grammar from Robert B. Heckendorn, University of Idaho
http://marvin.cs.uidaho.edu/Teaching/CS445/grammar.html

View Slide

“The boy owns a dog.”

View Slide

“The dog loves the boy.”

View Slide

“Loves boy the.”

View Slide

How could you programmatically determine
which of those are correct…
…and which are not?

View Slide

✨Context-free Grammar✨ 
(CFG)
A set of production rules that describe all possible
strings in a given formal language.
“What sentences are in the language
and what are not?”

View Slide

✨Backus-Naur Form✨ 
(BNF)
One of two main notation techniques for context-
free grammars, often used to describe computer
programming languages.

View Slide

::=
::=
::=
::=
::= THE | A
::= BOY | DOG
::= OWNS | LOVES | BITES

View Slide

::=
::=
::=
::=
::= THE | A
::= BOY | DOG
Symbols

View Slide

::=
::=
::=
::=
::= THE | A
::= BOY | DOG
< non-terminals >

View Slide

::=
::=
::=
::=
::= THE | A
::= BOY | DOG
TERMINALS

View Slide

::=
::=
::=
::=
::= THE | A
::= BOY | DOG
Symbol on left is replaced with expression on the right
‘OR’ (choose between expressions)
‘deﬁnitional operator’

View Slide

::=
::=
::=
::=
::= THE | A
::= BOY | DOG
The boy owns a dog.

View Slide

::=
::=
::=
::=
::= THE | A
::= BOY | DOG
The boy owns a dog.

Terminal found

View Slide

::=
::=
::=
::=
::= THE | A
::= BOY | DOG
The boy owns a dog.

View Slide

::=
::=
::=
::=
::= THE | A
::= BOY | DOG
The boy owns a dog.

Terminal found

View Slide

::=
::=
::=
::=
::= THE | A
::= BOY | DOG
The boy owns a dog.

View Slide

The boy owns a dog.

View Slide


THE BOY OWNS
A DOG
Parsing was successful!

View Slide


View Slide


THE DOG LOVES
THE BOY
Parsing was successful!

View Slide

Loves boy the.
“ ”

View Slide

::=
::=
::=
::=
::= THE | A
::= BOY | DOG
::= OWNS | LOVES
Loves boy the.
???
???
???

View Slide

“The boy owns a dog.” 
“Loves boy the.”
In programming terms…
OK
OK
SYNTAX ERROR

View Slide

How does Ruby know the
meaning of these characters?
class Person
def initialize(name)
@name = name
end
def say_hello
puts “Hi! My name is #{@name}!”
end
end
chris = Person.new(“Chris”)
chris.say_hello

View Slide

Figure 1-13, Ruby Under a Microscope by Pat Shaughnessy 
Used with permission
How Ruby tokenizes and parses code

View Slide

Figure 1-13, Ruby Under a Microscope by Pat Shaughnessy 
Used with permission
How Ruby tokenizes and parses code
Ruby uses a LALR parser generator called Bison
Look-Ahead, Left, Reversed Rightmost Derivation
LALR(k): k refers to the amount of look-ahead

View Slide

Racc
https://github.com/tenderlove/racc 
An LALR(1) parser generator
How we will tokenize and parse code
ruby_parser
https://github.com/seattlerb/ruby_parser 
A Ruby parser, written in Ruby, using racc

View Slide

class Person
def greet(name)
puts "Hello #{name}!"
end
end
person = Person.new
person.greet("RubyKaigi")

View Slide

class Person
def greet(name)
puts "Hello #{name}!"
end
end
person = Person.new
person.greet("RubyKaigi")
> parser = RubyParser.for_current_ruby
> parser.parse(File.read(“example1.rb”))
1 s(:block,
2 s(:class, :Person, nil,
3 s(:defn, :greet, s(:args, :name),
4 s(:call, nil, :puts,  
5 s(:dstr, "Hello ", s(:evstr, s(:lvar, :name)), s(:str, "!"))))),
6 s(:lasgn, :person, s(:call, s(:const, :Person), :new)),
7 s(:call, s(:lvar, :person), :greet, s(:str, "RubyKaigi")))
example1.rb

View Slide

1 s(:block,
2 s(:class, :Person, nil,
3 s(:defn, :greet, s(:args, :name),
4 s(:call, nil, :puts,  
5 s(:dstr, "Hello ", s(:evstr, …
6 s(:lasgn, :person, s(:call, …
7 s(:call, s(:lvar, :person), :greet, …

View Slide

ruby_parser
racc
Ruby code (.rb)
Ruby23Parser
Parsing
Generic processing
File.read()
s-expression (parse tree)
? ? ?

View Slide

Part II: Processing the s-expression

View Slide

class MinimalSexpProcessor

View Slide

def initialize
@processors ={}
public_methods.each do |name|
case name
when /^process_(.*)/ then
@processors[$1.to_sym] = name.to_sym
end
end
end
end

View Slide

process_defn

View Slide

defn =>:process_defn
:
@processors = {
}
Node type Corresponding
processor

View Slide

def process(exp)
return nil if exp.nil?
type = exp.first
meth = @processors[type] || default_method
return nil unless meth
if meth == default_method && warn_on_default
puts "WARNING: Using default method #{meth} for #{type}"
end
self.send(meth, exp) if meth
end

View Slide

attr_accessor :default_method
attr_accessor :warn_on_default
def initialize
@default_method = nil
@warn_on_default = true
@processors ={}
public_methods.each do |name|
case name
when /^process_(.*)/ then
@processors[$1.to_sym] = name.to_sym
end
end
end
end
def process(exp)
return nil if exp.nil?
type = exp.first
meth = @processors[type] || default_method
return nil unless meth
if meth == default_method && warn_on_default
puts "WARNING: Using default method #{meth} for #{type}"
end
self.send(meth, exp) if meth
end

View Slide


View Slide

class SillyProcessor < MinimalSexpProcessor

View Slide

def process_defn(exp)
method_name = exp[1]
puts "PROCESSING A METHOD DEFINITION NODE: #{method_name}"
process_until_empty(exp)
s()
end
def process_not_defn(exp)
node_type = exp[0]
puts "Processing node: #{node_type}"
s()
end

View Slide

def process_until_empty(exp)
until exp.empty?
sexp = exp.shift
process(sexp) if Sexp === sexp
end
end

View Slide

def initialize
super
@default_method = :process_not_defn
@warn_on_default = false
end

View Slide

class SillyProcessor < MinimalSexpProcessor
puts "PROCESSING A METHOD DEFINITION NODE: #{method_name}"
s()
end
def process_not_defn(exp)
node_type = exp[0]
puts "Processing node: #{node_type}"
s()
end
until exp.empty?
sexp = exp.shift
end
end
end
def initialize
super
@default_method = :process_not_defn
end

View Slide

1 require 'ruby_parser'
2 require_relative 'minimal_sexp_processor.rb'
3 require_relative 'silly_processor.rb'
4
5 parser = RubyParser.for_current_ruby
6 file = File.read(ARGV[0])
7 sexp = parser.parse(file)
8
9 processor = SillyProcessor.new
10 processor.process(sexp)
silly_processor_demo.rb

View Slide

$ ruby minimal_parser_demo.rb example1.rb
Processing node: block
Processing node: class
PROCESSING A METHOD DEFINITION NODE: greet
Processing node: args
Processing node: call
Processing node: dstr
Processing node: evstr
Processing node: lvar
Processing node: str
Processing node: lasgn
Processing node: const
Processing node: lvar
Processing node: str
1 class Person
2 def greet(name)
3 puts "Hello #{name}!"
4 end
5 end
6
7 person = Person.new
8 person.greet("RubyKaigi")
example1.rb

View Slide

class MethodTrackingProcessor < MinimalSexpProcessor

View Slide

attr_reader :method_locations
def initialize
super
@default_method = :process_until_empty
@class_stack = []
@method_stack = []
@method_locations = {}
end

View Slide

def initialize
super
@class_stack = []
@method_stack = []
end
exp.shift # node type
name = exp.shift
in_method(name, exp.file, exp.line) do
end
s()
end
def process_class(exp)
in_klass(exp.shift) do
end
s()
end
until exp.empty?
sexp = exp.shift
end
end

View Slide

def initialize
super
@class_stack = []
@method_stack = []
end
name = exp.shift
end
s()
end
end
s()
end
until exp.empty?
sexp = exp.shift
end
end
def in_method(name, file, line)
method_name = Regexp === name ? name.inspect : name.to_s
@method_stack.unshift(method_name)
@method_locations[signature] = "#{file}:#{line}"
yield
ensure
@method_stack.shift
end
def in_klass(name)
@class_stack.unshift(name)
with_new_method_stack do
yield
end
ensure
@class_stack.shift
end
def with_new_method_stack
old_method_stack, @method_stack = @method_stack, []
yield
ensure
@method_stack = old_method_stack
end

View Slide

def initialize
super
@class_stack = []
@method_stack = []
end
name = exp.shift
end
s()
end
end
s()
end
until exp.empty?
sexp = exp.shift
end
end
yield
ensure
@method_stack.shift
end
def in_klass(name)
yield
end
ensure
@class_stack.shift
end
yield
ensure
end
def klass_name
@class_stack.first
end
def method_name
@method_stack.first
end
def signature
"#{klass_name}##{method_name}"
end

View Slide

def initialize
super
@class_stack = []
@method_stack = []
end
name = exp.shift
end
s()
end
end
s()
end
until exp.empty?
sexp = exp.shift
end
end
yield
ensure
@method_stack.shift
end
def in_klass(name)
yield
end
ensure
@class_stack.shift
end
yield
ensure
end
def klass_name
@class_stack.first
end
def method_name
@method_stack.first
end
def signature
"#{klass_name}##{method_name}"
end
end

View Slide

1 class Person
2 def greet(name)
3 puts "Hello #{name}!"
4 end
5
6 def say_goodbye(name)
7 puts "Goodbye, #{name}!"
8 end
9 end
10
11 class Dog
12 def bark!
13 puts "Woof!"
14 end
15 end
16
17 person = Person.new
18 person.greet("RubyKaigi")
example2.rb

View Slide

5 path = ARGV[0]
6 file = File.read(path)
7 sexp = RubyParser.for_current_ruby.process(file, path)
8
9 processor = MethodTrackingProcessor.new
10 processor.process(sexp)
11 pp processor.method_locations
method_tracking_demo.rb
$ ruby minimal_parser_demo.rb example2.rb
{ "Person#greet" => "example2.rb:2",
"Person#say_goodbye" => "example2.rb:6",
“Dog#bark!" => "example2.rb:12" }

View Slide

ruby_parser
racc
Ruby code (.rb)
Ruby23Parser
MethodTrackingProcessor
MinimalSexpProcessor
Parsing
Generic processing
File.read()
Call processing and reporting
? ? ?

View Slide

Part III: Building the dead method ﬁnder 
(or, drawing the rest of the damn owl)

View Slide

ruby_parser
racc
Ruby code (.rb)
Ruby23Parser
Parsing
Generic processing
File.read()
Our tool DeadMethodFinder
MethodTrackingProcessor
MinimalSexpProcessor

View Slide

class DeadMethodFinder < MethodTrackingProcessor

View Slide

attr_reader :known
attr_reader :called
def initialize
super
@known = Hash.new { |h,k| h[k] = Set.new }
@called = Set.new
end

View Slide

attr_reader :known
attr_reader :called
def initialize
super
@called = Set.new
end
super do
known[plain_method_name] << klass_name
end
end
def process_call(exp)
called << method_name
exp
end

View Slide

attr_reader :known
attr_reader :called
def initialize
super
@called = Set.new
end
super do
end
end
exp
end
def uncalled
not_called = known.keys - called.to_a
by_class = Hash.new { |h,k| h[k] = [] }
not_called.each do |meth|
known[meth].each do |klass|
by_class[klass] << meth
end
end
by_class
end
# #method_name -> :method_name
def plain_method_name
method_name.to_s.sub(/^::|#/, "").to_sym
end

View Slide

attr_reader :known
attr_reader :called
def initialize
super
@called = Set.new
end
super do
end
end
exp
end
def uncalled
not_called = known.keys - called.to_a
by_class = Hash.new { |h,k| h[k] = [] }
not_called.each do |meth|
known[meth].each do |klass|
by_class[klass] << meth
end
end
by_class
end
# #method_name -> :method_name
def plain_method_name
method_name.to_s.sub(/^::|#/, "").to_sym
end
end

View Slide

1 class Person
2 def greet(name)
3 speak "Hello #{name}!"
4 end
5
6 # Never used
8 speak "Goodbye, #{name}!"
9 end
10
11 def speak(text)
12 puts text
13 end
14
15 def pet_dog(dog)
16 dog.send(:pet)
17 end
18 end
19
20 class Dog
21 attr_accessor :fed # Never used
22
23 def bark!
24 puts "Bark!"
25 end
26
27 def pet
28 puts "Ahhh..."
29 end
30 end
31
32
33 chris = Person.new
34 reuben = Dog.new
35
36 chris.greet("RubyKaigi")
37 reuben.bark!
38 chris.pet_dog(reuben)
example3.rb
=> Hello RubyKaigi! 
Bark!  
Ahhh…

View Slide

1 class Person
2 def greet(name)
4 end
5
6 # Never used
9 end
10
11 def speak(text)
12 puts text
13 end
14
15 def pet_dog(dog)
16 dog.send(:pet)
17 end
18 end
19
20 class Dog
22
23 def bark!
24 puts "Bark!"
25 end
26
27 def pet
28 puts "Ahhh..."
29 end
30 end
31
32
34 reuben = Dog.new
35
37 reuben.bark!
example3.rb
> processor = DeadMethodFinder.new
> processor.process(sexp)
> puts processor.uncalled
=> {"Person"=>[:say_goodbye], "Dog"=>[:pet]}

View Slide

s(:call, s(:lvar, :dog), :send, s(:lit, :pet))
dog.send(:pet)

View Slide

case method_name
when :send, :public_send, :__send__
msg_arg = exp[3]
if Sexp === msg_arg && [:lit, :str].include?(msg_arg.sexp_type)
called << msg_arg.last.to_sym
end
end
exp
end
s(:call, s(:lvar, :dog), :send, s(:lit, :pet))
0 1 2 3

View Slide

1 class Person
2 def greet(name)
4 end
5
6 # Never used
9 end
10
11 def speak(text)
12 puts text
13 end
14
15 def pet_dog(dog)
16 dog.send(:pet)
17 end
18 end
19
20 class Dog
22
23 def bark!
24 puts "Bark!"
25 end
26
27 def pet
28 puts "Ahhh..."
29 end
30 end
31
32
34 reuben = Dog.new
35
37 reuben.bark!
example3.rb
> processor.process(sexp)
> puts processor.uncalled
=> {"Person"=>[:say_goodbye]}

View Slide

s(:call, nil, :attr_accessor, s(:lit, :fed))
attr_accessor :fed

View Slide

s(:call, nil, :attr_accessor, s(:lit, :fed))
case method_name
when :attr_accessor
_, _, _, *args = exp
file, line = exp.file, exp.line
args.each do |(_, name)|
record_known_method(name, file, line)
record_known_method("#{name}=".to_sym, file, line)
end
when :send, :public_send, :__send__
_, _, _, msg_arg, * = exp
if Sexp === msg_arg && [:lit, :str].include?(msg_arg.sexp_type)
called << msg_arg.last.to_sym
end
end
exp
end
def record_known_method(name, file, line)
signature = "#{klass_name}##{name}"
method_locations[signature] = "#{file}:#{line}"
known[name] << klass_name
end
0 1 2 3

View Slide

def report
puts "These methods MIGHT not be called:"
uncalled.each do |klass, methods|
not_called_methods = methods.map do |method|
location = method_locations["#{klass}##{method}"]
" %-35s %s" % [method, location]
end
not_called_methods.compact!
next if not_called_methods.empty?
puts "\n#{klass}"
puts not_called_methods.join "\n"
end
end

View Slide

> processor.process(sexp) 
> processor.report

View Slide

1 class Person
2 def greet(name)
4 end
5
6 # Never used
9 end
10
11 def speak(text)
12 puts text
13 end
14
15 def pet_dog(dog)
16 dog.send(:pet)
17 end
18 end
19
20 class Dog
22
23 def bark!
24 puts "Bark!"
25 end
26
27 def pet
28 puts "Ahhh..."
29 end
30 end
31
32
34 reuben = Dog.new
35
37 reuben.bark!
example2.rb
These methods MIGHT not be called: 
 
Person
say_goodbye example2.rb:7
Dog
fed example2.rb:21
fed= example2.rb:21

View Slide

1 class Person
2 def greet(name)
4 end
5
6
7
8
9
10
11 def speak(text)
12 puts text
13 end
14
15 def pet_dog(dog)
16 dog.send(:pet)
17 end
18 end
19
20 class Dog
21
22
23 def bark!
24 puts "Bark!"
25 end
26
27 def pet
28 puts "Ahhh..."
29 end
30 end
31
32
34 reuben = Dog.new
35
37 reuben.bark!
# Never used
def say_goodbye(name)
speak "Goodbye, #{name}!"
end
attr_accessor :fed # Never used
-
-
-
-
-

View Slide

✌
Done?

View Slide


Done
Ruby is complex (to parse)

View Slide


Done?
Ruby is complex (to parse)
…but adding edge cases is easy.

View Slide

reuben.fed = true

View Slide

1 class Person
2 def greet(name)
4 end
5
6 # Never used
9 end
10
11 def speak(text)
12 puts text
13 end
14
15 def pet_dog(dog)
16 dog.send(:pet)
17 end
18 end
19
20 class Dog
22
23 def bark!
24 puts "Bark!"
25 end
26
27 def pet
28 puts "Ahhh..."
29 end
30 end
31
32
34 reuben = Dog.new
35
37 reuben.bark!
38 chris.pet_dog(reuben) 
39
example4.rb

Person
Dog
fed example2.rb:21
fed= example2.rb:21
reuben.fed = true

View Slide

Brought to you by: Julian Cheal

View Slide

s(:attrasgn, s(:lvar, :reuben), :fed=, s(:true))
def process_attrasgn(exp)
method_name = method_name.last if Sexp === method_name
process_until_empty exp
exp
end
reuben.fed = true

View Slide

1 class Person
2 def greet(name)
4 end
5
6 # Never used
9 end
10
11 def speak(text)
12 puts text
13 end
14
15 def pet_dog(dog)
16 dog.send(:pet)
17 end
18 end
19
20 class Dog
22
23 def bark!
24 puts "Bark!"
25 end
26
27 def pet
28 puts "Ahhh..."
29 end
30 end
31
32
34 reuben = Dog.new
35
37 reuben.bark!
38 chris.pet_dog(reuben) 
39 reuben.fed = true
example4.rb

Person
Dog
fed example2.rb:21

View Slide

What about Rails DSL?

View Slide

:after_commit
:before_create
:after_create
:before_destroy
:before_filter
:before_action
:after_validation
:before_update
:around_save
:validates
:validates_length_of
:validates_format_of
:validates_cuteness_of
:validates_confirmation_of
:validate

View Slide

What about my own DSL?

View Slide


View Slide

class Disk < ActiveRecord::Base
has_many :partitions
virtual_column :allocated_space,
type: :integer,
uses: :partitions
virtual_column :used_percent_of_provisioned,
type: :float
virtual_has_many :storage_systems,
class_name: "CimComputerSystem"
def allocated_space
# ...
end
def used_percent_of_provisioned
# ...
end
end

View Slide

RAILS_DSL_METHODS = [
:after_action,
:around_action,
:before_action,
... 
...
]
# ...
when *RAILS_DSL_METHODS + AR_VIRTUAL_DSL_METHODS
_, _, _, (_, new_name), possible_hash = exp
called << new_name
if Sexp === possible_hash && possible_hash.sexp_type == :hash
possible_hash.sexp_body.each_slice(2) do |key, val|
next unless Sexp === val
called << val.last if val.first == :lit
called << val.last.to_sym if val.first == :str
end
end
# ...
end

View Slide

As with most things, with the right tools,
the job isn’t very difﬁcult.
Customization is easy.

View Slide

Other things that are easy:
Executing this code on your project, right now.

View Slide

debride
https://github.com/seattlerb/debride
debride (v): To remove dead, contaminated, or
adherent tissue and/or foreign material.

View Slide

ruby_parser
racc
Ruby code (.rb)
Ruby23Parser
Parsing
File.read()

View Slide

ruby_parser
racc
sexp_processor
Ruby code (.rb)
Ruby23Parser
MethodBasedSexpProcessor
SexpProcessor
Parsing
Generic processing
File.read()

View Slide

ruby_parser
racc
sexp_processor
Ruby code (.rb)
Ruby23Parser
MethodBasedSexpProcessor
SexpProcessor
Parsing
Generic processing
File.read()
debride Debride

View Slide

$ debride [options] files_or_dirs
-e, --exclude FILE1,FILE2,ETC
-w, --whitelist PATTERN
-f, --focus PATH
-r, --rails
-v, --verbose

View Slide

- @called = Set.new
+ @called = Hash.new { |h,k| h[k] = 0 }
- called << method_name
+ called[method_name] += 1
- not_called = known.keys - called.to_a
+ called_once = called.select { |_, v| v == 1 }
API Quality Control - Single call methods

View Slide

Future considerations:
• Actually push the work I’ve done upstream
✴ Bug ﬁxes
✴ Cleanup
✴ Potential features

View Slide

But wait, there’s more!
Unused by Josh Clayton 
https://github.com/joshuaclayton/unused 
Written in Haskell, this project utilizes ctags
to statically ﬁnd unused code
OldeCodeFinder by Tom Copeland 
https://github.com/tcopeland/olde_code_ﬁnder 
A Ruby gem checking code content by date and authorship
Thanks, Juanito!
Here’s a few…

View Slide

No code is faster than no code.
No code has fewer bugs than no code.
No code is easier to understand than no code.
No code is more maintainable than no code.

View Slide

Thank you! 
Questions?
chrisarcand
chrisarcand
www.chrisarcand.com

View Slide

Deletion Driven Development: Code to delete code! (RubyKaigi edition)

Deletion Driven Development: Code to delete code! (RubyKaigi edition)

Chris Arcand

More Decks by Chris Arcand

Other Decks in Programming

Featured

Transcript