Slide 1
Slide 1 text
RubyKaigi
2024
Speeding Up
Instance Variables
in Ruby 3.3
Slide 2
Slide 2 text
Aaron Patterson
@tenderlove
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
Ruby Implementation
Details
Slide 5
Slide 5 text
Data Structures
Slide 6
Slide 6 text
Red Black Trees
Slide 7
Slide 7 text
Instance Variables
Slide 8
Slide 8 text
Object Shapes
Slide 9
Slide 9 text
What Makes IVs Slow
Slide 10
Slide 10 text
Speeding up IVs
Slide 11
Slide 11 text
Data Structures
Slide 12
Slide 12 text
Red-Black 🌲
Slide 13
Slide 13 text
Red-Black Tree is a
Binary Tree
Slide 14
Slide 14 text
Self Balancing
Slide 15
Slide 15 text
Red Black Tree
Left is < parent, Right is > parent
3
2
nil
nil
7
5
4
nil nil
nil
8
nil 9
nil
nil
Insert 5
Insert 3
Insert 2
Insert 4
Insert 7
Insert 8
Insert 9
Slide 16
Slide 16 text
Red Black Tree: Rule 1
All nodes are Red or Black
3
2
nil
nil
7
5
4
nil nil
nil
8
nil 9
nil
nil
Slide 17
Slide 17 text
Red Black Tree: Rule 2
All Leaf Nodes are Black
3
2
nil
nil
7
5
4
nil nil
nil
8
nil 9
nil
nil
Slide 18
Slide 18 text
Red Black Tree: Rule 3
Red Nodes Cannot Have Red Children
3
2
nil
nil
7
5
4
nil nil
nil
8
nil 9
nil
nil
Slide 19
Slide 19 text
Red Black Tree: Rule 4
Every path from a given node to a leaf passes through the same number of black nodes
3
2
nil
nil
7
5
4
nil nil
nil
8
nil 9
nil
nil
Slide 20
Slide 20 text
Red Black Tree: Rule 4
Every path from a given node to a leaf passes through the same number of black nodes
3
2
nil
nil
7
5
4
nil nil
nil
8
nil 9
nil
nil
Slide 21
Slide 21 text
Red Black Tree: Bonus Rule
The root node is always black.
3
2
nil
nil
7
5
4
nil nil
nil
8
nil 9
nil
nil
Slide 22
Slide 22 text
Let’s Implement It!
Slide 23
Slide 23 text
Okasaki Style
Functional Red Black Tree
https://www.cs.tufts.edu/comp/150FP/archive/chris-okasaki/redblack99.pdf
Slide 24
Slide 24 text
Chris Okasaki
Everybody learns about balanced search trees in their introductory computer science
classes, but even the stouthearted tremble at the thought of implementing such a
beast. […] we present an algorithm for insertion in to red-black trees that any
competent programmer should be able to implement in fifteen minutes or less
Slide 25
Slide 25 text
Competent
Programmer?
Slide 26
Slide 26 text
Blog in 15 min
Slide 27
Slide 27 text
Leaf Nodes
module RBTree
class Leaf
def color = :black
def leaf? = true
def deconstruct
[:black, nil, nil, nil]
end
def insert val
RBTree.insert self, val
end
end
LEAF = Leaf.new
def self.new; LEAF; end
end
tree = RBTree.new
Always
black
Used by pattern
matching
Helper “insert”
method
Leaf is a
singleton
Creating a new tree
just returns leaf
Slide 28
Slide 28 text
Regular Node
module RBTree
class Node
attr_reader :color, :key, :left, :right
def initialize color, key, left, right
@color = color
@key = key
@left = left
@right = right
end
def leaf? = false
def deconstruct
[color, key, left, right]
end
def insert val
RBTree.insert self, val
end
end
end
Used by pattern
matching
Slide 29
Slide 29 text
Inserting a Key
module RBTree
def self.insert tree, key
root = insert_aux(tree, key)
case root
in [:red, key, left, right]
Node.new(:black, key, left, right)
else
root
end
end
private_class_method def self.insert_aux tree, key
if tree.leaf?
Node.new :red, key, LEAF, LEAF
else
# FIXME!
end
end
end
tree = RBTree.new
tree = tree.insert(5)
pp tree
Slide 30
Slide 30 text
Inserting a Key
module RBTree
def self.insert tree, key
root = insert_aux(tree, key)
case root
in [:red, key, left, right]
Node.new(:black, key, left, right)
else
root
end
end
private_class_method def self.insert_aux tree, key
if tree.leaf?
Node.new :red, key, LEAF, LEAF
else
# FIXME!
end
end
end
tree = RBTree.new
tree = tree.insert(5)
pp tree
Slide 31
Slide 31 text
Inserting a Key
module RBTree
def self.insert tree, key
root = insert_aux(tree, key)
case root
in [:red, key, left, right]
Node.new(:black, key, left, right)
else
root
end
end
private_class_method def self.insert_aux tree, key
if tree.leaf?
Node.new :red, key, LEAF, LEAF
else
# FIXME!
end
end
end
tree = RBTree.new
tree = tree.insert(5)
pp tree
Deconstruct and
reassemble with
pattern matching
Slide 32
Slide 32 text
Inserting a Key
module RBTree
def self.insert tree, key
root = insert_aux(tree, key)
case root
in [:red, key, left, right]
Node.new(:black, key, left, right)
else
root
end
end
private_class_method def self.insert_aux tree, key
if tree.leaf?
Node.new :red, key, LEAF, LEAF
else
# FIXME!
end
end
end
tree = RBTree.new
tree = tree.insert(5)
pp tree
Slide 33
Slide 33 text
Inserting a Key
module RBTree
def self.insert tree, key
root = insert_aux(tree, key)
case root
in [:red, key, left, right]
Node.new(:black, key, left, right)
else
root
end
end
private_class_method def self.insert_aux tree, key
if tree.leaf?
Node.new :red, key, LEAF, LEAF
else
# FIXME!
end
end
end
tree = RBTree.new
tree = tree.insert(5)
pp tree
LEAF
LEAF LEAF
5
LEAF LEAF
5
LEAF
Slide 34
Slide 34 text
Inserting a Second Key
module RBTree
private_class_method def self.balance color, key, left, right
return Node.new(color, key, left, right)
end
private_class_method def self.insert_aux tree, key
if tree.leaf?
Node.new :red, key, LEAF, LEAF
else
# If the new key is smaller, we need to put it on the left
if key < tree.key
new_left = insert_aux(tree.left, key)
balance(tree.color, tree.key, new_left, tree.right)
elsif key > tree.key
# ...
else
tree
end
end
end
end
tree = RBTree.new
tree = tree.insert(5)
tree = tree.insert(3)
LEAF LEAF
5
LEAF LEAF
3
5
LEAF
Slide 35
Slide 35 text
Inserting a Second Key
module RBTree
private_class_method def self.balance color, key, left, right
return Node.new(color, key, left, right)
end
private_class_method def self.insert_aux tree, key
if tree.leaf?
Node.new :red, key, LEAF, LEAF
else
# If the new key is smaller, we need to put it on the left
if key < tree.key
new_left = insert_aux(tree.left, key)
balance(tree.color, tree.key, new_left, tree.right)
elsif key > tree.key
# ...
else
tree
end
end
end
end
tree = RBTree.new
tree = tree.insert(5)
tree = tree.insert(3)
LEAF LEAF
5
LEAF LEAF
3
5
LEAF
LEAF LEAF
5
Slide 36
Slide 36 text
module RBTree
private_class_method def self.balance color, key, left, right
return Node.new(color, key, left, right)
end
private_class_method def self.insert_aux tree, key
if tree.leaf?
Node.new :red, key, LEAF, LEAF
else
# If the new key is smaller, we need to put it on the left
if key < tree.key
new_left = insert_aux(tree.left, key)
balance(tree.color, tree.key, new_left, tree.right)
elsif key > tree.key
# ...
else
tree
end
end
end
end
tree = RBTree.new
tree = tree.insert(5)
tree = tree.insert(3)
tree = tree.insert(2)
Inserting a Third Key
LEAF LEAF
3
5
LEAF
2 < ?
LEAF LEAF
2
Slide 37
Slide 37 text
Inserting a Third Key
module RBTree
private_class_method def self.balance color, key, left, right
return Node.new(color, key, left, right)
end
private_class_method def self.insert_aux tree, key
if tree.leaf?
Node.new :red, key, LEAF, LEAF
else
# If the new key is smaller, we need to put it on the left
if key < tree.key
new_left = insert_aux(tree.left, key)
balance(tree.color, tree.key, new_left, tree.right)
elsif key > tree.key
# ...
else
tree
end
end
end
end
tree = RBTree.new
tree = tree.insert(5)
tree = tree.insert(3)
tree = tree.insert(2)
LEAF
3
5
LEAF
LEAF LEAF
2
Rule 2: Red Nodes
Cannot Have Red
Children
Slide 38
Slide 38 text
Rebalancing
Detect violations
LEAF
3
5
LEAF
LEAF LEAF
2
Slide 39
Slide 39 text
Rebalancing
Rotating the tree
LEAF
3
5
LEAF
LEAF LEAF
2
Must be a leaf OR
less than 5 and
greater than 3
Slide 40
Slide 40 text
Rebalancing
Rotating the tree
LEAF
3
5
LEAF
LEAF LEAF
2
LEAF
5
2
Slide 41
Slide 41 text
General Form
Z
Y
X
A B
C
D
Slide 42
Slide 42 text
Z
Y
General Form
X
A B C D
X
Slide 43
Slide 43 text
Detect Violations with Pattern Matching
Pattern matching detects violations and deconstructs nodes
private_class_method def self.balance color, key, left, right
# x, y, z are keys
# a, b, c, d are nodes
case [color, key, left, right]
in [:black, z, [:red, y, [:red, x, a, b], c], d]
else
return Node.new(color, key, left, right)
end
Node.new(:red, y, Node.new(:black, x, a, b), Node.new(:black, z, c, d))
end
Top
Node
Black?
Left Child
Red?
Left Child
Red?
Rotate Tree
Make a
new node
Slide 44
Slide 44 text
Only 4 Patterns
Slide 45
Slide 45 text
Four different violation patterns
Z
Y
X
A B
C
D
X
A
Y
B C
D
Z
X
A
B
C D
Z
Y
Z
X
A
B C
Y
D
Y
X
C
A B D
Z
Slide 46
Slide 46 text
Rotation Patterns
Node Placement and Color
Head Left Left Left
Head Left Left Right
Head Right Right Right
Head Right Right Left
Slide 47
Slide 47 text
Detect Violations with Pattern Matching
Handle all four cases
private_class_method def self.balance color, key, left, right
# x, y, z are keys
# a, b, c, d are nodes
case [color, key, left, right]
# HEAD LEFT LEFT-LEFT
in [:black, z, [:red, y, [:red, x, a, b], c], d]
# HEAD LEFT LEFT-RIGHT
in [:black, z, [:red, x, a, [:red, y, b, c]], d]
# HEAD RIGHT RIGHT-RIGHT
in [:black, x, a, [:red, y, b, [:red, z, c, d]]]
# HEAD RIGHT RIGHT-LEFT
in [:black, x, a, [:red, z, [:red, y, b, c], d]]
else
return Node.new(color, key, left, right)
end
Node.new(:red, y, Node.new(:black, x, a, b), Node.new(:black, z, c, d))
end
Slide 48
Slide 48 text
Finish Up insert_aux
We need to handle both smaller and larger numbers
private_class_method def self.insert_aux tree, key
if tree.leaf?
Node.new :red, key, LEAF, LEAF
else
# If the new key is smaller, we need to put it on the left
if key < tree.key
new_left = insert_aux(tree.left, key)
balance(tree.color, tree.key, new_left, tree.right)
# If the new key is larger, we need to put it on the right
elsif key > tree.key
new_right = insert_aux(tree.right, key)
balance(tree.color, tree.key, tree.left, new_right)
else
tree
end
end
end
If the tree is empty,
make a new node
If the number is
smaller, put it on the
left
If the number is
larger, put it on the
right
Slide 49
Slide 49 text
That’s It!
module RBTree
class Leaf
def color = :black
def leaf? = true
def deconstruct
[:black, nil, nil, nil]
end
def insert val
RBTree.insert self, val
end
end
class Node
attr_reader :color, :key, :left, :right
def initialize color, key, left, right
@color = color
@key = key
@left = left
@right = right
end
def leaf? = false
def deconstruct
[color, key, left, right]
end
def insert val
RBTree.insert self, val
end
end
LEAF = Leaf.new
def self.new; LEAF; end
def self.insert tree, key
root = insert_aux(tree, key)
case root
Get the code!
Slide 50
Slide 50 text
Using the Red Black Tree
tree = RBTree.new
tree = tree.insert(5)
tree = tree.insert(3)
tree = tree.insert(2)
p tree.key?(5) # => true
p tree.key?(10) # => false
Slide 51
Slide 51 text
Each Insert Returns a New Tree
Slide 52
Slide 52 text
Only rebalanced nodes are copied
Slide 53
Slide 53 text
Red Black Tree Advantages
Versions Share Memory
Fast Inserts O(log n)
Fast Lookups O(log n)
Slide 54
Slide 54 text
We can’t delete anything 😅
Slide 55
Slide 55 text
Object Shapes and
Instance Variables
Slide 56
Slide 56 text
Object Shapes:
Mapping Instance Variables to Indexes
Slide 57
Slide 57 text
How Object Shapes Work
Slide 58
Slide 58 text
Object Shapes: The Bad Parts
Slide 59
Slide 59 text
Instance Variable Storage
Instance variables are stored in an array on the object
class Node
def initialize color, key, left, right
@color = color
@key = key
@left = left
@right = right
end
def color
@color
end
end
n = Node.new(:red, 5, nil, nil)
n.color
Node
Instance
name index value
@color 0 :red
@key 1 5
@left 2 nil
@right 3 nil
Slide 60
Slide 60 text
Tree Data Structure
Slide 61
Slide 61 text
“Shape Tree”
Slide 62
Slide 62 text
Shape Tree Structure
Each node has an ID, each edge represents an instance variable
0
1
2
3
Tree Root:
all objects start here
@color
@key
@left
4
@right
class Node
def initialize a, b, c, d
@color = a
@key = b
@left = d
@right = c
end
end
Slide 63
Slide 63 text
class Node2
def initialize a, b, c, d
@color = a
@key = b
@right = c
@left = d
end
end
Shape Tree Structure
Each node has an ID, each edge represents an instance variable
0
1
2
3 5
@color
@key
@right
@left
4 6
@right @left
class Node
def initialize a, b, c, d
@color = a
@key = b
@left = d
@right = c
end
end
Slide 64
Slide 64 text
Many objects can share the same
shape
Slide 65
Slide 65 text
Shape Tree:
IV Name / Order
Slide 66
Slide 66 text
Setting an Instance Variable
Tree depth determines array index
class Node
def initialize color, key, left, right
# Start at tree root
#
@color = color
@key = key
@left = left
@right = right
end
end
n = Node.new(:red, 5, nil, nil)
Node
Instance
1
2
3
@color
@key
@left
4
@right
0
0
0 :red
1 5
2 nil
3 nil
1
2
3
4
Shape Tree
@key?
@left?
@right?
Shape ID
Slide 67
Slide 67 text
Instance Variable Index:
Tree Depth
Slide 68
Slide 68 text
Before Setting:
Check if an IV is defined
Slide 69
Slide 69 text
Bad News:
Checking IV’s is O(n)
Slide 70
Slide 70 text
Good News:
Inline Caches
Slide 71
Slide 71 text
Inline Caches
Cache Key is Shape ID
class Node
def initialize color, key, left, right
# Start at tree root
#
@color = color
@key = key
@left = left
@right = right
end
end
n = Node.new(:red, 5, nil, nil)
Node
Instance
1
2
3
@color
@key
@left
4
@right
0
0
0 :red
1 5
2 nil
3 nil
1
2
3
4
Shape Tree
Shape ID
shape: 0, next id: 1, iv index: 0
shape: 1, next id: 2, iv index: 1
shape: 2, next id: 3, iv index: 2
shape: 3, next id: 4, iv index: 3
Cache Key
Next Shape
IV Index
Slide 72
Slide 72 text
Inline Caches: Second Run
Cache Key is Shape ID
class Node
def initialize color, key, left, right
# Start at tree root
#
@color = color
@key = key
@left = left
@right = right
end
end
n = Node.new(:red, 5, nil, nil)
n = Node.new(:red, 5, nil, nil)
Node
Instance
1
2
3
@color
@key
@left
4
@right
0
0
0 :red
1 5
2 nil
3 nil
1
2
3
4
Shape Tree
Shape ID
shape: 0, next id: 1, iv index: 0
shape: 1, next id: 2, iv index: 1
shape: 2, next id: 3, iv index: 2
shape: 3, next id: 4, iv index: 3
Cache Key
Cache Key
Slide 73
Slide 73 text
Cache Hit: O(1)
Slide 74
Slide 74 text
Cache Miss: O(n)
Slide 75
Slide 75 text
Speeding up Cache Misses
Slide 76
Slide 76 text
When do cache misses occur?
Slide 77
Slide 77 text
Subclasses (on read)
Subclasses can add IVs, causing the shape to be di
ff
erent
class NodeWithManyIVs
def initialize
@a0 = 0
@a1 = 1
@a2 = 2
end
def read; @a0; end
end
class Subclass < NodeWithManyIVs
def initialize
super
@foo = 123
end
end
obj = NodeWithManyIVs.new # shape N
sub = Subclass.new # shape N + 1
obj.read # cache MISS in read
obj.read # cache HIT in read
sub.read # cache MISS in read
sub.read # cache HIT in read
Slide 78
Slide 78 text
Subclasses (on write)
Subclasses can add IVs, causing the shape to be di
ff
erent
class ManyIVs
def initialize
# Shape 0 or 1,
# depending on subclass
@a0 = 0
@a1 = 1
@a2 = 2
end
end
class Subclass
def initialize
# always starts at shape 0
@foo = 123 # makes a new shape
super
end
end
ManyIVs.new # All IVs MISS on set
Subclass.new # All IVs MISS on set
ManyIVs.new # All IVs MISS on set
Subclass.new # All IVs MISS on set
Slide 79
Slide 79 text
Lazy IV Initialization
Instance variable name and order is what matters
class LazyIvs
def initialize
@a = 1
end
def b
@b ||= 1
end
def c
@c ||= 1
end
def a
@a
end
end
LazyIvs.new.a # Shape 1
LazyIvs.new.tap { x.b }.a # Shape 2
LazyIvs.new.tap { x.c }.a # Shape 3
LazyIvs.new.tap { x.c; x.b }.a # Shape 4
LazyIvs.new.tap { x.b; x.c }.a # Shape 5
Creates a new shape
Creates a new shape
Slide 80
Slide 80 text
Fairly Rare
Slide 81
Slide 81 text
How can we speed this up?
Slide 82
Slide 82 text
1
2
3
@color
@key
@left
4
@right
0
Shape Tree
{right, left, key, color}
{left, key, color}
{key, color}
{color}
@color?
Ancestor Index
Slide 83
Slide 83 text
Functional Red Black Tree!
Slide 84
Slide 84 text
“You can’t delete from that tree”
Slide 85
Slide 85 text
Shapes are never deleted
Slide 86
Slide 86 text
Ruby 3.3
1
2
3
@color
@key
@left
4
@right
0
Shape Tree
{right, left, key, color}
{left, key, color}
{key, color}
{color}
Ancestor Index
(Red Black Tree)
Slide 87
Slide 87 text
Benchmark IV Read Performance
N = ARGV[0].to_i
class ManyIVs
class_eval "def initialize;" +
N.times.map { "@a#{_1} = #{_1}" }.join("\n") +
"end"
def read; @a0; end
end
class Subclass < ManyIVs
def initialize
super
@foo = 123
end
end
def always_miss
a = ManyIVs.new
b = Subclass.new
Benchmark.measure {
200000.times { a.read; b.read }
}.real
end
def always_hit
a = ManyIVs.new
b = ManyIVs.new
Benchmark.measure {
200000.times { a.read; b.read }
}.real
end
puts "#{N},#{always_hit},#{always_miss}"
Set N instance
variables
Compare
always
hitting with
never hitting
Slide 88
Slide 88 text
Ruby 3.2.2: Instance Variable Read Speed (Lower is Better)
Time to Read IV
0.0000
0.0200
0.0400
0.0600
0.0800
Number of Instance Variables
1 4 7 10 13 16 19 22 25 28 31
Always Cache Hit Always Cache Miss
Slide 89
Slide 89 text
Ruby 3.3.0: Instance Variable Read Speed (Lower is Better)
Time to Read IV
0.0000
0.0075
0.0150
0.0225
0.0300
Number of Instance Variables
1 4 7 10 13 16 19 22 25 28 31
Always Hit Always Miss
Slide 90
Slide 90 text
Benchmark IV Write Performance
require "benchmark"
N = ARGV[0].to_i
class ManyIVs
class_eval "def initialize;" +
N.times.map { "@a#{_1} = #{_1}" }.join("\n") +
"end"
def write; @a0 = 0; end
end
class Subclass < ManyIVs
def initialize
@foo = 123
super
end
end
def always_miss
a = ManyIVs.new
b = Subclass.new
Benchmark.measure {
500000.times { a.write; b.write }
}.real
end
def always_hit
a = ManyIVs.new
b = ManyIVs.new
Benchmark.measure {
500000.times { a.write; b.write }
}.real
end
puts "#{N},#{always_hit},#{always_miss}"
Change read to
write
Slide 91
Slide 91 text
Writing one IV on Ruby 3.2
Time to write one IV
0.0000
0.0200
0.0400
0.0600
0.0800
Number of Instance Variables on Object
1 4 7 10 13 16 19 22 25 28 31
Always Cache Hit
Always Cache Miss
Slide 92
Slide 92 text
Writing one IV on Ruby 3.3
Time to write one IV
0.0000
0.0120
0.0240
0.0360
0.0480
Number of Instance Variables on Object
1 4 7 10 13 16 19 22 25 28 31
Always Cache Hit
Always Cache Miss
Slide 93
Slide 93 text
Memoization
Set a variable only if it’s not de
fi
ned
class ManyIVs
def memoized
if defined?(@some_iv)
@some_iv
else
@some_iv = something_expensive
end
end
end
Slide 94
Slide 94 text
Benchmark Undefined IV
require "benchmark"
N = ARGV[0].to_i
class ManyIVs
class_eval "def initialize;" +
N.times.map { "@a#{_1} = #{_1}" }.join("\n") +
"end"
def defined; defined?(@not_defined); end
end
class Subclass < ManyIVs
def initialize
super
@foo = 123
end
end
def always_miss
a = ManyIVs.new
b = Subclass.new
Benchmark.measure {
500000.times { a.defined; b.defined }
}.real
end
def always_hit
a = ManyIVs.new
b = ManyIVs.new
Benchmark.measure {
500000.times { a.defined; b.defined }
}.real
end
puts "#{N},#{always_hit},#{always_miss}"
look for an IV that’s
never de
fi
ned
Slide 95
Slide 95 text
Check for Unde
fi
ned IV: Ruby 3.2
Time to Check for Unde
fi
ned IV
0.0000
0.0200
0.0400
0.0600
0.0800
Number of IVs on Object
1 4 7 10 13 16 19 22 25 28 31
Always Cache Hit
Always Cache Miss
No Cache on
Ruby 3.2 😅
Slide 96
Slide 96 text
Check for Unde
fi
ned IV: Ruby 3.3
Time to Check for Unde
fi
ned IV
0.0000
0.0225
0.0450
0.0675
0.0900
Number of IVs on Object
1 4 7 10 13 16 19 22 25 28 31
Always Cache Hit
Always Cache Miss
Not Using red
black tree😅
Slide 97
Slide 97 text
Check for Unde
fi
ned IV: Ruby 3.4
Time to Check for Unde
fi
ned IV
0.0000
0.0085
0.0170
0.0255
0.0340
Number of IVs on Object
1 4 7 10 13 16 19 22 25 28 31
Always Cache Hit
Always Cache Miss
Slide 98
Slide 98 text
Conclusion
Slide 99
Slide 99 text
O(n)
O(log n)
🙅
🙆
Slide 100
Slide 100 text
Data structures are fun!
Even if you never use them
Slide 101
Slide 101 text
Ruby 3.3 is Fast!
Slide 102
Slide 102 text
͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ