These are
horror stories
from the
GitHub Systems team
(the rainbows are just to make them less awful)
Slide 5
Slide 5 text
I HAVE A
LOT
OF SPARE
TIME
Slide 6
Slide 6 text
Ogre Magi is (C) 2013 by Valve Software and takes 0 skill to play
Slide 7
Slide 7 text
Mark Sweep
The Garba e
Collector
Slide 8
Slide 8 text
Garba e Collector
Slide 9
Slide 9 text
Conservative
Garba e Collector
Slide 10
Slide 10 text
Conservative
Garba e Collector
Non-deterministic
Slide 11
Slide 11 text
Conservative
Stop The World
Garba e Collector
Non-deterministic
Slide 12
Slide 12 text
Conservative
Stop The World
Mark and Sweep
Garba e Collector
Non-deterministic
Slide 13
Slide 13 text
Conservative
Stop The World
Mark and Sweep
Garba e Collector
Non-deterministic
Basically a shitshow.
Slide 14
Slide 14 text
No content
Slide 15
Slide 15 text
Mark
Slide 16
Slide 16 text
Mark 1. Walk the
object raph
2. Look for
thin s that
look like
Ruby objects
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
Ruby handles raw pointers
to C extensions.
Slide 19
Slide 19 text
Ruby handles raw pointers
to C extensions.
The Garba e Collector must find
roots in the stack, the heap and
in re isters.
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
“This kinda looks like a pointer”
Slide 22
Slide 22 text
“This kinda looks like a pointer”
“I uess…”
Slide 23
Slide 23 text
“This kinda looks like a pointer”
“I uess…”
*dramatization
Slide 24
Slide 24 text
No content
Slide 25
Slide 25 text
Sweep
Slide 26
Slide 26 text
Sweep
1. Go throu h
every sin le
object
2. Free the
ones that
have not been
marked
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
The Problem
Slide 29
Slide 29 text
The Problem
There is memory allocated by
Ruby that the Ruby GC cannot
detect.
Slide 30
Slide 30 text
char *
VALUE *
…
…
RStrin
MAGICAL
RUBY
THINGS
int a_ruby_function(VALUE rb_foo)
{
char *str = RSTRING_PTR(rb_foo);
return do_smthing(str);
}
Slide 31
Slide 31 text
char *
VALUE *
…
…
RStrin
MAGICAL
RUBY
THINGS
int a_ruby_function(VALUE rb_foo)
{
char *str = RSTRING_PTR(rb_foo);
return do_smthing(str);
}
H e l l o W o r l d \0
Slide 32
Slide 32 text
char *
…
…
H e l l o W o r l d \0
int do_smthing(char *str)
{
/* do stuff */
return strlen(str);
}
Slide 33
Slide 33 text
int do_smthing(char *str)
{
/* do stuff */
return strlen(str);
}
Un ! Un ! Mark o throu h
stack. Yes. Stack.
Slide 34
Slide 34 text
Un ! This no look like
Ruby object. Mark no touchie!
char *
…
…
…
H e l l o W o r l d \0
Slide 35
Slide 35 text
No root.
H e l l o W o r l d \0
RStrin
MAGICAL
RUBY
THINGS
Strin object with no roots?
Set fire. Yes. Fire.
Slide 36
Slide 36 text
No root.
H e l l o W o r l d \0
Strin object with no roots?
Set fire. Yes. Fire.
Slide 37
Slide 37 text
No root.
Strin object with no roots?
Set fire. Yes. Fire.
Slide 38
Slide 38 text
int do_smthing(char *str)
{
/* do stuff */
return strlen(str);
}
Slide 39
Slide 39 text
int do_smthing(char *str)
{
/* do stuff */
return strlen(str);
}
*sound of an explosion*
“What was that?”
“Dunno.”
Slide 40
Slide 40 text
HAPPY
ENDINGS:
(sorry about this slide)
Slide 41
Slide 41 text
1. Drink
Yourself To
Sleep Every
Ni ht
Slide 42
Slide 42 text
2. Stress the
Garba e
Collector in
Critical Paths
Slide 43
Slide 43 text
No content
Slide 44
Slide 44 text
mallopt(M_PERTURB, 0x33);
C land
Slide 45
Slide 45 text
mallopt(M_PERTURB, 0x33);
C land
GC.stress = true
Ruby land
Slide 46
Slide 46 text
Bonus Points
Use Val rind/ASan
instead of tamperin .
Slide 47
Slide 47 text
Bonus Points
Use Val rind/ASan
instead of tamperin .
Your Rails test suite
Heat death of the Universe
Slide 48
Slide 48 text
3. Static
Analysis
Slide 49
Slide 49 text
No content
Slide 50
Slide 50 text
How do you feel
about timeouts?
Slide 51
Slide 51 text
THERE ARE
NO
TIMEOUTS IN
RUBY
(just ask the Timeout Rabbit)
Slide 52
Slide 52 text
require ‘timeout’
Slide 53
Slide 53 text
require ‘timeout’
Timeout::timeout(5) {
# Do something
}
Slide 54
Slide 54 text
require ‘timeout’
Timeout::timeout(5) {
# Do something
}
…
Slide 55
Slide 55 text
No content
Slide 56
Slide 56 text
…
Slide 57
Slide 57 text
…
…
Slide 58
Slide 58 text
…
…
oh hey
Slide 59
Slide 59 text
“Sorry I’m late.
Couldn’t et here
any faster.”
Slide 60
Slide 60 text
No content
Slide 61
Slide 61 text
Beautiful
Slide 62
Slide 62 text
Beautiful
wron
also:
Slide 63
Slide 63 text
MRI 1.8.7
Ruby Process
Green Thread
Green Thread
Green Thread
THE
KERNEL
Slide 64
Slide 64 text
MRI 1.9.3
Ruby Process
THE
KERNEL
NNative Thread
NNative Thread
NNative Thread
Slide 65
Slide 65 text
MRI 1.9.3
Ruby Process
THE
KERNEL
NNative Thread
NNative Thread
NNative Thread
GVL
Slide 66
Slide 66 text
Use the
force,
kid
Slide 67
Slide 67 text
“Unix, you moron.
Unix is the force.”
Slide 68
Slide 68 text
“Unix, you moron.
Unix is the force.”
Slide 69
Slide 69 text
No content
Slide 70
Slide 70 text
MRI 1.8.7
Use si nals & the
power of Unix
Slide 71
Slide 71 text
MRI 1.8.7
Use si nals & the
power of Unix
MRI 1.9.3
Release the GVL
Slide 72
Slide 72 text
Mathiasman the Wise
Slide 73
Slide 73 text
No content
Slide 74
Slide 74 text
No content
Slide 75
Slide 75 text
No content
Slide 76
Slide 76 text
“I, too, had a
dreadful problem
with timeouts”
Slide 77
Slide 77 text
You see, I was
usin JRuby…
Slide 78
Slide 78 text
JRuby?
Sounds like you
have two
problems now.
Slide 79
Slide 79 text
(shut up, kid)
You see, I was
usin JRuby, and
the GitHub API
you uys keep
breakin …
Slide 80
Slide 80 text
Duh.
That’s probably
because the
thread is
sleepin or stuck
in the GVL.
Slide 81
Slide 81 text
You fool!
JRuby has native threads
and fine- rained
lockin !
*smack*
Slide 82
Slide 82 text
JRUBY 1.6.7
Timeout::timeout(5) {
Timeout::timeout(60) {
# Do something
}
}
Lon est timeout
clobbers the short one
Slide 83
Slide 83 text
JRUBY 1.6.7
Timeout::timeout(5) {
Timeout::timeout(60) {
# Do something
}
}
Lon est timeout
clobbers the short one
Slide 84
Slide 84 text
I LIKE THIS BUG
Because JRuby wins
Slide 85
Slide 85 text
Simple Bu
Complex Bu
Painful Bu
Kernel Bu
NESTED
TIMEOUTS
IN THE JVM
Slide 86
Slide 86 text
Now Leavin :
Realm of Bu s
Valley of Common Sense
Slide 87
Slide 87 text
mmon Sense
Mountains of Reason
Slide 88
Slide 88 text
Country of
WHAT THE
HELL ARE YOU
DOING
MRI Timeouts
in Green Threads
Slide 89
Slide 89 text
MORAL
OF
THE
STORY
Slide 90
Slide 90 text
BETTER
WORSE
IS
Sometimes
*this doesn’t apply to the MRI GC
Slide 91
Slide 91 text
Computers
break Ruby breaks
twice as often
Slide 92
Slide 92 text
If you build simpler
systems, they will
break
in simpler
ways.