Unicorns Die With Bullets Made of Glitter

The reduced, 30 minutes version of this collection of Ruby Systems Horror Stories, including:

- Mark & Sweep, a tale of a dumb garbage collector
- Our Time is Running Out, a tale of tricky timeouts

Originally presented at Rubyfuza.

Transcript

1. View Slide

2. Once upon
   a time…
   

   View Slide

3. View Slide

4. These are
   horror stories
   from the
   GitHub Systems team
   (the rainbows are just to make them less awful)
   

   View Slide

5. I HAVE A
   LOT
   OF SPARE
   TIME
   

   View Slide

6. Ogre Magi is (C) 2013 by Valve Software and takes 0 skill to play
   

   View Slide

7. Mark Sweep
   The Garba e
   Collector
   

   View Slide

8. Garba e Collector
   

   View Slide

9. Conservative
   Garba e Collector
   

   View Slide

10. Conservative
    Garba e Collector
    Non-deterministic
    

    View Slide

11. Conservative
    Stop The World
    Garba e Collector
    Non-deterministic
    

    View Slide

12. Conservative
    Stop The World
    Mark and Sweep
    Garba e Collector
    Non-deterministic
    

    View Slide

13. Conservative
    Stop The World
    Mark and Sweep
    Garba e Collector
    Non-deterministic
    Basically a shitshow.
    

    View Slide

14. View Slide

15. Mark
    

    View Slide

16. Mark 1. Walk the
    object raph
    2. Look for
    thin s that
    look like
    Ruby objects
    

    View Slide

17. View Slide

18. Ruby handles raw pointers
    to C extensions.
    

    View Slide

19. Ruby handles raw pointers
    to C extensions.
    The Garba e Collector must find
    roots in the stack, the heap and
    in re isters.
    

    View Slide

20. View Slide

21. “This kinda looks like a pointer”
    

    View Slide

22. “This kinda looks like a pointer”
    “I uess…”
    

    View Slide

23. “This kinda looks like a pointer”
    “I uess…”
    *dramatization
    

    View Slide

24. View Slide

25. Sweep
    

    View Slide

26. Sweep
    1. Go throu h
    every sin le
    object
    2. Free the
    ones that
    have not been
    marked
    

    View Slide

27. View Slide

28. The Problem
    

    View Slide

29. The Problem
    There is memory allocated by
    Ruby that the Ruby GC cannot
    detect.
    

    View Slide

30. char *
    VALUE *
    …
    …
    RStrin
    MAGICAL
    RUBY
    THINGS
    int a_ruby_function(VALUE rb_foo)
    {
    char *str = RSTRING_PTR(rb_foo);
    return do_smthing(str);
    }
    

    View Slide

31. char *
    VALUE *
    …
    …
    RStrin
    MAGICAL
    RUBY
    THINGS
    int a_ruby_function(VALUE rb_foo)
    {
    char *str = RSTRING_PTR(rb_foo);
    return do_smthing(str);
    }
    H e l l o W o r l d \0
    

    View Slide

32. char *
    …
    …
    H e l l o W o r l d \0
    int do_smthing(char *str)
    {
    /* do stuff */
    return strlen(str);
    }
    

    View Slide

33. int do_smthing(char *str)
    {
    /* do stuff */
    return strlen(str);
    }
    Un ! Un ! Mark o throu h
    stack. Yes. Stack.
    

    View Slide

34. Un ! This no look like
    Ruby object. Mark no touchie!
    char *
    …
    …
    …
    H e l l o W o r l d \0
    

    View Slide

35. No root.
    H e l l o W o r l d \0
    RStrin
    MAGICAL
    RUBY
    THINGS
    Strin object with no roots?
    Set fire. Yes. Fire.
    

    View Slide

36. No root.
    H e l l o W o r l d \0
    Strin object with no roots?
    Set fire. Yes. Fire.
    

    View Slide

37. No root.
    Strin object with no roots?
    Set fire. Yes. Fire.
    

    View Slide

38. int do_smthing(char *str)
    {
    /* do stuff */
    return strlen(str);
    }
    

    View Slide

39. int do_smthing(char *str)
    {
    /* do stuff */
    return strlen(str);
    }
    *sound of an explosion*
    “What was that?”
    “Dunno.”
    

    View Slide

40. HAPPY
    ENDINGS:
    (sorry about this slide)
    

    View Slide

41. 1. Drink
    Yourself To
    Sleep Every
    Ni ht
    

    View Slide

42. 2. Stress the
    Garba e
    Collector in
    Critical Paths
    

    View Slide

43. View Slide

44. mallopt(M_PERTURB, 0x33);
    C land
    

    View Slide

45. mallopt(M_PERTURB, 0x33);
    C land
    GC.stress = true
    Ruby land
    

    View Slide

46. Bonus Points
    Use Val rind/ASan
    instead of tamperin .
    

    View Slide

47. Bonus Points
    Use Val rind/ASan
    instead of tamperin .
    Your Rails test suite
    Heat death of the Universe
    

    View Slide

48. 3. Static
    Analysis
    

    View Slide

49. View Slide

50. How do you feel
    about timeouts?
    

    View Slide

51. THERE ARE
    NO
    TIMEOUTS IN
    RUBY
    (just ask the Timeout Rabbit)
    

    View Slide

52. require ‘timeout’
    

    View Slide

53. require ‘timeout’
    Timeout::timeout(5) {
    # Do something
    }
    

    View Slide

54. require ‘timeout’
    Timeout::timeout(5) {
    # Do something
    }
    …
    

    View Slide

55. View Slide

56. …
    

    View Slide

57. …
    …
    

    View Slide

58. …
    …
    oh hey
    

    View Slide

59. “Sorry I’m late.
    Couldn’t et here
    any faster.”
    

    View Slide

60. View Slide

61. Beautiful
    

    View Slide

62. Beautiful
    wron
    also:
    

    View Slide

63. MRI 1.8.7
    Ruby Process
    Green Thread
    Green Thread
    Green Thread
    THE
    KERNEL
    

    View Slide

64. MRI 1.9.3
    Ruby Process
    THE
    KERNEL
    NNative Thread
    NNative Thread
    NNative Thread
    

    View Slide

65. MRI 1.9.3
    Ruby Process
    THE
    KERNEL
    NNative Thread
    NNative Thread
    NNative Thread
    GVL
    

    View Slide

66. Use the
    force,
    kid
    

    View Slide

67. “Unix, you moron.
    Unix is the force.”
    

    View Slide

68. “Unix, you moron.
    Unix is the force.”
    

    View Slide

69. View Slide

70. MRI 1.8.7
    Use si nals & the
    power of Unix
    

    View Slide

71. MRI 1.8.7
    Use si nals & the
    power of Unix
    MRI 1.9.3
    Release the GVL
    

    View Slide

72. Mathiasman the Wise
    

    View Slide

73. View Slide

74. View Slide

75. View Slide

76. “I, too, had a
    dreadful problem
    with timeouts”
    

    View Slide

77. You see, I was
    usin JRuby…
    

    View Slide

78. JRuby?
    Sounds like you
    have two
    problems now.
    

    View Slide

79. (shut up, kid)
    You see, I was
    usin JRuby, and
    the GitHub API
    you uys keep
    breakin …
    

    View Slide

80. Duh.
    That’s probably
    because the
    thread is
    sleepin or stuck
    in the GVL.
    

    View Slide

81. You fool!
    JRuby has native threads
    and fine- rained
    lockin !
    *smack*
    

    View Slide

82. JRUBY 1.6.7
    Timeout::timeout(5) {
    Timeout::timeout(60) {
    # Do something
    }
    }
    Lon est timeout
    clobbers the short one
    

    View Slide

83. JRUBY 1.6.7
    Timeout::timeout(5) {
    Timeout::timeout(60) {
    # Do something
    }
    }
    Lon est timeout
    clobbers the short one
    

    View Slide

84. I LIKE THIS BUG
    Because JRuby wins
    

    View Slide

85. Simple Bu
    Complex Bu
    Painful Bu
    Kernel Bu
    NESTED
    TIMEOUTS
    IN THE JVM
    

    View Slide

86. Now Leavin :
    Realm of Bu s
    Valley of Common Sense
    

    View Slide

87. mmon Sense
    Mountains of Reason
    

    View Slide

88. Country of
    WHAT THE
    HELL ARE YOU
    DOING
    MRI Timeouts
    in Green Threads
    

    View Slide

89. MORAL
    OF
    THE
    STORY
    

    View Slide

90. BETTER
    WORSE
    IS
    Sometimes
    *this doesn’t apply to the MRI GC
    

    View Slide

91. Computers
    break Ruby breaks
    twice as often
    

    View Slide

92. If you build simpler
    systems, they will
    break
    in simpler
    ways.
    

    View Slide

93. View Slide