Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Unicorns Die With Bullets Made of Glitter

6804f1775cb4babfcc3851298566fbce?s=47 Vicent Martí
September 20, 2013

Unicorns Die With Bullets Made of Glitter

The reduced, 30 minutes version of this collection of Ruby Systems Horror Stories, including:

- Mark & Sweep, a tale of a dumb garbage collector
- Our Time is Running Out, a tale of tricky timeouts

Originally presented at Rubyfuza.

6804f1775cb4babfcc3851298566fbce?s=128

Vicent Martí

September 20, 2013
Tweet

Transcript

  1. None
  2. Once upon a time…

  3. None
  4. These are horror stories from the GitHub Systems team (the

    rainbows are just to make them less awful)
  5. I HAVE A LOT OF SPARE TIME

  6. Ogre Magi is (C) 2013 by Valve Software and takes

    0 skill to play
  7. Mark Sweep The Garba e Collector

  8. Garba e Collector

  9. Conservative Garba e Collector

  10. Conservative Garba e Collector Non-deterministic

  11. Conservative Stop The World Garba e Collector Non-deterministic

  12. Conservative Stop The World Mark and Sweep Garba e Collector

    Non-deterministic
  13. Conservative Stop The World Mark and Sweep Garba e Collector

    Non-deterministic Basically a shitshow.
  14. None
  15. Mark

  16. Mark 1. Walk the object raph 2. Look for thin

    s that look like Ruby objects
  17. None
  18. Ruby handles raw pointers to C extensions.

  19. Ruby handles raw pointers to C extensions. The Garba e

    Collector must find roots in the stack, the heap and in re isters.
  20. None
  21. “This kinda looks like a pointer”

  22. “This kinda looks like a pointer” “I uess…”

  23. “This kinda looks like a pointer” “I uess…” *dramatization

  24. None
  25. Sweep

  26. Sweep 1. Go throu h every sin le object 2.

    Free the ones that have not been marked
  27. None
  28. The Problem

  29. The Problem There is memory allocated by Ruby that the

    Ruby GC cannot detect.
  30. char * VALUE * … … RStrin MAGICAL RUBY THINGS

    int a_ruby_function(VALUE rb_foo) { char *str = RSTRING_PTR(rb_foo); return do_smthing(str); }
  31. char * VALUE * … … RStrin MAGICAL RUBY THINGS

    int a_ruby_function(VALUE rb_foo) { char *str = RSTRING_PTR(rb_foo); return do_smthing(str); } H e l l o W o r l d \0
  32. char * … … H e l l o W

    o r l d \0 int do_smthing(char *str) { /* do stuff */ return strlen(str); }
  33. int do_smthing(char *str) { /* do stuff */ return strlen(str);

    } Un ! Un ! Mark o throu h stack. Yes. Stack.
  34. Un ! This no look like Ruby object. Mark no

    touchie! char * … … … H e l l o W o r l d \0
  35. No root. H e l l o W o r

    l d \0 RStrin MAGICAL RUBY THINGS Strin object with no roots? Set fire. Yes. Fire.
  36. No root. H e l l o W o r

    l d \0 Strin object with no roots? Set fire. Yes. Fire.
  37. No root. Strin object with no roots? Set fire. Yes.

    Fire.
  38. int do_smthing(char *str) { /* do stuff */ return strlen(str);

    }
  39. int do_smthing(char *str) { /* do stuff */ return strlen(str);

    } *sound of an explosion* “What was that?” “Dunno.”
  40. HAPPY ENDINGS: (sorry about this slide)

  41. 1. Drink Yourself To Sleep Every Ni ht

  42. 2. Stress the Garba e Collector in Critical Paths

  43. None
  44. mallopt(M_PERTURB, 0x33); C land

  45. mallopt(M_PERTURB, 0x33); C land GC.stress = true Ruby land

  46. Bonus Points Use Val rind/ASan instead of tamperin .

  47. Bonus Points Use Val rind/ASan instead of tamperin . Your

    Rails test suite Heat death of the Universe
  48. 3. Static Analysis

  49. None
  50. How do you feel about timeouts?

  51. THERE ARE NO TIMEOUTS IN RUBY (just ask the Timeout

    Rabbit)
  52. require ‘timeout’

  53. require ‘timeout’ Timeout::timeout(5) { # Do something }

  54. require ‘timeout’ Timeout::timeout(5) { # Do something } …

  55. None
  56. … …

  57. … … oh hey

  58. “Sorry I’m late. Couldn’t et here any faster.”

  59. None
  60. Beautiful

  61. Beautiful wron also:

  62. MRI 1.8.7 Ruby Process Green Thread Green Thread Green Thread

    THE KERNEL
  63. MRI 1.9.3 Ruby Process THE KERNEL NNative Thread NNative Thread

    NNative Thread
  64. MRI 1.9.3 Ruby Process THE KERNEL NNative Thread NNative Thread

    NNative Thread GVL
  65. Use the force, kid

  66. “Unix, you moron. Unix is the force.”

  67. “Unix, you moron. Unix is the force.”

  68. None
  69. MRI 1.8.7 Use si nals & the power of Unix

  70. MRI 1.8.7 Use si nals & the power of Unix

    MRI 1.9.3 Release the GVL
  71. Mathiasman the Wise

  72. None
  73. None
  74. None
  75. “I, too, had a dreadful problem with timeouts”

  76. You see, I was usin JRuby…

  77. JRuby? Sounds like you have two problems now.

  78. (shut up, kid) You see, I was usin JRuby, and

    the GitHub API you uys keep breakin …
  79. Duh. That’s probably because the thread is sleepin or stuck

    in the GVL.
  80. You fool! JRuby has native threads and fine- rained lockin

    ! *smack*
  81. JRUBY 1.6.7 Timeout::timeout(5) { Timeout::timeout(60) { # Do something }

    } Lon est timeout clobbers the short one
  82. JRUBY 1.6.7 Timeout::timeout(5) { Timeout::timeout(60) { # Do something }

    } Lon est timeout clobbers the short one
  83. I LIKE THIS BUG Because JRuby wins

  84. Simple Bu Complex Bu Painful Bu Kernel Bu NESTED TIMEOUTS

    IN THE JVM
  85. Now Leavin : Realm of Bu s Valley of Common

    Sense
  86. mmon Sense Mountains of Reason

  87. Country of WHAT THE HELL ARE YOU DOING MRI Timeouts

    in Green Threads
  88. MORAL OF THE STORY

  89. BETTER WORSE IS Sometimes *this doesn’t apply to the MRI

    GC
  90. Computers break Ruby breaks twice as often

  91. If you build simpler systems, they will break in simpler

    ways.
  92. None