Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Methods of Memory Management in MRI

Methods of Memory Management in MRI

This is my RubyConf 2016 talk about Ruby's GC

Aaron Patterson

November 16, 2016
Tweet

More Decks by Aaron Patterson

Other Decks in Technology

Transcript

  1. Interactive
    Portion

    View Slide

  2. Unlock your
    phones

    View Slide

  3. Selfie
    Sticks

    View Slide

  4. Two
    Purposes

    View Slide

  5. Help you identify people
    who like fun stuff and to
    have a fun time

    View Slide

  6. Let you know who
    doesn’t like to have
    fun or like fun things

    View Slide

  7. Have you
    seen this?

    View Slide

  8. View Slide

  9. It’s me!

    View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. Methods of
    Memory
    Management in
    MRI

    View Slide

  17. mmmmRuby

    View Slide

  18. Aaron Patterson
    @tenderlove
    PGP Fingerprint: 4CE9 1B75 A798 28E8 6B1A A8BB 9531 70BC B4FF AFC6

    View Slide

  19. View Slide

  20. I am from Seattle

    View Slide

  21. Which is not Ohio

    View Slide

  22. !

    View Slide

  23. "Ohio"

    View Slide

  24. h
    GitHub

    View Slide

  25. LeGit

    View Slide

  26. GitHub Certified®
    Engineer

    View Slide

  27. Bear Metal

    View Slide

  28. x tenderlove

    View Slide

  29. Please call
    me "Aaron"

    View Slide

  30. ("tenderlove" is fine too)

    View Slide

  31. I love cats!

    View Slide

  32. Cats are the
    best!

    View Slide

  33. View Slide

  34. View Slide

  35. View Slide

  36. View Slide

  37. View Slide

  38. I have stickers of
    my cats

    View Slide

  39. I also have
    GitHub stickers!

    View Slide

  40. I was in the news
    recently

    View Slide

  41. View Slide

  42. Keyboards

    View Slide

  43. View Slide

  44. View Slide

  45. New Ruby
    Features

    View Slide

  46. Soft Typing

    View Slide

  47. Dynamic Typing

    View Slide

  48. Static Typing

    View Slide

  49. GC

    View Slide

  50. Garbage Collector

    View Slide

  51. Memory Terms

    View Slide

  52. Stack Heap

    View Slide

  53. Heap
    Ruby Heap

    View Slide

  54. GC in MRI

    View Slide

  55. Apps in
    Production

    View Slide

  56. Scaling Issues

    View Slide

  57. Tuning Issues

    View Slide

  58. What I want you
    to learn

    View Slide

  59. If you don’t know
    much about GC

    View Slide

  60. If you know about
    GC terminology

    View Slide

  61. If you already know
    the algorithms

    View Slide

  62. If you already
    know the new stuff

    View Slide

  63. GC Algorithms
    (in MRI)

    View Slide

  64. Two sides of a GC

    View Slide

  65. Collection
    Algorithm

    View Slide

  66. Allocation
    Algorithm

    View Slide

  67. Introspection API

    View Slide

  68. Tuning Variables

    View Slide

  69. Collection
    Algorithm

    View Slide

  70. View Slide

  71. View Slide

  72. What type is the
    MRI collector?

    View Slide

  73. Generational
    Incremental
    Mark & Sweep

    View Slide

  74. High level:
    What is a GC?

    View Slide

  75. Tree of Objects
    Root
    A
    B
    C D
    a = [
    { c: 'd' }
    ]
    Ruby
    Array
    Hash
    String
    Symbol

    View Slide

  76. Tree of Objects
    Root
    A
    B
    C D
    a = [
    { c: 'd' }
    ]
    a = nil
    Ruby
    Array
    Hash
    String
    Symbol

    View Slide

  77. Important Words!

    View Slide

  78. Root Set

    View Slide

  79. Garbage

    View Slide

  80. Live Data

    View Slide

  81. GC’s job:
    Find unlinked nodes,
    then free them

    View Slide

  82. How to find
    unlinked nodes

    View Slide

  83. Mark & Sweep

    View Slide

  84. 2 distinct phases

    View Slide

  85. Mark Phase
    Root
    D
    A
    B
    C
    F
    E
    H
    G

    View Slide

  86. Mark Phase
    Root
    D
    A
    B
    C
    F
    E
    H
    G

    View Slide

  87. Mark Phase
    Root
    D
    A
    B
    C
    F
    E
    H
    G

    View Slide

  88. Mark Phase
    Root
    D
    A
    B
    C
    F
    E
    H
    G

    View Slide

  89. Sweep Phase
    Root
    D
    A
    B
    C
    F
    E
    H
    G

    View Slide

  90. Sweep Phase
    Root
    D
    A
    B
    C
    F

    View Slide

  91. Mark & Sweep
    Very Easy!
    Too Slow

    View Slide

  92. "Stop the world"

    View Slide

  93. Visits every
    object, every time

    View Slide

  94. Walk every object
    every time

    View Slide

  95. Generational

    View Slide

  96. Objects die young

    View Slide

  97. Divide objects in
    to "old" and "new"

    View Slide

  98. Generational
    Root
    A
    B
    D
    C
    Gen 0 Gen 1

    View Slide

  99. Generational
    Root
    A
    B
    D
    C
    Gen 0 Gen 1

    View Slide

  100. Generational
    Root
    A
    B
    D
    C
    Gen 0 Gen 1

    View Slide

  101. Generational
    Root
    B
    D
    Gen 0 Gen 1

    View Slide

  102. Generational
    Root
    B
    D
    Gen 0 Gen 1
    E
    F
    G

    View Slide

  103. Generational
    Root
    B
    D
    Gen 0 Gen 1
    E
    F
    G

    View Slide

  104. Generational
    Root
    B
    D
    Gen 0 Gen 1
    E
    F
    G

    View Slide

  105. Generational
    Root
    B
    D
    Gen 0 Gen 1
    F
    G

    View Slide

  106. Generational
    Root
    B
    D
    Gen 0 Gen 1
    F
    G

    View Slide

  107. We didn’t have to
    touch "B"

    View Slide

  108. One Slight
    Problem

    View Slide

  109. Generational
    Root
    B
    D
    Gen 0 Gen 1

    View Slide

  110. Generational
    Root
    B
    D
    Gen 0 Gen 1
    E
    U
    nused!

    View Slide

  111. Remembered Set
    Root
    B
    D
    Gen 0 Gen 1
    E

    View Slide

  112. Remembered Set
    Root
    B
    D
    Gen 0 Gen 1
    E
    Remember!

    View Slide

  113. Remembered Set
    Root
    B
    D
    Gen 0 Gen 1
    E

    View Slide

  114. Remembered Set
    Root
    B
    D
    Gen 0 Gen 1
    E

    View Slide

  115. Remembered Set
    Root
    B
    D
    Gen 0 Gen 1
    E

    View Slide

  116. Important Words

    View Slide

  117. Write Barrier

    View Slide

  118. Remembered Set

    View Slide

  119. Generational
    Fast(er)!
    Not so easy

    View Slide

  120. "Stop the world"

    View Slide

  121. Incremental GC

    View Slide

  122. Tri-Color
    Marking

    View Slide

  123. Object Colors
    • White: will be collected
    • Black: No reference to white objects, but
    reachable from the root
    • Gray: Reachable from root, but not yet
    scanned

    View Slide

  124. Algorithm
    1. Pick an object from the gray set and move it
    to the black set
    2. For each object it references, move it to the
    gray set
    3. Repeat 1 and 2 until the gray set is empty

    View Slide

  125. Tri-Color Marking
    Root
    H
    G
    F
    C
    B
    D
    A
    E

    View Slide

  126. Tri-Color Marking
    Root
    H
    G
    F
    C
    B
    D
    A
    E

    View Slide

  127. Tri-Color Marking
    Root
    H
    G
    F
    C
    B
    D
    A
    E

    View Slide

  128. Tri-Color Marking
    Root
    H
    G
    F
    C
    B
    D
    A
    E

    View Slide

  129. Tri-Color Marking
    Root
    H
    G
    F
    C
    B
    D
    A
    E

    View Slide

  130. View Slide

  131. What is the
    benefit?

    View Slide

  132. We can interrupt
    steps!

    View Slide

  133. Each step can be
    performed
    incrementally

    View Slide

  134. Halting time is
    reduced

    View Slide

  135. One Slight
    Problem

    View Slide

  136. Tri-Color Marking
    Root
    F
    C
    B
    D
    A

    View Slide

  137. Tri-Color Marking
    Root
    F
    C
    B
    D
    A
    G

    View Slide

  138. Tri-Color Marking
    Root
    F
    C
    B
    D
    A
    G

    View Slide

  139. Tri-Color Marking
    Root
    F
    C
    B
    D
    A
    G
    Write!

    View Slide

  140. Important Words

    View Slide

  141. Incremental

    View Slide

  142. Write Barrier

    View Slide

  143. Remembered Set

    View Slide

  144. Minimize tracing

    View Slide

  145. Decrease halting

    View Slide

  146. Things our GC
    is not

    View Slide

  147. Parallel

    View Slide

  148. Real-Time

    View Slide

  149. Compacting

    View Slide

  150. Allocation
    Algorithm

    View Slide

  151. View Slide

  152. View Slide

  153. Heap layout

    View Slide

  154. malloc isn’t free
    GET IT?????

    View Slide

  155. Large Chunk:
    Page (or Slab)

    View Slide

  156. Page memory is
    contiguous

    View Slide

  157. Each page holds a
    linked list

    View Slide

  158. Nodes are called
    "slots"

    View Slide

  159. Each slot is a
    Ruby object

    View Slide

  160. Page Layout
    Page
    Ruby Object
    Ruby Object
    Ruby Object
    Ruby Object
    Ruby Object

    View Slide

  161. "Free List"

    View Slide

  162. Find the first open
    slot!

    View Slide

  163. Move to the next
    space in the free list

    View Slide

  164. "Bump Pointer"
    allocation

    View Slide

  165. Full Pages
    Object
    Object
    Object
    Object
    Page Page

    View Slide

  166. "Eden" pages are
    searched

    View Slide

  167. GC Time
    Object
    Object
    Object
    Object

    View Slide

  168. Important Words

    View Slide

  169. Slot

    View Slide

  170. Page

    View Slide

  171. Eden

    View Slide

  172. Tomb

    View Slide

  173. Interesting
    Allocation Hacks

    View Slide

  174. Not every object
    requires allocation

    View Slide

  175. 1 Page: 16k

    View Slide

  176. 1 Object: 40 bytes

    View Slide

  177. Pages are
    "aligned"

    View Slide

  178. Not `malloc` but
    "aligned malloc"
    `posix_memalign`

    View Slide

  179. Choose "40" as a
    multiple

    View Slide

  180. Start = 40
    >> start = 40
    => 40
    >> (start * 1).to_s(2).rjust(10, '0')
    => "0000101000"
    >> (start * 2).to_s(2).rjust(10, '0')
    => "0001010000"
    >> (start * 3).to_s(2).rjust(10, '0')
    => "0001111000"
    >> (start * 4).to_s(2).rjust(10, '0')
    => "0010100000"
    >> (start * 5).to_s(2).rjust(10, '0')
    => "0011001000"
    >> (start * 6).to_s(2).rjust(10, '0')
    => "0011110000"
    >> (start * 7).to_s(2).rjust(10, '0')
    => "0100011000"

    View Slide

  181. "0000101000"
    "0001010000"
    "0001111000"
    "0010100000"
    "0011001000"
    "0011110000"
    "0100011000"

    View Slide

  182. Use these bits to
    add meaning

    View Slide

  183. Represent Integers
    Without Allocation

    View Slide

  184. Encode Number 2
    >> INT_FLAG = 0x1
    => 1
    >> 2.to_s(2)
    => "10"
    >> ((2 << 1) | INT_FLAG).to_s(2)
    => "101"

    View Slide

  185. Decode Number 2
    >> 0b101
    => 5
    >> 0b101 >> 1
    => 2

    View Slide

  186. Biggest Fixnum
    >> ((2 ** 64) - 1).to_s(2)
    =>
    "11111111111111111111111111111111111111111111111
    11111111111111111"
    >> ((2 ** 64) - 1).class
    => Bignum
    >> ((2 ** 63) - 1).class
    => Bignum
    >> ((2 ** 62) - 1).class
    => Fixnum
    >> ((2 ** 62)).class
    => Bignum

    View Slide

  187. Biggest Before Heap
    Allocation
    >> ((2 ** 64) - 1).to_s(2)
    =>
    "11111111111111111111111111111111111111111111111
    11111111111111111"
    >> ((2 ** 64) - 1).class
    => Integer
    >> ((2 ** 63) - 1).class
    => Integer
    >> ((2 ** 62) - 1).class
    => Integer
    >> ((2 ** 62)).class
    => Integer
    R
    uby
    2.4

    View Slide

  188. Fixnums are
    singletons
    >> ((2 ** 62) - 1).object_id
    => 9223372036854775807
    >> ((2 ** 62) - 1).object_id
    => 9223372036854775807
    >> ((2 ** 62)).object_id
    => 70213216135940
    >> ((2 ** 62)).object_id
    => 70213216117840

    View Slide

  189. Tagged Pointer

    View Slide

  190. Tagged Pointers
    • Fixnum
    • Floats
    • True / False / Nil
    • Symbols

    View Slide

  191. Allocation
    Problems

    View Slide

  192. Poor Reclamation
    Object
    Object
    Object
    Object
    Object
    Object
    Object
    Object
    Object
    Object
    Object
    Object

    View Slide

  193. Poor Reclamation
    Object
    Object
    Object
    Object
    Object
    Object
    Object
    Object
    NOPE

    View Slide

  194. Poor Reclamation
    Object
    Object
    Object
    Object
    Object
    Object
    Object
    Object

    View Slide

  195. CoW problems

    View Slide

  196. 1 Ruby Memory Page !=
    1 OS Memory Page

    View Slide

  197. 1 Ruby Page: 16k
    1 OS Page: 4k

    View Slide

  198. Object
    Object
    Object
    Parent
    Process
    Child
    Process
    Object

    View Slide

  199. OS Copies
    1 OS Page

    View Slide

  200. We wrote 40 bytes,
    but 4kb got copied.

    View Slide

  201. Solution:
    Group Old Objects

    View Slide

  202. Two Page Types
    Probably
    Old Page
    Page
    Object
    Object

    View Slide

  203. What is going to
    be old?

    View Slide

  204. Probably Old
    class Foo
    end
    module Bar
    CONSTANT = Object.new
    def foo
    "frozen string".freeze
    end
    end

    View Slide

  205. Statistically
    Determined

    View Slide

  206. Efficiently use
    space

    View Slide

  207. Reduce GC time

    View Slide

  208. CoW Friendly

    View Slide

  209. GC Work
    G
    GitHub

    View Slide

  210. View Slide

  211. View Slide

  212. Key
    Objects
    Class / Module
    Empty

    View Slide

  213. View Slide

  214. View Slide

  215. View Slide

  216. View Slide

  217. View Slide

  218. View Slide

  219. ~17%
    Smaller Heap

    View Slide

  220. View Slide

  221. github.com/
    github/ruby

    View Slide

  222. Future Work

    View Slide

  223. Moving Objects

    View Slide

  224. Poor Reclamation
    Object
    Object
    Object
    Object
    Object
    Object
    Object
    Object
    NOPE
    YEP

    View Slide

  225. Stack Scanning,
    Forward Pointers

    View Slide

  226. GC
    Introspection

    View Slide

  227. Seeing GC info

    View Slide

  228. GC.stat
    {:count=>21,
    :heap_allocated_pages=>87,
    :heap_sorted_length=>87,
    :heap_allocatable_pages=>0,
    :heap_available_slots=>35460,
    :heap_live_slots=>35342,
    :heap_free_slots=>118,
    :heap_final_slots=>0,
    :heap_marked_slots=>22207,
    :heap_eden_pages=>87,
    :heap_tomb_pages=>0,
    :total_allocated_pages=>87,
    :total_freed_pages=>0,
    :total_allocated_objects=>208577,
    :total_freed_objects=>173235,
    :malloc_increase_bytes=>5152,
    :malloc_increase_bytes_limit=>16777216,
    :minor_gc_count=>19,
    :major_gc_count=>2,
    :remembered_wb_unprotected_objects=>189,
    :remembered_wb_unprotected_objects_limit=>374,
    :old_objects=>21977,
    :old_objects_limit=>39876,
    :oldmalloc_increase_bytes=>485600,
    :oldmalloc_increase_bytes_limit=>16777216}

    View Slide

  229. GC Performance

    View Slide

  230. GC::Profiler
    >> GC::Profiler.enable
    => nil
    >> GC::Profiler.report
    => nil
    >> GC.start
    => nil
    >> GC::Profiler.report
    GC 22 invokes.
    Index Invoke Time(sec) Use Size(byte) Total Size(byte) Total Object
    GC Time(ms)
    1 0.143 906920 1419840 35496
    3.72500000000000586198
    => nil
    >> GC.start
    => nil
    >> GC::Profiler.report
    GC 23 invokes.
    Index Invoke Time(sec) Use Size(byte) Total Size(byte) Total Object
    GC Time(ms)
    1 0.143 906920 1419840 35496
    3.72500000000000586198
    2 0.148 906920 1419840 35496
    3.47800000000000864020

    View Slide

  231. GC::Profiler.enable

    View Slide

  232. GC::Profiler.report

    View Slide

  233. Heap Inspection

    View Slide

  234. ObjectSpace.dump_all

    View Slide

  235. ObjectSpace.dump
    >> require 'objspace'
    => false
    >> x = Object.new
    => #
    >> ObjectSpace.dump x
    => "{\"address\":\"0x007fbcd09334a8\",
    \"type\":\"OBJECT\", \"class\":
    \"0x007fbcd08dd878\", \"ivars\":0,
    \"memsize\":40, \"flags\":
    {\"wb_protected\":true}}\n"

    View Slide

  236. ObjectSpace.dump
    >> x = Object.new
    => #
    >> JSON.parse(ObjectSpace.dump(x))['flags']
    => {"wb_protected"=>true}
    >> GC.start
    => nil
    >> JSON.parse(ObjectSpace.dump(x))['flags']
    => {"wb_protected"=>true}
    >> GC.start
    => nil
    >> JSON.parse(ObjectSpace.dump(x))['flags']
    => {"wb_protected"=>true}
    >> GC.start
    => nil
    >> JSON.parse(ObjectSpace.dump(x))['flags']
    => {"wb_protected"=>true, "old"=>true, "uncollectible"=>true,
    "marked"=>true}

    View Slide

  237. Object have 3
    generations

    View Slide

  238. ObjectSpace

    View Slide

  239. trace_object_allocations

    View Slide

  240. GC Tuning

    View Slide

  241. RUBY_GC_HEAP
    _FREE_SLOTS
    Number of free slots available after a GC

    View Slide

  242. RUBY_GC_HEAP
    _INIT_SLOTS
    Number of free slots to initialize the GC with

    View Slide

  243. RUBY_GC_HEAP_GR
    OWTH_MAX_SLOTS
    Never grow more than this many objects

    View Slide

  244. THANKS!

    View Slide

  245. View Slide