Methods of Memory Management in MRI

This is my RubyConf 2016 talk about Ruby's GC

Transcript

1. Interactive
   Portion
   

   View Slide

2. Unlock your
   phones
   

   View Slide

3. Selfie
   Sticks
   

   View Slide

4. Two
   Purposes
   

   View Slide

5. Help you identify people
   who like fun stuff and to
   have a fun time
   

   View Slide

6. Let you know who
   doesn’t like to have
   fun or like fun things
   

   View Slide

7. Have you
   seen this?
   

   View Slide

8. View Slide

9. It’s me!
   

   View Slide

10. View Slide

11. View Slide

12. View Slide

13. View Slide

14. View Slide

15. View Slide

16. Methods of
    Memory
    Management in
    MRI
    

    View Slide

17. mmmmRuby
    

    View Slide

18. Aaron Patterson
    @tenderlove
    PGP Fingerprint: 4CE9 1B75 A798 28E8 6B1A A8BB 9531 70BC B4FF AFC6
    

    View Slide

19. View Slide

20. I am from Seattle
    

    View Slide

21. Which is not Ohio
    

    View Slide

22. !
    

    View Slide

23. "Ohio"
    

    View Slide

24. h
    GitHub
    

    View Slide

25. LeGit
    

    View Slide

26. GitHub Certified®
    Engineer
    

    View Slide

27. Bear Metal
    

    View Slide

28. x tenderlove
    

    View Slide

29. Please call
    me "Aaron"
    

    View Slide

30. ("tenderlove" is fine too)
    

    View Slide

31. I love cats!
    

    View Slide

32. Cats are the
    best!
    

    View Slide

33. View Slide

34. View Slide

35. View Slide

36. View Slide

37. View Slide

38. I have stickers of
    my cats
    

    View Slide

39. I also have
    GitHub stickers!
    

    View Slide

40. I was in the news
    recently
    

    View Slide

41. View Slide

42. Keyboards
    

    View Slide

43. View Slide

44. View Slide

45. New Ruby
    Features
    

    View Slide

46. Soft Typing
    

    View Slide

47. Dynamic Typing
    

    View Slide

48. Static Typing
    

    View Slide

49. GC
    

    View Slide

50. Garbage Collector
    

    View Slide

51. Memory Terms
    

    View Slide

52. Stack Heap
    

    View Slide

53. Heap
    Ruby Heap
    

    View Slide

54. GC in MRI
    

    View Slide

55. Apps in
    Production
    

    View Slide

56. Scaling Issues
    

    View Slide

57. Tuning Issues
    

    View Slide

58. What I want you
    to learn
    

    View Slide

59. If you don’t know
    much about GC
    

    View Slide

60. If you know about
    GC terminology
    

    View Slide

61. If you already know
    the algorithms
    

    View Slide

62. If you already
    know the new stuff
    

    View Slide

63. GC Algorithms
    (in MRI)
    

    View Slide

64. Two sides of a GC
    

    View Slide

65. Collection
    Algorithm
    

    View Slide

66. Allocation
    Algorithm
    

    View Slide

67. Introspection API
    

    View Slide

68. Tuning Variables
    

    View Slide

69. Collection
    Algorithm
    

    View Slide

70. View Slide

71. View Slide

72. What type is the
    MRI collector?
    

    View Slide

73. Generational
    Incremental
    Mark & Sweep
    

    View Slide

74. High level:
    What is a GC?
    

    View Slide

75. Tree of Objects
    Root
    A
    B
    C D
    a = [
    { c: 'd' }
    ]
    Ruby
    Array
    Hash
    String
    Symbol
    

    View Slide

76. Tree of Objects
    Root
    A
    B
    C D
    a = [
    { c: 'd' }
    ]
    a = nil
    Ruby
    Array
    Hash
    String
    Symbol
    

    View Slide

77. Important Words!
    

    View Slide

78. Root Set
    

    View Slide

79. Garbage
    

    View Slide

80. Live Data
    

    View Slide

81. GC’s job:
    Find unlinked nodes,
    then free them
    

    View Slide

82. How to find
    unlinked nodes
    

    View Slide

83. Mark & Sweep
    

    View Slide

84. 2 distinct phases
    

    View Slide

85. Mark Phase
    Root
    D
    A
    B
    C
    F
    E
    H
    G
    

    View Slide

86. Mark Phase
    Root
    D
    A
    B
    C
    F
    E
    H
    G
    

    View Slide

87. Mark Phase
    Root
    D
    A
    B
    C
    F
    E
    H
    G
    

    View Slide

88. Mark Phase
    Root
    D
    A
    B
    C
    F
    E
    H
    G
    

    View Slide

89. Sweep Phase
    Root
    D
    A
    B
    C
    F
    E
    H
    G
    

    View Slide

90. Sweep Phase
    Root
    D
    A
    B
    C
    F
    

    View Slide

91. Mark & Sweep
    Very Easy!
    Too Slow
    

    View Slide

92. "Stop the world"
    

    View Slide

93. Visits every
    object, every time
    

    View Slide

94. Walk every object
    every time
    

    View Slide

95. Generational
    

    View Slide

96. Objects die young
    

    View Slide

97. Divide objects in
    to "old" and "new"
    

    View Slide

98. Generational
    Root
    A
    B
    D
    C
    Gen 0 Gen 1
    

    View Slide

99. Generational
    Root
    A
    B
    D
    C
    Gen 0 Gen 1
    

    View Slide

100. Generational
     Root
     A
     B
     D
     C
     Gen 0 Gen 1
     

     View Slide

101. Generational
     Root
     B
     D
     Gen 0 Gen 1
     

     View Slide

102. Generational
     Root
     B
     D
     Gen 0 Gen 1
     E
     F
     G
     

     View Slide

103. Generational
     Root
     B
     D
     Gen 0 Gen 1
     E
     F
     G
     

     View Slide

104. Generational
     Root
     B
     D
     Gen 0 Gen 1
     E
     F
     G
     

     View Slide

105. Generational
     Root
     B
     D
     Gen 0 Gen 1
     F
     G
     

     View Slide

106. Generational
     Root
     B
     D
     Gen 0 Gen 1
     F
     G
     

     View Slide

107. We didn’t have to
     touch "B"
     

     View Slide

108. One Slight
     Problem
     

     View Slide

109. Generational
     Root
     B
     D
     Gen 0 Gen 1
     

     View Slide

110. Generational
     Root
     B
     D
     Gen 0 Gen 1
     E
     U
     nused!
     

     View Slide

111. Remembered Set
     Root
     B
     D
     Gen 0 Gen 1
     E
     

     View Slide

112. Remembered Set
     Root
     B
     D
     Gen 0 Gen 1
     E
     Remember!
     

     View Slide

113. Remembered Set
     Root
     B
     D
     Gen 0 Gen 1
     E
     

     View Slide

114. Remembered Set
     Root
     B
     D
     Gen 0 Gen 1
     E
     

     View Slide

115. Remembered Set
     Root
     B
     D
     Gen 0 Gen 1
     E
     

     View Slide

116. Important Words
     

     View Slide

117. Write Barrier
     

     View Slide

118. Remembered Set
     

     View Slide

119. Generational
     Fast(er)!
     Not so easy
     

     View Slide

120. "Stop the world"
     

     View Slide

121. Incremental GC
     

     View Slide

122. Tri-Color
     Marking
     

     View Slide

123. Object Colors
     • White: will be collected
     • Black: No reference to white objects, but
     reachable from the root
     • Gray: Reachable from root, but not yet
     scanned
     

     View Slide

124. Algorithm
     1. Pick an object from the gray set and move it
     to the black set
     2. For each object it references, move it to the
     gray set
     3. Repeat 1 and 2 until the gray set is empty
     

     View Slide

125. Tri-Color Marking
     Root
     H
     G
     F
     C
     B
     D
     A
     E
     

     View Slide

126. Tri-Color Marking
     Root
     H
     G
     F
     C
     B
     D
     A
     E
     

     View Slide

127. Tri-Color Marking
     Root
     H
     G
     F
     C
     B
     D
     A
     E
     

     View Slide

128. Tri-Color Marking
     Root
     H
     G
     F
     C
     B
     D
     A
     E
     

     View Slide

129. Tri-Color Marking
     Root
     H
     G
     F
     C
     B
     D
     A
     E
     

     View Slide

130. View Slide

131. What is the
     benefit?
     

     View Slide

132. We can interrupt
     steps!
     

     View Slide

133. Each step can be
     performed
     incrementally
     

     View Slide

134. Halting time is
     reduced
     

     View Slide

135. One Slight
     Problem
     

     View Slide

136. Tri-Color Marking
     Root
     F
     C
     B
     D
     A
     

     View Slide

137. Tri-Color Marking
     Root
     F
     C
     B
     D
     A
     G
     

     View Slide

138. Tri-Color Marking
     Root
     F
     C
     B
     D
     A
     G
     

     View Slide

139. Tri-Color Marking
     Root
     F
     C
     B
     D
     A
     G
     Write!
     

     View Slide

140. Important Words
     

     View Slide

141. Incremental
     

     View Slide

142. Write Barrier
     

     View Slide

143. Remembered Set
     

     View Slide

144. Minimize tracing
     

     View Slide

145. Decrease halting
     

     View Slide

146. Things our GC
     is not
     

     View Slide

147. Parallel
     

     View Slide

148. Real-Time
     

     View Slide

149. Compacting
     

     View Slide

150. Allocation
     Algorithm
     

     View Slide

151. View Slide

152. View Slide

153. Heap layout
     

     View Slide

154. malloc isn’t free
     GET IT?????
     

     View Slide

155. Large Chunk:
     Page (or Slab)
     

     View Slide

156. Page memory is
     contiguous
     

     View Slide

157. Each page holds a
     linked list
     

     View Slide

158. Nodes are called
     "slots"
     

     View Slide

159. Each slot is a
     Ruby object
     

     View Slide

160. Page Layout
     Page
     Ruby Object
     Ruby Object
     Ruby Object
     Ruby Object
     Ruby Object
     

     View Slide

161. "Free List"
     

     View Slide

162. Find the first open
     slot!
     

     View Slide

163. Move to the next
     space in the free list
     

     View Slide

164. "Bump Pointer"
     allocation
     

     View Slide

165. Full Pages
     Object
     Object
     Object
     Object
     Page Page
     

     View Slide

166. "Eden" pages are
     searched
     

     View Slide

167. GC Time
     Object
     Object
     Object
     Object
     ☠
     

     View Slide

168. Important Words
     

     View Slide

169. Slot
     

     View Slide

170. Page
     

     View Slide

171. Eden
     

     View Slide

172. Tomb
     

     View Slide

173. Interesting
     Allocation Hacks
     

     View Slide

174. Not every object
     requires allocation
     

     View Slide

175. 1 Page: 16k
     

     View Slide

176. 1 Object: 40 bytes
     

     View Slide

177. Pages are
     "aligned"
     

     View Slide

178. Not `malloc` but
     "aligned malloc"
     `posix_memalign`
     

     View Slide

179. Choose "40" as a
     multiple
     

     View Slide

180. Start = 40
     >> start = 40
     => 40
     >> (start * 1).to_s(2).rjust(10, '0')
     => "0000101000"
     >> (start * 2).to_s(2).rjust(10, '0')
     => "0001010000"
     >> (start * 3).to_s(2).rjust(10, '0')
     => "0001111000"
     >> (start * 4).to_s(2).rjust(10, '0')
     => "0010100000"
     >> (start * 5).to_s(2).rjust(10, '0')
     => "0011001000"
     >> (start * 6).to_s(2).rjust(10, '0')
     => "0011110000"
     >> (start * 7).to_s(2).rjust(10, '0')
     => "0100011000"
     

     View Slide

181. "0000101000"
     "0001010000"
     "0001111000"
     "0010100000"
     "0011001000"
     "0011110000"
     "0100011000"
     

     View Slide

182. Use these bits to
     add meaning
     

     View Slide

183. Represent Integers
     Without Allocation
     

     View Slide

184. Encode Number 2
     >> INT_FLAG = 0x1
     => 1
     >> 2.to_s(2)
     => "10"
     >> ((2 << 1) | INT_FLAG).to_s(2)
     => "101"
     

     View Slide

185. Decode Number 2
     >> 0b101
     => 5
     >> 0b101 >> 1
     => 2
     

     View Slide

186. Biggest Fixnum
     >> ((2 ** 64) - 1).to_s(2)
     =>
     "11111111111111111111111111111111111111111111111
     11111111111111111"
     >> ((2 ** 64) - 1).class
     => Bignum
     >> ((2 ** 63) - 1).class
     => Bignum
     >> ((2 ** 62) - 1).class
     => Fixnum
     >> ((2 ** 62)).class
     => Bignum
     

     View Slide

187. Biggest Before Heap
     Allocation
     >> ((2 ** 64) - 1).to_s(2)
     =>
     "11111111111111111111111111111111111111111111111
     11111111111111111"
     >> ((2 ** 64) - 1).class
     => Integer
     >> ((2 ** 63) - 1).class
     => Integer
     >> ((2 ** 62) - 1).class
     => Integer
     >> ((2 ** 62)).class
     => Integer
     R
     uby
     2.4
     

     View Slide

188. Fixnums are
     singletons
     >> ((2 ** 62) - 1).object_id
     => 9223372036854775807
     >> ((2 ** 62) - 1).object_id
     => 9223372036854775807
     >> ((2 ** 62)).object_id
     => 70213216135940
     >> ((2 ** 62)).object_id
     => 70213216117840
     

     View Slide

189. Tagged Pointer
     

     View Slide

190. Tagged Pointers
     • Fixnum
     • Floats
     • True / False / Nil
     • Symbols
     

     View Slide

191. Allocation
     Problems
     

     View Slide

192. Poor Reclamation
     Object
     Object
     Object
     Object
     Object
     Object
     Object
     Object
     Object
     Object
     Object
     Object
     

     View Slide

193. Poor Reclamation
     Object
     Object
     Object
     Object
     Object
     Object
     Object
     Object
     NOPE
     

     View Slide

194. Poor Reclamation
     Object
     Object
     Object
     Object
     Object
     Object
     Object
     Object
     

     View Slide

195. CoW problems
     

     View Slide

196. 1 Ruby Memory Page !=
     1 OS Memory Page
     

     View Slide

197. 1 Ruby Page: 16k
     1 OS Page: 4k
     

     View Slide

198. Object
     Object
     Object
     Parent
     Process
     Child
     Process
     Object
     

     View Slide

199. OS Copies
     1 OS Page
     

     View Slide

200. We wrote 40 bytes,
     but 4kb got copied.
     

     View Slide

201. Solution:
     Group Old Objects
     

     View Slide

202. Two Page Types
     Probably
     Old Page
     Page
     Object
     Object
     

     View Slide

203. What is going to
     be old?
     

     View Slide

204. Probably Old
     class Foo
     end
     module Bar
     CONSTANT = Object.new
     def foo
     "frozen string".freeze
     end
     end
     

     View Slide

205. Statistically
     Determined
     

     View Slide

206. Efficiently use
     space
     

     View Slide

207. Reduce GC time
     

     View Slide

208. CoW Friendly
     

     View Slide

209. GC Work
     G
     GitHub
     

     View Slide

210. View Slide

211. View Slide

212. Key
     Objects
     Class / Module
     Empty
     

     View Slide

213. View Slide

214. View Slide

215. View Slide

216. View Slide

217. View Slide

218. View Slide

219. ~17%
     Smaller Heap
     

     View Slide

220. View Slide

221. github.com/
     github/ruby
     

     View Slide

222. Future Work
     

     View Slide

223. Moving Objects
     

     View Slide

224. Poor Reclamation
     Object
     Object
     Object
     Object
     Object
     Object
     Object
     Object
     NOPE
     YEP
     

     View Slide

225. Stack Scanning,
     Forward Pointers
     

     View Slide

226. GC
     Introspection
     

     View Slide

227. Seeing GC info
     

     View Slide

228. GC.stat
     {:count=>21,
     :heap_allocated_pages=>87,
     :heap_sorted_length=>87,
     :heap_allocatable_pages=>0,
     :heap_available_slots=>35460,
     :heap_live_slots=>35342,
     :heap_free_slots=>118,
     :heap_final_slots=>0,
     :heap_marked_slots=>22207,
     :heap_eden_pages=>87,
     :heap_tomb_pages=>0,
     :total_allocated_pages=>87,
     :total_freed_pages=>0,
     :total_allocated_objects=>208577,
     :total_freed_objects=>173235,
     :malloc_increase_bytes=>5152,
     :malloc_increase_bytes_limit=>16777216,
     :minor_gc_count=>19,
     :major_gc_count=>2,
     :remembered_wb_unprotected_objects=>189,
     :remembered_wb_unprotected_objects_limit=>374,
     :old_objects=>21977,
     :old_objects_limit=>39876,
     :oldmalloc_increase_bytes=>485600,
     :oldmalloc_increase_bytes_limit=>16777216}
     

     View Slide

229. GC Performance
     

     View Slide

230. GC::Profiler
     >> GC::Profiler.enable
     => nil
     >> GC::Profiler.report
     => nil
     >> GC.start
     => nil
     >> GC::Profiler.report
     GC 22 invokes.
     Index Invoke Time(sec) Use Size(byte) Total Size(byte) Total Object
     GC Time(ms)
     1 0.143 906920 1419840 35496
     3.72500000000000586198
     => nil
     >> GC.start
     => nil
     >> GC::Profiler.report
     GC 23 invokes.
     Index Invoke Time(sec) Use Size(byte) Total Size(byte) Total Object
     GC Time(ms)
     1 0.143 906920 1419840 35496
     3.72500000000000586198
     2 0.148 906920 1419840 35496
     3.47800000000000864020
     

     View Slide

231. GC::Profiler.enable
     

     View Slide

232. GC::Profiler.report
     

     View Slide

233. Heap Inspection
     

     View Slide

234. ObjectSpace.dump_all
     

     View Slide

235. ObjectSpace.dump
     >> require 'objspace'
     => false
     >> x = Object.new
     => #
     >> ObjectSpace.dump x
     => "{\"address\":\"0x007fbcd09334a8\",
     \"type\":\"OBJECT\", \"class\":
     \"0x007fbcd08dd878\", \"ivars\":0,
     \"memsize\":40, \"flags\":
     {\"wb_protected\":true}}\n"
     

     View Slide

236. ObjectSpace.dump
     >> x = Object.new
     => #
     >> JSON.parse(ObjectSpace.dump(x))['flags']
     => {"wb_protected"=>true}
     >> GC.start
     => nil
     >> JSON.parse(ObjectSpace.dump(x))['flags']
     => {"wb_protected"=>true}
     >> GC.start
     => nil
     >> JSON.parse(ObjectSpace.dump(x))['flags']
     => {"wb_protected"=>true}
     >> GC.start
     => nil
     >> JSON.parse(ObjectSpace.dump(x))['flags']
     => {"wb_protected"=>true, "old"=>true, "uncollectible"=>true,
     "marked"=>true}
     

     View Slide

237. Object have 3
     generations
     

     View Slide

238. ObjectSpace
     

     View Slide

239. trace_object_allocations
     

     View Slide

240. GC Tuning
     

     View Slide

241. RUBY_GC_HEAP
     _FREE_SLOTS
     Number of free slots available after a GC
     

     View Slide

242. RUBY_GC_HEAP
     _INIT_SLOTS
     Number of free slots to initialize the GC with
     

     View Slide

243. RUBY_GC_HEAP_GR
     OWTH_MAX_SLOTS
     Never grow more than this many objects
     

     View Slide

244. THANKS!
     

     View Slide

245. View Slide