Compacting GC in MRI v2

About compacting GC in MRI (v2). For Ruby Kaigi 2019

Transcript

1. Compacting GC for MRI Version 2?
2. HELLO!!!
3. Aaron Patterson @tenderlove
7. I have cat stickers!
8. G GitHub
9. Ruby && Rails
10. 10 Years On Ruby Core! W ow !!!
11. presentation << "joke"
12. Compacting GC for MRI
13. ~3 years to complete
14. What is Compaction?
15. Compaction Allocated Memory Computer Memory Allocated Memory Allocated Memory Free Memory Free Memory
16. Compaction Allocated Memory Computer Memory Allocated Memory Allocated Memory Free Memory Free Memory
17. Compaction Allocated Memory Computer Memory Free Memory
19. Why Compact?
20. Efficient Memory Usage
21. Efficient Memory Usage Allocated Memory Computer Memory Allocated Memory Allocated Memory Free Memory Free Memory Want To Allocate
22. Efficient Memory Usage Allocated Memory Computer Memory Allocated Memory Allocated Memory Free Memory Free Memory Want To Allocate
23. Efficient Memory Usage Allocated Memory Computer Memory Allocated Memory Allocated Memory Wanted To Allocate
24. CPU Caches
25. CPU Cache Hits Allocated Memory Computer Memory Allocated Memory Allocated Memory Free Memory Free Memory In CPU Cache
26. CPU Cache Hits Allocated Memory Computer Memory Allocated Memory Allocated Memory Free Memory Free Memory In CPU Cache
27. "Good Locality"
28. CoW Friendliness CoW is "Copy on Write"
29. CoW Friendliness Computer Memory Allocated Memory Allocated Memory Allocated Memory Free Memory Free Memory Allocated Memory Allocated Memory Allocated Memory Free Memory Free Memory Parent Process Child Process
30. CoW Friendliness Computer Memory Allocated Memory Allocated Memory Allocated Memory Free Memory Free Memory Allocated Memory Allocated Memory Allocated Memory Free Memory Free Memory Parent Process Child Process
31. CoW Friendliness Computer Memory Allocated Memory Allocated Memory Allocated Memory Free Memory Free Memory Allocated Memory Allocated Memory Allocated Memory Free Memory Free Memory Parent Process Child Process
32. CoW Friendliness Computer Memory Allocated Memory Allocated Memory Allocated Memory Free Memory Free Memory Allocated Memory Allocated Memory Allocated Memory Free Memory Free Memory Parent Process Child Process
33. Eliminating Fragmentation
34. Fragmented Memory Allocated Memory Computer Memory Allocated Memory Allocated Memory Free Memory Free Memory
35. No Fragmentation Allocated Memory Computer Memory Allocated Memory Allocated Memory Free Memory Free Memory
36. Two Heaps
37. Ruby Heaps System Memory Malloc Heap Ruby’s Object Heap
38. Ruby Heaps System Memory Malloc Heap Ruby’s Object Heap Object.new
39. Ruby Heaps System Memory Malloc Heap Ruby’s Object Heap String.new "The Quick Brown Fox Jumps Over The Lazy Dog"
40. Fragmentation Can Occur in Both Heaps
41. For Malloc Heap: jemalloc
42. For Ruby Heap: GC.compact
43. Ruby’s Heap
44. Ruby Heaps System Memory Malloc Heap Ruby’s Object Heap
45. Ruby’s Heap Layout 40 bytes Each chunk is a "slot" Em pty Filled Empty Filled Moved
46. Ruby’s Heap Layout ~16 kb Contiguous slots make a "page"
47. Ruby’s Heap Layout
48. Ruby’s Heap Layout malloc( malloc( malloc( ) ) )
49. Ruby’s Heap Layout Each slot has a unique address 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
50. Compaction Algorithm
51. Two Finger Compaction The Programming Language LISP: Its Operation and Applications (1964)
52. Move Objects Update References
53. Moving Objects A B C D E F Free Scan 1 2 3 4 5 6 7 8 9 10 Empty Filled Moved 4 5
54. Updating References A B C D E F 1 2 3 4 5 6 7 8 9 10 Empty Filled Moved Before Moving Objects
55. Updating References A B C D 5 4 1 2 3 4 5 6 7 8 9 10 Empty Filled F Moved E After Moving Objects
56. Updating References A B C D 5 4 1 2 3 4 5 6 7 8 9 10 Empty Filled F Moved E After Moving Objects
57. Object Movement def compact heap = [ ... ] # many slots left = 0 right = heap.length - 1 while left < right left_slot = heap[left] right_slot = heap[right] if is_empty?(left_slot) && !is_empty?(right_slot) && can_move?(right_slot) swap(left, right) heap[right] = T_MOVED.new(left) # leave forwarding address end while !is_empty?(heap[left]) left += 1 end while is_empty?(heap[right]) || !can_move?(heap[right]) right -= 1 end end end Pointers Met? Copy / Forward Advance "free" Retreat "scan"
58. Reference Updating def update_references heap.each do |slot| next if is_empty?(slot) || is_moved?(slot) slot.references.each_with_index do |child, i| if is_moved?(child) slot.set_reference(i, child.new_location) end end end end How are references stored?
59. Finding References • How do Hashes hold references? • How do Arrays hold references? • How do Objects hold references? • … • … • …
60. Reference Updating static void gc_ref_update_array(rb_objspace_t * objspace, VALUE v) { long i, len; if (FL_TEST(v, ELTS_SHARED)) return; len = RARRAY_LEN(v); if (len > 0) { VALUE *ptr = (VALUE *)RARRAY_CONST_PTR_TRANSIENT(v); for(i = 0; i < len; i++) { UPDATE_IF_MOVED(objspace, ptr[i]); } } } static void gc_ref_update_object(rb_objspace_t * objspace, VALUE v) { uint32_t i, len = ROBJECT_NUMIV(v); VALUE *ptr = ROBJECT_IVPTR(v); for (i = 0; i < len; i++) { UPDATE_IF_MOVED(objspace, ptr[i]); } } static int hash_replace_ref(st_data_t *key, st_data_t *value, st_data_t argp, int existing)
61. Reference Updating
62. Supporting C Extensions
63. Where Are References Stored?
64. Array Array VALUE* Some Object Some Object Some Object
65. Hashes Hash VALUE* Some Object Some Object VALUE* Some Object Some Object Keys Values
66. Strings, Classes, Modules, Symbols, etc
67. GC Can Update All "Known Types"
68. "Known Types" Are Types Implemented by Ruby
69. What About "Unknown Types"?
70. Unknown Types are Types Implemented in C
71. Yajl typedef struct { VALUE builderStack; VALUE parse_complete_callback; int nestedArrayLevel; int nestedHashLevel; int objectsFound; int symbolizeKeys; yajl_handle parser; } yajl_parser_wrapper; C Code (yajl_ext.h) malloc(yajl_parser_wrapper) Ruby Object T_DATA Ruby Object Ruby Object builderStack parse_complete_callback
72. Yajl typedef struct { VALUE builderStack; VALUE parse_complete_callback; int nestedArrayLevel; int nestedHashLevel; int objectsFound; int symbolizeKeys; yajl_handle parser; } yajl_parser_wrapper; C Code (yajl_ext.h) malloc(yajl_parser_wrapper) Ruby Object T_DATA Ruby Object Ruby Object builderStack parse_complete_callback GC: "idk, " MOVED!
73. Yajl Mark Function void yajl_parser_wrapper_mark(void * wrapper) { yajl_parser_wrapper * w = wrapper; if (w) { rb_gc_mark(w->builderStack); rb_gc_mark(w->parse_complete_callback); } } malloc(yajl_parser_wrapper) Ruby Object T_DATA Ruby Object Ruby Object rb_gc_mark(builderStack) rb_gc_mark(parse_complete_callback)
74. Anything Marked With `rb_gc_mark` Cannot Move
75. Pinning Bits 1 2 3 4 5 6 7 8 9 10 Yajl [ ] ? "foo" "bar" ? Address Content Pinned x = [ "foo", "bar" ] y = Yajl.new Ruby Code rb_gc_m ark rb_gc_m ark gc_m ark_no_pin gc_m ark_no_pin
76. Pinning Bits 1 2 3 4 5 6 7 8 9 10 Yajl [ ] ? ? Address Content Pinned x = [ "foo", "bar" ] y = Yajl.new Ruby Code Free Scan "bar" "foo" 4 5 Move Step
77. Pinning Bits 1 2 3 4 5 6 7 8 9 10 Yajl [ ] ? ? Address Content Pinned x = [ "foo", "bar" ] y = Yajl.new Ruby Code 4 5 Reference Update Step "bar" "foo" Update
78. Known Types Use `gc_mark_no_pin`
79. Unknown Types Use `rb_gc_mark`
80. Allowing Movement in C Extensions
81. Compaction Callback "No Pin" Marking New Location Function
82. GC Cannot Update a C Extension
83. C Extension Can Update Itself
84. Compaction Callback static const rb_data_type_t yajl_parser_type = { "Yajl/parser", {yajl_parser_wrapper_mark, yajl_parser_wrapper_free, NULL,}, 0, 0, RUBY_TYPED_FREE_IMMEDIATELY, }; Mark No Compaction Callback static const rb_data_type_t yajl_parser_type = { "Yajl/parser", {yajl_parser_wrapper_mark, yajl_parser_wrapper_free, NULL, yajl_parser_compact}, 0, 0, RUBY_TYPED_FREE_IMMEDIATELY, }; Compact With Compaction Callback Sweep
85. "No Pin" Marking void yajl_parser_wrapper_mark(void * wrapper) { yajl_parser_wrapper * w = wrapper; if (w) { rb_gc_mark(w->builderStack); rb_gc_mark(w->parse_complete_callback); } } No Compaction Support void yajl_parser_wrapper_mark(void * wrapper) { yajl_parser_wrapper * w = wrapper; if (w) { rb_gc_mark_no_pin(w->builderStack); rb_gc_mark_no_pin(w->parse_complete_callback); } } With Compaction Support
86. Compaction Callback void yajl_parser_compact(void *wrapper) { yajl_parser_wrapper * w = wrapper; if (w) { w->builderStack = rb_gc_new_location(w->builderStack); w->parse_complete_callback = rb_gc_new_location(w->parse_complete_callback); } } New Location
87. Known Issue
88. Problem Object Graph Object Implemented in Ruby Object Implemented in C Some Object Automatically Marked!! (gc_mark_no_pin) Not Marked
89. Compaction 1 2 3 4 5 6 7 8 9 10 Ruby Obj C Obj ? 4 5 3
90. Maybe Not Common
91. RubyVM Instruction Sequence ISeq Object (in C) def foo "bar" end Code Mark Array (Ruby) "bar" Marked Marked NOT Marked
92. "Direct Marking" in Ruby 2.6 ISeq Object (in C) def foo "bar" end Code "bar" Marked https://bugs.ruby-lang.org/issues/14370
93. MsgPack Object Implemented in Ruby Object Implemented in C Some Object Automatically Marked!! (gc_mark) Not Marked
94. Pure Ruby Shouldn’t Crash https://github.com/msgpack/msgpack-ruby/issues/133
95. If you hold a reference, you must mark the reference
96. More Challenges Object#object_id
97. Direct Memory Access Prevents Movement
98. object_id is based on location 1 2 3 4 5 6 7 8 9 10 Ruby Obj Ruby Obj Ruby Obj 5 object#object_id => 1 object#object_id => 2 object#object_id => 9 object#object_id => ?
99. Object ID After Move x = Object.new GC.compact x.object_id 1 2 3 4 X x.object_id => 1 Heap
100. Object ID After Move x = Object.new x.object_id GC.compact x.object_id 1 2 3 4 X x.object_id => 4 Heap
101. "Seen" Object IDs $seen_object_id = {} class Object def object_id $seen_object_id[memory_location] ||= memory_location end end
102. Object ID After Move x = Object.new x.object_id GC.compact x.object_id 1 2 3 4 X Heap Object ID Table Memory Location Object ID 4 4 Updated Object ID Table Memory Location Object ID 1 4
103. Object ID Collisions x = Object.new x.object_id GC.compact x.object_id y = Object.new y.object_id 1 2 3 4 X Heap Object ID Table Memory Location Object ID 4 4 Updated Object ID Table Memory Location Object ID 1 4 y.object_id => ??? x.object_id => 4 Y
104. Collision Resolution $seen_object_id = {} $location_to_object_id = {} class Object def object_id id = memory_location while $seen_object_id[id] id += 1 end $seen_object_id[id] = id $location_to_object_id[memory_location] = id end end
105. Object ID Collisions x = Object.new x.object_id GC.compact x.object_id y = Object.new y.object_id 1 2 3 4 X Heap Object ID Table Memory Location Object ID 4 4 Updated Object ID Table Memory Location Object ID 1 4 y.object_id => 5 x.object_id => 4 Y Updated Object ID Table- Memory Location Object ID 1 4 4 5
106. GC Cleanup $seen_object_id = {} $location_to_object_id = {} def free_obj(obj) if $location_to_object_id[obj.memory_location] id = $location_to_object_id.delete(obj.memory_location) $seen_object_id.delete(id) end end
107. Don’t Use Object ID!
109. Compaction Impact Patch Results
110. Basic Rails Application Before Compaction
111. Basic Rails Application After Compaction
112. GitHub Before Compaction After Compaction
113. Future Plans
114. Performance Improvements
115. Full GC Move Objects Update References Full GC
116. Sliding Compaction
117. Sliding Compaction 1 2 3 4 5 6 7 8 9 10 Yajl [ ] ? "foo" "bar" ? Address Content
118. Better Locality
119. Supports Variable Widths
120. Variable Width Allocation
121. END http://bugs.ruby-lang.org/issues/15626