Mark Compact GC in MRI - Speaker Deck

Slide 1

Slide 1 text

How people build software ! " Mark Compact GC in MRI

Slide 2

Slide 2 text

How people build software ! ͜Μʹͪ͸ʂʂʂ 2

Slide 3

Slide 3 text

How people build software ! ࣗݾ঺հ 3

Slide 4

Slide 4 text

How people build software ! ΞʔϩϯͰ͢ɻ! 4

Slide 5

Slide 5 text

How people build software ! ˏtenderlove 5

Slide 6

Slide 6 text

How people build software ! ! ˏtenderlove 6

Slide 7

Slide 7 text

How people build software ! ఱက͕େ޷͖Ͱɺ 7 ఱကϥϒΛݺΜͰ΋͍͍Ͱ͢

Slide 8

Slide 8 text

How people build software ! Ruby Core Team Rails Core Team 8

Slide 9

Slide 9 text

How people build software ! ೔ຊޠೳྗࢼݧ 9

Slide 10

Slide 10 text

How people build software ! ೣɺೋඖ 10

Slide 11

Slide 11 text

How people build software ! 11 SEATAC Airport Facebook YouTube

Slide 12

Slide 12 text

How people build software ! 12

Slide 13

Slide 13 text

How people build software ! 13 Gorbachev Puff Puff Thunderhorse ΰϒͪΌΜ

Slide 14

Slide 14 text

How people build software ! GitHub ͷࣾһ 14

Slide 15

Slide 15 text

How people build software ! Ruby & Rails Developer 15

Slide 16

Slide 16 text

How people build software ! GitHub & OSS 16

Slide 17

Slide 17 text

How people build software ! ! GitHub Core Values 17 • Collaboration • Empathy • Quality • Positive Impact • Shipping

Slide 18

Slide 18 text

How people build software ! Collaboration 18 !

Slide 19

Slide 19 text

How people build software ! Empathy 19 !

Slide 20

Slide 20 text

How people build software ! Quality 20 !

Slide 21

Slide 21 text

How people build software ! Positive Impact 21 !

Slide 22

Slide 22 text

How people build software ! Shipping 22 !

Slide 23

Slide 23 text

How people build software ! GitHub ❤ OSS 23 !

Slide 24

Slide 24 text

How people build software ! GitHub & OSS 24 !

Slide 25

Slide 25 text

How people build software ! Mark / Compact GC 25

Slide 26

Slide 26 text

How people build software ! Mark / Compact GC 26

Slide 27

Slide 27 text

How people build software ! ! Roadmap 27 Ξϩέʔγϣϯ ίϨΫγϣϯ ίϯύΫτ

Slide 28

Slide 28 text

How people build software ! Ξϩέʔγϣϯ 28

Slide 29

Slide 29 text

How people build software ! ! Ξϩέʔγϣϯ 29 def hoge loop do Object.new end end hoge

Slide 30

Slide 30 text

How people build software ! Ξϩέʔγϣϯ΋ GCͷ੹೚ 30

Slide 31

Slide 31 text

How people build software ! `rb_newobj` in gc.c 31

Slide 32

Slide 32 text

How people build software ! ! ϝϞϦྖҬͷϨΠΞ΢τ 32 ώʔϓྖҬ ϖʔδ ϖʔδ ϖʔδ ϖʔδ ϖʔδ ϖʔδ ϖʔδ ϖʔδ Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj

Slide 33

Slide 33 text

How people build software ! Free Space = "Fast Path" 33

Slide 34

Slide 34 text

How people build software ! New Page = "Slow Path" 34

Slide 35

Slide 35 text

How people build software ! ! ϖʔδͷϨΠΞ΢τ 35 ϖʔδ FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE SLOTS Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj

Slide 36

Slide 36 text

How people build software ! ! ϖʔδͷϨΠΞ΢τ 36 ϖʔδ FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE SLOTS Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj

Slide 37

Slide 37 text

How people build software ! ཁ໿ 37

Slide 38

Slide 38 text

How people build software ! heap has_many :pages 38

Slide 39

Slide 39 text

How people build software ! page has_many :slots 39

Slide 40

Slide 40 text

How people build software ! slot.is_a?(Empty) || slot.is_a?(Object) 40

Slide 41

Slide 41 text

How people build software ! ίϨΫγϣϯ 41

Slide 42

Slide 42 text

How people build software ! ϚʔΫεΠʔϓ GC Mark Sweep GC 42 !

Slide 43

Slide 43 text

How people build software ! ϚʔΫεΠʔϓͷ ΞϧΰϦζϜ 43

Slide 44

Slide 44 text

How people build software ! ! Tree of Objects 44 Root A B C D a = [ { c: 'd' } ] Ruby Array Hash String Symbol

Slide 45

Slide 45 text

How people build software ! ! Tree of Objects 45 Root A B C D a = [ { c: 'd' } ] a = nil Ruby

Slide 46

Slide 46 text

How people build software ! ϚʔΫɹεΠʔϓ 46 ͔ͩΒʮϚʔΫεΠʔϓʯɹ tenderlove

Slide 47

Slide 47 text

How people build software ! ! Mark Phase 47 Root D A B C F E H G

Slide 48

Slide 48 text

How people build software ! ! Mark Phase 48 Root D A B C F E H G

Slide 49

Slide 49 text

How people build software ! ! Mark Phase 49 Root D A B C F E H G

Slide 50

Slide 50 text

How people build software ! ! Mark Phase 50 Root D A B C F E H G

Slide 51

Slide 51 text

How people build software ! Incremental / Lazy 51

Slide 52

Slide 52 text

How people build software ! ίϯύΫγϣϯͷؔ܎ 52

Slide 53

Slide 53 text

How people build software ! ! Free Space 53 ϖʔδ FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj ϖʔδ FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj ϖʔδ FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj

Slide 54

Slide 54 text

How people build software ! ! Free Space 54 FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj

Slide 55

Slide 55 text

How people build software ! ! Free Space 55 FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE Obj Obj Obj Obj Obj Obj FREE FREE FREE FREE FREE FREE FREE FREE FREE FREE Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj Obj

Slide 56

Slide 56 text

How people build software ! ΦϒδΣΫτ͸Ҡಈग़དྷͳ͍ 56

Slide 57

Slide 57 text

How people build software ! ίϯύΫγϣϯͱ͸ 57

Slide 58

Slide 58 text

How people build software ! github.com/github/ruby/tree/gc-compact 58

Slide 59

Slide 59 text

How people build software ! ίϯύΫγϣϯͷ ΞϧΰϦζϜ 59

Slide 60

Slide 60 text

How people build software ! "2 Finger Compaction" 60 ☝ ☝

Slide 61

Slide 61 text

How people build software ! ! Compaction Steps 61 ΦϒδΣΫτͷҠಈ ϦϑΝϨϯεͷߋ৽

Slide 62

Slide 62 text

How people build software ! ! ΦϒδΣΫτͷҠಈ 62 1 2 3 4 5 6 7 8 9 a b Free Free Free Obj Free Obj Obj Obj Free Free Obj ☝ ☝ Free Pointer Scan Pointer 1 2 3 5 Done! Address Heap

Slide 63

Slide 63 text

How people build software ! ! ΦϒδΣΫτҠಈͷલ 63 1 2 3 4 5 6 7 8 9 a b Free Free Free Obj Free Obj Obj Obj Free Free Obj Address Heap a = { c: 'd' } Ruby {} :c 'd'

Slide 64

Slide 64 text

How people build software ! ! ΦϒδΣΫτҠಈͷޙ 64 1 2 3 4 5 6 7 8 9 a b Obj Free Obj Obj Obj Free Free Obj Address Heap a = { c: 'd' } Ruby {} :c 'd' 1 2 3 5

Slide 65

Slide 65 text

How people build software ! ! ϦϑΝϨϯεͷߋ৽ 65 1 2 3 4 5 6 7 8 9 a b Obj Obj Obj Obj Free Free Obj Address Heap a = { c: 'd' } Ruby {} :c 'd' 1 2 3 5 ☝ Done!

Slide 66

Slide 66 text

How people build software ! ! ϦϑΝϨϯεͷߋ৽ 66 1 2 3 4 5 6 7 8 9 a b Obj Obj Obj Obj Free Free Obj Address Heap a = { c: 'd' } Ruby {} :c 'd' 1 2 3 5 Free Free Free Free

Slide 67

Slide 67 text

How people build software ! ίϯύΫτ׬ྃʂʂ 67

Slide 68

Slide 68 text

How people build software ! ίϯύΫγϣϯͷ໰୊ 68

Slide 69

Slide 69 text

How people build software ! ! ී௨ͷϦϑΝϨϯε 69 Hash Symbol String C code

Slide 70

Slide 70 text

How people build software ! ! C֦ுͷϦϑΝϨϯε 70 Hash Symbol String C code rb_gc_mark ෆҠಈ Ҡಈ Ҡ ಈ

Slide 71

Slide 71 text

How people build software ! άϩόϧม਺ 71

Slide 72

Slide 72 text

How people build software ! ! ﬁle.c in MRI 72 separator = rb_obj_freeze(rb_usascii_str_new2("/")); /* separates directory parts in path */ rb_define_const(rb_cFile, "Separator", separator); rb_define_const(rb_cFile, "SEPARATOR", separator); static VALUE rb_file_s_join(VALUE klass, VALUE args) { return rb_file_join(args, separator); } άϩόϧม਺

Slide 73

Slide 73 text

How people build software ! ! άϩόϧม਺ - SEGV 73 $ ruby -e'File::Separator = File::SEPARATOR = nil; GC.start; File.join("a", "b")' # BOOM $ jruby -e'File::Separator = File::SEPARATOR = nil; GC.start; File.join("a", "b")'

Slide 74

Slide 74 text

How people build software ! General Problem 74

Slide 75

Slide 75 text

How people build software ! ݁Ռ 75

Slide 76

Slide 76 text

How people build software ! Rails Heap 76

Slide 77

Slide 77 text

How people build software ! ! Compaction Code 77 GC.start File.open("pre_compact.json", "wb") { |f| ObjectSpace.dump_all(output: f) } GC.start p GC.compact GC.start File.open("post_compact.json", "wb") { |f| ObjectSpace.dump_all(output: f) } $ ruby bin/rails r script.rb

Slide 78

Slide 78 text

ίϯύΫτͷલ F U P Pages Number of slots 0 100 200 300 400

Slide 79

Slide 79 text

ίϯύΫτͷޙ F U P Pages Number of slots 0 100 200 300 400

Slide 80

Slide 80 text

How people build software ! ίϯύΫτͷલ: 528 / 552 80

Slide 81

Slide 81 text

How people build software ! ίϯύΫτͷޙ: 167 / 552 81

Slide 82

Slide 82 text

How people build software ! Ͱ΋ɺվળͰ͖·͢ 82

Slide 83

Slide 83 text

How people build software ! GitHub Heap Graphs 83

Slide 84

Slide 84 text

No content

Slide 85

Slide 85 text

How people build software ! ·ͱΊ 85 !

Slide 86

Slide 86 text

How people build software ! Why Compact? 86 !

Slide 87

Slide 87 text

How people build software ! OSS Software https://github.com/github/ruby/tree/gc-compact 87 !