Consistency types of Riak

Slide 1

Slide 1 text

Consistency types in Riak 2014/6/4 Riak Meetup #4 @kuenishi

Slide 2

Slide 2 text

ࣗݾ঺հ •UENISHI Kota @ kuenishi •github, twitter, … •Erlangྺ5೥ɺ෼ࢄγεςϜྺ6೥ •࠷ۙ͸ Riak CS ͷ։ൃ

Slide 3

Slide 3 text

Consistencyɹ੔߹ੑ •ʮNoSQL͸ίϯγεςϯγʔ͕ͳ͍ʯ •ʮSQLʹ͸ίϯγεςϯγʔ͕͋Δʯ •ʢ; °Д°ʣ

Slide 4

Slide 4 text

SQL => ੔߹ੑʁ NO

Slide 5

Slide 5 text

੔߹ੑ㲈 τϥϯβΫγϣϯ ˚

Slide 6

Slide 6 text

੔߹ੑʹෆม৚݅

Slide 7

Slide 7 text

ෆม৚݅ •࿦ཧతͳ੔߹ੑ •ςʔϒϧؒͷ੍໿৚݅ɺΠϯσοΫε •෺ཧతͳ੔߹ੑ •ෳ੡͕ඞͣಉ͡Α͏ʹ؍ଌ͞ΕΔ͔

Slide 8

Slide 8 text

RiakͰ͸ •Siblings •CRDT (2.0~) •Strong Consistency (2.0~)

Slide 9

Slide 9 text

Why Siblings?

Slide 10

Slide 10 text

Consistent Hashing • 160-bit Ωʔۭؒ • ۭؒΛ౳෼͢Δ • ύʔςΟγϣϯ͸ϊʔ υ͕ݸผ؅ཧ • ϨϓϦΧ͸Nݸͷύʔ ςΟγϣϯʹίϐʔ͞ ΕΔ OPEF OPEF OPEF OPEF hash(“meetups/spamham”) N=3

Slide 11

Slide 11 text

CAPఆཧͱཧ૝ͷDB •ͲΜͳނোʹରͯ͠΋ (partition tolerance) •σʔλ͸ৗʹ੔߹͓ͯ͠Γ (consistency) •γεςϜ͕ࢭ·Δ͜ͱ͸ͳ͍ (availability) ͜ͷ3ͭΛಉ࣌ʹຬͨ͢γεςϜ͸ଘࡏ͠ͳ͍

Slide 12

Slide 12 text

Consistency͸೉͍͠ •ߋ৽ΛࢭΊΔʢAvailabilityΛԼ͛Δʣ͔ɺߋ৽ͷ্ॻ͖Λ ڐ͢ʢσʔλΛࣦ͏ʣ͔͔͠બ୒ࢶ͕ͳ͍ Server2 Server1 Server3 PUT V=42 PUT V=0 V=?

Slide 13

Slide 13 text

Consistencyͷ୅ΘΓʹ •ͱΓ͋͑ͣෳ਺ͷόʔδϣϯͷڞଘΛڐ͢ •Ͳͷόʔδϣϯ͕ਖ਼͍͔͠ɺ΋͘͠͸Ϛʔδ͢Δ͔ΛRead࣌ʹܾఆ Server2 Server1 Server3 PUT V=42 PUT V=0 V=0 or 42 V=0 V=0 or 42 V=42

Slide 14

Slide 14 text

APΛ࣮ݱ •ωοτϫʔΫ෼அ͕ى͖͍ͯͯ΋ͱΓ͋͑ͣॻ͖ࠐΈΛڐ͢ Server2 Server1 Server3 PUT V=42 PUT V=0 Server4 ෮چͨ͠Βॻ͖໭͢ ྆ํ͓࣋ͬͯ͘

Slide 15

Slide 15 text

γϣοϐϯάΧʔτͷྫ •UnionΛͱΕ͹Α͍ Server2 Server1 Server3 PUT cart=[a,b,d] PUT cart=[a,b,c] union([a,b,c], [a,b,d]) => [a,b,c,d] [a,b,c] [a,b,c] or [a,b,d] [a,b,d]

Slide 16

Slide 16 text

Siblings riak_object = Riak.fetch(bucket, key)! riak_object.version! riak_object.has_siblings! for value in riak_object.values: …! riak_object.data = new_value! riak_object.store!

Slide 17

Slide 17 text

Siblings ͷෆม৚݅ •ಛʹͳ͠…͍͋͑ͯ͏ͱϨϓϦΧͷ࿨ू߹ •Data = R1 ∪ R2 ∪ R3

Slide 18

Slide 18 text

ෳ਺όʔδϣϯΛ ڐ͢͜ͱͷ೉఺ •ϓϩάϥϛϯά͕೉͍͠ʢτϥϯβΫγϣϯ͸ૉ੖ Β͍͠ʣ •ݱ࣮ੈք͸γϣοϐϯάΧʔτͱΧ΢ϯλʔ͚ͩ Ͱ͸ͳ͍ •҆શͳMerge, update͕Ͱ͖Δσʔλߏ଄Λຖճ ߟ͑ͳ͚Ε͹ͳΒͳ͍ •࢖͍ͬͯΔ͏ͪʹࣅͨΑ͏ͳϥΠϒϥϦ͕͋ͪ͜ ͪͰग़དྷ্͕Δ

Slide 19

Slide 19 text

ͳͥ೉͍͠ͷ͔ʁ •σʔλͷWriteͱWrite͕ೖΕସΘΓ͏ ΔʹSerializableͲ͜Ζ͔Write΋Ұ؏ ͨ͠ঢ়ଶʹͰ͖ͳ͍ Server2 Server1 Server3 w1 w2 w1 w2 w2 (w1 lost)

Slide 20

Slide 20 text

౴͑: CRDT •ʮෳ੡ՄೳͳՄ׵σʔλܕʯ •Conﬂict-Free Replicated Data Types •Commutative Replicated Data Types •… •(Going to be included in Riak 2.0) ஫) CRDTͷ࡞ऀ͸Logical Monotinicy ͱ͍͏ݴ༿͸࢖͍ͬͯͳ͍

Slide 21

Slide 21 text

CRDTͷෆม৚݅ •σʔλʹର͢ΔՄ׵ͳૢ࡞ͷΈΛڐ͢ʂ Data = update(w2, update(w1, Data0)) = update(w1, update(w2, Data0)) Data = merge(update(w2, Data0), Data)

Slide 22

Slide 22 text

CRDT in Riak 2.0 •KVSͷVʹʮܕʯΛ࣋ͨͤͯɺܕʹΑͬͯ UpdateͱMergeͷϩδοΫΛܾΊΔ •Read࣌ʹMerge͕αʔόʔଆͰࣗಈతʹ࣮ ߦ͞ΕΔ •ΞϓϦέʔγϣϯ͸ܕΛࢦఆ͢Δ͚ͩͰΑ͘ɺ ෳ਺όʔδϣϯͷϋϯυϦϯά͕ෆཁʹͳΔ

Slide 23

Slide 23 text

CRDT riak_object = Riak.fetch(bucket, key)! riak_object.type => counter|set|…! riak_object.set << element! riak_object.set.delete(old_element)! riak_object.store!

Slide 24

Slide 24 text

CRDT example •PN-Counter •Set •OR-sets •LWW-register •Graph…

Slide 25

Slide 25 text

PN-Counter • merge • {a: {1,-1}, b: {1,0}, c: {2,0}} • {a: {0,0}, b: {2, 0}, c: {0, -2}} • => {a: {1,-1}, b:{2,0}, c:{2,-2}} => 2 • update • a͕ {increment, 3} Λड͚෇͚Δͱ • {a: {4,-1}, b: {1,0}, c: {2,0}}

Slide 26

Slide 26 text

OR-Sets • merge • {a:{“foo”:true}, b:{“bar”:false}} • + {a:{“foo”:true}, b:{“foo”:false, “bar”:false}} • => {a:{“foo”:true}, b:{“foo”:false, “bar”:true}} • => [“bar”] • update • add: {a:{}} => +”foo” => {a:{“foo”:false}} • remove: {a: {“foo”:false}} => {a: {“foo”:true}}

Slide 27

Slide 27 text

Ϣʔεέʔε •ΫϦοΫ਺ͷΧ΢ϯτ (G-counter) • riak-server/types/counters/buckets/likes/datatypes/basho.com -d 1 •γϣοϐϯάΧʔτ (OR-sets) •ϩάΠϯϢʔβʔ਺ (PN-counter) •͜ΕΒͷ૊Έ߹Θͤ (map & LWW-register, boolean) •{ name : “basho.com”, likes: 20000, users: 3000, links: [ “basho.co.jp”, “basho.co.uk” ], cool: true }

Slide 28

Slide 28 text

Ͱ͖ͳ͍͜ͱ •ʮ0Ҏ্ʯͷPN-counter •ϢχʔΫͳIDൃߦ •ͦͷଞCAS͕ඞཁͳσʔλߏ଄ͱૢ࡞ •RESTfulͳૢ࡞

Slide 29

Slide 29 text

Strong Consistency •ෆม৚݅: t(w1) > t(w2) or t(w2) > t(w1) •Sequencial Consistencyʹ͍ۙ Riak c1 c2 get get v1 v1 w1(v1) ok w2(v1) fail

Slide 30

Slide 30 text

Strong Consistency •͍ͪͲGETͯ͠ɺͦͷόʔδϣϯʹର͢ Δૢ࡞ΛૹΔ •ͦͷؒʹߋ৽͞Ε͍ͯͨΒࣦഊɺ੒ޭ •MultiPaxos

Slide 31

Slide 31 text

Strong Consistency do {! riak_object = Riak.fetch(bucket, key)! riak_object.data = new_value! } while (riak_object.store != ok)

Slide 32

Slide 32 text

·ͱΊ •Riak 1.x ͸Մ༻ੑͷ͋Δ෼ࢄσʔλϕʔε •RESTfulͳઃܭΛ͢ΔͳΒSiblings •2.0 ͔Β͸ෳ਺ͷ੔߹ੑϞσϧΛબ୒Ͱ͖ΔΑ ͏ʹͳͬͨ •ΞϓϦΛ؆୯ʹ࡞Γ͍ͨͳΒCRDT •CASతߋ৽Λ͍ͨ͠ͳΒStrong Consistency

Slide 33

Slide 33 text

Questions? •Riak 2.0 Λָ͠Έʹ͍ͯͩ͘͠͞ •Web: http://basho.co.jp •Twitter: @BashoJapan •Me: [email protected] •ML: [email protected]