Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Consistency types of Riak
Search
UENISHI Kota
June 04, 2014
Technology
2
340
Consistency types of Riak
Riakの整合性は2.0からもっといろいろ調整できるようになるので、その解説(25分)
UENISHI Kota
June 04, 2014
Tweet
Share
More Decks by UENISHI Kota
See All by UENISHI Kota
Metadata Management in Distributed File Systems
kuenishi
2
410
Behind The Scenes: Cloud Native Storage System for AI
kuenishi
2
270
Apache Ozone behind Simulation and AI Industries
kuenishi
0
210
Distributed Deep Learning with Chainer and Hadoop
kuenishi
3
990
A Few Ways to Accelerate Deep Learning
kuenishi
0
940
Introducing Retz
kuenishi
5
920
Introducing Retz and how to develop practical frameworks
kuenishi
3
590
Formalization and Proof of Distributed Systems (ja)
kuenishi
10
6.1k
Mesos Frameworkの作り方 (How to Make Mesos Framework)
kuenishi
7
2.2k
Other Decks in Technology
See All in Technology
Azure犬駆動開発の記録/GlobalAzureFukuoka2024_20240420
nina01
1
210
反実仮想機械学習とは何か
usaito
PRO
11
4.6k
KubeConにproposalを送りたい人へのアドバイス
sat
PRO
3
250
web-application-security
matsuihidetoshi
0
170
エンジニア候補者向け資料2024.04.24.pdf
macloud
0
3.3k
Databricks における 『MLOps』
databricksjapan
2
170
APIファーストなプロダクトマネジメントの実践 〜SaaSus Platformでの例〜 / "Practicing API-First Product Management - An Example with SaaSus Platform
oztick139
0
100
Google Cloud Next '24でブログを10本書いた方法と勉強会を沸かせた方法
yasumuusan
0
290
推しは推せるときに推せ! プロダクトにフィードバックしていこう
nakasho
0
310
リテール金融(キャッシュレス・ネット銀行・ネット証券)の競争環境と経済圏
8maki
0
1.1k
MySQL の SQL クエリチューニングの要所を掴む勉強会
andpad
3
6.3k
本当のAWS基礎
toru_kubota
0
520
Featured
See All Featured
In The Pink: A Labor of Love
frogandcode
138
21k
Build The Right Thing And Hit Your Dates
maggiecrowley
24
2k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
659
120k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
Optimising Largest Contentful Paint
csswizardry
8
2.4k
Keith and Marios Guide to Fast Websites
keithpitt
408
22k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
322
20k
The Pragmatic Product Professional
lauravandoore
25
5.8k
Build your cross-platform service in a week with App Engine
jlugia
225
17k
The Art of Programming - Codeland 2020
erikaheidi
42
12k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
116
18k
Product Roadmaps are Hard
iamctodd
44
9.7k
Transcript
Consistency types in Riak 2014/6/4 Riak Meetup #4 @kuenishi
ࣗݾհ •UENISHI Kota @ kuenishi •github, twitter, … •Erlangྺ5ɺࢄγεςϜྺ6 •࠷ۙ
Riak CS ͷ։ൃ
Consistencyɹ߹ੑ •ʮNoSQLίϯγεςϯγʔ͕ͳ͍ʯ •ʮSQLʹίϯγεςϯγʔ͕͋Δʯ •ʢ; °Д°ʣ
SQL => ߹ੑʁ NO
߹ੑ㲈 τϥϯβΫγϣϯ ˚
߹ੑʹෆม݅
ෆม݅ •ཧతͳ߹ੑ •ςʔϒϧؒͷ੍݅ɺΠϯσοΫε •ཧతͳ߹ੑ •ෳ͕ඞͣಉ͡Α͏ʹ؍ଌ͞ΕΔ͔
RiakͰ •Siblings •CRDT (2.0~) •Strong Consistency (2.0~)
Why Siblings?
Consistent Hashing • 160-bit Ωʔۭؒ • ۭؒΛ͢Δ • ύʔςΟγϣϯϊʔ υ͕ݸผཧ
• ϨϓϦΧNݸͷύʔ ςΟγϣϯʹίϐʔ͞ ΕΔ OPEF OPEF OPEF OPEF hash(“meetups/spamham”) N=3
CAPఆཧͱཧͷDB •ͲΜͳނোʹରͯ͠ (partition tolerance) •σʔλৗʹ߹͓ͯ͠Γ (consistency) •γεςϜ͕ࢭ·Δ͜ͱͳ͍ (availability) ͜ͷ3ͭΛಉ࣌ʹຬͨ͢γεςϜଘࡏ͠ͳ͍
Consistency͍͠ •ߋ৽ΛࢭΊΔʢAvailabilityΛԼ͛Δʣ͔ɺߋ৽ͷ্ॻ͖Λ ڐ͢ʢσʔλΛࣦ͏ʣ͔͔͠બࢶ͕ͳ͍ Server2 Server1 Server3 PUT V=42 PUT V=0
V=?
ConsistencyͷΘΓʹ •ͱΓ͋͑ͣෳͷόʔδϣϯͷڞଘΛڐ͢ •Ͳͷόʔδϣϯ͕ਖ਼͍͔͠ɺ͘͠Ϛʔδ͢Δ͔ΛRead࣌ʹܾఆ Server2 Server1 Server3 PUT V=42 PUT V=0
V=0 or 42 V=0 V=0 or 42 V=42
APΛ࣮ݱ •ωοτϫʔΫஅ͕ى͖͍ͯͯͱΓ͋͑ͣॻ͖ࠐΈΛڐ͢ Server2 Server1 Server3 PUT V=42 PUT V=0 Server4
෮چͨ͠Βॻ͖͢ ྆ํ͓࣋ͬͯ͘
γϣοϐϯάΧʔτͷྫ •UnionΛͱΕΑ͍ Server2 Server1 Server3 PUT cart=[a,b,d] PUT cart=[a,b,c] union([a,b,c],
[a,b,d]) => [a,b,c,d] [a,b,c] [a,b,c] or [a,b,d] [a,b,d]
Siblings riak_object = Riak.fetch(bucket, key)! riak_object.version! riak_object.has_siblings! for value in
riak_object.values: …! riak_object.data = new_value! riak_object.store!
Siblings ͷෆม݅ •ಛʹͳ͠…͍͋͑ͯ͏ͱϨϓϦΧͷू߹ •Data = R1 ∪ R2 ∪ R3
ෳόʔδϣϯΛ ڐ͢͜ͱͷ •ϓϩάϥϛϯά͕͍͠ʢτϥϯβΫγϣϯૉ Β͍͠ʣ •ݱ࣮ੈքγϣοϐϯάΧʔτͱΧϯλʔ͚ͩ Ͱͳ͍ •҆શͳMerge, update͕Ͱ͖ΔσʔλߏΛຖճ ߟ͑ͳ͚ΕͳΒͳ͍ •͍ͬͯΔ͏ͪʹࣅͨΑ͏ͳϥΠϒϥϦ͕͋ͪ͜
ͪͰग़དྷ্͕Δ
ͳ͍ͥ͠ͷ͔ʁ •σʔλͷWriteͱWrite͕ೖΕସΘΓ͏ ΔʹSerializableͲ͜Ζ͔WriteҰ؏ ͨ͠ঢ়ଶʹͰ͖ͳ͍ Server2 Server1 Server3 w1 w2 w1
w2 w2 (w1 lost)
͑: CRDT •ʮෳՄೳͳՄσʔλܕʯ •Conflict-Free Replicated Data Types •Commutative Replicated Data
Types •… •(Going to be included in Riak 2.0) ) CRDTͷ࡞ऀLogical Monotinicy ͱ͍͏ݴ༿͍ͬͯͳ͍
CRDTͷෆม݅ •σʔλʹର͢ΔՄͳૢ࡞ͷΈΛڐ͢ʂ Data = update(w2, update(w1, Data0)) = update(w1, update(w2,
Data0)) Data = merge(update(w2, Data0), Data)
CRDT in Riak 2.0 •KVSͷVʹʮܕʯΛ࣋ͨͤͯɺܕʹΑͬͯ UpdateͱMergeͷϩδοΫΛܾΊΔ •Read࣌ʹMerge͕αʔόʔଆͰࣗಈతʹ࣮ ߦ͞ΕΔ •ΞϓϦέʔγϣϯܕΛࢦఆ͢Δ͚ͩͰΑ͘ɺ ෳόʔδϣϯͷϋϯυϦϯά͕ෆཁʹͳΔ
CRDT riak_object = Riak.fetch(bucket, key)! riak_object.type => counter|set|…! riak_object.set <<
element! riak_object.set.delete(old_element)! riak_object.store!
CRDT example •PN-Counter •Set •OR-sets •LWW-register •Graph…
PN-Counter • merge • {a: {1,-1}, b: {1,0}, c: {2,0}}
• {a: {0,0}, b: {2, 0}, c: {0, -2}} • => {a: {1,-1}, b:{2,0}, c:{2,-2}} => 2 • update • a͕ {increment, 3} Λड͚͚Δͱ • {a: {4,-1}, b: {1,0}, c: {2,0}}
OR-Sets • merge • {a:{“foo”:true}, b:{“bar”:false}} • + {a:{“foo”:true}, b:{“foo”:false,
“bar”:false}} • => {a:{“foo”:true}, b:{“foo”:false, “bar”:true}} • => [“bar”] • update • add: {a:{}} => +”foo” => {a:{“foo”:false}} • remove: {a: {“foo”:false}} => {a: {“foo”:true}}
Ϣʔεέʔε •ΫϦοΫͷΧϯτ (G-counter) • riak-server/types/counters/buckets/likes/datatypes/basho.com -d 1 •γϣοϐϯάΧʔτ (OR-sets) •ϩάΠϯϢʔβʔ
(PN-counter) •͜ΕΒͷΈ߹Θͤ (map & LWW-register, boolean) •{ name : “basho.com”, likes: 20000, users: 3000, links: [ “basho.co.jp”, “basho.co.uk” ], cool: true }
Ͱ͖ͳ͍͜ͱ •ʮ0Ҏ্ʯͷPN-counter •ϢχʔΫͳIDൃߦ •ͦͷଞCAS͕ඞཁͳσʔλߏͱૢ࡞ •RESTfulͳૢ࡞
Strong Consistency •ෆม݅: t(w1) > t(w2) or t(w2) > t(w1)
•Sequencial Consistencyʹ͍ۙ Riak c1 c2 get get v1 v1 w1(v1) ok w2(v1) fail
Strong Consistency •͍ͪͲGETͯ͠ɺͦͷόʔδϣϯʹର͢ Δૢ࡞ΛૹΔ •ͦͷؒʹߋ৽͞Ε͍ͯͨΒࣦഊɺޭ •MultiPaxos
Strong Consistency do {! riak_object = Riak.fetch(bucket, key)! riak_object.data =
new_value! } while (riak_object.store != ok)
·ͱΊ •Riak 1.x Մ༻ੑͷ͋Δࢄσʔλϕʔε •RESTfulͳઃܭΛ͢ΔͳΒSiblings •2.0 ͔Βෳͷ߹ੑϞσϧΛબͰ͖ΔΑ ͏ʹͳͬͨ •ΞϓϦΛ؆୯ʹ࡞Γ͍ͨͳΒCRDT •CASతߋ৽Λ͍ͨ͠ͳΒStrong
Consistency
Questions? •Riak 2.0 Λָ͠Έʹ͍ͯͩ͘͠͞ •Web: http://basho.co.jp •Twitter: @BashoJapan •Me:
[email protected]
•ML:
[email protected]