Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Consistency types of Riak
Search
UENISHI Kota
June 04, 2014
Technology
2
350
Consistency types of Riak
Riakの整合性は2.0からもっといろいろ調整できるようになるので、その解説(25分)
UENISHI Kota
June 04, 2014
Tweet
Share
More Decks by UENISHI Kota
See All by UENISHI Kota
Storage Systems in Preferred Networks
kuenishi
0
3
Metadata Management in Distributed File Systems
kuenishi
2
480
Behind The Scenes: Cloud Native Storage System for AI
kuenishi
2
360
Apache Ozone behind Simulation and AI Industries
kuenishi
0
330
Distributed Deep Learning with Chainer and Hadoop
kuenishi
3
1.2k
A Few Ways to Accelerate Deep Learning
kuenishi
0
1k
Introducing Retz
kuenishi
5
1.1k
Introducing Retz and how to develop practical frameworks
kuenishi
3
700
Formalization and Proof of Distributed Systems (ja)
kuenishi
10
6.3k
Other Decks in Technology
See All in Technology
Two Blades, One Journey: Engineering While Managing
ohbarye
4
1.9k
AIエージェント入門
minorun365
PRO
31
17k
2/18 Making Security Scale: メルカリが考えるセキュリティ戦略 - Coincheck x LayerX x Mercari
jsonf
0
190
Active Directory攻防
cryptopeg
PRO
8
5.4k
PHPカンファレンス名古屋-テックリードの経験から学んだ設計の教訓
hayatokudou
2
540
株式会社Awarefy(アウェアファイ)会社説明資料 / Awarefy-Company-Deck
awarefy
3
11k
AI Agent時代なのでAWSのLLMs.txtが欲しい!
watany
2
220
遷移の高速化 ヤフートップの試行錯誤
narirou
6
1.1k
MIMEと文字コードの闇
hirachan
2
1.4k
依存パッケージの更新はコツコツが勝つコツ! / phpcon_nagoya2025
blue_goheimochi
3
210
大規模アジャイルフレームワークから学ぶエンジニアマネジメントの本質
staka121
PRO
3
1.1k
LINE NEWSにおけるバックエンド開発
lycorptech_jp
PRO
0
230
Featured
See All Featured
BBQ
matthewcrist
87
9.5k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
49k
We Have a Design System, Now What?
morganepeng
51
7.4k
The Power of CSS Pseudo Elements
geoffreycrofte
75
5.5k
Practical Orchestrator
shlominoach
186
10k
The Pragmatic Product Professional
lauravandoore
32
6.4k
YesSQL, Process and Tooling at Scale
rocio
172
14k
Fireside Chat
paigeccino
34
3.2k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
30
4.6k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Documentation Writing (for coders)
carmenintech
67
4.6k
Transcript
Consistency types in Riak 2014/6/4 Riak Meetup #4 @kuenishi
ࣗݾհ •UENISHI Kota @ kuenishi •github, twitter, … •Erlangྺ5ɺࢄγεςϜྺ6 •࠷ۙ
Riak CS ͷ։ൃ
Consistencyɹ߹ੑ •ʮNoSQLίϯγεςϯγʔ͕ͳ͍ʯ •ʮSQLʹίϯγεςϯγʔ͕͋Δʯ •ʢ; °Д°ʣ
SQL => ߹ੑʁ NO
߹ੑ㲈 τϥϯβΫγϣϯ ˚
߹ੑʹෆม݅
ෆม݅ •ཧతͳ߹ੑ •ςʔϒϧؒͷ੍݅ɺΠϯσοΫε •ཧతͳ߹ੑ •ෳ͕ඞͣಉ͡Α͏ʹ؍ଌ͞ΕΔ͔
RiakͰ •Siblings •CRDT (2.0~) •Strong Consistency (2.0~)
Why Siblings?
Consistent Hashing • 160-bit Ωʔۭؒ • ۭؒΛ͢Δ • ύʔςΟγϣϯϊʔ υ͕ݸผཧ
• ϨϓϦΧNݸͷύʔ ςΟγϣϯʹίϐʔ͞ ΕΔ OPEF OPEF OPEF OPEF hash(“meetups/spamham”) N=3
CAPఆཧͱཧͷDB •ͲΜͳނোʹରͯ͠ (partition tolerance) •σʔλৗʹ߹͓ͯ͠Γ (consistency) •γεςϜ͕ࢭ·Δ͜ͱͳ͍ (availability) ͜ͷ3ͭΛಉ࣌ʹຬͨ͢γεςϜଘࡏ͠ͳ͍
Consistency͍͠ •ߋ৽ΛࢭΊΔʢAvailabilityΛԼ͛Δʣ͔ɺߋ৽ͷ্ॻ͖Λ ڐ͢ʢσʔλΛࣦ͏ʣ͔͔͠બࢶ͕ͳ͍ Server2 Server1 Server3 PUT V=42 PUT V=0
V=?
ConsistencyͷΘΓʹ •ͱΓ͋͑ͣෳͷόʔδϣϯͷڞଘΛڐ͢ •Ͳͷόʔδϣϯ͕ਖ਼͍͔͠ɺ͘͠Ϛʔδ͢Δ͔ΛRead࣌ʹܾఆ Server2 Server1 Server3 PUT V=42 PUT V=0
V=0 or 42 V=0 V=0 or 42 V=42
APΛ࣮ݱ •ωοτϫʔΫஅ͕ى͖͍ͯͯͱΓ͋͑ͣॻ͖ࠐΈΛڐ͢ Server2 Server1 Server3 PUT V=42 PUT V=0 Server4
෮چͨ͠Βॻ͖͢ ྆ํ͓࣋ͬͯ͘
γϣοϐϯάΧʔτͷྫ •UnionΛͱΕΑ͍ Server2 Server1 Server3 PUT cart=[a,b,d] PUT cart=[a,b,c] union([a,b,c],
[a,b,d]) => [a,b,c,d] [a,b,c] [a,b,c] or [a,b,d] [a,b,d]
Siblings riak_object = Riak.fetch(bucket, key)! riak_object.version! riak_object.has_siblings! for value in
riak_object.values: …! riak_object.data = new_value! riak_object.store!
Siblings ͷෆม݅ •ಛʹͳ͠…͍͋͑ͯ͏ͱϨϓϦΧͷू߹ •Data = R1 ∪ R2 ∪ R3
ෳόʔδϣϯΛ ڐ͢͜ͱͷ •ϓϩάϥϛϯά͕͍͠ʢτϥϯβΫγϣϯૉ Β͍͠ʣ •ݱ࣮ੈքγϣοϐϯάΧʔτͱΧϯλʔ͚ͩ Ͱͳ͍ •҆શͳMerge, update͕Ͱ͖ΔσʔλߏΛຖճ ߟ͑ͳ͚ΕͳΒͳ͍ •͍ͬͯΔ͏ͪʹࣅͨΑ͏ͳϥΠϒϥϦ͕͋ͪ͜
ͪͰग़དྷ্͕Δ
ͳ͍ͥ͠ͷ͔ʁ •σʔλͷWriteͱWrite͕ೖΕସΘΓ͏ ΔʹSerializableͲ͜Ζ͔WriteҰ؏ ͨ͠ঢ়ଶʹͰ͖ͳ͍ Server2 Server1 Server3 w1 w2 w1
w2 w2 (w1 lost)
͑: CRDT •ʮෳՄೳͳՄσʔλܕʯ •Conflict-Free Replicated Data Types •Commutative Replicated Data
Types •… •(Going to be included in Riak 2.0) ) CRDTͷ࡞ऀLogical Monotinicy ͱ͍͏ݴ༿͍ͬͯͳ͍
CRDTͷෆม݅ •σʔλʹର͢ΔՄͳૢ࡞ͷΈΛڐ͢ʂ Data = update(w2, update(w1, Data0)) = update(w1, update(w2,
Data0)) Data = merge(update(w2, Data0), Data)
CRDT in Riak 2.0 •KVSͷVʹʮܕʯΛ࣋ͨͤͯɺܕʹΑͬͯ UpdateͱMergeͷϩδοΫΛܾΊΔ •Read࣌ʹMerge͕αʔόʔଆͰࣗಈతʹ࣮ ߦ͞ΕΔ •ΞϓϦέʔγϣϯܕΛࢦఆ͢Δ͚ͩͰΑ͘ɺ ෳόʔδϣϯͷϋϯυϦϯά͕ෆཁʹͳΔ
CRDT riak_object = Riak.fetch(bucket, key)! riak_object.type => counter|set|…! riak_object.set <<
element! riak_object.set.delete(old_element)! riak_object.store!
CRDT example •PN-Counter •Set •OR-sets •LWW-register •Graph…
PN-Counter • merge • {a: {1,-1}, b: {1,0}, c: {2,0}}
• {a: {0,0}, b: {2, 0}, c: {0, -2}} • => {a: {1,-1}, b:{2,0}, c:{2,-2}} => 2 • update • a͕ {increment, 3} Λड͚͚Δͱ • {a: {4,-1}, b: {1,0}, c: {2,0}}
OR-Sets • merge • {a:{“foo”:true}, b:{“bar”:false}} • + {a:{“foo”:true}, b:{“foo”:false,
“bar”:false}} • => {a:{“foo”:true}, b:{“foo”:false, “bar”:true}} • => [“bar”] • update • add: {a:{}} => +”foo” => {a:{“foo”:false}} • remove: {a: {“foo”:false}} => {a: {“foo”:true}}
Ϣʔεέʔε •ΫϦοΫͷΧϯτ (G-counter) • riak-server/types/counters/buckets/likes/datatypes/basho.com -d 1 •γϣοϐϯάΧʔτ (OR-sets) •ϩάΠϯϢʔβʔ
(PN-counter) •͜ΕΒͷΈ߹Θͤ (map & LWW-register, boolean) •{ name : “basho.com”, likes: 20000, users: 3000, links: [ “basho.co.jp”, “basho.co.uk” ], cool: true }
Ͱ͖ͳ͍͜ͱ •ʮ0Ҏ্ʯͷPN-counter •ϢχʔΫͳIDൃߦ •ͦͷଞCAS͕ඞཁͳσʔλߏͱૢ࡞ •RESTfulͳૢ࡞
Strong Consistency •ෆม݅: t(w1) > t(w2) or t(w2) > t(w1)
•Sequencial Consistencyʹ͍ۙ Riak c1 c2 get get v1 v1 w1(v1) ok w2(v1) fail
Strong Consistency •͍ͪͲGETͯ͠ɺͦͷόʔδϣϯʹର͢ Δૢ࡞ΛૹΔ •ͦͷؒʹߋ৽͞Ε͍ͯͨΒࣦഊɺޭ •MultiPaxos
Strong Consistency do {! riak_object = Riak.fetch(bucket, key)! riak_object.data =
new_value! } while (riak_object.store != ok)
·ͱΊ •Riak 1.x Մ༻ੑͷ͋Δࢄσʔλϕʔε •RESTfulͳઃܭΛ͢ΔͳΒSiblings •2.0 ͔Βෳͷ߹ੑϞσϧΛબͰ͖ΔΑ ͏ʹͳͬͨ •ΞϓϦΛ؆୯ʹ࡞Γ͍ͨͳΒCRDT •CASతߋ৽Λ͍ͨ͠ͳΒStrong
Consistency
Questions? •Riak 2.0 Λָ͠Έʹ͍ͯͩ͘͠͞ •Web: http://basho.co.jp •Twitter: @BashoJapan •Me:
[email protected]
•ML:
[email protected]