Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Consistency types of Riak
Search
UENISHI Kota
June 04, 2014
Technology
2
360
Consistency types of Riak
Riakの整合性は2.0からもっといろいろ調整できるようになるので、その解説(25分)
UENISHI Kota
June 04, 2014
Tweet
Share
More Decks by UENISHI Kota
See All by UENISHI Kota
Storage Systems in Preferred Networks
kuenishi
0
30
Metadata Management in Distributed File Systems
kuenishi
2
510
Behind The Scenes: Cloud Native Storage System for AI
kuenishi
2
400
Apache Ozone behind Simulation and AI Industries
kuenishi
0
370
Distributed Deep Learning with Chainer and Hadoop
kuenishi
3
1.2k
A Few Ways to Accelerate Deep Learning
kuenishi
0
1.1k
Introducing Retz
kuenishi
5
1.1k
Introducing Retz and how to develop practical frameworks
kuenishi
3
730
Formalization and Proof of Distributed Systems (ja)
kuenishi
10
6.4k
Other Decks in Technology
See All in Technology
Zero Data Loss Autonomous Recovery Service サービス概要
oracle4engineer
PRO
2
7.8k
関数型プログラミングで 「脳がバグる」を乗り越える
manabeai
2
210
赤煉瓦倉庫勉強会「Databricksを選んだ理由と、絶賛真っ只中のデータ基盤移行体験記」
ivry_presentationmaterials
2
380
AWS CDKの仕組み / how-aws-cdk-works
gotok365
8
290
クラウド開発の舞台裏とSRE文化の醸成 / SRE NEXT 2025 Lunch Session
kazeburo
1
290
React開発にStorybookとCopilotを導入して、爆速でUIを編集・確認する方法
yu_kod
1
300
TableauLangchainとは何か?
cielo1985
1
120
How to Quickly Call American Airlines®️ U.S. Customer Care : Full Guide
flyaahelpguide
0
140
Getting to Know Your Legacy (System) with AI-Driven Software Archeology (WeAreDevelopers World Congress 2025)
feststelltaste
1
160
VS CodeとGitHub Copilotで爆速開発!アップデートの波に乗るおさらい会 / Rapid Development with VS Code and GitHub Copilot: Catch the Latest Wave
yamachu
2
190
DBのスキルで生き残る技術 - AI時代におけるテーブル設計の勘所
soudai
PRO
54
21k
Operating Operator
shhnjk
1
620
Featured
See All Featured
Art, The Web, and Tiny UX
lynnandtonic
299
21k
Agile that works and the tools we love
rasmusluckow
329
21k
The World Runs on Bad Software
bkeepers
PRO
69
11k
The Cost Of JavaScript in 2023
addyosmani
51
8.5k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
Optimizing for Happiness
mojombo
379
70k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Visualization
eitanlees
146
16k
Become a Pro
speakerdeck
PRO
29
5.4k
A designer walks into a library…
pauljervisheath
207
24k
Thoughts on Productivity
jonyablonski
69
4.7k
Transcript
Consistency types in Riak 2014/6/4 Riak Meetup #4 @kuenishi
ࣗݾհ •UENISHI Kota @ kuenishi •github, twitter, … •Erlangྺ5ɺࢄγεςϜྺ6 •࠷ۙ
Riak CS ͷ։ൃ
Consistencyɹ߹ੑ •ʮNoSQLίϯγεςϯγʔ͕ͳ͍ʯ •ʮSQLʹίϯγεςϯγʔ͕͋Δʯ •ʢ; °Д°ʣ
SQL => ߹ੑʁ NO
߹ੑ㲈 τϥϯβΫγϣϯ ˚
߹ੑʹෆม݅
ෆม݅ •ཧతͳ߹ੑ •ςʔϒϧؒͷ੍݅ɺΠϯσοΫε •ཧతͳ߹ੑ •ෳ͕ඞͣಉ͡Α͏ʹ؍ଌ͞ΕΔ͔
RiakͰ •Siblings •CRDT (2.0~) •Strong Consistency (2.0~)
Why Siblings?
Consistent Hashing • 160-bit Ωʔۭؒ • ۭؒΛ͢Δ • ύʔςΟγϣϯϊʔ υ͕ݸผཧ
• ϨϓϦΧNݸͷύʔ ςΟγϣϯʹίϐʔ͞ ΕΔ OPEF OPEF OPEF OPEF hash(“meetups/spamham”) N=3
CAPఆཧͱཧͷDB •ͲΜͳނোʹରͯ͠ (partition tolerance) •σʔλৗʹ߹͓ͯ͠Γ (consistency) •γεςϜ͕ࢭ·Δ͜ͱͳ͍ (availability) ͜ͷ3ͭΛಉ࣌ʹຬͨ͢γεςϜଘࡏ͠ͳ͍
Consistency͍͠ •ߋ৽ΛࢭΊΔʢAvailabilityΛԼ͛Δʣ͔ɺߋ৽ͷ্ॻ͖Λ ڐ͢ʢσʔλΛࣦ͏ʣ͔͔͠બࢶ͕ͳ͍ Server2 Server1 Server3 PUT V=42 PUT V=0
V=?
ConsistencyͷΘΓʹ •ͱΓ͋͑ͣෳͷόʔδϣϯͷڞଘΛڐ͢ •Ͳͷόʔδϣϯ͕ਖ਼͍͔͠ɺ͘͠Ϛʔδ͢Δ͔ΛRead࣌ʹܾఆ Server2 Server1 Server3 PUT V=42 PUT V=0
V=0 or 42 V=0 V=0 or 42 V=42
APΛ࣮ݱ •ωοτϫʔΫஅ͕ى͖͍ͯͯͱΓ͋͑ͣॻ͖ࠐΈΛڐ͢ Server2 Server1 Server3 PUT V=42 PUT V=0 Server4
෮چͨ͠Βॻ͖͢ ྆ํ͓࣋ͬͯ͘
γϣοϐϯάΧʔτͷྫ •UnionΛͱΕΑ͍ Server2 Server1 Server3 PUT cart=[a,b,d] PUT cart=[a,b,c] union([a,b,c],
[a,b,d]) => [a,b,c,d] [a,b,c] [a,b,c] or [a,b,d] [a,b,d]
Siblings riak_object = Riak.fetch(bucket, key)! riak_object.version! riak_object.has_siblings! for value in
riak_object.values: …! riak_object.data = new_value! riak_object.store!
Siblings ͷෆม݅ •ಛʹͳ͠…͍͋͑ͯ͏ͱϨϓϦΧͷू߹ •Data = R1 ∪ R2 ∪ R3
ෳόʔδϣϯΛ ڐ͢͜ͱͷ •ϓϩάϥϛϯά͕͍͠ʢτϥϯβΫγϣϯૉ Β͍͠ʣ •ݱ࣮ੈքγϣοϐϯάΧʔτͱΧϯλʔ͚ͩ Ͱͳ͍ •҆શͳMerge, update͕Ͱ͖ΔσʔλߏΛຖճ ߟ͑ͳ͚ΕͳΒͳ͍ •͍ͬͯΔ͏ͪʹࣅͨΑ͏ͳϥΠϒϥϦ͕͋ͪ͜
ͪͰग़དྷ্͕Δ
ͳ͍ͥ͠ͷ͔ʁ •σʔλͷWriteͱWrite͕ೖΕସΘΓ͏ ΔʹSerializableͲ͜Ζ͔WriteҰ؏ ͨ͠ঢ়ଶʹͰ͖ͳ͍ Server2 Server1 Server3 w1 w2 w1
w2 w2 (w1 lost)
͑: CRDT •ʮෳՄೳͳՄσʔλܕʯ •Conflict-Free Replicated Data Types •Commutative Replicated Data
Types •… •(Going to be included in Riak 2.0) ) CRDTͷ࡞ऀLogical Monotinicy ͱ͍͏ݴ༿͍ͬͯͳ͍
CRDTͷෆม݅ •σʔλʹର͢ΔՄͳૢ࡞ͷΈΛڐ͢ʂ Data = update(w2, update(w1, Data0)) = update(w1, update(w2,
Data0)) Data = merge(update(w2, Data0), Data)
CRDT in Riak 2.0 •KVSͷVʹʮܕʯΛ࣋ͨͤͯɺܕʹΑͬͯ UpdateͱMergeͷϩδοΫΛܾΊΔ •Read࣌ʹMerge͕αʔόʔଆͰࣗಈతʹ࣮ ߦ͞ΕΔ •ΞϓϦέʔγϣϯܕΛࢦఆ͢Δ͚ͩͰΑ͘ɺ ෳόʔδϣϯͷϋϯυϦϯά͕ෆཁʹͳΔ
CRDT riak_object = Riak.fetch(bucket, key)! riak_object.type => counter|set|…! riak_object.set <<
element! riak_object.set.delete(old_element)! riak_object.store!
CRDT example •PN-Counter •Set •OR-sets •LWW-register •Graph…
PN-Counter • merge • {a: {1,-1}, b: {1,0}, c: {2,0}}
• {a: {0,0}, b: {2, 0}, c: {0, -2}} • => {a: {1,-1}, b:{2,0}, c:{2,-2}} => 2 • update • a͕ {increment, 3} Λड͚͚Δͱ • {a: {4,-1}, b: {1,0}, c: {2,0}}
OR-Sets • merge • {a:{“foo”:true}, b:{“bar”:false}} • + {a:{“foo”:true}, b:{“foo”:false,
“bar”:false}} • => {a:{“foo”:true}, b:{“foo”:false, “bar”:true}} • => [“bar”] • update • add: {a:{}} => +”foo” => {a:{“foo”:false}} • remove: {a: {“foo”:false}} => {a: {“foo”:true}}
Ϣʔεέʔε •ΫϦοΫͷΧϯτ (G-counter) • riak-server/types/counters/buckets/likes/datatypes/basho.com -d 1 •γϣοϐϯάΧʔτ (OR-sets) •ϩάΠϯϢʔβʔ
(PN-counter) •͜ΕΒͷΈ߹Θͤ (map & LWW-register, boolean) •{ name : “basho.com”, likes: 20000, users: 3000, links: [ “basho.co.jp”, “basho.co.uk” ], cool: true }
Ͱ͖ͳ͍͜ͱ •ʮ0Ҏ্ʯͷPN-counter •ϢχʔΫͳIDൃߦ •ͦͷଞCAS͕ඞཁͳσʔλߏͱૢ࡞ •RESTfulͳૢ࡞
Strong Consistency •ෆม݅: t(w1) > t(w2) or t(w2) > t(w1)
•Sequencial Consistencyʹ͍ۙ Riak c1 c2 get get v1 v1 w1(v1) ok w2(v1) fail
Strong Consistency •͍ͪͲGETͯ͠ɺͦͷόʔδϣϯʹର͢ Δૢ࡞ΛૹΔ •ͦͷؒʹߋ৽͞Ε͍ͯͨΒࣦഊɺޭ •MultiPaxos
Strong Consistency do {! riak_object = Riak.fetch(bucket, key)! riak_object.data =
new_value! } while (riak_object.store != ok)
·ͱΊ •Riak 1.x Մ༻ੑͷ͋Δࢄσʔλϕʔε •RESTfulͳઃܭΛ͢ΔͳΒSiblings •2.0 ͔Βෳͷ߹ੑϞσϧΛબͰ͖ΔΑ ͏ʹͳͬͨ •ΞϓϦΛ؆୯ʹ࡞Γ͍ͨͳΒCRDT •CASతߋ৽Λ͍ͨ͠ͳΒStrong
Consistency
Questions? •Riak 2.0 Λָ͠Έʹ͍ͯͩ͘͠͞ •Web: http://basho.co.jp •Twitter: @BashoJapan •Me:
[email protected]
•ML:
[email protected]