Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Consistency types of Riak
Search
Sponsored
·
Ship Features Fearlessly
Turn features on and off without deploys. Used by thousands of Ruby developers.
→
UENISHI Kota
June 04, 2014
Technology
390
2
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Consistency types of Riak
Riakの整合性は2.0からもっといろいろ調整できるようになるので、その解説(25分)
UENISHI Kota
June 04, 2014
Transcript
33 slides
Consistency types in Riak 2014/6/4 Riak Meetup #4 @kuenishi
ࣗݾհ •UENISHI Kota @ kuenishi •github, twitter, … •Erlangྺ5ɺࢄγεςϜྺ6 •࠷ۙ
Riak CS ͷ։ൃ
Consistencyɹ߹ੑ •ʮNoSQLίϯγεςϯγʔ͕ͳ͍ʯ •ʮSQLʹίϯγεςϯγʔ͕͋Δʯ •ʢ; °Д°ʣ
SQL => ߹ੑʁ NO
߹ੑ㲈 τϥϯβΫγϣϯ ˚
߹ੑʹෆม݅
ෆม݅ •ཧతͳ߹ੑ •ςʔϒϧؒͷ੍݅ɺΠϯσοΫε •ཧతͳ߹ੑ •ෳ͕ඞͣಉ͡Α͏ʹ؍ଌ͞ΕΔ͔
RiakͰ •Siblings •CRDT (2.0~) •Strong Consistency (2.0~)
Why Siblings?
Consistent Hashing • 160-bit Ωʔۭؒ • ۭؒΛ͢Δ • ύʔςΟγϣϯϊʔ υ͕ݸผཧ
• ϨϓϦΧNݸͷύʔ ςΟγϣϯʹίϐʔ͞ ΕΔ OPEF OPEF OPEF OPEF hash(“meetups/spamham”) N=3
CAPఆཧͱཧͷDB •ͲΜͳނোʹରͯ͠ (partition tolerance) •σʔλৗʹ߹͓ͯ͠Γ (consistency) •γεςϜ͕ࢭ·Δ͜ͱͳ͍ (availability) ͜ͷ3ͭΛಉ࣌ʹຬͨ͢γεςϜଘࡏ͠ͳ͍
Consistency͍͠ •ߋ৽ΛࢭΊΔʢAvailabilityΛԼ͛Δʣ͔ɺߋ৽ͷ্ॻ͖Λ ڐ͢ʢσʔλΛࣦ͏ʣ͔͔͠બࢶ͕ͳ͍ Server2 Server1 Server3 PUT V=42 PUT V=0
V=?
ConsistencyͷΘΓʹ •ͱΓ͋͑ͣෳͷόʔδϣϯͷڞଘΛڐ͢ •Ͳͷόʔδϣϯ͕ਖ਼͍͔͠ɺ͘͠Ϛʔδ͢Δ͔ΛRead࣌ʹܾఆ Server2 Server1 Server3 PUT V=42 PUT V=0
V=0 or 42 V=0 V=0 or 42 V=42
APΛ࣮ݱ •ωοτϫʔΫஅ͕ى͖͍ͯͯͱΓ͋͑ͣॻ͖ࠐΈΛڐ͢ Server2 Server1 Server3 PUT V=42 PUT V=0 Server4
෮چͨ͠Βॻ͖͢ ྆ํ͓࣋ͬͯ͘
γϣοϐϯάΧʔτͷྫ •UnionΛͱΕΑ͍ Server2 Server1 Server3 PUT cart=[a,b,d] PUT cart=[a,b,c] union([a,b,c],
[a,b,d]) => [a,b,c,d] [a,b,c] [a,b,c] or [a,b,d] [a,b,d]
Siblings riak_object = Riak.fetch(bucket, key)! riak_object.version! riak_object.has_siblings! for value in
riak_object.values: …! riak_object.data = new_value! riak_object.store!
Siblings ͷෆม݅ •ಛʹͳ͠…͍͋͑ͯ͏ͱϨϓϦΧͷू߹ •Data = R1 ∪ R2 ∪ R3
ෳόʔδϣϯΛ ڐ͢͜ͱͷ •ϓϩάϥϛϯά͕͍͠ʢτϥϯβΫγϣϯૉ Β͍͠ʣ •ݱ࣮ੈքγϣοϐϯάΧʔτͱΧϯλʔ͚ͩ Ͱͳ͍ •҆શͳMerge, update͕Ͱ͖ΔσʔλߏΛຖճ ߟ͑ͳ͚ΕͳΒͳ͍ •͍ͬͯΔ͏ͪʹࣅͨΑ͏ͳϥΠϒϥϦ͕͋ͪ͜
ͪͰग़དྷ্͕Δ
ͳ͍ͥ͠ͷ͔ʁ •σʔλͷWriteͱWrite͕ೖΕସΘΓ͏ ΔʹSerializableͲ͜Ζ͔WriteҰ؏ ͨ͠ঢ়ଶʹͰ͖ͳ͍ Server2 Server1 Server3 w1 w2 w1
w2 w2 (w1 lost)
͑: CRDT •ʮෳՄೳͳՄσʔλܕʯ •Conflict-Free Replicated Data Types •Commutative Replicated Data
Types •… •(Going to be included in Riak 2.0) ) CRDTͷ࡞ऀLogical Monotinicy ͱ͍͏ݴ༿͍ͬͯͳ͍
CRDTͷෆม݅ •σʔλʹର͢ΔՄͳૢ࡞ͷΈΛڐ͢ʂ Data = update(w2, update(w1, Data0)) = update(w1, update(w2,
Data0)) Data = merge(update(w2, Data0), Data)
CRDT in Riak 2.0 •KVSͷVʹʮܕʯΛ࣋ͨͤͯɺܕʹΑͬͯ UpdateͱMergeͷϩδοΫΛܾΊΔ •Read࣌ʹMerge͕αʔόʔଆͰࣗಈతʹ࣮ ߦ͞ΕΔ •ΞϓϦέʔγϣϯܕΛࢦఆ͢Δ͚ͩͰΑ͘ɺ ෳόʔδϣϯͷϋϯυϦϯά͕ෆཁʹͳΔ
CRDT riak_object = Riak.fetch(bucket, key)! riak_object.type => counter|set|…! riak_object.set <<
element! riak_object.set.delete(old_element)! riak_object.store!
CRDT example •PN-Counter •Set •OR-sets •LWW-register •Graph…
PN-Counter • merge • {a: {1,-1}, b: {1,0}, c: {2,0}}
• {a: {0,0}, b: {2, 0}, c: {0, -2}} • => {a: {1,-1}, b:{2,0}, c:{2,-2}} => 2 • update • a͕ {increment, 3} Λड͚͚Δͱ • {a: {4,-1}, b: {1,0}, c: {2,0}}
OR-Sets • merge • {a:{“foo”:true}, b:{“bar”:false}} • + {a:{“foo”:true}, b:{“foo”:false,
“bar”:false}} • => {a:{“foo”:true}, b:{“foo”:false, “bar”:true}} • => [“bar”] • update • add: {a:{}} => +”foo” => {a:{“foo”:false}} • remove: {a: {“foo”:false}} => {a: {“foo”:true}}
Ϣʔεέʔε •ΫϦοΫͷΧϯτ (G-counter) • riak-server/types/counters/buckets/likes/datatypes/basho.com -d 1 •γϣοϐϯάΧʔτ (OR-sets) •ϩάΠϯϢʔβʔ
(PN-counter) •͜ΕΒͷΈ߹Θͤ (map & LWW-register, boolean) •{ name : “basho.com”, likes: 20000, users: 3000, links: [ “basho.co.jp”, “basho.co.uk” ], cool: true }
Ͱ͖ͳ͍͜ͱ •ʮ0Ҏ্ʯͷPN-counter •ϢχʔΫͳIDൃߦ •ͦͷଞCAS͕ඞཁͳσʔλߏͱૢ࡞ •RESTfulͳૢ࡞
Strong Consistency •ෆม݅: t(w1) > t(w2) or t(w2) > t(w1)
•Sequencial Consistencyʹ͍ۙ Riak c1 c2 get get v1 v1 w1(v1) ok w2(v1) fail
Strong Consistency •͍ͪͲGETͯ͠ɺͦͷόʔδϣϯʹର͢ Δૢ࡞ΛૹΔ •ͦͷؒʹߋ৽͞Ε͍ͯͨΒࣦഊɺޭ •MultiPaxos
Strong Consistency do {! riak_object = Riak.fetch(bucket, key)! riak_object.data =
new_value! } while (riak_object.store != ok)
·ͱΊ •Riak 1.x Մ༻ੑͷ͋Δࢄσʔλϕʔε •RESTfulͳઃܭΛ͢ΔͳΒSiblings •2.0 ͔Βෳͷ߹ੑϞσϧΛબͰ͖ΔΑ ͏ʹͳͬͨ •ΞϓϦΛ؆୯ʹ࡞Γ͍ͨͳΒCRDT •CASతߋ৽Λ͍ͨ͠ͳΒStrong
Consistency
Questions? •Riak 2.0 Λָ͠Έʹ͍ͯͩ͘͠͞ •Web: http://basho.co.jp •Twitter: @BashoJapan •Me:
[email protected]
•ML:
[email protected]
More Decks by UENISHI Kota
See All by UENISHI Kota
Storage Systems in Preferred Networks
kuenishi
0
120
Metadata Management in Distributed File Systems
kuenishi
2
590
Behind The Scenes: Cloud Native Storage System for AI
kuenishi
2
490
Apache Ozone behind Simulation and AI Industries
kuenishi
0
520
Distributed Deep Learning with Chainer and Hadoop
kuenishi
3
1.4k
A Few Ways to Accelerate Deep Learning
kuenishi
0
1.3k
Introducing Retz
kuenishi
5
1.3k
Introducing Retz and how to develop practical frameworks
kuenishi
3
870
Formalization and Proof of Distributed Systems (ja)
kuenishi
10
6.6k
Other Decks in Technology
See All in Technology
ぼくのClaude Codeの使い方
falken
PRO
5
7.6k
Biohub - Cell Tracking During Development 5th place solution
rist
0
180
認知の質も文化醸成も。数年取り組んで見えてきた技術広報のKPI設計 / Tech PR KPIs: Lessons on Awareness Quality and Culture After Several Years
bitkey
PRO
0
110
こんなに変わった! AWSネットワークの常識
shotashiratori
16
11k
人の努力に頼らないセキュリティをエンジニアリングする在り方 / An approach to engineering security that does not rely on human effort
carta_engineering
0
230
専門家に使ってもらうAIプロダクトをつくる - 病院の医事課向けDPCコーディング支援システムの設計と評価
hagino3000
4
950
CloudTrail を推したい
kazzpapa3
1
150
AWS Emulator を評価する: 手元で動く AWS の API の仕組み、始め方、確かめ方
inariku
PRO
2
780
Taking back control of your AI development
inesmontani
PRO
0
300
メルカリの小規模チーム「Pod」で、 エンジニアがデータアナリストなしで、 AIを使ってデータ分析の自走を目指す取り組み
jackojacko_
0
280
re:Inventを楽しむために、 1日1つ「これが楽しみ」を決めよう
beli68
0
120
攻撃と守りは何が変わり、何が変わらないのか~最近の不正アクセスから考える、リスク評価と標準の使いどころ
oidfj
2
250
Featured
See All Featured
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
31
3.4k
Reality Check: Gamification 10 Years Later
codingconduct
0
2.4k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
35
3.9k
B2B Lead Gen: Tactics, Traps & Triumph
marketingsoph
1
310
Code Review Best Practice
trishagee
74
20k
Code Reviewing Like a Champion
maltzj
529
40k
A Soul's Torment
seathinner
9
3.9k
Designing for Performance
lara
612
70k
Designing Experiences People Love
moore
143
24k
Into the Great Unknown - MozCon
thekraken
41
2.8k
GitHub's CSS Performance
jonrohan
1033
470k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
254
23k