Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scaling Your Data Safely with Riak
Search
Alex Moore
January 10, 2014
Programming
3
140
Scaling Your Data Safely with Riak
CodeMash 2014 Presentation.
Alex Moore
January 10, 2014
Tweet
Share
Other Decks in Programming
See All in Programming
KotlinConf 2025 現地で感じたServer-Side Kotlin
n_takehata
1
230
Result型で“失敗”を型にするPHPコードの書き方
kajitack
4
390
Haskell でアルゴリズムを抽象化する / 関数型言語で競技プログラミング
naoya
17
4.9k
技術同人誌をMCP Serverにしてみた
74th
1
370
High-Level Programming Languages in AI Era -Human Thought and Mind-
hayat01sh1da
PRO
0
390
「ElixirでIoT!!」のこれまでとこれから
takasehideki
0
370
Railsアプリケーションと パフォーマンスチューニング ー 秒間5万リクエストの モバイルオーダーシステムを支える事例 ー Rubyセミナー 大阪
falcon8823
4
950
なぜ「共通化」を考え、失敗を繰り返すのか
rinchoku
1
530
PHPでWebSocketサーバーを実装しよう2025
kubotak
0
160
git worktree × Claude Code × MCP ~生成AI時代の並列開発フロー~
hisuzuya
1
480
エンジニア向け採用ピッチ資料
inusan
0
160
0626 Findy Product Manager LT Night_高田スライド_speaker deck用
mana_takada
0
110
Featured
See All Featured
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
130
19k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Adopting Sorbet at Scale
ufuk
77
9.4k
Agile that works and the tools we love
rasmusluckow
329
21k
Writing Fast Ruby
sferik
628
61k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Balancing Empowerment & Direction
lara
1
370
Reflections from 52 weeks, 52 projects
jeffersonlam
351
20k
jQuery: Nuts, Bolts and Bling
dougneiner
63
7.8k
The Cost Of JavaScript in 2023
addyosmani
51
8.5k
Transcript
with Riak @alexmoore Scaling Data Safely
Internet 1 (2) (3)
Internet 503 Timeout 1 (2) (3)
Internet 1 (2) 3
Internet 1 (2) 3
Internet (2) 3 2 3 1
Internet 4 2 3 (2) 1 3 3
Internet CDN 2 6 7 2 4 3 1 3
3
Internet CDN 5 6 2 4 3 2 1 3
3 7 3
Internet CDN 6 7 2 3 2 1 3 3
Internet CDN P S S 3 6 7 2 5
3 2 1 4 4 3 3 7
Internet CDN 6 8 2 3 2 1 3 3
Internet CDN 6 8 2 3 2 1 3 3
3? 4? q User Session Widgets Lolcats Stuff 8? magic
Internet CDN 6 8 2 3 2 1 3 3
3? 4? q User Session Widgets Lolcats Stuff 8? magic
Internet CDN 6 8 2 3 2 1 3 3
3? 4? q User Session Widgets Lolcats Stuff 8? magic
Internet CDN 6 8 2 3 2 1 3 3
3? 4? q User Session Widgets Lolcats Stuff 8? magic Who’s on first?
Internet CDN 6 8 2 3 2 1 3 3
3? 4? q User Session Widgets Lolcats Stuff 8? magic What the hell have you built? Who’s on first?
Enter Riak
Riak is a distributed, fault-tolerant, highly available database
None
Distributed
Distributed a cluster of nodes each node has an identical
role
Fault- tolerant
Fault- Tolerant symmetrical no Single Point of Failure
Highly Available
Highly Available replicas & redundancy
Scalable
Scalable add or remove nodes linear scalability
Scalable add or remove nodes linear scalability
Multiple Platforms
Multiple Platforms
Multiple Platforms
Predictable Performance
None
Internet CDN 5 6 2 3 2 1 3 3
Internet CDN 5 6 2 3 2 1 3 3
3 3 4 4
1 2 3 4 5
6 7 8 9 10 11 12 Building A Cluster # All Nodes $ riak start ! # Nodes 2..n.each $ riak-‐admin join
[email protected]
! # Any Node $ riak-‐admin cluster plan $ riak-‐admin cluster commit !
None
None
Architecture
Riak is the ops-friendly database
Riak is distributed
Consistent Hashing hash(“bucket/key”)
hash ring
tokenize it
node 0 node 1 node 2
node 0 node 1 node 2 hash(key)
node 0 node 1 node 2 Replicas are stored to
the N - 1 contiguous partitions
node 0 node 1 node 2 hash(“companies/GE”) Replicas are stored
to the N - 1 contiguous partitions
node 0 node 1 node 2 hash(“companies/GE”) Replicas are stored
to the N - 1 contiguous partitions
node 0 node 1 node 2
node 0 node 1 node 2 node 3 + Scaling
out
Quorum requests N R W PR/PW DW
get(“users/clay-davis”)
get(“users/clay-davis”) client Riak
get(“users/clay-davis”) Get Handler (FSM) client Riak
get(“users/clay-davis”) Get Handler (FSM) client Riak hash(“users/clay-davis”) == 10, 11,
12
get(“users/clay-davis”) Get Handler (FSM) client Riak hash(“users/clay-davis”) == 10, 11,
12 Coordinating node Cluster 6 7 8 9 10 11 12 13 14 15 16 The Ring
get(“users/clay-davis”) Get Handler (FSM) client Riak get(“users/clay-davis”) Coordinating node Cluster
6 7 8 9 10 11 12 13 14 15 16 The Ring
get(“users/clay-davis”) Get Handler (FSM) client Riak Coordinating node Cluster 6
7 8 9 10 11 12 13 14 15 16 The Ring R=2
get(“users/clay-davis”) Get Handler (FSM) client Riak Coordinating node Cluster 6
7 8 9 10 11 12 13 14 15 16 The Ring R=2 obj
get(“users/clay-davis”) Get Handler (FSM) client Riak R=2 obj obj
get(“users/clay-davis”) Get Handler (FSM) client Riak R=2 obj obj
get(“users/clay-davis”) obj
Read Repair (Anti-Entropy)
replica replica replica
replica replica replica X
replica replica replica replica replica replica
Active Anti-Entropy (self healing clusters)
real-time updates persistent non-blocking disk-based
merkle tree to track changes coordinated at the vnode level
runs as a background process exchange with neighbor vnodes for inconsistencies resolution semantics: trigger read-repair
= hashes marked “dirty”
None
None
None
None
= keys to read-repair
Using Riak
Riak is the ops-friendly database
Riak is friendly to Developers, too!
{“key”: “value”}
key value key value key value key value key value
key value key value key value key value Keys are namespaced into Buckets
Buckets aren’t “real”
✘ ✘ ✘ Buckets aren’t “real”
Metadata Bucket Properties
Aitch Tee Tee Pee GET /buckets/bucket/keys/key PUT /buckets/bucket/keys/key DEL /buckets/bucket/keys/key
1 2 3 4 5
6 7 8 9 10 11 12 Working With Ruby - Starting $ gem install riak-‐client ! book = { isbn: '1111979723', title: 'Moby Dick', author: 'Herman Melville', body: 'Call me Ishmael. Some years ago...', copies_owned: 3 }
1 2 3 4 5
6 7 8 9 10 11 12 Working With Ruby - Create # Starting Client client = Riak::Client.new(protocol: 'pbc', pb_port: 8087, host: '10.0.0.1') ! books_bucket = client.bucket('books') ! robj_moby = books_bucket.new(book[:isbn]) robj_moby.data = book robj_moby.store ! !
1 2 3 4 5
6 7 8 9 10 11 12 Working With Ruby - Read fetched_book = books_bucket.get('1111979723') ! puts fetched_book.raw_data ! => {"isbn":"1111979723","title":"Moby Dick”, "author":"Herman Melville", "body":"Call me Ishmael. Some years ago...","copies_owned":3} ! fetched_book.data['isbn'] ! => "1111979723"
1 2 3 4 5
6 7 8 9 10 11 12 Working With Ruby - Delete fetched_book.delete ! # or ! books_bucket.delete('1111979723')
1 2 3 4 5
6 7 8 9 10 11 12 Working With Ruby - 2I ! fetched_book.indexes['author_bin'] = [] fetched_book.indexes['author_bin'] << fetched_book.data['author'] ! fetched_book.store ! books_bucket.get_index('author_bin','Herman Melville') ! => ["1111979723"]
etc...
JUST ONE MORE THING
Internet CDN 5 6 2 3 2 1 3 3
3 3 4 4
Internet CDN 5 6 2 3 2 1 3 3
3 4
Internet CDN 5 6 2 3 2 1 3 3
3 4
in riak.conf:
search = off in riak.conf:
search = off search = on in riak.conf:
1 2 3 4 5
6 7 8 9 10 11 12 client.create_search_index(“tweets_idx”) ! bucket = Riak::Bucket.new(client, "tweets") bucket.props = {"search_index" => "tweets_idx"} ! # load some tweets into riak… ! results = client.search("tweets", "text:who") ! puts results[‘docs'].first
1 2 3 4 5
6 7 8 9 10 11 12 pp results[‘docs'].first ! {"score"=>"2.50804900000000019489e+00", "_yz_rb"=>"tweets", "_yz_rk"=>"421672897250226176", ...}
More resources: docs.basho.com “Taste of Riak”
Multi-Datacenter Replication (E.G. - Nuke proofing)
Full Sync
Real-Time
None
put(“users/jimmy”)
put(“users/jimmy”)
put(“users/jimmy”)
put(“users/jimmy”)
put(“users/jimmy”)
Internet CDN 5 6 2 3 2 1 3 3
3 4
Internet CDN 5 6 2 3 2 1 3 3
3 4 ?
Internet CDN 5 6 2 3 2 1 3 3
3 4 ?
Q & A Alex Moore - Client Services Engineer, Basho
@alexmoore
[email protected]
http://alexmoore.io