Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
New Intro to Riak
Search
Joel Jacobson
August 21, 2013
Technology
1
44
New Intro to Riak
Joel Jacobson
August 21, 2013
Tweet
Share
More Decks by Joel Jacobson
See All by Joel Jacobson
Microsoft Azure Meetup
joeljacobson
0
71
CRDTs and Eventual Consistency
joeljacobson
0
70
Killing Pigs and Saving Danish Bacon
joeljacobson
0
75
Conflict-Free Replicated Data Types in Eventually Consistent Systems
joeljacobson
0
110
Intro to Riak
joeljacobson
0
85
Other Decks in Technology
See All in Technology
データ戦略部門 紹介資料
sansan33
PRO
1
3.7k
OCI Network Firewall 概要
oracle4engineer
PRO
2
7.9k
20201008_ファインディ_品質意識を育てる役目は人かAIか___2_.pdf
findy_eventslides
2
640
ソースを読むプロセスの例
sat
PRO
12
5.1k
OAuthからOIDCへ ― 認可の仕組みが認証に拡張されるまで
yamatai1212
0
120
AI Agent Dojo #2 watsonx Orchestrateフローの作成
oniak3ibm
PRO
0
120
WEBサービスを成り立たせるAWSサービス
takano0131
1
160
"プロポーザルってなんか怖そう"という境界を超えてみた@TSUDOI by giftee Tech #1
shilo113
0
200
Data Hubグループ 紹介資料
sansan33
PRO
0
2.2k
このままAIが発展するだけでAGI達成可能な理由
frievea
0
100
Wasmのエコシステムを使った ツール作成方法
askua
0
180
リセラー企業のテクサポ担当が考える、生成 AI 時代のトラブルシュート 2025
kazzpapa3
1
350
Featured
See All Featured
Typedesign – Prime Four
hannesfritz
42
2.8k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
33
2.3k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.4k
The Cult of Friendly URLs
andyhume
79
6.6k
BBQ
matthewcrist
89
9.8k
Making Projects Easy
brettharned
120
6.4k
The World Runs on Bad Software
bkeepers
PRO
72
11k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
359
30k
GitHub's CSS Performance
jonrohan
1032
470k
Art, The Web, and Tiny UX
lynnandtonic
303
21k
Into the Great Unknown - MozCon
thekraken
40
2.1k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
9.7k
Transcript
Introduction to Riak 2013
Who am I? Joel Jacobson Technical Evangelist @Basho @joeljacobson
Distributed computing is hard COncurrency scaLIng Latency consIstency avaILabILIty MuLtI
Tenancy faILover SLA’s
What is Riak? Key Value store + extras Distributed /
Horizontally Scalable Fault Tolerant Highly available built for the web
inspired by amazon dynamo white paper released to describe a
database system to be used for their shopping cart Masterless, peer coordinated replication Consistent Hashing Eventually Consistent
Riak Key-value store Simple operations; GET, PUT, DELETE Value is
Opaque, with metadata Extras; Secondary indexes MapReduce full text search
Horizontal Scalability Near linear Scalability Query load and data are
spread evenly Add more nodes and get more; Ops/second storage capacity compute power (mapreduce)
Fault tolerant no Single point of failure (SPOF) All Data
is replicated CLusters self heal; Handoff, Active Anti Entropy cluster transparently survives Node Failure Network partition
Highly Available Any Node Can Serve Client requests Fallbacks are
used when nodes are down Always available for read and write requests Per-request quorums
Quorums n = 3 r / w = 2 R
= 1 - faster response time, less likely consistent r = all - slower response, greater consistency
the ring
Replication replicated to 3 nodes by default (n_val , which
is configurable)
Node fails Request goes to fallback Handoff - data retuned
to recovered node X X X X X X X X hash(“user_id”) Disaster scenario
Automatically repair inconsistencies in data runs as a background process
or Can be configured as a manual process active anti-entropy
Network partitions or concurrent actors modifying the same data Riak
provides two solutions to manage this: Last Write Wins Vector Clocks Conflict resolution
Vector Clocks Every node has an ID Send last-seen vector
clock in every “put” request Can be viewed as ‘commit history’ e.g. Git Lets you decide conflicts
sibling creation 0 3 2 1 Object v1 Object v1
0 3 2 1 Object v1 Siblings can be created by: Simultaneous writes Anti-entropy [{a,3}] [{a,2},{b,1}] [{a,3}] Object v1 Object v1 [{a,2},{b,1}]
storage backends Bitcask Leveldb memory multi
bitcask A fast, append-only key-value store Key space must fit
in memory Suitable for bounded data, e.g. reference data
Leveldb Append-only for very large data sets multiple levels Allows
for more advanced querying (2i) includes compression (Snappy algorithm) Suitable for unbounded data
memory Data is never persisted to disk Definable memory limits
per vnode Configurable object expiry Useful for highly transient data supports secondary indexes
multi Configure multiple storage engines for different types of data
Choose storage engine on per bucket basis
clients apis Protocol Buffers REST based HTTP Interface
client libraries Client libraries supported by Basho: Community supported languages
and frameworks: C/C++, Clojure, Common Lisp, Dart, Django, Go, Grails, Griffon, Groovy, Haskell, .NET, Node.js, OCaml , Perl, PHP, Play, Racket, Scala, Smalltalk
Using Riak as datastore for all back-end systems supporting Angry
Birds Game-state storage, ID/Login, Payments, Push notifications, analytics, advertisements 9 clusters in use with over 100 nodes 263 million active monthly users
Spine2 - storing 80 million+ patient data 500 complex messages
per second 20,000 integrated end points 0% data loss 99.9% availability SLA
Push to talk application Billions of requests daily > 50
dedicated servers Everything stored in Riak
MDC Allows data to be replicated between clusters in different
data centers real-time and full sync uni-directional or bi-directional replication global load-balancing backups
riak-cs S3 compatible object store Supports Objects of Arbitrary Content
Type Up to 5TB multi-tenancy Per-tenant usage data and statistics on network I/O supports MDC
try it? http://docs.basho.com/riak/latest/references/appendices/ community/Sample-Applications/ https://github.com/basho/riak-dev-cluster
thanks
[email protected]