Practical Eventually-Consistent Web Apps on Riak

Slide 1

Slide 1 text

Practical Eventually-Consistent Web Apps on Sean Cribbs @seancribbs #AYB13 #riak

Slide 2

Slide 2 text

About me 3.5yrs @ Basho Technologies Software Engineer Erlang / Ruby Python / JavaScript Riak + Clients + Tools

Slide 3

Slide 3 text

Eventual Consistency Scalability Schema-free

Slide 4

Slide 4 text

How many of you...?

Slide 5

Slide 5 text

How many of you...? rolling upgrade

Slide 6

Slide 6 text

How many of you...? memcache / Redis rolling upgrade

Slide 7

Slide 7 text

How many of you...? memcache / Redis CDN rolling upgrade

Slide 8

Slide 8 text

How many of you...? memcache / Redis CDN search engine rolling upgrade

Slide 9

Slide 9 text

How many of you...? memcache / Redis CDN search engine rolling upgrade background queue

Slide 10

Slide 10 text

Consistency?

Slide 11

Slide 11 text

Inconsistency!

Slide 12

Slide 12 text

Already non-ACID Photo by Julie Jacobson

Slide 13

Slide 13 text

Already non-ACID Photo by Julie Jacobson data everywhere

Slide 14

Slide 14 text

Already non-ACID Photo by Julie Jacobson data everywhere availability = $

Slide 15

Slide 15 text

Already non-ACID Photo by Julie Jacobson data everywhere availability = $ blind writes

Slide 16

Slide 16 text

Already non-ACID Photo by Julie Jacobson data everywhere availability = $ blind writes DB not “ACID”

Slide 17

Slide 17 text

Already non-ACID Photo by Julie Jacobson data everywhere availability = $ blind writes DB not “ACID” application invariants

Slide 18

Slide 18 text

Eventual Consistency • Available • Responsive • Failure resistant • Convergent

Slide 19

Slide 19 text

Safety & Liveness Leslie Lamport 1977

Slide 20

Slide 20 text

Safety vs. Liveness “Bad things never happen.” “Good things eventually happen.”

Slide 21

Slide 21 text

Eventual consistency is not safe “...it’s easy to satisfy liveness without being useful... If all replicas return the value 42 in response to every request, the system is eventually consistent.” http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/ Peter Bailis

Slide 22

Slide 22 text

so convergent liveness amaze safety very facepalm wow such databas

Slide 23

Slide 23 text

Scalability

Slide 24

Slide 24 text

Scalability

Slide 25

Slide 25 text

Scalability http://www.youtube.com/watch?v=b2F-DItXtZs

Slide 26

Slide 26 text

Scalability COST BENEFIT throughput, capacity, latency, hit-rate money, time = =

Slide 27

Slide 27 text

Schema Pain http://www.webassist.com/tutorials/Free-eCommerce-MySQL-Database • Restrictive formalism • Excessive normalization • Cost of changes • Maintaining relationships • Expensive joins

Slide 28

Slide 28 text

Schema-Free • Data is denormalized • Single-access reads • Great for write-heavy loads • More duplication • Harder to query • Harder to evolve cold data

Slide 29

Slide 29 text

distributed replicated highly available eventually-consistent key-value + extras free / open-source paid add-ons + support

Slide 30

Slide 30 text

1 2 3 Riak Eventual Consistency • Peer-to-peer replication • F+1 failure tolerance • Hinted handoff • Anti-entropy • Logical time (vector clocks)

Slide 31

Slide 31 text

Riak Scalability • More nodes → more throughput • More replicas → greater reliability • More RAM → lower latency • Faster disk → more throughput, lower latency

Slide 32

Slide 32 text

Riak Schema-Free • It’s just a “blob” • Key-Value access • “Bucket” namespaces • Indexing/search optional

Slide 33

Slide 33 text

Riak Deployments

Slide 34

Slide 34 text

Don’t Use Riak if... • Data fits in RAM on one machine • Data is tightly-coupled or highly structured • Application can’t handle eventual consistency* *Riak 2.0 will have more conflict-free datatypes

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

Practical

Slide 37

Slide 37 text

Social Timelines • Write new updates to various collection points • Aggregate and de- dupe on read • Send notifications • Yammer, Formspring, Voxer http://basho.com/riak-and-scala-at-yammer/

Slide 38

Slide 38 text

Session Storage • Big sessions • Quintessential “Dynamo” use-case (shopping cart) • Wikia, ideeli

Slide 39

Slide 39 text

Content Management • Documents, design assets, large files • Riak CS - run your own S3 • AT&T, IDC Frontier

Slide 40

Slide 40 text

Persistent Cache • Store rendered pages and snippets or coalesced records • Posterous, fast.ly • “A Riak machine was off and no one noticed.”

Slide 41

Slide 41 text

Game Data • Backup mobile game data, play from another device • Record matchup history and stats • Rovio, Riot Games

Slide 42

Slide 42 text

RICON & Riak 2.0 • RICON|WEST Jeff Dean Pat Helland • Riak 2.0 - Strong consistency, New search system, More datatypes, Security features

Slide 43

Slide 43 text

Thanks @seancribbs #AYB13 #riak hugs!