Practical Eventually-Consistent Web Apps on Riak

"Eventual consistency! Scalability! Schema-free!" is the rallying call of the NoSQL world. But what does it mean to be eventually-consistent, to be scalable, to be schema-free? Why should you, as a web developer, care? Let's demystify these terms and then explore some ways that Riak, an "eventually-consistent, scalable, schema-free" datastore, can be put to practical use in web applications. You might just find that you've been using eventual consistency all along!

Transcript

1. Practical Eventually-Consistent Web Apps on Sean Cribbs @seancribbs #AYB13 #riak
2. About me 3.5yrs @ Basho Technologies Software Engineer Erlang / Ruby Python / JavaScript Riak + Clients + Tools
3. Eventual Consistency Scalability Schema-free
4. How many of you...?
5. How many of you...? rolling upgrade
6. How many of you...? memcache / Redis rolling upgrade
7. How many of you...? memcache / Redis CDN rolling upgrade
8. How many of you...? memcache / Redis CDN search engine rolling upgrade
9. How many of you...? memcache / Redis CDN search engine rolling upgrade background queue
10. Consistency?
11. Inconsistency!
12. Already non-ACID Photo by Julie Jacobson
13. Already non-ACID Photo by Julie Jacobson data everywhere
14. Already non-ACID Photo by Julie Jacobson data everywhere availability = $
15. Already non-ACID Photo by Julie Jacobson data everywhere availability = $ blind writes
16. Already non-ACID Photo by Julie Jacobson data everywhere availability = $ blind writes DB not “ACID”
17. Already non-ACID Photo by Julie Jacobson data everywhere availability = $ blind writes DB not “ACID” application invariants
18. Eventual Consistency • Available • Responsive • Failure resistant • Convergent
19. Safety & Liveness Leslie Lamport 1977
20. Safety vs. Liveness “Bad things never happen.” “Good things eventually happen.”
21. Eventual consistency is not safe “...it’s easy to satisfy liveness without being useful... If all replicas return the value 42 in response to every request, the system is eventually consistent.” http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/ Peter Bailis
22. so convergent liveness amaze safety very facepalm wow such databas
23. Scalability
24. Scalability
25. Scalability http://www.youtube.com/watch?v=b2F-DItXtZs
26. Scalability COST BENEFIT throughput, capacity, latency, hit-rate money, time = =
27. Schema Pain http://www.webassist.com/tutorials/Free-eCommerce-MySQL-Database • Restrictive formalism • Excessive normalization • Cost of changes • Maintaining relationships • Expensive joins
28. Schema-Free • Data is denormalized • Single-access reads • Great for write-heavy loads • More duplication • Harder to query • Harder to evolve cold data
29. distributed replicated highly available eventually-consistent key-value + extras free / open-source paid add-ons + support
30. 1 2 3 Riak Eventual Consistency • Peer-to-peer replication • F+1 failure tolerance • Hinted handoff • Anti-entropy • Logical time (vector clocks)
31. Riak Scalability • More nodes → more throughput • More replicas → greater reliability • More RAM → lower latency • Faster disk → more throughput, lower latency
32. Riak Schema-Free • It’s just a “blob” • Key-Value access • “Bucket” namespaces • Indexing/search optional
33. Riak Deployments
34. Don’t Use Riak if... • Data fits in RAM on one machine • Data is tightly-coupled or highly structured • Application can’t handle eventual consistency* *Riak 2.0 will have more conflict-free datatypes
36. Practical
37. Social Timelines • Write new updates to various collection points • Aggregate and de- dupe on read • Send notifications • Yammer, Formspring, Voxer http://basho.com/riak-and-scala-at-yammer/
38. Session Storage • Big sessions • Quintessential “Dynamo” use-case (shopping cart) • Wikia, ideeli
39. Content Management • Documents, design assets, large files • Riak CS - run your own S3 • AT&T, IDC Frontier
40. Persistent Cache • Store rendered pages and snippets or coalesced records • Posterous, fast.ly • “A Riak machine was off and no one noticed.”
41. Game Data • Backup mobile game data, play from another device • Record matchup history and stats • Rovio, Riot Games
42. RICON & Riak 2.0 • RICON|WEST Jeff Dean Pat Helland • Riak 2.0 - Strong consistency, New search system, More datatypes, Security features
43. Thanks @seancribbs #AYB13 #riak hugs!