Practical Eventually-Consistent Web Apps on Riak

Practical Eventually-Consistent Web Apps on Riak

"Eventual consistency! Scalability! Schema-free!" is the rallying call of the NoSQL world. But what does it mean to be eventually-consistent, to be scalable, to be schema-free? Why should you, as a web developer, care? Let's demystify these terms and then explore some ways that Riak, an "eventually-consistent, scalable, schema-free" datastore, can be put to practical use in web applications. You might just find that you've been using eventual consistency all along!

Ce461b57b6a1f65ce5b6cc1c124673e3?s=128

Sean Cribbs

October 18, 2013
Tweet

Transcript

  1. Practical Eventually-Consistent Web Apps on Sean Cribbs @seancribbs #AYB13 #riak

  2. About me 3.5yrs @ Basho Technologies Software Engineer Erlang /

    Ruby Python / JavaScript Riak + Clients + Tools
  3. Eventual Consistency Scalability Schema-free

  4. How many of you...?

  5. How many of you...? rolling upgrade

  6. How many of you...? memcache / Redis rolling upgrade

  7. How many of you...? memcache / Redis CDN rolling upgrade

  8. How many of you...? memcache / Redis CDN search engine

    rolling upgrade
  9. How many of you...? memcache / Redis CDN search engine

    rolling upgrade background queue
  10. Consistency?

  11. Inconsistency!

  12. Already non-ACID Photo by Julie Jacobson

  13. Already non-ACID Photo by Julie Jacobson data everywhere

  14. Already non-ACID Photo by Julie Jacobson data everywhere availability =

    $
  15. Already non-ACID Photo by Julie Jacobson data everywhere availability =

    $ blind writes
  16. Already non-ACID Photo by Julie Jacobson data everywhere availability =

    $ blind writes DB not “ACID”
  17. Already non-ACID Photo by Julie Jacobson data everywhere availability =

    $ blind writes DB not “ACID” application invariants
  18. Eventual Consistency • Available • Responsive • Failure resistant •

    Convergent
  19. Safety & Liveness Leslie Lamport 1977

  20. Safety vs. Liveness “Bad things never happen.” “Good things eventually

    happen.”
  21. Eventual consistency is not safe “...it’s easy to satisfy liveness

    without being useful... If all replicas return the value 42 in response to every request, the system is eventually consistent.” http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/ Peter Bailis
  22. so convergent liveness amaze safety very facepalm wow such databas

  23. Scalability

  24. Scalability

  25. Scalability http://www.youtube.com/watch?v=b2F-DItXtZs

  26. Scalability COST BENEFIT throughput, capacity, latency, hit-rate money, time =

    =
  27. Schema Pain http://www.webassist.com/tutorials/Free-eCommerce-MySQL-Database • Restrictive formalism • Excessive normalization •

    Cost of changes • Maintaining relationships • Expensive joins
  28. Schema-Free • Data is denormalized • Single-access reads • Great

    for write-heavy loads • More duplication • Harder to query • Harder to evolve cold data
  29. distributed replicated highly available eventually-consistent key-value + extras free /

    open-source paid add-ons + support
  30. 1 2 3 Riak Eventual Consistency • Peer-to-peer replication •

    F+1 failure tolerance • Hinted handoff • Anti-entropy • Logical time (vector clocks)
  31. Riak Scalability • More nodes → more throughput • More

    replicas → greater reliability • More RAM → lower latency • Faster disk → more throughput, lower latency
  32. Riak Schema-Free • It’s just a “blob” • Key-Value access

    • “Bucket” namespaces • Indexing/search optional
  33. Riak Deployments

  34. Don’t Use Riak if... • Data fits in RAM on

    one machine • Data is tightly-coupled or highly structured • Application can’t handle eventual consistency* *Riak 2.0 will have more conflict-free datatypes
  35. None
  36. Practical

  37. Social Timelines • Write new updates to various collection points

    • Aggregate and de- dupe on read • Send notifications • Yammer, Formspring, Voxer http://basho.com/riak-and-scala-at-yammer/
  38. Session Storage • Big sessions • Quintessential “Dynamo” use-case (shopping

    cart) • Wikia, ideeli
  39. Content Management • Documents, design assets, large files • Riak

    CS - run your own S3 • AT&T, IDC Frontier
  40. Persistent Cache • Store rendered pages and snippets or coalesced

    records • Posterous, fast.ly • “A Riak machine was off and no one noticed.”
  41. Game Data • Backup mobile game data, play from another

    device • Record matchup history and stats • Rovio, Riot Games
  42. RICON & Riak 2.0 • RICON|WEST Jeff Dean Pat Helland

    • Riak 2.0 - Strong consistency, New search system, More datatypes, Security features
  43. Thanks @seancribbs #AYB13 #riak hugs!