$30 off During Our Annual Pro Sale. View Details »

Practical Eventually-Consistent Web Apps on Riak

Practical Eventually-Consistent Web Apps on Riak

"Eventual consistency! Scalability! Schema-free!" is the rallying call of the NoSQL world. But what does it mean to be eventually-consistent, to be scalable, to be schema-free? Why should you, as a web developer, care? Let's demystify these terms and then explore some ways that Riak, an "eventually-consistent, scalable, schema-free" datastore, can be put to practical use in web applications. You might just find that you've been using eventual consistency all along!

Sean Cribbs

October 18, 2013
Tweet

More Decks by Sean Cribbs

Other Decks in Programming

Transcript

  1. Practical
    Eventually-Consistent
    Web Apps on
    Sean Cribbs
    @seancribbs #AYB13 #riak

    View Slide

  2. About me
    3.5yrs @ Basho Technologies
    Software Engineer
    Erlang / Ruby
    Python / JavaScript
    Riak + Clients + Tools

    View Slide

  3. Eventual
    Consistency
    Scalability
    Schema-free

    View Slide

  4. How many of you...?

    View Slide

  5. How many of you...?
    rolling upgrade

    View Slide

  6. How many of you...?
    memcache / Redis
    rolling upgrade

    View Slide

  7. How many of you...?
    memcache / Redis
    CDN
    rolling upgrade

    View Slide

  8. How many of you...?
    memcache / Redis
    CDN
    search engine
    rolling upgrade

    View Slide

  9. How many of you...?
    memcache / Redis
    CDN
    search engine
    rolling upgrade
    background queue

    View Slide

  10. Consistency?

    View Slide

  11. Inconsistency!

    View Slide

  12. Already
    non-ACID
    Photo by Julie Jacobson

    View Slide

  13. Already
    non-ACID
    Photo by Julie Jacobson
    data everywhere

    View Slide

  14. Already
    non-ACID
    Photo by Julie Jacobson
    data everywhere
    availability = $

    View Slide

  15. Already
    non-ACID
    Photo by Julie Jacobson
    data everywhere
    availability = $
    blind writes

    View Slide

  16. Already
    non-ACID
    Photo by Julie Jacobson
    data everywhere
    availability = $
    blind writes
    DB not “ACID”

    View Slide

  17. Already
    non-ACID
    Photo by Julie Jacobson
    data everywhere
    availability = $
    blind writes
    DB not “ACID”
    application invariants

    View Slide

  18. Eventual Consistency
    • Available
    • Responsive
    • Failure resistant
    • Convergent

    View Slide

  19. Safety &
    Liveness
    Leslie Lamport
    1977

    View Slide

  20. Safety vs. Liveness
    “Bad things
    never happen.”
    “Good things
    eventually happen.”

    View Slide

  21. Eventual
    consistency is
    not safe
    “...it’s easy to satisfy liveness
    without being useful... If all
    replicas return the value 42 in
    response to every request, the
    system is eventually
    consistent.”
    http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/
    Peter Bailis

    View Slide

  22. so convergent
    liveness
    amaze
    safety
    very facepalm
    wow such databas

    View Slide

  23. Scalability

    View Slide

  24. Scalability

    View Slide

  25. Scalability
    http://www.youtube.com/watch?v=b2F-DItXtZs

    View Slide

  26. Scalability
    COST
    BENEFIT throughput, capacity, latency, hit-rate
    money, time
    =
    =

    View Slide

  27. Schema Pain
    http://www.webassist.com/tutorials/Free-eCommerce-MySQL-Database
    • Restrictive formalism
    • Excessive normalization
    • Cost of changes
    • Maintaining
    relationships
    • Expensive joins

    View Slide

  28. Schema-Free
    • Data is denormalized
    • Single-access reads
    • Great for write-heavy loads
    • More duplication
    • Harder to query
    • Harder to evolve cold data

    View Slide

  29. distributed
    replicated
    highly available
    eventually-consistent
    key-value + extras
    free / open-source
    paid add-ons + support

    View Slide

  30. 1
    2
    3
    Riak Eventual Consistency
    • Peer-to-peer
    replication
    • F+1 failure tolerance
    • Hinted handoff
    • Anti-entropy
    • Logical time
    (vector clocks)

    View Slide

  31. Riak Scalability
    • More nodes →
    more throughput
    • More replicas →
    greater reliability
    • More RAM →
    lower latency
    • Faster disk →
    more throughput,
    lower latency

    View Slide

  32. Riak Schema-Free
    • It’s just a “blob”
    • Key-Value access
    • “Bucket” namespaces
    • Indexing/search
    optional

    View Slide

  33. Riak Deployments

    View Slide

  34. Don’t Use Riak if...
    • Data fits in RAM on
    one machine
    • Data is tightly-coupled
    or highly structured
    • Application can’t
    handle eventual
    consistency*
    *Riak 2.0 will have more
    conflict-free datatypes

    View Slide

  35. View Slide

  36. Practical

    View Slide

  37. Social Timelines
    • Write new updates to
    various collection
    points
    • Aggregate and de-
    dupe on read
    • Send notifications
    • Yammer, Formspring,
    Voxer
    http://basho.com/riak-and-scala-at-yammer/

    View Slide

  38. Session Storage
    • Big sessions
    • Quintessential
    “Dynamo” use-case
    (shopping cart)
    • Wikia, ideeli

    View Slide

  39. Content Management
    • Documents, design
    assets, large files
    • Riak CS - run your own
    S3
    • AT&T, IDC Frontier

    View Slide

  40. Persistent Cache
    • Store rendered pages
    and snippets or
    coalesced records
    • Posterous, fast.ly
    • “A Riak machine was
    off and no one
    noticed.”

    View Slide

  41. Game Data
    • Backup mobile game
    data, play from another
    device
    • Record matchup
    history and stats
    • Rovio, Riot Games

    View Slide

  42. RICON & Riak 2.0
    • RICON|WEST
    Jeff Dean
    Pat Helland
    • Riak 2.0 - Strong
    consistency, New
    search system, More
    datatypes, Security
    features

    View Slide

  43. Thanks
    @seancribbs
    #AYB13 #riak
    hugs!

    View Slide