Practical Eventually-Consistent Web Apps on Riak

"Eventual consistency! Scalability! Schema-free!" is the rallying call of the NoSQL world. But what does it mean to be eventually-consistent, to be scalable, to be schema-free? Why should you, as a web developer, care? Let's demystify these terms and then explore some ways that Riak, an "eventually-consistent, scalable, schema-free" datastore, can be put to practical use in web applications. You might just find that you've been using eventual consistency all along!

Transcript

1. Practical
   Eventually-Consistent
   Web Apps on
   Sean Cribbs
   @seancribbs #AYB13 #riak
   

   View Slide

2. About me
   3.5yrs @ Basho Technologies
   Software Engineer
   Erlang / Ruby
   Python / JavaScript
   Riak + Clients + Tools
   

   View Slide

3. Eventual
   Consistency
   Scalability
   Schema-free
   

   View Slide

4. How many of you...?
   

   View Slide

5. How many of you...?
   rolling upgrade
   

   View Slide

6. How many of you...?
   memcache / Redis
   rolling upgrade
   

   View Slide

7. How many of you...?
   memcache / Redis
   CDN
   rolling upgrade
   

   View Slide

8. How many of you...?
   memcache / Redis
   CDN
   search engine
   rolling upgrade
   

   View Slide

9. How many of you...?
   memcache / Redis
   CDN
   search engine
   rolling upgrade
   background queue
   

   View Slide

10. Consistency?
    

    View Slide

11. Inconsistency!
    

    View Slide

12. Already
    non-ACID
    Photo by Julie Jacobson
    

    View Slide

13. Already
    non-ACID
    Photo by Julie Jacobson
    data everywhere
    

    View Slide

14. Already
    non-ACID
    Photo by Julie Jacobson
    data everywhere
    availability = $
    

    View Slide

15. Already
    non-ACID
    Photo by Julie Jacobson
    data everywhere
    availability = $
    blind writes
    

    View Slide

16. Already
    non-ACID
    Photo by Julie Jacobson
    data everywhere
    availability = $
    blind writes
    DB not “ACID”
    

    View Slide

17. Already
    non-ACID
    Photo by Julie Jacobson
    data everywhere
    availability = $
    blind writes
    DB not “ACID”
    application invariants
    

    View Slide

18. Eventual Consistency
    • Available
    • Responsive
    • Failure resistant
    • Convergent
    

    View Slide

19. Safety &
    Liveness
    Leslie Lamport
    1977
    

    View Slide

20. Safety vs. Liveness
    “Bad things
    never happen.”
    “Good things
    eventually happen.”
    

    View Slide

21. Eventual
    consistency is
    not safe
    “...it’s easy to satisfy liveness
    without being useful... If all
    replicas return the value 42 in
    response to every request, the
    system is eventually
    consistent.”
    http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/
    Peter Bailis
    

    View Slide

22. so convergent
    liveness
    amaze
    safety
    very facepalm
    wow such databas
    

    View Slide

23. Scalability
    

    View Slide

24. Scalability
    

    View Slide

25. Scalability
    http://www.youtube.com/watch?v=b2F-DItXtZs
    

    View Slide

26. Scalability
    COST
    BENEFIT throughput, capacity, latency, hit-rate
    money, time
    =
    =
    

    View Slide

27. Schema Pain
    http://www.webassist.com/tutorials/Free-eCommerce-MySQL-Database
    • Restrictive formalism
    • Excessive normalization
    • Cost of changes
    • Maintaining
    relationships
    • Expensive joins
    

    View Slide

28. Schema-Free
    • Data is denormalized
    • Single-access reads
    • Great for write-heavy loads
    • More duplication
    • Harder to query
    • Harder to evolve cold data
    

    View Slide

29. distributed
    replicated
    highly available
    eventually-consistent
    key-value + extras
    free / open-source
    paid add-ons + support
    

    View Slide

30. 1
    2
    3
    Riak Eventual Consistency
    • Peer-to-peer
    replication
    • F+1 failure tolerance
    • Hinted handoff
    • Anti-entropy
    • Logical time
    (vector clocks)
    

    View Slide

31. Riak Scalability
    • More nodes →
    more throughput
    • More replicas →
    greater reliability
    • More RAM →
    lower latency
    • Faster disk →
    more throughput,
    lower latency
    

    View Slide

32. Riak Schema-Free
    • It’s just a “blob”
    • Key-Value access
    • “Bucket” namespaces
    • Indexing/search
    optional
    

    View Slide

33. Riak Deployments
    

    View Slide

34. Don’t Use Riak if...
    • Data fits in RAM on
    one machine
    • Data is tightly-coupled
    or highly structured
    • Application can’t
    handle eventual
    consistency*
    *Riak 2.0 will have more
    conflict-free datatypes
    

    View Slide

35. View Slide

36. Practical
    

    View Slide

37. Social Timelines
    • Write new updates to
    various collection
    points
    • Aggregate and de-
    dupe on read
    • Send notifications
    • Yammer, Formspring,
    Voxer
    http://basho.com/riak-and-scala-at-yammer/
    

    View Slide

38. Session Storage
    • Big sessions
    • Quintessential
    “Dynamo” use-case
    (shopping cart)
    • Wikia, ideeli
    

    View Slide

39. Content Management
    • Documents, design
    assets, large files
    • Riak CS - run your own
    S3
    • AT&T, IDC Frontier
    

    View Slide

40. Persistent Cache
    • Store rendered pages
    and snippets or
    coalesced records
    • Posterous, fast.ly
    • “A Riak machine was
    off and no one
    noticed.”
    

    View Slide

41. Game Data
    • Backup mobile game
    data, play from another
    device
    • Record matchup
    history and stats
    • Rovio, Riot Games
    

    View Slide

42. RICON & Riak 2.0
    • RICON|WEST
    Jeff Dean
    Pat Helland
    • Riak 2.0 - Strong
    consistency, New
    search system, More
    datatypes, Security
    features
    

    View Slide

43. Thanks
    @seancribbs
    #AYB13 #riak
    hugs!
    

    View Slide