Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Riak Lunch'n'learn June 2013

Riak Lunch'n'learn June 2013

A short talk about Riak, Basho & Dynamo, Presented at a Techcube Lunch'n'learn.

Sam Elliott

June 25, 2013
Tweet

More Decks by Sam Elliott

Other Decks in Technology

Transcript

  1. Sam Elliott - Basho
    Techcube - 2013/06

    View full-size slide

  2. Riak
    What is Riak?
    Riak is an open source distributed, key-value database.
    Riak is architected for Availability, Fault-Tolerance, Operational
    Simplicity, and Scalability.

    View full-size slide

  3. Basho & Riak
    • Basho originally wrote Riak
    • Riak is now Open-Source
    • Developers employed by Basho
    • Basho sell Enterprise Features &
    Support

    View full-size slide

  4. Dynamo
    Dynamo: Amazon’s Highly Available Key-value Store
    This paper presents the design and implementation of Dynamo, a
    highly available key-value storage system that some of Amazons
    core services use to provide an always-on experience.

    View full-size slide

  5. CAP Theorem
    • Consistency
    • Availability
    • Partition-tolerance

    View full-size slide

  6. CAP Theorem
    • Consistency
    • Availability
    • Partition-tolerance

    View full-size slide

  7. CAP Theorem
    • Consistency
    • Availability
    • Partition-tolerance

    View full-size slide

  8. CAP Theorem
    • Consistency
    • Availability
    • Partition-tolerance
    Eventual Consistency
    A value will eventually see all its updates, once all partition
    repairs are complete.

    View full-size slide

  9. Consistent Hashing
    0
    2160

    View full-size slide

  10. Consistent Hashing
    0
    2160 hash(<>,<>)

    View full-size slide

  11. Consistent Hashing
    0
    2160 hash(<>,<>)
    riak1.example.com

    View full-size slide

  12. Consistent Hashing
    0
    2160 hash(<>,<>)
    riak1.example.com

    View full-size slide

  13. Consistent Hashing
    0
    2160
    riak1.example.com

    View full-size slide

  14. Replicas & Quorums
    0
    2160 hash(<>,<>)
    read(<>, <>)

    View full-size slide

  15. Replicas & Quorums
    0
    2160 hash(<>,<>)
    read(<>, <>, N=3)

    View full-size slide

  16. Replicas & Quorums
    0
    2160 hash(<>,<>)
    read(<>, <>, N=3)

    View full-size slide

  17. Replicas & Quorums
    0
    2160 hash(<>,<>)
    read(<>, <>, N=3, R=2)

    View full-size slide

  18. Replicas & Quorums
    0
    2160 hash(<>,<>)
    read(<>, <>, N=3, R=2)

    View full-size slide

  19. Quorum Options
    n r w
    Also: Primary & Durable

    View full-size slide

  20. Nodes & Virtual Nodes
    riak1.example.com
    riak2.example.com
    riak3.example.com
    riak4.example.com

    View full-size slide

  21. Nodes & Virtual Nodes
    riak1.example.com
    riak2.example.com
    riak3.example.com
    riak4.example.com

    View full-size slide

  22. Nodes & Virtual Nodes
    riak1.example.com 11
    riak2.example.com 11
    riak3.example.com 10
    riak4.example.com 0

    View full-size slide

  23. Nodes & Virtual Nodes
    riak1.example.com 8
    riak2.example.com 8
    riak3.example.com 8
    riak4.example.com 8

    View full-size slide

  24. Nodes & Virtual Nodes
    riak1.example.com
    riak2.example.com
    riak3.example.com
    riak4.example.com

    View full-size slide

  25. Extras
    Not only a Key-Value store. We also have:
    • MapReduce (Erlang & JS)
    • Links & Link-walking
    • Secondary Indexes
    • Fulltext Search (& More Soon)
    • CRDTs

    View full-size slide

  26. Extras
    Not only a Key-Value store. We also have:
    • MapReduce (Erlang & JS)
    • Links & Link-walking
    • Secondary Indexes
    • Fulltext Search (& More Soon)
    • CRDTs

    View full-size slide

  27. Riak Search
    A hand-rolled Solr-like Search Engine, built into Riak

    View full-size slide

  28. Riak Search - Mk II (Coming Soon)
    Yokozuna
    Integrate Solr itself with a Riak Cluster: use Riak for distribution
    and Solr for searching.
    https://github.com/basho/yokozuna

    View full-size slide

  29. Concurrent Updates
    1

    View full-size slide

  30. Concurrent Updates
    1 2

    View full-size slide

  31. Concurrent Updates
    1 2
    3a

    View full-size slide

  32. Concurrent Updates
    1 2
    3a
    3b

    View full-size slide

  33. Concurrent Updates
    1 2
    3a
    3b
    4

    View full-size slide

  34. Concurrent Updates
    1 2
    3a
    3b
    4

    View full-size slide

  35. Concurrent Updates
    1 2
    3a
    3b
    4

    View full-size slide

  36. An Approach
    One Approach to Eventual Consistency:
    • Use allow_mult=true (not the default)
    • Teach your application how to resolve conflicts
    • Fetch all siblings
    • Replace siblings with merged value

    View full-size slide

  37. Another Approach
    • Each “actor” maintains their own per-actor data
    • Upon fetch, merged into a single object
    In General:
    • Store the data by “actor”
    • Teach your application how to resolve that structure into an
    object

    View full-size slide

  38. Another Approach
    • Each “actor” maintains their own per-actor data
    • Upon fetch, merged into a single object
    In General:
    • Store the data by “actor”
    • Teach your application how to resolve that structure into an
    object
    • CRDTs: Convergent Replicated Data Types

    View full-size slide

  39. CRDTs
    Convergent Replicated Data Types
    Data Structures that Riak knows how to merge, preserving all
    operations.
    • Counters (PN-Counter, ...)
    • Sets (OR-Set, ...)
    • Graphs
    • TreeDoc (Collaborative Doc)
    • Maps (field name → CRDT)

    View full-size slide

  40. Riak EE
    Enterprise Edition
    Riak with:
    • Multi Data Centre Replication
    • Better Statistics

    View full-size slide

  41. Riak CS
    Riak CS
    An S3-compatible Multi-tenant Storage System

    View full-size slide

  42. Conclusions
    • Distributed databases aren’t trivial
    • Riak doesn’t hide anything
    • CRDTs are where it’s at for EC

    View full-size slide