Understanding Eventual Consistency and Riak

The success of applications and businesses on the Internet is often measured in converting visitors into customers, retaining user attentions and providing rich experiences, even to mobile devices.
All of these things require responsiveness -- low latency and constant availability.

Riak is a networked key-value database that enables applications to be highly-available and keep latencies low, but requires the developer to grasp eventual consistency. I'll discuss why this tradeoff to eventual consistency is necessary (including reviewing the CAP theorem), how Riak implements eventual consistency, and how developers can harness Riak to tackle hard availability problems.

Transcript

1. basho Understanding Eventual Consistency and Riak Sean Cribbs Basho Technologies @seancribbs
2. Approaching Eventual Consistency
3. Internet
4. Internet
5. Internet
6. Internet
7. Internet CDN
8. Internet CDN
9. Internet CDN M S S
10. Internet CDN
11. Is this system consistent?
12. You sacrificed ACID as soon as you added caching. Woops.
13. Theoretical Digression
14. FLP Impossibility http://the-paper-trail.org/blog/a-brief-tour-of-flp-impossibility/ Fischer, Lynch, Patterson - 1985
15. FLP Impossibility Perfect asynchronous consensus is impossible with even one fault. http://the-paper-trail.org/blog/a-brief-tour-of-flp-impossibility/ Fischer, Lynch, Patterson - 1985
16. FLP Impossibility Perfect asynchronous consensus is impossible with even one fault. http://the-paper-trail.org/blog/a-brief-tour-of-flp-impossibility/ Fischer, Lynch, Patterson - 1985
17. FLP Impossibility Perfect asynchronous consensus is impossible with even one fault. Failure and indefinite delay are indistinguishable. http://the-paper-trail.org/blog/a-brief-tour-of-flp-impossibility/ Fischer, Lynch, Patterson - 1985
18. CAP Brewer, et al. - 1997, 2001
19. CAP Under partition (failure), a system cannot maintain both availability and consistency. Brewer, et al. - 1997, 2001
20. CAP Under partition (failure), a system cannot maintain both availability and consistency. Brewer, et al. - 1997, 2001
21. CAP Under partition (failure), a system cannot maintain both availability and consistency. Practical systems favor one or the other. Brewer, et al. - 1997, 2001
22. Harvest & Yield Fox, Brewer - 2003?
23. Harvest & Yield Harvest: how much of dataset is reflected in a response (C) Fox, Brewer - 2003?
24. Harvest & Yield Harvest: how much of dataset is reflected in a response (C) Yield: how likely is the datastore to complete request (A) Fox, Brewer - 2003?
25. Harvest & Yield Harvest: how much of dataset is reflected in a response (C) Yield: how likely is the datastore to complete request (A) http://codahale.com/you-cant-sacrifice-partition-tolerance/ Fox, Brewer - 2003?
26. Safety & Liveness Leslie Lamport, 1977
27. Safety & Liveness Safety: bad things don’t happen agreement, durability, integrity Leslie Lamport, 1977
28. Safety & Liveness Safety: bad things don’t happen agreement, durability, integrity Liveness: good things eventually happen progress, responsiveness, resilience Leslie Lamport, 1977
29. Failure Tolerance
30. Failure Tolerance F + 1 at least one Eventual 2F + 1 majority Paxos, 2PC 3F + 1 super-majority Byzantine
31. Failure Tolerance F + 1 at least one Eventual 2F + 1 majority Paxos, 2PC 3F + 1 super-majority Byzantine
32. Failure Tolerance F + 1 at least one Eventual 2F + 1 majority Paxos, 2PC 3F + 1 super-majority Byzantine
33. Failure Tolerance F + 1 at least one Eventual 2F + 1 majority Paxos, 2PC 3F + 1 super-majority Byzantine “Any sufficiently large system is in a constant state of partial failure.” Justin Sheehy, Basho CTO
34. Failure Tolerance F + 1 at least one Eventual 2F + 1 majority Paxos, 2PC 3F + 1 super-majority Byzantine
35. Internet CDN
36. Eventual Consistency
37. Eventual Consistency Tolerates temporary inconsistency
38. Eventual Consistency Tolerates temporary inconsistency Moves toward consistency automatically
39. Eventual Consistency Tolerates temporary inconsistency Moves toward consistency automatically Resilient, maintains availability
40. Eventual Consistency Tolerates temporary inconsistency Moves toward consistency automatically Resilient, maintains availability “Eventual” in seconds, not days
41. Eventual Consistency Tolerates temporary inconsistency Moves toward consistency automatically Resilient, maintains availability “Eventual” in seconds, not days
42. What is Riak?
43. Riak is...
44. Riak is... a scalable
45. Riak is... a scalable highly-available
46. Riak is... a scalable highly-available networked/distributed
47. Riak is... a scalable highly-available networked/distributed key-value store
49. Riak’s Data Model
50. Riak’s Data Model Stores values against keys
51. Riak’s Data Model Stores values against keys Values are opaque
52. Riak’s Data Model Stores values against keys Values are opaque Keys are namespaced in buckets
53. Values Have Metadata
54. Values Have Metadata Content-Type
55. Values Have Metadata Content-Type Last-Modified
56. Values Have Metadata Content-Type Last-Modified Link
57. Values Have Metadata Content-Type Last-Modified Link Secondary index values
58. Values Have Metadata Content-Type Last-Modified Link Secondary index values Custom metadata
59. Values Have Metadata Content-Type Last-Modified Link Secondary index values Custom metadata Value + Metadata == “Object”
60. Basic Operations
61. Basic Operations GET /buckets/B/keys/K
62. Basic Operations GET /buckets/B/keys/K PUT /buckets/B/keys/K
63. Basic Operations GET /buckets/B/keys/K PUT /buckets/B/keys/K DELETE /buckets/B/keys/K
64. Basic Operations GET /buckets/B/keys/K PUT /buckets/B/keys/K DELETE /buckets/B/keys/K HTTP or Protocol Buffers
65. Other Query Methods
66. Other Query Methods Full-text Search
67. Other Query Methods Full-text Search Secondary Indexes
68. Other Query Methods Full-text Search Secondary Indexes MapReduce
69. Other Query Methods Full-text Search Secondary Indexes MapReduce Link-traversal (“walking”)
70. That’s kind of cool. But what about Eventual Consistency?
71. Replication
72. By default, 3 copies are stored (N). Replication
73. By default, 3 copies are stored (N). Consistent hashing and virtual nodes decouple storage from physical machines. Replication
74. By default, 3 copies are stored (N). Consistent hashing and virtual nodes decouple storage from physical machines. “Sloppy” quorum increases availability. Replication
75. By default, 3 copies are stored (N). Consistent hashing and virtual nodes decouple storage from physical machines. “Sloppy” quorum increases availability. Hinted handoff ensures eventual delivery. Replication
76. Eventual Consistency requires convergence. Vector clocks provide that.
77. Vector Clocks
78. Vector Clocks Each read includes a token of the current state (vector clock)
79. Vector Clocks Each read includes a token of the current state (vector clock) Clients send the token back with writes
80. Vector Clocks Each read includes a token of the current state (vector clock) Clients send the token back with writes Riak updates the token and detects stale versions and conflicts by comparison
81. Vector Clocks Each read includes a token of the current state (vector clock) Clients send the token back with writes Riak updates the token and detects stale versions and conflicts by comparison Conflicts are exposed to your application
82. Conflicts? That sounds bad.
83. It happens frequently without strong consistency. For example, have two threads write at the same time. Riak detects this conflict for you.
84. Your application is also part of the Eventually Consistent system. It makes the hard decisions that Riak can’t.
85. Ok. So how do I resolve conflicts?
86. Riak will give you all values that can’t be resolved automatically. Just write back the resolved value.
87. Think of it like merging or rebasing git branches. Sometimes you MUST use git mergetool to fix things.
88. Two Ways to Resolve
89. Two Ways to Resolve Semantic Resolution
90. Two Ways to Resolve Semantic Resolution CRDTs
91. Semantic Resolution
92. Semantic Resolution Given any number of values, compute the resolved version using business logic.
93. Example
94. Example If your value is a shopping cart, merge all items and sum quantities into a single cart.
95. Example If your value is a shopping cart, merge all items and sum quantities into a single cart. Maybe the customer buys more that way!
96. Conflict-Free Replicated Data-Types http://hal.inria.fr/inria-00397981/en/
97. Conflict-Free Replicated Data-Types Values that converge automatically http://hal.inria.fr/inria-00397981/en/
98. Conflict-Free Replicated Data-Types Values that converge automatically aka Bounded Join Semi-Lattices http://hal.inria.fr/inria-00397981/en/
99. Conflict-Free Replicated Data-Types Values that converge automatically aka Bounded Join Semi-Lattices Registers, counters, sets, graphs, etc http://hal.inria.fr/inria-00397981/en/
100. Conflict-Free Replicated Data-Types Values that converge automatically aka Bounded Join Semi-Lattices Registers, counters, sets, graphs, etc Active area of research in EU http://hal.inria.fr/inria-00397981/en/
101. A CRDT: G-Set
102. A CRDT: G-Set A set that only grows in elements
103. A CRDT: G-Set A set that only grows in elements Converges via set-union
104. A CRDT: G-Set A set that only grows in elements Converges via set-union Allows add and member operations
105. A CRDT: G-Set A set that only grows in elements Converges via set-union Allows add and member operations {A} ⋃ {B} = {A,B}
106. Example
107. Example A “like” button could use an Observed-Remove Set that contains all the users who clicked it.
108. Example A “like” button could use an Observed-Remove Set that contains all the users who clicked it. Each “like” is a separate add entry. To “unlike”, tombstone all the “likes” the user has added.
109. Riak 1.4 will likely have Counters Even more to come in later releases
110. Use-Cases
111. Use-Cases Session storage: Wikia
112. Use-Cases Session storage: Wikia Sensor data: Boundary, Temetra
113. Use-Cases Session storage: Wikia Sensor data: Boundary, Temetra Social: Yammer, Voxer
114. Use-Cases Session storage: Wikia Sensor data: Boundary, Temetra Social: Yammer, Voxer Gaming: Rovio
115. Use-Cases Session storage: Wikia Sensor data: Boundary, Temetra Social: Yammer, Voxer Gaming: Rovio Infrastructure: Comcast, IDC Frontier
116. Conclusion
117. Conclusion Any sufficiently large system is inconsistent and constantly failing
118. Conclusion Any sufficiently large system is inconsistent and constantly failing Being consistent or available under failure is a practical tradeoff and spectrum of choices
119. Conclusion Any sufficiently large system is inconsistent and constantly failing Being consistent or available under failure is a practical tradeoff and spectrum of choices Riak remains available during failures, progresses toward consistency