Papers We Love: Managing Update Conflicts in Bayou

17 December 2014
http://www.meetup.com/papers-we-love-too/events/197678922/

Video TBA

Transcript

1. Papers We Love SF: Bayou Peter Bailis @pbailis UC Berkeley 17 December 2014
2. Who am I? Ph.D. candidate, 4th year student at Berkeley Study high performance distributed databases » When do we need coordination? » What happens if we don’t coordinate? Graduating 2015! http://bailis.org/
4. Why Bayou? “NoSQL” before “NoSQL” Learn from our predecessors Rethink application, system boundaries Lucid discussion of systems challenges
5. Talk Outline Background and system architecture Conflict detection and resolution APIs “Correctness” and ordering Lessons for all of us
6. Talk Outline Background and system architecture Conflict detection and resolution APIs “Correctness” and ordering Lessons for all of us
8. Bayou Goals Storage system for mobile devices:
9. Bayou Goals Storage system for mobile devices: Handle frequent disconnections
13. Bayou Goals Storage system for mobile devices: Handle frequent disconnections
14. Bayou Goals Storage system for mobile devices: Handle frequent disconnections Reason about distribution explicitly
16. Bayou Goals Storage system for mobile devices: Handle frequent disconnections Reason about distribution explicitly
17. Bayou Goals Storage system for mobile devices: Handle frequent disconnections Reason about distribution explicitly Facilitate conflict detection and resolution
18. How do we build a system that allows update- anywhere but allows users to easily build correct applications?
20. Basic Architecture
21. Basic Architecture Optimistically replicated distributed database
23. Basic Architecture Optimistically replicated distributed database Update-anywhere; CAP “AP” » Guarantees availability » Low latency and scalability! Use anti-entropy to exchange updates » Pick your favorite
25. Talk Outline Background and system architecture Conflict detection and resolution APIs “Correctness” and ordering Lessons for all of us
26. Talk Outline Background and system architecture Conflict detection and resolution APIs “Correctness” and ordering Lessons for all of us
27. Main problem: “conflicts” What happens if two clients simultaneously update the same piece of data?
28. Example: Peter wants to book Fastly HQ on 12/17 at 7PM Ines wants to book Fastly HQ on 12/17 at 7PM
29. Example: Peter wants to book Fastly HQ on 12/17 at 7PM Ines wants to book Fastly HQ on 12/17 at 7PM Peter and Ines are connected to different servers
30. Example: Peter wants to book Fastly HQ on 12/17 at 7PM Ines wants to book Fastly HQ on 12/17 at 7PM Peter and Ines are connected to different servers What will happen?
31. Example: Peter wants to book Fastly HQ on 12/17 at 7PM Ines wants to book Fastly HQ on 12/17 at 7PM Peter and Ines are connected to different servers What will happen? Example:
32. Example: Peter wants to book Fastly HQ on 12/17 at 7PM Ines wants to book Fastly HQ on 12/17 at 7PM Peter and Ines are connected to different servers What will happen? Example: Mike wants to give Peter $100 Carol wants to give Peter $100
33. Example: Peter wants to book Fastly HQ on 12/17 at 7PM Ines wants to book Fastly HQ on 12/17 at 7PM Peter and Ines are connected to different servers What will happen? Example: Mike wants to give Peter $100 Carol wants to give Peter $100 Mike and Carol are connected to different servers
34. Example: Peter wants to book Fastly HQ on 12/17 at 7PM Ines wants to book Fastly HQ on 12/17 at 7PM Peter and Ines are connected to different servers What will happen? Example: Mike wants to give Peter $100 Carol wants to give Peter $100 Mike and Carol are connected to different servers What will happen?
35. Main problem: “conflicts” What happens if two clients simultaneously update the same piece of data?
36. Main problem: “conflicts” It depends on the application! What happens if two clients simultaneously update the same piece of data?
37. Main problem: “conflicts” It depends on the application! Traditional DB (“serializability”): always* matters! What happens if two clients simultaneously update the same piece of data?
38. Main problem: “conflicts” It depends on the application! Traditional DB (“serializability”): always* matters! Bayou: let the application help us out! What happens if two clients simultaneously update the same piece of data?
39. Main problem: “conflicts” Bayou writes have three components:
40. Main problem: “conflicts” Bayou writes have three components: » An update function: the actual write
41. Main problem: “conflicts” Bayou writes have three components: » An update function: the actual write » A dependency check: conflict detection
42. Main problem: “conflicts” Bayou writes have three components: » An update function: the actual write » A dependency check: conflict detection » A merge function: conflict compensation
43. Main problem: “conflicts” Bayou writes have three components: » An update function: the actual write » A dependency check: conflict detection » A merge function: conflict compensation If dependency check passes: apply update function Else: apply merge function
44. Example: Booking tonight’s meetup:
45. Example: Booking tonight’s meetup: » Update function: insert 12/17/14, “Fastly HQ”, Ines
46. Example: Booking tonight’s meetup: » Update function: insert 12/17/14, “Fastly HQ”, Ines » Dependency check: Is the requested time and location available?
47. Example: Booking tonight’s meetup: » Update function: insert 12/17/14, “Fastly HQ”, Ines » Dependency check: Is the requested time and location available? » Merge function: try to find another time based on provided alternates
52. Inside the server: Date Location User Time 12/16 Fastly HQ Fred 2:30-5PM 12/18 Fastly HQ Artur 10AM-12PM 12/17 FastlyHQ Inez 6:30-9PM
53. Inside the server: Date Location User Time 12/16 Fastly HQ Fred 2:30-5PM 12/18 Fastly HQ Artur 10AM-12PM 12/17 FastlyHQ Inez 6:30-9PM
54. Inside the server: Date Location User Time 12/16 Fastly HQ Fred 2:30-5PM 12/18 Fastly HQ Artur 10AM-12PM 12/17 FastlyHQ Inez 6:30-9PM Update function: insert 12/17/14, “Fastly HQ”, Ines
55. Inside the server: Date Location User Time 12/16 Fastly HQ Fred 2:30-5PM 12/18 Fastly HQ Artur 10AM-12PM 12/17 FastlyHQ Inez 6:30-9PM Update function: insert 12/17/14, “Fastly HQ”, Ines Dependency check: Is the requested time and location available?
56. Inside the server: Date Location User Time 12/16 Fastly HQ Fred 2:30-5PM 12/18 Fastly HQ Artur 10AM-12PM 12/17 FastlyHQ Inez 6:30-9PM Update function: insert 12/17/14, “Fastly HQ”, Ines Dependency check: Is the requested time and location available?
66. Another example: Money transfer:
67. Another example: Money transfer: » Update function: transfer $100 from Jan to Mary
68. Another example: Money transfer: » Update function: transfer $100 from Jan to Mary » Dependency check: does Jan have $100 in her account?
69. Another example: Money transfer: » Update function: transfer $100 from Jan to Mary » Dependency check: does Jan have $100 in her account? » Merge function: log an error
70. User Balance Jan $110 Marsha $10 Peter $42 » Update function: transfer $100 from Jan to Marsha » Dependency check: does Jan have $100 in her account? » Merge function: log an error Errors
71. » Update function: transfer $100 from Jan to Marsha » Dependency check: does Jan have $100 in her account? » Merge function: log an error » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error User Balance Jan $10 Marsha $110 Peter $42 Errors
72. » Update function: transfer $100 from Jan to Marsha » Dependency check: does Jan have $100 in her account? » Merge function: log an error » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error Errors Jan->Peter failed! User Balance Jan $10 Marsha $110 Peter $42
86. Write stability How do we ensure that all servers eventually agree?
87. Write stability How do we ensure that all servers eventually agree? Decide on a “stable” prefix of writes:
88. Write stability How do we ensure that all servers eventually agree? Decide on a “stable” prefix of writes: » Strawman: order writes by timestamp
89. Write stability How do we ensure that all servers eventually agree? Decide on a “stable” prefix of writes: » Strawman: order writes by timestamp • Drawbacks?
90. » Update function: transfer $100 from Jan to Marsha » Dependency check: does Jan have $100 in her account? » Merge function: log an error User Balance Jan $10 Marsha $110 Peter $42 Errors Timestamp 10
91. » Update function: transfer $100 from Jan to Marsha » Dependency check: does Jan have $100 in her account? » Merge function: log an error » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error User Balance Jan $10 Marsha $110 Peter $42 Errors Timestamp 10
92. » Update function: transfer $100 from Jan to Marsha » Dependency check: does Jan have $100 in her account? » Merge function: log an error » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error User Balance Jan $10 Marsha $110 Peter $42 Errors Timestamp 10 Timestamp 2
93. User Balance Jan $110 Marsha $10 Peter $42 Errors
94. User Balance Jan $110 Marsha $10 Peter $42 Errors » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error Timestamp 2
95. User Balance Jan $10 Marsha $10 Peter $142 » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error Timestamp 2 Errors Jan->Marsha failed!
96. User Balance Jan $10 Marsha $10 Peter $142 » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error Timestamp 2 » Update function: transfer $100 from Jan to Marsha » Dependency check: does Jan have $100 in her account? » Merge function: log an error Timestamp 10 Errors Jan->Marsha failed!
98. 11 22
99. 11 22 28
100. 11 22 28 34
101. 11 22 28 34 13
102. 11 22 28 34 13
103. 11 22 28 34 13
104. 11 22 28 34 13 NEED TO RE-EXECUTE!
105. 11 22 28 34 13 NEED TO RE-EXECUTE! Also, makes GC hard
106. Write stability How do we ensure that all servers eventually agree? Decide on a “stable” prefix of writes: » Strawman: order writes by timestamp » Bayou: uses master to determine ordering
107. MASTER
108. MASTER
109. MASTER TENTATIVE COMMITTED T C T C T C
110. MASTER TENTATIVE COMMITTED T C T C T C
111. MASTER TENTATIVE COMMITTED T C T C T C
112. MASTER TENTATIVE COMMITTED T C T C T C
113. MASTER TENTATIVE COMMITTED T C T C T C
114. MASTER TENTATIVE COMMITTED T C T C T C
115. MASTER TENTATIVE COMMITTED T C T C T C
116. MASTER TENTATIVE COMMITTED T C T C T C
117. MASTER TENTATIVE COMMITTED T C T C T C
118. MASTER TENTATIVE COMMITTED T C T C T C
119. MASTER TENTATIVE COMMITTED T C T C T C
120. MASTER TENTATIVE COMMITTED T C T C T C
121. MASTER TENTATIVE COMMITTED T C T C T C
122. Write stability How do we ensure that all servers eventually agree? Decide on a “stable” prefix of writes: » Strawman: order writes by timestamp • Drawbacks? » Bayou: uses master to determine ordering • Benefits? Note: don’t require commutative updates!
123. Read API Can read from: » Stable storage: only committed writes » Tentative storage: all writes seen so far
124. Read API Can read from: » Stable storage: only committed writes » Tentative storage: all writes seen so far Why read from tentative storage at all?
127. Talk Outline Background and system architecture Conflict detection and resolution APIs “Correctness” and ordering Lessons for all of us
128. Talk Outline Background and system architecture Conflict detection and resolution APIs “Correctness” and ordering Lessons for all of us
129. Bayou’s Secret Sauce Push app logic into updates:
130. Bayou’s Secret Sauce Push app logic into updates: » Read and write are insufficiently expressive!
131. » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error » Update function: transfer $100 from Jan to Mary » Dependency check: does Jan have $100 in her account? » Merge function: log an error
132. » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error » Update function: transfer $100 from Jan to Mary » Dependency check: does Jan have $100 in her account? » Merge function: log an error WRITE: Jan = 10; Peter = 42 WRITE: Jan = 10; Mary = 110
133. » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error » Update function: transfer $100 from Jan to Mary » Dependency check: does Jan have $100 in her account? » Merge function: log an error WRITE: Jan = 10; Peter = 42 WRITE: Jan = 10; Mary = 110 » decrement Jan by $100; increment Peter by 100 » decrement Jan by $100; increment Mary by 100
134. Bayou’s Secret Sauce Push app logic into updates: » Read and write are insufficiently expressive!
135. Bayou’s Secret Sauce Push app logic into updates: » Read and write are insufficiently expressive! Capture transaction intent:
136. Bayou’s Secret Sauce Push app logic into updates: » Read and write are insufficiently expressive! Capture transaction intent: » Dependency checks encode preconditions » e.g., guarded atomic actions
137. What does Bayou guarantee? Eventually all updates are applied in the same order on all servers
138. What does Bayou guarantee? Eventually all updates are applied in the same order on all servers » Kind of like serializable/ACID transactions!
139. What does Bayou guarantee? Eventually all updates are applied in the same order on all servers » Kind of like serializable/ACID transactions! » Dependency checks enforce invariants
140. What does Bayou guarantee? Eventually all updates are applied in the same order on all servers » Kind of like serializable/ACID transactions! » Dependency checks enforce invariants » Did we just “beat CAP”?
141. What does Bayou guarantee? Eventually all updates are applied in the same order on all servers » Kind of like serializable/ACID transactions! » Dependency checks enforce invariants » Did we just “beat CAP”? Key: eventually means we have to wait
142. H-Store, VoltDB, Granola, Calvin, Atomic Broadcast Idea sketch: pre-schedule transactions, then execute them sequentially on respective servers
143. H-Store, VoltDB, Granola, Calvin, Atomic Broadcast Idea sketch: pre-schedule transactions, then execute them sequentially on respective servers Totally ordered outcomes on each replica
144. H-Store, VoltDB, Granola, Calvin, Atomic Broadcast Idea sketch: pre-schedule transactions, then execute them sequentially on respective servers Totally ordered outcomes on each replica …but ordering determined up front!
146. Bayou 㱺 “ACID”
147. Bayou 㱺 “ACID” 1.) Given transaction T, issue write W:
148. Bayou 㱺 “ACID” 1.) Given transaction T, issue write W: » Update function: // do nothing
149. Bayou 㱺 “ACID” 1.) Given transaction T, issue write W: » Update function: // do nothing » Dependency check: false
150. Bayou 㱺 “ACID” 1.) Given transaction T, issue write W: » Update function: // do nothing » Dependency check: false » Merge function: execute T
151. Bayou 㱺 “ACID” 1.) Given transaction T, issue write W: » Update function: // do nothing » Dependency check: false » Merge function: execute T 2.) Wait for write W to commit.
152. Bayou 㱺 “ACID” 1.) Given transaction T, issue write W: » Update function: // do nothing » Dependency check: false » Merge function: execute T 2.) Wait for write W to commit. 3.) Notify success.
154. What does Bayou guarantee? Eventually all writes are applied in the same order on all servers » Kind of like serializable/ACID transactions! » Dependency checks enforce invariants » Did we just “beat CAP”? Key: eventually means we have to wait
155. “#is meeting room scheduling program is intended for use after a group of people have already decided that they want to meet in a certain room and have determined a set of acceptable times for the meeting. It does not help them to determine a mutually agreeable place and time for the meeting, it only allows them to reserve the room.”
156. “#is meeting room scheduling program is intended for use after a group of people have already decided that they want to meet in a certain room and have determined a set of acceptable times for the meeting. It does not help them to determine a mutually agreeable place and time for the meeting, it only allows them to reserve the room.”
157. “#is meeting room scheduling program is intended for use after a group of people have already decided that they want to meet in a certain room and have determined a set of acceptable times for the meeting. It does not help them to determine a mutually agreeable place and time for the meeting, it only allows them to reserve the room.”
158. When can we avoid “waiting”?
159. When can we avoid “waiting”? Commutative logic need not be re-executed in the log! (Paper discusses this.)
160. A note on commutativity Commutative logic: decisions made in application logic are insensitive to ordering
161. A note on commutativity Commutative logic: decisions made in application logic are insensitive to ordering Commutative datatypes: operations on datatypes are insensitive to ordering #e latter are useful, but won’t ensure correctness! http://www.bailis.org/blog/data-integrity-and-problems-of-scope/
162. » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error » Update function: transfer $100 from Jan to Marsha » Dependency check: does Jan have $100 in her account? » Merge function: log an error » decrement Jan by $100; increment Peter by 100 » decrement Jan by $100; increment Marsha by 100 WRITE: Jan = 10; Peter = 42 WRITE: Jan = 10; Marsha = 110
163. A note on commutativity Commutative logic: decisions made in application logic are insensitive to ordering Commutative datatypes: operations on datatypes are insensitive to ordering #e latter are useful, but won’t ensure correctness! http://www.bailis.org/blog/data-integrity-and-problems-of-scope/
164. When can we avoid “waiting”? Commutative logic need not be re-executed in the log! (Paper discusses this.)
165. When can we avoid “waiting”? CALM !eorem: monotonic logic <=> determinism despite different orders [CIDR 2011] See also Kuper’s LVars Commutative logic need not be re-executed in the log! (Paper discusses this.)
166. When can we avoid “waiting”? CALM !eorem: monotonic logic <=> determinism despite different orders [CIDR 2011] See also Kuper’s LVars I-confluence: guarantees “safe” tentative reads with convergent and safe outcomes [VLDB 2015] Commutative logic need not be re-executed in the log! (Paper discusses this.)
167. A note on immutability
168. A note on immutability Bayou’s stable log is “immutable” » e.g., event sourcing, Lambda architecture
169. A note on immutability Bayou’s stable log is “immutable” » e.g., event sourcing, Lambda architecture But what guarantees can we make about the outcomes in the log?
170. A note on immutability Bayou’s stable log is “immutable” » e.g., event sourcing, Lambda architecture But what guarantees can we make about the outcomes in the log? » “Immutable” writes are the easy part! » Reasoning about outcomes is the challenge
171. Why have “merge” at all?
172. Why have “merge” at all? Why not just ship the stored procedures and re-execute them instead?
173. Why have “merge” at all? Why not just ship the stored procedures and re-execute them instead? » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error
174. Why have “merge” at all? Why not just ship the stored procedures and re-execute them instead? » Update function: transfer $100 from Jan to Peter » Dependency check: does Jan have $100 in her account? » Merge function: log an error If Jan has $100: transfer $100 from Jan to Peter Else: Log an error
175. a
176. ELE Edelweiss
177. Durability? In this context, durability and consistency are effectively orthogonal » Durability: “survive F faults, need F+1 servers” » Strong consistency: usually requires “majority”
178. Durability? In this context, durability and consistency are effectively orthogonal » Durability: “survive F faults, need F+1 servers” » Strong consistency: usually requires “majority” Bayou: local updates may not survive faults » But the (arguably) more fundamental part of the system is maintaining updates
179. Bayou vs. Dynamo Fully replicated Update-one In-order per-server sequenced anti-entropy Server-side merge Global stability detection No “strong” consistency Partially replicated Update-N Merkle tree-based anti- entropy Client-side merge No notion of, API for stability Regular registers via majority quorum
180. Talk Outline Background and system architecture Conflict detection and resolution APIs “Correctness” and ordering Lessons for all of us
181. Talk Outline Background and system architecture Conflict detection and resolution APIs “Correctness” and ordering Lessons for all of us
182. Why Bayou? “NoSQL” before “NoSQL” Learn from smart predecessors Rethink application, system boundaries Lucid discussion of systems challenges
183. Bayou Goals Handle frequent disconnections » Embrace replication, update-anywhere
184. Bayou Goals Handle frequent disconnections » Embrace replication, update-anywhere Reason about distribution explicitly » Require application semantics
185. Bayou Goals Handle frequent disconnections » Embrace replication, update-anywhere Reason about distribution explicitly » Require application semantics Facilitate conflict detection and resolution » Use merge and dependency APIs
186. “Weakly consistent replication has been used previously for availability, simplicity, and scalability in a variety of systems [3, 7, 10, 12, 15, 19].”
187. “Weakly consistent replication has been used previously for availability, simplicity, and scalability in a variety of systems [3, 7, 10, 12, 15, 19].”
188. Simplicity for whom?
189. Simplicity for whom? Architects Systems programmers Operators
190. Simplicity for whom? Architects Systems programmers Operators Application writers? Users?
191. Simplicity for whom? Architects Systems programmers Operators Application writers? Users? Analogous issues in RDBMS design! http://www.bailis.org/blog/understanding-weak- isolation-is-a-serious-problem/
192. Unfortunately, not always a choice!
193. THOSE LIGHT CONES_
194. 2.6 Billion Internet users in 2013 7.1 Billion humans on planet Earth Who cares about scale? Coordination? [Mary Meeker]
195. 2.6 Billion Internet users in 2013 7.1 Billion humans on planet Earth Who cares about scale? Coordination? Will data keep increasing? Why? Hint: not humans. [Mary Meeker]
202. Xerox PARC Laser printers Computer-generated bitmap graphics #e Graphical user interface, featuring windows and icons, operated with a mouse #e WYSIWYG text editor #e precursor to PostScript Ethernet as a local-area computer network Fully formed object-oriented programming in the Smalltalk programming language and integrated development environment. Model–view–controller software architecture Bayou! [Wikipedia]
203. Xerox PARC DEC SRC MSR SVC ???
204. What can we learn? 1.) Integrating application logic is key to “correct” execution of “AP” distributed systems. R/W is bad. 2.) Lack of app-specific mechanisms in a coordination- free system is a recipe for data corruption. 3.) Merge/repair is a narrow API for developers to express their application conflicts. 4.) Alternatives like commutativity, I-confluence, and research tools like Bloom can help limit overhead.
205. !anks Fastly & Ines!