Bridging the Gap: Opportunities in Coordination-Avoiding Database Systems

Talk from Principles and Practice of Eventual Consistency workshop (co-located with EuroSys 2014):
http://eventos.fct.unl.pt/papec/pages/program

Transcript

1. BRIDGING THE GAP: OPPORTUNITIES IN COORDINATION-AVOIDING DATABASE SYSTEMS Peter Bailis, Alan Fekete, Mike Franklin, Ali Ghodsi, Joe Hellerstein, Ion Stoica UC Berkeley and University of Sydney
2. Renaissance of EC? …but embrace is not unanimous!
3. “Designing applications to cope with concurrency anomalies in their data is very error-prone, time-consuming, and ultimately not worth the performance gains.” VLDB 2013
4. “Once you give up serializability, you fall off a cliff.” -A Certain Five-Star Wizard of the DB Community Personal communication, 2013
7. These accounts (and many others) suggest: a.) the benefits of EC are misunderstood b.) we underestimate programmability costs c.) some combination of the two OR OR
8. This talk: • A proposal: coordination-free execution is unifying principle behind availability, low latency, scalability • Coordination-avoidance is key: offer tight bound, successful application to high value applications These accounts (and many others) suggest: a.) the benefits of EC are misunderstood b.) we underestimate programmability costs c.) some combination of the two OR OR
9. EC Benefits: Availability Low Latency Scale-out/Efficiency
10. EC Benefits: Availability Low Latency Scale-out/Efficiency
11. Availability Stonebraker, CACM 2010 “The justification for giving up C is so that the A and P can be preserved.”
12. Availability “The justification for giving up C is so that the A and P can be preserved.” Stonebraker, CACM 2010 Danger: majority availability is often okay for practitioners (cf. F1, Spanner, many NewSQL) Danger: as stated, only applies to failure scenarios
13. EC Benefits: Availability Low Latency Scale-out/Efficiency
14. EC Benefits: Availability Low Latency Scale-out/Efficiency
15. Low Latency Any live replica can respond! Abadi, IEEE Computer 2013 LAN VS WAN up to 720x faster on EC2 (on average) Bailis et al. VLDB 2014
16. EC Benefits: Availability Low Latency Scale-out/Efficiency
17. EC Benefits: Availability Low Latency Scale-out/Efficiency
18. Efficiency/Scalability Can process requests without coordination, even for individual records “CP” distributed throughput: < 1/RTT 1 2 3 4 5 6 7 Number of Items per Transaction Throughput (txns/s) No Conflict Conflict “CP” “AP” “Coordination-Avoiding Database Systems” arXiv:1402.2237
19. +OR +CA +IR +SP +TO +SI +SY Participating Datacenters (+VA) 2 4 6 8 10 12 Maximum Throughput (txn/s) 2 4 6 8 10 12 14 16 18 20 Number of Servers in 2PC 0 200 400 600 800 1000 1200 Maximum Throughput (txns/s) LOCAL! DATACENTER MULTI-! DATACENTER! max 1200 txn/s max 12 txn/s decentralized (optimized) 2PC decentralized (optimized) 2PC Implementation-Independent “Coordination-Avoiding Database Systems” arXiv:1402.2237
20. EC Benefits: Availability Latency Scale-out/Efficiency In isolation, each under-represents the whole
21. EC Benefits: Availability Latency Scale-out/Efficiency One core mechanism: COORDINATION-FREE EXECUTION In isolation, each under-represents the whole
22. One core mechanism: COORDINATION-FREE EXECUTION Concurrent operations execute without communicating Like executing operations on separate, partitioned replicas “Coordination-Avoiding Databases” arXiv:1402.2237
23. One core mechanism: COORDINATION-FREE EXECUTION Coordination avoidance becomes goal of EC systems and programming Perhaps “just rhetoric” a.) EC benefits are many, but complicated b.) one mechanism captures all benefits c.) it’s implementation-agnostic , but:
24. When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants
25. When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants
26. “When is EC okay for my application?” “When do I have to give up EC benefits?”
27. When is coordination strictly required for correctness? (In both formal and practical senses.) Depends on application’s invariants and operations
28. SAFETY correctness always guaranteed LIVENESS database states agree (converge)
29. Convergence (e.g., CRDTs) and confluence (e.g., CALM) provide useful liveness guarantees SAFETY correctness always guaranteed LIVENESS database states agree (converge) When is coordination strictly required for correctness?
30. Convergence (e.g., CRDTs) and confluence (e.g., CALM) provide useful liveness guarantees SAFETY correctness always guaranteed LIVENESS database states agree (converge) Invariant Confluence ensures application-level safety properties also hold (in addition to liveness) When is coordination strictly required for correctness? “Coordination-Avoiding Database Systems” arXiv:1402.2237
31. Invariant Confluence is necessary and sufficient for ensuring safety, convergence, availability, and coordination-free execution. Invariant Confluence holds?! A safe, c-free execution strategy exists. Invariant Confluence fails?! No safe, c-free mechanism exists. When is coordination strictly required for correctness? “Coordination-Avoiding Database Systems” arXiv:1402.2237
33. Invariant Operation C.F. Equality, Inequality Any ??? Generate unique ID Any ??? Specify unique ID Insert ??? >! Increment ??? >! Decrement ??? < Decrement ??? < Increment ??? Foreign Key Insert ??? Foreign Key Delete ??? Secondary Indexing Any ??? Materialized Views Any ??? AUTO_INCREMENT Insert ??? Typical DB! operations and ! invariants! (SQL) “Coordination-Avoiding Database Systems” arXiv:1402.2237
34. Test fails? Cannot avoid coordination Invariant Operation C.F. Equality, Inequality Any Y Generate unique ID Any Y Specify unique ID Insert N >! Increment Y >! Decrement N < Decrement Y < Increment N Foreign Key Insert Y Foreign Key Delete Y* Secondary Indexing Any Y Materialized Views Any Y! AUTO_INCREMENT Insert N MANY TRADITIONAL DB APPS OK Typical DB! operations and ! invariants! (SQL) “Coordination-Avoiding Database Systems” arXiv:1402.2237
35. Applicable to real applications scale to! over 25x! prior best 0 50 100 150 200 2M 4M 6M 8M 10M 12M 14M Total Throughput (txn/s) 0 50 100 150 200 Number of Servers 0 20K 40K 60K 80K Throughput (txn/s/server) 6-11x faster than serializability 8 16 32 48 64 Number of Warehouses 40K 100K 600K Throughput (txns/s) Coordination-Avoiding Serializable (2PL) 10 of 12 invariants in TPC-C benchmark are I-confluent “Coordination-Avoiding Database Systems” arXiv:1402.2237
36. Opportunity • I-confluence offers tight bound on when coordination is actually required • SQL applications often use invariants already • Many invariants and operations are I-confluent; amenable to coordination-free execution
37. When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants
38. When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants
39. When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants
40. do not support! SSI/serializability HANA Actian Ingres YES Aerospike NO! N Persistit NO! N Clustrix NO! N Greenplum YES IBM DB2 YES IBM Informix YES MySQL YES MemSQL NO! N MS SQL Server YES NuoDB NO! N Oracle 11G NO! N Oracle BDB YES Oracle BDB JE YES Postgres 9.2.2 YES SAP Hana NO! N ScaleDB NO! N VoltDB YES 8/18 databases! surveyed did not 15/18 used! weaker models! by default “Highly Available Transactions: Virtues and Limitations” VLDB 2014
41. So which require coordination? Implementations typically use coordination (e.g. locking) But is this fundamental to the isolation levels?
42. So which require coordination? Implementations typically use coordination (e.g. locking) But is this fundamental to the isolation levels? NO! Unavailable Sticky Available Highly Available Legend prevents lost update†! prevents write skew‡! requires recency guarantees⊕ Sticky Available Unavailable Highly Available “Highly Available Transactions: Virtues and Limitations” VLDB 2014
43. So which require coordination? Implementations typically use coordination (e.g. locking) But is this fundamental to the isolation levels? NO! Unavailable Sticky Available Highly Available Legend prevents lost update†! prevents write skew‡! requires recency guarantees⊕ Sticky Available Unavailable Highly Available “Highly Available Transactions: Virtues and Limitations” VLDB 2014 Many popular isolation levels are achievable without coordination
44. • Weak isolation models are prevalent • Many weak isolation models are achievable without coordination • Applications running on weak isolation are excellent candidates for porting to an EC or coordination-free environment Opportunity
45. When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants
46. When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants
47. When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants
48. ACID Internals Highly concurrent algorithms allow read/write races but hide effects from end users Key: well-defined external specification allows flexibility in execution strategy From Srivinvasan and Carey, SIGMOD 1991 End-user semantics unaffected!
49. Example: RAMP Transactions Atomic visibility is sufficient for correctly maintaining foreign key constraints, secondary indexes, materialized views Traditional strategy: use mutual exclusion Coordination-free solution: multi-versioning and metadata C1 PREPARE PREPARE lastCommit[x]=⊥ versions={x1 } x1 , md={y} y1 , md={x} lastCommit[y]=⊥ versions={y1 } COMMIT tsc =1 COMMIT ts c =1 GET i=x, tsreq =⊥ GET lastCommit[x]=1 versions={x1 } GET x1 , md={y} y⊥, md={} y1 , md={x} lastCommit[y]=1 versions={y1 } vlatest ←{x:1, y:1} prepared RESPONSE prepared i=y, tsreq =⊥ i=y, tsreq =1 lastCommit[x]=⊥ versions={} lastCommit[y]=⊥ versions={} BEGIN T1 committed committed BEGIN T2 COMMIT T1 COMMIT T2 resp={x 1 ,y 1 } RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE C2 Px Py [w(x1 ), w(y1 )] [r(x), r(y)] SIGMOD 2014
50. • Leverage coordination-avoiding techniques in internal database design • End-user applications are faster but unaffected • Traditional mechanisms (often) not designed for distribution or partitioned environment Opportunity
51. When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants
52. When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants
53. When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants EXISTING PRACTICES RIPE FOR IMPROVEMENT NEW PRINCIPLE FOR OPTIMIZATION
54. Takeaways • Sole focus on availability, latency, or scale-out is myopic: coordination-free execution unifies them • Invariant confluence as tight bound is a powerful measure of optimization potential • Our take: scope for EC can be greatly expanded with study of existing application invariants, use of weak isolation, and well-defined system interfaces