Slide 1

Slide 1 text

BRIDGING THE GAP: OPPORTUNITIES IN COORDINATION-AVOIDING DATABASE SYSTEMS Peter Bailis, Alan Fekete, Mike Franklin, Ali Ghodsi, Joe Hellerstein, Ion Stoica UC Berkeley and University of Sydney

Slide 2

Slide 2 text

Renaissance of EC? …but embrace is not unanimous!

Slide 3

Slide 3 text

“Designing applications to cope with concurrency anomalies in their data is very error-prone, time-consuming, and ultimately not worth the performance gains.” VLDB 2013

Slide 4

Slide 4 text

“Once you give up serializability, you fall off a cliff.” -A Certain Five-Star Wizard of the DB Community Personal communication, 2013

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

These accounts (and many others) suggest: a.) the benefits of EC are misunderstood b.) we underestimate programmability costs c.) some combination of the two OR OR

Slide 8

Slide 8 text

This talk: • A proposal: coordination-free execution is unifying principle behind availability, low latency, scalability • Coordination-avoidance is key: offer tight bound, successful application to high value applications These accounts (and many others) suggest: a.) the benefits of EC are misunderstood b.) we underestimate programmability costs c.) some combination of the two OR OR

Slide 9

Slide 9 text

EC Benefits: Availability Low Latency Scale-out/Efficiency

Slide 10

Slide 10 text

EC Benefits: Availability Low Latency Scale-out/Efficiency

Slide 11

Slide 11 text

Availability Stonebraker, CACM 2010 “The justification for giving up C is so that the A and P can be preserved.”

Slide 12

Slide 12 text

Availability “The justification for giving up C is so that the A and P can be preserved.” Stonebraker, CACM 2010 Danger: majority availability is often okay for practitioners (cf. F1, Spanner, many NewSQL) Danger: as stated, only applies to failure scenarios

Slide 13

Slide 13 text

EC Benefits: Availability Low Latency Scale-out/Efficiency

Slide 14

Slide 14 text

EC Benefits: Availability Low Latency Scale-out/Efficiency

Slide 15

Slide 15 text

Low Latency Any live replica can respond! Abadi, IEEE Computer 2013 LAN VS WAN up to 720x faster on EC2 (on average) Bailis et al. VLDB 2014

Slide 16

Slide 16 text

EC Benefits: Availability Low Latency Scale-out/Efficiency

Slide 17

Slide 17 text

EC Benefits: Availability Low Latency Scale-out/Efficiency

Slide 18

Slide 18 text

Efficiency/Scalability Can process requests without coordination, even for individual records “CP” distributed throughput: < 1/RTT 1 2 3 4 5 6 7 Number of Items per Transaction Throughput (txns/s) No Conflict Conflict “CP” “AP” “Coordination-Avoiding Database Systems” arXiv:1402.2237

Slide 19

Slide 19 text

+OR +CA +IR +SP +TO +SI +SY Participating Datacenters (+VA) 2 4 6 8 10 12 Maximum Throughput (txn/s) 2 4 6 8 10 12 14 16 18 20 Number of Servers in 2PC 0 200 400 600 800 1000 1200 Maximum Throughput (txns/s) LOCAL! DATACENTER MULTI-! DATACENTER! max 1200 txn/s max 12 txn/s decentralized (optimized) 2PC decentralized (optimized) 2PC Implementation-Independent “Coordination-Avoiding Database Systems” arXiv:1402.2237

Slide 20

Slide 20 text

EC Benefits: Availability Latency Scale-out/Efficiency In isolation, each under-represents the whole

Slide 21

Slide 21 text

EC Benefits: Availability Latency Scale-out/Efficiency One core mechanism: COORDINATION-FREE EXECUTION In isolation, each under-represents the whole

Slide 22

Slide 22 text

One core mechanism: COORDINATION-FREE EXECUTION Concurrent operations execute without communicating Like executing operations on separate, partitioned replicas “Coordination-Avoiding Databases” arXiv:1402.2237

Slide 23

Slide 23 text

One core mechanism: COORDINATION-FREE EXECUTION Coordination avoidance becomes goal of EC systems and programming Perhaps “just rhetoric” a.) EC benefits are many, but complicated b.) one mechanism captures all benefits c.) it’s implementation-agnostic , but:

Slide 24

Slide 24 text

When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants

Slide 25

Slide 25 text

When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants

Slide 26

Slide 26 text

“When is EC okay for my application?” “When do I have to give up EC benefits?”

Slide 27

Slide 27 text

When is coordination strictly required for correctness? (In both formal and practical senses.) Depends on application’s invariants and operations

Slide 28

Slide 28 text

SAFETY correctness always guaranteed LIVENESS database states agree (converge)

Slide 29

Slide 29 text

Convergence (e.g., CRDTs) and confluence (e.g., CALM) provide useful liveness guarantees SAFETY correctness always guaranteed LIVENESS database states agree (converge) When is coordination strictly required for correctness?

Slide 30

Slide 30 text

Convergence (e.g., CRDTs) and confluence (e.g., CALM) provide useful liveness guarantees SAFETY correctness always guaranteed LIVENESS database states agree (converge) Invariant Confluence ensures application-level safety properties also hold (in addition to liveness) When is coordination strictly required for correctness? “Coordination-Avoiding Database Systems” arXiv:1402.2237

Slide 31

Slide 31 text

Invariant Confluence is necessary and sufficient for ensuring safety, convergence, availability, and coordination-free execution. Invariant Confluence holds?! A safe, c-free execution strategy exists. Invariant Confluence fails?! No safe, c-free mechanism exists. When is coordination strictly required for correctness? “Coordination-Avoiding Database Systems” arXiv:1402.2237

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

Invariant Operation C.F. Equality, Inequality Any ??? Generate unique ID Any ??? Specify unique ID Insert ??? >! Increment ??? >! Decrement ??? < Decrement ??? < Increment ??? Foreign Key Insert ??? Foreign Key Delete ??? Secondary Indexing Any ??? Materialized Views Any ??? AUTO_INCREMENT Insert ??? Typical DB! operations and ! invariants! (SQL) “Coordination-Avoiding Database Systems” arXiv:1402.2237

Slide 34

Slide 34 text

Test fails? Cannot avoid coordination Invariant Operation C.F. Equality, Inequality Any Y Generate unique ID Any Y Specify unique ID Insert N >! Increment Y >! Decrement N < Decrement Y < Increment N Foreign Key Insert Y Foreign Key Delete Y* Secondary Indexing Any Y Materialized Views Any Y! AUTO_INCREMENT Insert N MANY TRADITIONAL DB APPS OK Typical DB! operations and ! invariants! (SQL) “Coordination-Avoiding Database Systems” arXiv:1402.2237

Slide 35

Slide 35 text

Applicable to real applications scale to! over 25x! prior best 0 50 100 150 200 2M 4M 6M 8M 10M 12M 14M Total Throughput (txn/s) 0 50 100 150 200 Number of Servers 0 20K 40K 60K 80K Throughput (txn/s/server) 6-11x faster than serializability 8 16 32 48 64 Number of Warehouses 40K 100K 600K Throughput (txns/s) Coordination-Avoiding Serializable (2PL) 10 of 12 invariants in TPC-C benchmark are I-confluent “Coordination-Avoiding Database Systems” arXiv:1402.2237

Slide 36

Slide 36 text

Opportunity • I-confluence offers tight bound on when coordination is actually required • SQL applications often use invariants already • Many invariants and operations are I-confluent; amenable to coordination-free execution

Slide 37

Slide 37 text

When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants

Slide 38

Slide 38 text

When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants

Slide 39

Slide 39 text

When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants

Slide 40

Slide 40 text

do not support! SSI/serializability HANA Actian Ingres YES Aerospike NO! N Persistit NO! N Clustrix NO! N Greenplum YES IBM DB2 YES IBM Informix YES MySQL YES MemSQL NO! N MS SQL Server YES NuoDB NO! N Oracle 11G NO! N Oracle BDB YES Oracle BDB JE YES Postgres 9.2.2 YES SAP Hana NO! N ScaleDB NO! N VoltDB YES 8/18 databases! surveyed did not 15/18 used! weaker models! by default “Highly Available Transactions: Virtues and Limitations” VLDB 2014

Slide 41

Slide 41 text

So which require coordination? Implementations typically use coordination (e.g. locking) But is this fundamental to the isolation levels?

Slide 42

Slide 42 text

So which require coordination? Implementations typically use coordination (e.g. locking) But is this fundamental to the isolation levels? NO! Unavailable Sticky Available Highly Available Legend prevents lost update†! prevents write skew‡! requires recency guarantees⊕ Sticky Available Unavailable Highly Available “Highly Available Transactions: Virtues and Limitations” VLDB 2014

Slide 43

Slide 43 text

So which require coordination? Implementations typically use coordination (e.g. locking) But is this fundamental to the isolation levels? NO! Unavailable Sticky Available Highly Available Legend prevents lost update†! prevents write skew‡! requires recency guarantees⊕ Sticky Available Unavailable Highly Available “Highly Available Transactions: Virtues and Limitations” VLDB 2014 Many popular isolation levels are achievable without coordination

Slide 44

Slide 44 text

• Weak isolation models are prevalent • Many weak isolation models are achievable without coordination • Applications running on weak isolation are excellent candidates for porting to an EC or coordination-free environment Opportunity

Slide 45

Slide 45 text

When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants

Slide 46

Slide 46 text

When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants

Slide 47

Slide 47 text

When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants

Slide 48

Slide 48 text

ACID Internals Highly concurrent algorithms allow read/write races but hide effects from end users Key: well-defined external specification allows flexibility in execution strategy From Srivinvasan and Carey, SIGMOD 1991 End-user semantics unaffected!

Slide 49

Slide 49 text

Example: RAMP Transactions Atomic visibility is sufficient for correctly maintaining foreign key constraints, secondary indexes, materialized views Traditional strategy: use mutual exclusion Coordination-free solution: multi-versioning and metadata C1 PREPARE PREPARE lastCommit[x]=⊥ versions={x1 } x1 , md={y} y1 , md={x} lastCommit[y]=⊥ versions={y1 } COMMIT tsc =1 COMMIT ts c =1 GET i=x, tsreq =⊥ GET lastCommit[x]=1 versions={x1 } GET x1 , md={y} y⊥, md={} y1 , md={x} lastCommit[y]=1 versions={y1 } vlatest ←{x:1, y:1} prepared RESPONSE prepared i=y, tsreq =⊥ i=y, tsreq =1 lastCommit[x]=⊥ versions={} lastCommit[y]=⊥ versions={} BEGIN T1 committed committed BEGIN T2 COMMIT T1 COMMIT T2 resp={x 1 ,y 1 } RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE C2 Px Py [w(x1 ), w(y1 )] [r(x), r(y)] SIGMOD 2014

Slide 50

Slide 50 text

• Leverage coordination-avoiding techniques in internal database design • End-user applications are faster but unaffected • Traditional mechanisms (often) not designed for distribution or partitioned environment Opportunity

Slide 51

Slide 51 text

When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants

Slide 52

Slide 52 text

When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants

Slide 53

Slide 53 text

When can we avoid coordination? 3.) ACID databases aren’t built using ACID transactions This talk: Three opportunities 1.) I-Confluence offers tight bound/answer 2.) ACID-backed apps aren’t always ACID-backed Directly analyze application invariants Leverage existing use of weak isolation Exploit existing interfaces and system invariants EXISTING PRACTICES RIPE FOR IMPROVEMENT NEW PRINCIPLE FOR OPTIMIZATION

Slide 54

Slide 54 text

Takeaways • Sole focus on availability, latency, or scale-out is myopic: coordination-free execution unifies them • Invariant confluence as tight bound is a powerful measure of optimization potential • Our take: scope for EC can be greatly expanded with study of existing application invariants, use of weak isolation, and well-defined system interfaces