When Eventual Consistency is Not Enough: from Causality to Transactions

Slides for talk at RICON 2015, where I discuss how to provide extra guarantees on key-value stores beyond eventual consistency and the associated prices.

Transcript

1. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary When Weak Consistency is not Enough: From Causality to Transactions RICON 2015 Jo˜ ao Leit˜ ao November 5th 2015
2. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Roadmap 1 Introduction 2 More than eventual consistency: ChainReaction 3 Strong Consistency and Transactional Support: Blotter 4 Summary
3. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction Who am I?
4. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction Who am I? RICON 2014
5. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction Who am I? Live and Work in Portugal
6. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction Who am I? Live and Work in Portugal (Lisbon)
7. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction Who am I? Postdoctoral Researcher and Integrated Member at the NOVA Laboratory for Computer Science and Informatics (NOVA LINCS) Web page: http://novalincs.fct.unl.pt Follow us on Twitter: @novalincs
8. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction Who am I? Invited Assistant Professor at the Computer Science Department (DI) of the School of Sciences and Technology in the NOVA University of Lisbon Web page: http://di.fct.unl.pt
9. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction Who am I? Research: Large-Scale Systems: Geo-Distributed/Geo-Replicated Systems. Peer-to-Peer Systems. My focus: Scalability. Fault-Tolerance and Dependability. Performance.
10. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction Who am I? Research: Large-Scale Systems: Geo-Distributed/Geo-Replicated Systems. Peer-to-Peer Systems. My focus: Scalability. Fault-Tolerance and Dependability. Performance.
11. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction Who am I? I have two cats...
12. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction Motivation Large Scale Applications: Large number of applications based on the Cloud Computing paradigm. Huge number of users. Large amount of data. High system load (and different applications exhibit different types of load).
13. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction Data Storage Systems Data Storage Systems Require the following mechanisms: Geo-Replication. Support concurrent reads and updates. Conflicting Goals High performance. Semantics that enable developers to write correct applications. Consistency Guarantees. Transactional Support.
14. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction Data Storage Systems Data Storage Systems Require the following mechanisms: Geo-Replication. Support concurrent reads and updates. Conflicting Goals High performance. Semantics that enable developers to write correct applications. Consistency Guarantees. Transactional Support.
15. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough If special care is not taken, eventual consistency can yield unexpected behaviours. Chat example!
16. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$
17. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$
18. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$ B$please$tell$us$a$lie!$
19. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$ B$please$tell$us$a$lie!$
20. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$ B$please$tell$us$a$lie!$ B$please$tell$us$a$lie!$
21. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$ B$please$tell$us$a$lie!$ B$please$tell$us$a$lie!$ C$is$$#”%!$
22. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$ B$please$tell$us$a$lie!$ B$please$tell$us$a$lie!$ C$is$$#”%!$ C$is$$#”%!$
23. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$ B$please$tell$us$a$lie!$ B$please$tell$us$a$lie!$ C$is$$#”%!$ C$is$$#”%!$
24. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough What went wrong here? There was a relationship of causality between those two events. However the datastore is unaware of this and hence does take any special care on this situation. What’s the solution? Provide causal consistency (or stronger).
25. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough What went wrong here? There was a relationship of causality between those two events. However the datastore is unaware of this and hence does take any special care on this situation. What’s the solution? Provide causal consistency (or stronger).
26. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough If special care is not taken, eventual consistency can yield completely wrong behaviours. Ticket sale example!
27. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$
28. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$ THERE%IS%ONLY%1% TICKET%LEFT%TO%BE% SOLD%
29. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$
30. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$
31. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$
32. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$
33. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough DC1$ DC2$ DC3$ User$A$ User$B$ User$C$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$ User:$Is$there$a$2cket?$ System:$yes!$ User:$Buy$the$2cket!$ System:$ok$
34. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough What went wrong here? There was a single resource at dispute here, so all operations that attempt to manipulate it must be coordinated in some way at the datastore level. What’s the solution? Offer strong consistency. Provide support for transactions.
35. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough What went wrong here? There was a single resource at dispute here, so all operations that attempt to manipulate it must be coordinated in some way at the datastore level. What’s the solution? Offer strong consistency. Provide support for transactions.
36. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction When eventual consistency is not enough What went wrong here? There was a single resource at dispute here, so all operations that attempt to manipulate it must be coordinated in some way at the datastore level. What’s the solution? Offer strong consistency. Provide support for transactions.
37. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction In This Talk More than eventual consistency: ChainReaction Provide Causal+ consistency on operations over a single object. Both in single datacenter and multiple datacenter deployments. Focus: efficient metadata and performance. Strong Consistency and Transactional Support: Blotter Provide strong consistency with transactional support. Both in single datacenter and multiple datacenter deployments. Focus: efficient coordination and performance.
38. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Introduction In This Talk More than eventual consistency: ChainReaction Provide Causal+ consistency on operations over a single object. Both in single datacenter and multiple datacenter deployments. Focus: efficient metadata and performance. Strong Consistency and Transactional Support: Blotter Provide strong consistency with transactional support. Both in single datacenter and multiple datacenter deployments. Focus: efficient coordination and performance.
39. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Roadmap 1 Introduction 2 More than eventual consistency: ChainReaction Preliminaries Architecture Overview of Replication Mechanisms Extending to Geo-distributed Environment Experimental Evaluation Lessons Learned 3 Strong Consistency and Transactional Support: Blotter 4 Summary
40. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: Causal+ Credits Sérgio'Almeida' Luís'Rodrigues'
41. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: Causal+ Credits ChainReaction a Causal+ Consistent Datastore based on Chain Replication. S. Almeida, J. Leit˜ ao, and L. Rodrigues. Proceedings of the EuroSys 2013, Prague, Czech Republic, April 2013.
42. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Replication and Consistency Consistency Models: Causal+ Consistency. Replication Mechanisms: Chain Replication (variant).
43. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Replication and Consistency Consistency Models: Causal+ Consistency. Replication Mechanisms: Chain Replication (variant).
44. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication Simple replication mechanism. Provides linearizability (for a single object). Network friendly (point-to-point communication). Pipelined writes.
45. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,0>$ <A,0>$ <A,0>$ <A,0>$ <A,0>$ App.$ Proxy$ Client$$$
46. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,0>   <A,0>   <A,0>   <A,0>   <A,0>   App.   Proxy   Client       HEAD   TAIL  
47. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,0>   <A,0>   <A,0>   <A,0>   <A,0>   App.   Proxy   Client       HEAD   TAIL   Chain   Index   0   1   2   3   4  
48. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,0>$ <A,0>$ <A,0>$ <A,0>$ <A,0>$ App.$ Proxy$ Client$$$ <PUT,Key:A,1>$
49. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,0>$ <A,0>$ <A,0>$ <A,0>$ <A,0>$ App.$ Proxy$ Client$$$ <PUT,Key:A,1>$ <PUT,Key:A,1>$
50. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,1>$ <A,0>$ <A,0>$ <A,0>$ <A,0>$ App.$ Proxy$ Client$$$ <PUT,Key:A,1>$ <PUT,Key:A,1>$
51. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,1>$ <A,1>$ <A,0>$ <A,0>$ <A,0>$ App.$ Proxy$ Client$$$ <PUT,Key:A,1>$ <PUT,Key:A,1>$
52. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,1>$ <A,1>$ <A,1>$ <A,0>$ <A,0>$ App.$ Proxy$ Client$$$ <PUT,Key:A,1>$ <PUT,Key:A,1>$
53. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,1>$ <A,1>$ <A,1>$ <A,1>$ <A,0>$ App.$ Proxy$ Client$$$ <PUT,Key:A,1>$ <PUT,Key:A,1>$
54. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,1>$ <A,1>$ <A,1>$ <A,1>$ <A,1>$ App.$ Proxy$ Client$$$ <ANS,Key:A,1>$
55. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,1>$ <A,1>$ <A,1>$ <A,1>$ <A,1>$ App.$ Proxy$ Client$$$ <ANS,Key:A,1>$ <ANS,Key:A,1>$
56. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,0>$ <A,0>$ <A,0>$ <A,0>$ <A,0>$ App.$ Proxy$ Client$$$
57. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,1>$ <A,1>$ <A,1>$ <A,1>$ <A,1>$ App.$ Proxy$ Client$$$ <GET,Key:A>$
58. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,1>$ <A,1>$ <A,1>$ <A,1>$ <A,1>$ App.$ Proxy$ Client$$$ <GET,Key:A>$ <GET,Key:A>$
59. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,1>$ <A,1>$ <A,1>$ <A,1>$ <A,1>$ App.$ Proxy$ Client$$$ <ANS,Key:A,1>$ <ANS,Key:A,1>$
60. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Chain Replication <A,1>$ <A,1>$ <A,1>$ <A,1>$ <A,1>$ App.$ Proxy$ Client$$$
61. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Main Contribution ChainReaction: a novel key-value distributed datastore Provides Causal+ consistency (single site and in a geo-replicated scenario). Based on specialized version of Chain Replication. Removes the tail-bottleneck by distributing reads. Relaxes the replication-level associated with write operations. Relaxes consistency. Offers high performance (optimized for reads).
62. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Main Contribution ChainReaction: a novel key-value distributed datastore Provides Causal+ consistency (single site and in a geo-replicated scenario). Based on specialized version of Chain Replication. Removes the tail-bottleneck by distributing reads. Relaxes the replication-level associated with write operations. Relaxes consistency. Offers high performance (optimized for reads).
63. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Main Contribution ChainReaction: a novel key-value distributed datastore Provides Causal+ consistency (single site and in a geo-replicated scenario). Based on specialized version of Chain Replication. Removes the tail-bottleneck by distributing reads. Relaxes the replication-level associated with write operations. Relaxes consistency. Offers high performance (optimized for reads).
64. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Main Contribution ChainReaction: a novel key-value distributed datastore Provides Causal+ consistency (single site and in a geo-replicated scenario). Based on specialized version of Chain Replication. Removes the tail-bottleneck by distributing reads. Relaxes the replication-level associated with write operations. Relaxes consistency. Offers high performance (optimized for reads).
65. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Architecture !""#$ %&'()$ !""#$ %&'()$ !""#$ %&'()$ !""#$ %&'()$ !"#$%$#$& *+,-./$$$ !"#$%&' '(#)*+*,' 01/1$2-&3-&4$ $ 5&61.,7-8$,.$1$ 5.-9:'"$0;<$ 01/1=-./-&$$ >'?.81&)$
66. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Architecture App.   Proxy   App.   Proxy   App.   Proxy   App.   Proxy   METADATA   Client       Client    Library   Data  Servers     Organized  in  a   One-­‐hop  DHT   Datacenter     Boundary   [Object  ID,  Version  ,Chain  Index]  
67. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Architecture !""#$ %&'()$ !""#$ %&'()$ !""#$ %&'()$ !""#$ %&'()$ !"#$%$#$& *+,-./$$$ !"#$%&' '(#)*+*,' 01/1$2-&3-&4$ $ 5&61.,7-8$,.$1$ 5.-9:'"$0;<$ 01/1=-./-&$$ >'?.81&)$
68. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Architecture !""#$ %&'()$ !""#$ %&'()$ !""#$ %&'()$ !""#$ %&'()$ !"#$%$#$& *+,-./$$$ !"#$%&' '(#)*+*,' 01/1$2-&3-&4$ $ 5&61.,7-8$,.$1$ 5.-9:'"$0;<$ 01/1=-./-&$$ >'?.81&)$
69. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Architecture !""#$ %&'()$ !""#$ %&'()$ !""#$ %&'()$ !""#$ %&'()$ !"#$%$#$& *+,-./$$$ !"#$%&' '(#)*+*,' 01/1$2-&3-&4$ $ 5&61.,7-8$,.$1$ 5.-9:'"$0;<$ 01/1=-./-&$$ >'?.81&)$
70. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Overview of Replication Mechanisms We allow writes to return before reaching the tail. We support reads on all nodes of the chain. We trade write efficiency for metadata efficiency.
71. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Overview of Replication Mechanisms We allow writes to return before reaching the tail. We support reads on all nodes of the chain. We trade write efficiency for metadata efficiency.
72. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We allow writes to return before reaching the tail We relax the number of nodes that have to process a write operation. This is controlled by a parameter K that can be configured locally for each datacenter. Allows for additional replicas to distribute the load of reads without significative penalisations to write operations performance.
73. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Execution Example: Put operation without dependencies <A,0>$ <A,0>$ <A,0>$ <A,0>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$
74. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Execution Example: Put operation without dependencies <A,0>$ <A,0>$ <A,0>$ <A,0>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ <PUT,Key:A,3,_>$
75. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Execution Example: Put operation without dependencies <A,0>$ <A,0>$ <A,0>$ <A,0>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ <PUT,Key:A,3,_>$
76. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Execution Example: Put operation without dependencies <A,0>$ <A,0>$ <A,0>$ <A,0>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ <PUT,Key:A,3,_>$
77. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Execution Example: Put operation without dependencies <A,3>$ <A,0>$ <A,0>$ <A,0>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$
78. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Execution Example: Put operation without dependencies <A,3>$ <A,3>$ <A,0>$ <A,0>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ K$=$3$
79. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Execution Example: Put operation without dependencies <A,3>$ <A,3>$ <A,3>$ <A,0>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ K$=$3$ <ANS,Key:A,3,2>$
80. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Execution Example: Put operation without dependencies <A,3>$ <A,3>$ <A,3>$ <A,0>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ K$=$3$ <ANS,Key:A,3,2>$
81. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Execution Example: Put operation without dependencies <A,3>$ <A,3>$ <A,3>$ <A,0>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ K$=$3$ <ANS,Key:A,3,2>$ <ANS,Key:A,3,2>$ [A,Ver:3,ChainIndex:2]&
82. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Execution Example: Put operation without dependencies <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [A,Ver:3,ChainIndex:2]&
83. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Overview of Replication Mechanisms We allow writes to return before reaching the tail. We support reads on all nodes of the chain. We trade write efficiency for metadata efficiency.
84. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain We allow read operations to be distributed across all nodes in the chain. Clients have to be aware of which node served the last read.
85. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [A,Ver:3,ChainIndex:2]&
86. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [A,Ver:3,ChainIndex:2]& <GET,Key:A,3,2>$
87. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>   <A,3>   <A,3>   <A,3>   <A,0>   App.   Proxy   App.   Proxy   App.   Proxy   App.   Proxy   <B,2>   <B,2>   <B,2>   <B,0>   <B,0>   METADATA   Client   1     METADATA   Client   2   Datacenter     Boundary   [A,Ver:3,ChainIndex:2]   <GET,Key:A,3,2>  
88. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [A,Ver:3,ChainIndex:2]& <GET,Key:A,3,2>$
89. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [A,Ver:3,ChainIndex:2]& <ANS,Key:A,3,1>$
90. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [A,Ver:3,ChainIndex:2]& <ANS,Key:A,3,1>$
91. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [A,Ver:3,ChainIndex:2]&
92. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [A,Ver:3,ChainIndex:2]& <GET,Key:A,3,2>$
93. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,3>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [A,Ver:3,ChainIndex:2]& <GET,Key:A,3,2>$ <ACK,Key:A,3,DC9Stable>$
94. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,3>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [A,Ver:3,ChainIndex:2]& <GET,Key:A,3,2>$ <ACK,Key:A,3,DC9Stable>$
95. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,3>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [A,Ver:3,ChainIndex:2]& <ACK,Key:A,3,DC9Stable>$ <ANS,Key:A,3,4>$
96. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,3>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ <ACK,Key:A,3,DC9Stable>$ <ANS,Key:A,3,4>$
97. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>   <A,3>   <A,3>   <A,3>   <A,3>   App.   Proxy   App.   Proxy   App.   Proxy   App.   Proxy   <B,2>   <B,2>   <B,2>   <B,0>   <B,0>   METADATA   Client   1     METADATA   Client   2   Datacenter     Boundary   <GET,Key:A,_,_>  
98. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We support reads on all nodes of the chain: Example <A,3>   <A,3>   <A,3>   <A,3>   <A,3>   App.   Proxy   App.   Proxy   App.   Proxy   App.   Proxy   <B,2>   <B,2>   <B,2>   <B,0>   <B,0>   METADATA   Client   1     METADATA   Client   2   Datacenter     Boundary   <GET,Key:A,_,_>  
99. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Overview of Replication Mechanisms We allow writes to return before reaching the tail. We support reads on all nodes of the chain. We trade write efficiency for metadata efficiency.
100. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We trade write efficiency for metadata efficiency We rely on a stabilization procedure performed before each individual write which: Enforces causal+ consistency. Lowers the amount of metadata maintained by clients.
101. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We trade write efficiency for metadata efficiency: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [B,Ver:2,ChainIndex:1]& [A,Ver:3,ChainIndex:2]&
102. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We trade write efficiency for metadata efficiency: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [B,Ver:2,ChainIndex:1]& [A,Ver:3,ChainIndex:2]& <PUT,Key:A,4,[A:3,B:2]>$
103. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We trade write efficiency for metadata efficiency: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [B,Ver:2,ChainIndex:1]& [A,Ver:3,ChainIndex:2]& <PUT,Key:A,4,[A:3,B:2]>$
104. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We trade write efficiency for metadata efficiency: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,0>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,0>$ <B,0>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [B,Ver:2,ChainIndex:1]& [A,Ver:3,ChainIndex:2]& <PUT,Key:A,4,[A:3,B:2]>$ StabilizaTon$ Procedure$ <CHECK,Key:A,3>$ <CHECK,Key:B,2>$
105. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We trade write efficiency for metadata efficiency: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,3>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,2>$ <B,2>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [B,Ver:2,ChainIndex:1]& [A,Ver:3,ChainIndex:2]& <PUT,Key:A,4,[A:3,B:2]>$ StabilizaTon$ Procedure$ <CHECK,Key:A,3>$ <CHECK,Key:B,2>$
106. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We trade write efficiency for metadata efficiency: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,3>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,2>$ <B,2>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [B,Ver:2,ChainIndex:1]& [A,Ver:3,ChainIndex:2]& <PUT,Key:A,4,[A:3,B:2]>$ StabilizaTon$ Procedure$ <OK,Key:A,3>$ <OK,Key:B,2>$
107. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We trade write efficiency for metadata efficiency: Example <A,3>$ <A,3>$ <A,3>$ <A,3>$ <A,3>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,2>$ <B,2>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ [B,Ver:2,ChainIndex:1]& [A,Ver:3,ChainIndex:2]& <PUT,Key:A,4,[A:3,B:2]>$ <PUT,Key:A,4>$
108. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We trade write efficiency for metadata efficiency: Example <A,4>   <A,4>   <A,4>   <A,3>   <A,3>   App.   Proxy   App.   Proxy   App.   Proxy   App.   Proxy   <B,2>   <B,2>   <B,2>   <B,2>   <B,2>   METADATA   Client   1     METADATA   Client   2   Datacenter     Boundary   [B,Ver:2,ChainIndex:1]   [A,Ver:3,ChainIndex:2]   <ANS,Key:A,4,2>   <ANS,Key:A,4,2>   K  =  3  
109. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We trade write efficiency for metadata efficiency: Example <A,4>   <A,4>   <A,4>   <A,3>   <A,3>   App.   Proxy   App.   Proxy   App.   Proxy   App.   Proxy   <B,2>   <B,2>   <B,2>   <B,2>   <B,2>   METADATA   Client   1     METADATA   Client   2   Datacenter     Boundary   <ANS,Key:A,4,2>   <ANS,Key:A,4,2>   K  =  3  
110. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction We trade write efficiency for metadata efficiency: Example <A,4>$ <A,4>$ <A,4>$ <A,3>$ <A,3>$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ <B,2>$ <B,2>$ <B,2>$ <B,2>$ <B,2>$ METADATA& Client$ 1$$ METADATA& Client$ 2$ Datacenter$$ Boundary$ <ANS,Key:A,4,2>$ <ANS,Key:A,4,2>$ [A,Ver:4,ChainIndex:2]& K$=$3$
111. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Outline 1 Introduction 2 More than eventual consistency: ChainReaction Preliminaries Architecture Overview of Replication Mechanisms Extending to Geo-distributed Environment Experimental Evaluation Lessons Learned 3 Strong Consistency and Transactional Support: Blotter 4 Summary
112. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Extending to Geo-distributed Environment App.   Proxy   App.   Proxy   App.   Proxy   App.   Proxy   METADATA   Client       Client    Library   Data  Servers     Organized  in  a   One-­‐hop  DHT   Datacenter     Boundary   [Object  ID,  Version  ,Chain  Index]  
113. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Extending to Geo-distributed Environment App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ METADATA& Client$$$ Client' 'Library' Data$Servers$ $ Organized$in$a$ One9hop$DHT$ Datacenter$$ Boundary$ [Object$ID,$Version$,Chain$Index]$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ METADATA& Client$$$ Client' 'Library' Data$Servers$ $ Organized$in$a$ One9hop$DHT$ Datacenter$$ Boundary$ [Object$ID,$Version$,Chain$Index]$ DATACENTER(1( DATACENTER(2(
114. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Extending to Geo-distributed Environment App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ METADATA& Client$$$ Client' 'Library' Data$Servers$ $ Organized$in$a$ One9hop$DHT$ Datacenter$$ Boundary$ [Object$ID,$Version$,Chain$Index]$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ METADATA& Client$$$ Client' 'Library' Data$Servers$ $ Organized$in$a$ One9hop$DHT$ Datacenter$$ Boundary$ [Object$ID,$Version$,Chain$Index]$ DATACENTER(1( DATACENTER(2( App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ METADATA& Client$$$ Client' 'Library' Data$Servers$ $ Organized$in$a$ One9hop$DHT$ Datacenter$$ Boundary$ [Object$ID,$Version[dc1,dc2]$,Chain$Index[dc1,dc2]]$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ App.$ Proxy$ METADATA& Client$$$ Client' 'Library' Data$Servers$ $ Organized$in$a$ One9hop$DHT$ Datacenter$$ Boundary$ [Object$ID,$Version[dc1,dc2]$,Chain$Index[dc1,dc2]]$
115. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Extending to Geo-distributed Environment Extend techniques for Geo-distribution Read operations: Processed in a single datacenter. Write operations: Return when applied to K nodes in a single datacenter. Propagated in background across datacenters. Rely on last-writer-wins to merge divergent versions. Learn with RIAK: Use CRDT’s to avoid losing updates.
116. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Extending to Geo-distributed Environment Extend techniques for Geo-distribution Read operations: Processed in a single datacenter. Write operations: Return when applied to K nodes in a single datacenter. Propagated in background across datacenters. Rely on last-writer-wins to merge divergent versions. Learn with RIAK: Use CRDT’s to avoid losing updates.
117. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Experimental Evaluation: Overview We implemented ChainReaction prototype: Used an optimized version of FAWN-KV (Fast Array of Wimpy Nodes) as basis. Prototype required approximately 26, 000 lines of C++ and Java code.
118. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Experimental Evaluation: Overview Comparative performance study ChainReaction. Optimized version of FAWN-KV. Apache Cassandra 0.8.10. COPS.
119. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Experimental Evaluation: Overview Results Summary: Throughput in single datacenter. Throughput across multiple datacenters. Latency of operations in both scenarios for particular workloads. Throughput and latency of workloads exercising Get-Transactions. Effect of failures over throughput. Metadata overhead.
120. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Experimental Evaluation: Overview Yahoo Cloud Serving Benchmark 0.1.3 (YCSB). 200 clients (i.e, threads). YCSB Standard Workloads: Workload A: Update heavy (50W/50R, zipf access dist.). Workload B: Read intensive (5W/95R), zipf access dist.). Workload C: Read only (0W/100R, zipf access dist.). Workload D: Read Latest (5W/95R, bias to recently inserted). Workload F: Read-modity-write (50W/50R, zipf access dist.). Additional custom workloads where only a single object is accessed.
121. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Experimental Evaluation: Geo-Replicated Scenario Setup Tests were executed in 2 virtual datacenters (delay between datacenters ∼ 120 ms) Each datacenter has 4 machines. 1.000.000 objects (1KByte). 200 clients (100 for each datacenter).
122. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Experimental Evaluation: Geo-Replicated Scenario System Configurations ChainReaction: Chain size: 4; K : 2 (in each datacenter). FAWN-KV: Chain size: 4. Cassandra: Replication factor: 6; Write set: 2; Read set: 1. CR-COPS: Version of ChainReaction that mimics COPS system; Chain size: 4.
123. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Experimental Evaluation: Geo-Replicated Scenario 0" 10" 20" 30" 40" 50" 60" 70" 80" Workload"A"(50W/50R)" Workload"B"(5W/95R)" Workload"C"(0W/100R)" Througput"(thousands"ops/s)" Cassandra" FAWNDKV" COPSDCR" ChainReacMon" Figure : Standard Workloads (Geo-Replicated Scenarios)
124. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Experimental Evaluation: Metadata Overhead Setup Tests were executed in 2 virtual datacenters. Exercised workload A: Update heavy (50W/50R, zipf access dist.). 10 clients. System Configurations ChainReaction: Chain size: 1. COPS: Chain size: 1.
125. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Experimental Evaluation: Metadata Overhead Setup Tests were executed in 2 virtual datacenters. Exercised workload A: Update heavy (50W/50R, zipf access dist.). 10 clients. System Configurations ChainReaction: Chain size: 1. COPS: Chain size: 1.
126. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Experimental Evaluation: Metadata Overhead 0" 5000" 10000" 15000" 20000" 25000" 5" 15" 25" 35" 45" 55" 65" 75" 85" 95" Size""(Bytes)" Time"Elapsed"(seconds)" ChainReacConMessage" ChainReacConMetadata" COPSMessage" COPSMetadata" Figure : Metadata Overhead (Comparison with COPS)
127. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Lessons Learned How to provide Causal+ Consistency in an efficient manner: Low metadata overhead (in exchange for write performance). Enhanced scalability for read operations. A well designed replication protocol for a single datacenter scenario... ...is actually easy to be expanded to a geo-replicated environment. What do we pay: The system is no longer always available, it is actually sticky available.
128. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Lessons Learned How to provide Causal+ Consistency in an efficient manner: Low metadata overhead (in exchange for write performance). Enhanced scalability for read operations. A well designed replication protocol for a single datacenter scenario... ...is actually easy to be expanded to a geo-replicated environment. What do we pay: The system is no longer always available, it is actually sticky available.
129. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Lessons Learned How to provide Causal+ Consistency in an efficient manner: Low metadata overhead (in exchange for write performance). Enhanced scalability for read operations. A well designed replication protocol for a single datacenter scenario... ...is actually easy to be expanded to a geo-replicated environment. What do we pay: The system is no longer always available, it is actually sticky available.
130. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary More than eventual consistency: ChainReaction Lessons Learned How to provide Causal+ Consistency in an efficient manner: Low metadata overhead (in exchange for write performance). Enhanced scalability for read operations. A well designed replication protocol for a single datacenter scenario... ...is actually easy to be expanded to a geo-replicated environment. What do we pay: The system is no longer always available, it is actually sticky available.
131. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Outline 1 Introduction 2 More than eventual consistency: ChainReaction 3 Strong Consistency and Transactional Support: Blotter Preliminaries Overview Single Datacenter Multiple Datacenter Experimental Results Lessons Learned 4 Summary
132. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Credits Henrique(Moniz( Ricardo(J.(Dias( Johannes(Gehrke( Nuno(Preguiça( Rodrigo(Rodrigues(
133. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Credits Blotter: Low Latency Transactions for Geo-Replicated Storage. H. Moniz, J. Leit˜ ao, R. J. Dias, J. Gehrke, N. Pregui¸ ca, and R. Rodrigues. currently under submission to VLDB.
134. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Consistency and Isolation Consistency Model: Strong Consistency. Isolation Level: Non-Monotonic Snapshot Isolation.
135. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Consistency and Isolation Consistency Model: Strong Consistency. Isolation Level: Non-Monotonic Snapshot Isolation.
136. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Non-Monotonic Snapshot Isolation Snapshot Isolation Transactions operate on a virtual snapshot of the database created when they start. A transaction T cannot observe something written by a transaction T that committed after the start of T. Allows the Write-Skew anomaly.
137. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Non-Monotonic Snapshot Isolation Snapshot Isolation Transactions operate on a virtual snapshot of the database created when they start. A transaction T cannot observe something written by a transaction T that committed after the start of T. Allows the Write-Skew anomaly.
138. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Non-Monotonic Snapshot Isolation Snapshot Isolation Transactions operate on a virtual snapshot of the database created when they start. A transaction T cannot observe something written by a transaction T that committed after the start of T. Allows the Write-Skew anomaly.
139. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Non-Monotonic Snapshot Isolation Non-Monotonic Snapshot Isolation Relaxation of Snapshot Isolation. Allows for two additional anomalies: Long Forks anomaly. Forward Freshness anomaly.
140. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Non-Monotonic Snapshot Isolation Non-Monotonic Snapshot Isolation Relaxation of Snapshot Isolation. Allows for two additional anomalies: Long Forks anomaly. Forward Freshness anomaly.
141. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Non-Monotonic Snapshot Isolation Non-Monotonic Snapshot Isolation Relaxation of Snapshot Isolation. Allows for two additional anomalies: Long Forks anomaly. Forward Freshness anomaly.
142. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Overview of Blotter A key-value store (such as RIAK) in each datacenter. In each datacenter the datastore provides one-copy serializability per object. We assume that the datastore provides multi-version per object.
143. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Overview of Blotter Concurrency Protocol that leverages metadata to allow conflicts to be detected. 2-Phase Commit protocol that biggybacks metadata. Inter-DC Replication Scheme based on a Special configuration of Paxos.
144. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Overview of Blotter Concurrency Protocol that leverages metadata to allow conflicts to be detected. 2-Phase Commit protocol that biggybacks metadata. Inter-DC Replication Scheme based on a Special configuration of Paxos.
145. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … …
146. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … BeginTx()' BeginTx()'
147. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … BeginTx()' BeginTx()' TxID'='T1' TxID'='T2'
148. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … Read(X,'T1)' TxID'='T1' TxID'='T2'
149. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … Read(X,'T1)' TxID'='T1' TxID'='T2'
150. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … Read(X,'T1)'='B'(V2)' TxID'='T1' TxID'='T2'
151. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … Read(X,'T1)'='B'(V2)' TxID'='T1' X'='B'(V2)' TxID'='T2'
152. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … Read(X,'T1)'='B'(V2)' TxID'='T1' X'='B'(V2)' TxID'='T2' T1'
153. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)' TxID'='T2' T1' Read(X,'T2)'
154. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='B'(V2)'[T1]' T1' Read(X,'T2)'='B'(V2)'[T1]' T2'
155. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='B'(V2)'[T1]' T1' Write(X,'Value,'T2)' T2'
156. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='Value'(V3)'[T1]'*' T1' Write(X,'Value,'T2)' T2'
157. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(?)'[]'*'' T1' Write(Y,'Value,'T2)' T2'
158. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(?)'[]'*'' T1' Commit(T2)' T2'
159. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(?)'[]'*'' T1' Commit(T2)' T2'
160. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(?)'[]'*'' T1' Commit(T2)' T2' Read(Y,'T2)'='D'(V2)'[]'
161. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(V3)'[]'*'' T1' Commit(T2)' T2' Read(Y,'T2)'='D'(V2)'[]' T2'
162. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(V3)'[]'*'' T1' Commit(T2)' T2' T2' Prewrite(Y,Value'(V3))' Prewrite(X,Value'(V3))'''
163. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(V3)'[]'*'' T1' Commit(T2)' T2' T2' OK'[]' OK'[T1]'
164. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(V3)'[]'*'' T1' Commit(T2)' Metadata:'[T1]' T2' T2' OK'[]' OK'[T1]'
165. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(V3)'[]'*'' T1' Commit(T2)' Metadata:'[T1]' T2' T2' Write(Y,Value'(V3),'[T1])' Write(X,Value'(V3),'[T1])'''
166. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(V3)'[]'*'' T1' Commit(T2)' Metadata:'[T1]' T2' T2' Write(Y,Value'(V3),'[T1])' Write(X,Value'(V3),'[T1])''' T1'
167. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(V3)'[]'*'' T1' Commit(T2)' Metadata:'[T1]' T2' T2' Write(Y,Value'(V3),'[T1])' Write(X,Value'(V3),'[T1])''' V3:'Value' V3:'Value' T1'
168. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(V3)'[]'*'' T1' Commit(T2)' Metadata:'[T1]' T2' T2' OK' V3:'Value' V3:'Value' T1' OK'
169. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' TxID'='T2' X'='V'(V3)'[T1]'*' Y'='Value'(V3)'[]'*'' T1' OK' T2' T2' OK' V3:'Value' V3:'Value' T1' OK'
170. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' T1' T2' T2' V3:'Value' V3:'Value' T1'
171. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' T1' T2' T2' V3:'Value' V3:'Value' T1'
172. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' T1' V3:'Value' V3:'Value' T1'
173. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' T1' V3:'Value' V3:'Value' T1' Read(Y,'T1)'
174. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' T1' V3:'Value' V3:'Value' T1' Read(Y,'T1)'
175. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' Y'='D'(V2)'[]' T1' V3:'Value' V3:'Value' T1' Read(Y,'T1)'='D'(V2)'
176. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='B'(V2)'[]' Y'='D'(V2)'[]' T1' V3:'Value' V3:'Value' T1' Write(X,'V,'T1)'
177. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='V'(V3)'[]'*' Y'='D'(V2)'[]' T1' V3:'Value' V3:'Value' T1' Write(X,'V,'T1)'
178. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='V'(V3)'[]'*' Y'='D'(V2)'[]' T1' V3:'Value' V3:'Value' T1' Commit(T1)' Prewrite(X,V'(V3))'''
179. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='V'(V3)'[]'*' Y'='D'(V2)'[]' T1' V3:'Value' V3:'Value' T1' Commit(T1)' Prewrite(X,V'(V3))'''
180. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … TxID'='T1' X'='V'(V3)'[]'*' Y'='D'(V2)'[]' T1' V3:'Value' V3:'Value' T1' ABORT' ABORT'
181. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Single Datacenter Object'X' Object'Y' Client' 1' Client' 2' V1:'A' V2:'B' V1:'C' V2:'D' Datacenter'Boundary' Applica<on' Proxy' Applica<on' Proxy' Applica<on' Proxy' … … V3:'Value' V3:'Value'
182. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Overview of Blotter Concurrency Protocol that leverages metadata to allow conflicts to be detected. 2-Phase Commit protocol that biggybacks metadata. Inter-DC Replication Scheme based on a Special configuration of Paxos.
183. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Multiple Datacenter Layer the commit execution on Paxos to execute a transaction across multiple datacenters. Use one Paxos instance per object (i.e, key). Reads remain local (executed at the closest datacenter). Decision of commit or abort is deterministic (does not depend on metadata exclusive to the local datacenter).
184. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Multiple Datacenter Layer the commit execution on Paxos to execute a transaction across multiple datacenters. Use one Paxos instance per object (i.e, key). Reads remain local (executed at the closest datacenter). Decision of commit or abort is deterministic (does not depend on metadata exclusive to the local datacenter).
185. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Multiple Datacenter Special Variant of Paxos Goal: Efficient (minimize communication over long latency links). We do this by using a configuration of Paxos that tolerates a single failure. Result: Commit concludes within a single round trip (in the common case) across two datacenters.
186. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Multiple Datacenter Special Variant of Paxos Goal: Efficient (minimize communication over long latency links). We do this by using a configuration of Paxos that tolerates a single failure. Result: Commit concludes within a single round trip (in the common case) across two datacenters.
187. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Experimental Results: Setup Tests were executed on 3 datacenters (Amazon EC2). Each datacenter has 4 machines (Large instances). 1.000.000 objects (256 Byte). 360 clients (120 for each datacenter). Variable workloads.
188. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Experimental Results: Setup We performed experiments with: Cassandra (local-quorum). Cassandra (each-quorum). Blotter. 2 Phase Locking solution (2PL) - similar to Spanner.
189. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Experimental Results Figure : Throughput in a deployment with 3 datacenters.
190. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Lessons Learned How to provide full ACID transactions (over a key-value store): Picking up the right isolation level. Allowing transactions to coordinate themselves by leveraging the datastore. A well designed concurrency control protocol for a single datacenter scenario... ...can be extended (mostly) in a easy way to geo-replicated scenarios. What do we pay: The system is no longer available (but we still tolerate the failure of a single datacenter).
191. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Lessons Learned How to provide full ACID transactions (over a key-value store): Picking up the right isolation level. Allowing transactions to coordinate themselves by leveraging the datastore. A well designed concurrency control protocol for a single datacenter scenario... ...can be extended (mostly) in a easy way to geo-replicated scenarios. What do we pay: The system is no longer available (but we still tolerate the failure of a single datacenter).
192. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Lessons Learned How to provide full ACID transactions (over a key-value store): Picking up the right isolation level. Allowing transactions to coordinate themselves by leveraging the datastore. A well designed concurrency control protocol for a single datacenter scenario... ...can be extended (mostly) in a easy way to geo-replicated scenarios. What do we pay: The system is no longer available (but we still tolerate the failure of a single datacenter).
193. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Strong Consistency and Transactional Support: Blotter Lessons Learned How to provide full ACID transactions (over a key-value store): Picking up the right isolation level. Allowing transactions to coordinate themselves by leveraging the datastore. A well designed concurrency control protocol for a single datacenter scenario... ...can be extended (mostly) in a easy way to geo-replicated scenarios. What do we pay: The system is no longer available (but we still tolerate the failure of a single datacenter).
194. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Outline 1 Introduction 2 More than eventual consistency: ChainReaction 3 Strong Consistency and Transactional Support: Blotter 4 Summary
195. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Summary Discussed Alternatives to Eventual Consistency in the context of Datastores: Deployed on Single Datacenters. Deployed across multiple (distant) Datacenters (Geo-Replicated Scenario). A well designed replication algorithm for single site deployment should be naturally extensible to geo-replicated scenarios. There are no free lunches! a price has to be paid to go beyond eventual consistency. Be flexible, support multiple paradigms (still some things to be done here).
196. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Summary Discussed Alternatives to Eventual Consistency in the context of Datastores: Deployed on Single Datacenters. Deployed across multiple (distant) Datacenters (Geo-Replicated Scenario). A well designed replication algorithm for single site deployment should be naturally extensible to geo-replicated scenarios. There are no free lunches! a price has to be paid to go beyond eventual consistency. Be flexible, support multiple paradigms (still some things to be done here).
197. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Summary Discussed Alternatives to Eventual Consistency in the context of Datastores: Deployed on Single Datacenters. Deployed across multiple (distant) Datacenters (Geo-Replicated Scenario). A well designed replication algorithm for single site deployment should be naturally extensible to geo-replicated scenarios. There are no free lunches! a price has to be paid to go beyond eventual consistency. Be flexible, support multiple paradigms (still some things to be done here).
198. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary Thanks for your attention. http://asc.di.fct.unl.pt/~jleitao jleitao@fct.unl.pt Twitter: @jcaleitao
199. When Weak Consistency is not Enough: From Causality to Transactions Jo˜ ao Leit˜ ao Introduction More than eventual consistency: ChainReaction Strong Consistency and Transactional Support: Blotter Summary