Non-uniform replication for replicated objects (Thesis defense)

https://gmcabrita.com/pubs/thesis.pdf

Transcript

1. 1 Non-uniform replication for replicated objects Gonçalo Cabrita, 41743
2. 2 Context • User activity has forced services to find new ways to scale • Services store their data in geo-replicated key-value stores • These data stores sacrifice consistency for higher availability • Some of these data stores have adopted CRDTs to solve the problem of data convergence
3. 3 Problem • Information maintained by these data stores increases rapidly • Often impossible to keep all data in all replicas • Some systems adopt a partial replication model ◦ Queries may need to contact more than one replica to provide a response
4. 4 Can we create a replication model where any single replica can answer all reads without storing all the data?
5. 5 Example 10 3 2 A B C
6. 6 Example 10 3 2 10 3 2 10 3 2 A B C
7. 7 Example 10 5 3 2 10 3 2 1 10 3 2 A B C
8. 8 Example 10 5 3 2 10 3 2 1 10 3 2 A B C
9. 9 Example 5 3 2 10 3 2 1 10 3 2 A B C
10. 10 Example 5 3 2 3 2 1 3 2 A B C
11. 11 Example 5 3 2 5 3 2 1 5 3 2 A B C
12. 12 Road Map • Non-uniform replication • NuCRDTs • Simulation • Integration in AntidoteDB • Evaluation • Conclusion and Future Work
13. 13 Non-uniform replication • A replication model where all replicas can answer all queries, while maintaining only a subset of the data • Replicas of the same object are not required to have equivalent states, instead they are required to have observable equivalent states • For two states to be observable equivalent a read operation must return the same value for both
14. 14 Non-uniform replication 5 3 2 5 3 2 1 5 3 2 A B C
15. 15 Non-uniform replication 5 3 2 5 3 2 1 5 3 2 A B C
16. 16 Non-uniform replication 5 3 2 5 3 2 1 5 3 2 A B C
17. 17 Eventual consistency A replicated system provides eventual consistency in a quiescent state iff: 1. Every replica executed all operations 2. The state of any pair of replicas is equivalent
18. 18 Non-uniform eventual consistency A replicated system provides non-uniform eventual consistency in a quiescent state iff: 1. Every replica executed a set of operations that impact the final observable state 2. The state of any pair of replicas is observable equivalent
19. 19 Conditions for NuEC Having: • Core, the subset of essential operations in some execution. • Masked, the subset of operations in some execution that are irrelevant to the final state of the replica Theorem (Sufficient conditions for NuEC). A replication system provides non-uniform eventual consistency (NuEC) if, for the set of operations O, in their execution the following conditions hold: • every replica executes a set of core operations of O and • all operations commute.
20. Primary goal is to divide operations into 4 groups: 1. Operations that are core 2. Operations that are forever masked 3. Operations that are masked but can become core 4. Operations that are masked but in the context of the entire system would be considered core 20 Algorithm for NuEC
21. 21 Road Map • Non-uniform replication • NuCRDTs • Simulation • Integration in AntidoteDB • Evaluation • Conclusion and Future Work
22. 22 NuCRDTs • Top-K • Top-K with removals
23. 23 Top-K • Set of tuples (id, score) • Single write operation ◦ ADD(id, score)
24. 24 Top-1 A B
25. 25 Top-1 (John, 1000) A B
26. 26 Top-1 (John, 1000) A B (John, 1000)
27. 27 Top-1 (John, 1000) A B (John, 1000) (John, 900)
28. 28 Top-1 (John, 1000) A B (John, 1000) (John, 900)
29. 29 Top-K with removals • More complicated than regular Top-K • Two write operations ◦ ADD(id, score) ◦ RMV(id)
30. 30 Top-1 with removals B A
31. 31 Top-1 with removals (John, 1000) B A
32. 32 Top-1 with removals (John, 1000) B A (John, 1000)
33. 33 Top-1 with removals (John, 1000) (Meg, 900) B A (John, 1000)
34. 34 Top-1 with removals (John, 1000) (Meg, 900) B A (John, 1000)
35. 35 Top-1 with removals (John, 1000) (Meg, 950) (Meg, 900) B A (John, 1000)
36. 36 Top-1 with removals (John, 1000) (Meg, 950) (Meg, 900) B A (John, 1000)
37. 37 Top-1 with removals (John, 1000) (Meg, 950) B A (John, 1000)
38. 38 Top-1 with removals (Meg, 950) B A (John, 1000)
39. 39 Top-1 with removals (Meg, 950) B A
40. 40 Top-1 with removals (Meg, 950) B A (Meg, 950)
41. 41 Road Map • Non-uniform replication • NuCRDTs • Simulations • Integration in AntidoteDB • Evaluation • Conclusion and Future Work
42. 42 Simulations • Simulation of 5 replicas for each data type • Up to 500,000 operations are generated and executed • Values used are randomly selected using uniform distribution • Comparison with Delta CRDTs and Computational CRDTs
43. 43 Top-K
44. 44 Top-K
45. 45 Top-K with removals Workload: 95% adds, 5% removes
46. 46 Top-K with removals Workload: 95% adds, 5% removes
47. 47 Road Map • Non-uniform replication • NuCRDTs • Simulations • Integration in AntidoteDB • Evaluation • Conclusion and Future Work
48. 48 AntidoteDB Architecture DC2 DC3 Client Client Node3 Node2 Transaction Manager Materializer InterDC Replication Log DC1 Client Client Node1
49. 49 AntidoteDB Architecture DC2 DC3 Client Client Node3 Node2 Transaction Manager Materializer InterDC Replication Log DC1 Client Client Node1
50. 50 AntidoteDB Architecture DC2 DC3 Client Client Node3 Node2 Transaction Manager Materializer InterDC Replication Log DC1 Client Client Node1
51. 51 AntidoteDB Architecture DC2 DC3 Client Client Node3 Node2 Transaction Manager Materializer InterDC Replication Log DC1 Client Client Node1
52. 52 AntidoteDB Architecture DC2 DC3 Client Client Node3 Node2 Transaction Manager Materializer InterDC Replication Log DC1 Client Client Node1
53. 53 AntidoteDB Architecture DC2 DC3 Client Client Node3 Node2 Transaction Manager Materializer InterDC Replication Log DC1 Client Client Node1
54. 54 Implemented data types • Among the data types we implement: ◦ Top-K ◦ Top-K with removals • To use these data types in AntidoteDB we had to apply some changes
55. 55 Discarding no-ops Problem: AntidoteDB assumes operations always have effects Solution: Return a no-op effect by the data type and discard this operation in the Materializer and Log
56. 56 Support for compaction Problem: Transactions are immediately propagated once they commit Solution: Buffer transactions for a short duration and then compact them
57. 57 Durability of Masked ops Problem: Masked operations are only kept in a single data center Solution: Create two versions of the transaction before propagating, one with all the operations and one with only core operations
58. 58 Masked ops that turn Core Problem: When an operation executes it may cause other operations to become core, how do we propagate these? Solution: Once the operation is executed in the Materializer we send the new operations to the InterDC replication for propagation
59. 59 Road Map • Non-uniform replication • NuCRDTs • Simulations • Integration in AntidoteDB • Evaluation • Conclusion and Future Work
60. 60 Evaluation in AntidoteDB EU-W EU-C US-E AP-NW US-W
61. 61 Evaluation in AntidoteDB B1 B2 B3 B4 B5 EU-W EU-C US-E AP-NW US-W
62. 62 Top-K
63. 63 Top-K
64. 64 Top-K
65. 65 Top-K with removals Workload: 95% adds, 5% removes
66. 66 Top-K with removals Workload: 95% adds, 5% removes
67. 67 Top-K with removals Workload: 95% adds, 5% removes
68. 68 Road Map • Non-uniform replication • NuCRDTs • Simulations • Integration in AntidoteDB • Evaluation • Conclusion and Future Work
69. 69 Conclusion • Introduced the non-uniform replication model and formalized its semantics for an eventually consistent system • Showed how the model is applied to different data type designs • Implemented our designs in the AntidoteDB key-value store
70. 70 Future Work • Evaluate the usefulness of non-uniform replication for other consistency models, such as linearizability and serializability • Design more useful data types, particularly for Big Data and Machine Learning environments • Cleanup and merge our changes into AntidoteDB’s main branch
71. 71 Questions?