CRDTs: An UPDATE (or just a PUT)

Riak 2.0's Data Types, presented at RICON West 2013, in SF.

A Few links for the interested:
- Dotted Version Vector Sets, a nice explanation: https://github.com/ricardobcl/Dotted-Version-Vectors#readme
- Our datatype repo: https://github.com/basho/riak_dt
- Comprehensive study of Convergent and Commutative Replicated Data Types (tech report): http://hal.upmc.fr/inria-00555588
- Optimised OR-Set (paper): http://arxiv.org/abs/1210.3368
- Dotted Version Vectors (paper): http://arxiv.org/abs/1011.5808
- Semilattices (wiki): https://en.wikipedia.org/wiki/Semilattice

Transcript

1. CRDTs An UPDATE (or maybe just a PUT)
2. Engineer, Bashtronaut Sam Elliott
3. Not Only Me Russell Brown Sean Cribbs Chris Meiklejohn Carlos Baquero (UMinho)
4. CRDTs Co-what Replicated Data Types?!
5. CRDTs Data Types
6. This Talk Usecases, Interface & Implementation
7. Usecases What can I do with them?
8. Counters Since 1.4
9. Counters Increment & Decrement
10. Counters Events!
11. Burger Counter
12. Sets New in 2.0!
13. Sets Add & Remove Elements
14. Sets Shopping Basket!
15. Shopping Cart
16. Maps New in 2.0!
17. Maps Values are CRDTs
18. Maps Values are Counters Sets
19. Maps Values are Counters Sets Registers Booleans
20. Maps Values are Counters Sets Registers Booleans Maps
21. Maps For all your Big Data™ needs!
22. Address Book
23. Interface How can I do that with them?
24. Counters Updates NOT Idempotent (Sorry Pat)
25. Counters Size: O(Actors)
26. Sets { Add | Remove } ~> Add Wins
27. Sets Size: O(Operations) ?
28. Sets Size: O(Actors + Elements)
29. Maps Entries keyed by Name & Type
30. Maps { Update | Remove } ~> Update Wins
31. Maps Size: O(Actors + Entries)
32. Registers Last Write Wins
33. Registers Some Write Loses
34. Booleans { Enable | Disable } ~> Enable
35. Booleans Size: O(Actors)
36. Operations PB & HTTP
37. Increment(1) Add(“Cuttlefish”) Update(“name”, register, Set(“Sam Elliott”))
38. Operations Atomic
39. Operations Unordered
40. Operations Causality
41. Operations Context Required for Correct Removal
42. Bucket Types Required for All Using 2.0 API
43. Bucket Types Counters from 1.4 API Use Default
44. Not in 2.0 2i, Custom Metadata
45. Not in 2.0 YZ Extractors
46. Not in 2.0 Type-level Queries
47. Implementation How do they work underneath?
48. CRDTs Convergent Replicated Data Types
49. Semilattices S ≤ ∨ ⊥ Values Partial Order Join Bottom
50. Counters PN-Counters
51. PN-Counters [ Node → ( Increments, Decrements ) ]
52. PN-Counters: Inc Increase Increments for Current Node
53. PN-Counters: Dec Increase Decrements for Current Node
54. PN-Counters Merge: Pairwise Maximum per-Node
55. Sets OR-Set
56. OR-Set [ Element → ( Tokens , Tombstones ) ]
57. Sets ORSWOT: Without Tombstones!
58. ORSWOT VV ⊠ [ Element → [ Dot ] ]
59. ORSWOT: Add Increment VV Add Dot to Element’s Dot
60. ORSWOT: Remove Leave VV Erase Element & Dots
61. ORSWOT: Merge Element In Both: Merge Dots Element In Only One: Drop when Dominated by Other’s VV Keep Otherwise
62. Maps OR-Map
63. Maps VV ⊠ ( Field , Type ) → ( [ Dot ] , Value )
64. Map: Update Apply Update to Value Increment VV Add Dot to Field’s Dots
65. Map: Add Like Update, Inserting ⊥
66. Map: Remove Leave VV Erase Field & Dots
67. Map: Merge Field In Both: Merge Dots Field In Only One: Drop when Dominated by Other’s VV Keep Otherwise
68. Causality Context & CRDTs
69. Causality Concurrent Remove
70. Concurrent Remove add(a) remove(a) {a,b} {a,b} {a,b} ? ?
71. C. Remove: What We Want add(a) remove(a) {a,b} {a,b} {a,b} {a,b} {a,b} Sᵢ₊₁ Sᵢ
72. C. Remove: The Incorrect Solution add(a) remove(a) {a,b} {a,b} {b} {a,b} {a,b} Sᵢ₊₁ Sᵢ Sᵢ₊₁ = op(Sᵢ) Sᵢ
73. C. Remove: The Correct Solution add(a) remove(a) {a,b} {a,b} {a,b} {a,b} {a,b} Sᵢ₊₁ Sᵢ Sᵢ₊₁ = opᵤ(merge( Sᵢ , opᵣ() ) )
74. Causality Split Brain
75. Split Brain remove(a) {a,b} {b} ? ? (eventually) {a,b}
76. Split Brain: What We Want remove(a) {a,b} {b} {b} {b} (eventually) {a,b}
77. Split Brain: The Incorrect Solution remove(a) {a,b} {b} {a,b} {b} Error Sᵢ₊₁ = op(Sᵢ) (eventually) {a,b} Sᵢ₊₁ Sᵢ
78. Split Brain: The Correct Solution remove(a) {b} {a,b} {b} (eventually) {a,b} {a,b} Sᵢ₊₁ = opᵤ(merge( Sᵢ , opᵣ() ) ) Sᵢ₊₁ Sᵢ
79. QuickCheck Is Awesome!
80. QuickCheck Generalised Checks for CRDTs in riak_dt
81. SyncFree CRDTs, European Style Funded by the European Union
82. SyncFree Dead Actor Pruning Funded by the European Union
83. SyncFree Useful Invariants Funded by the European Union
84. References I’m a Real Academic!
85. http://hal.upmc.fr/inria-00555588
86. http://arxiv.org/abs/1210.3368
87. http://arxiv.org/abs/1011.5808
88. https://github.com/basho/riak_dt
89. Recap Halp, I forgot something!
90. CRDTs Data Types
91. Sets & Maps In 2.0
92. Sets & Maps “If in doubt, Data is Kept”
93. Operations Don’t forget Context for Removes
94. No 2i, Metadata Yet
95. Bucket Types Except 1.4-Style Counters
96. No More Siblings! !!!!!!!!111111ONEONEONE
97. Magical Unless You Know How They Work (Sorry)
98. Question Time! Sam Elliott: sam.elliott@basho.com riak-users@lists.basho.com riak-core@lists.basho.com