Sam Elliott
October 30, 2013
2.7k

# CRDTs: An UPDATE (or just a PUT)

Riak 2.0's Data Types, presented at RICON West 2013, in SF.

A Few links for the interested:
- Dotted Version Vector Sets, a nice explanation: https://github.com/ricardobcl/Dotted-Version-Vectors#readme
- Our datatype repo: https://github.com/basho/riak_dt
- Comprehensive study of Convergent and Commutative Replicated Data Types (tech report): http://hal.upmc.fr/inria-00555588
- Optimised OR-Set (paper): http://arxiv.org/abs/1210.3368
- Dotted Version Vectors (paper): http://arxiv.org/abs/1011.5808
- Semilattices (wiki): https://en.wikipedia.org/wiki/Semilattice

October 30, 2013

## Transcript

1. CRDTs
An UPDATE (or maybe just a PUT)

2. Engineer, Bashtronaut
Sam Elliott

3. Not Only Me
Russell Brown
Sean Cribbs
Chris Meiklejohn
Carlos Baquero (UMinho)

4. CRDTs
Co-what Replicated Data Types?!

5. CRDTs
Data Types

6. This Talk
Usecases, Interface & Implementation

7. Usecases
What can I do with them?

8. Counters
Since 1.4

9. Counters
Increment & Decrement

10. Counters
Events!

11. Burger Counter

12. Sets
New in 2.0!

13. Sets

14. Sets

15. Shopping Cart

16. Maps
New in 2.0!

17. Maps
Values are CRDTs

18. Maps
Values are Counters
Sets

19. Maps
Values are Counters
Sets
Registers
Booleans

20. Maps
Values are Counters
Sets
Registers
Booleans
Maps

21. Maps
For all your Big Data™ needs!

23. Interface
How can I do that with them?

24. Counters
(Sorry Pat)

25. Counters
Size: O(Actors)

26. Sets

27. Sets
Size: O(Operations) ?

28. Sets
Size: O(Actors + Elements)

29. Maps
Entries keyed by Name & Type

30. Maps
{ Update | Remove } ~> Update Wins

31. Maps
Size: O(Actors + Entries)

32. Registers
Last Write Wins

33. Registers
Some Write Loses

34. Booleans
{ Enable | Disable } ~> Enable

35. Booleans
Size: O(Actors)

36. Operations
PB & HTTP

37. Increment(1)
Update(“name”,
register,
Set(“Sam Elliott”))

38. Operations
Atomic

39. Operations
Unordered

40. Operations
Causality

41. Operations
Context Required for Correct Removal

42. Bucket Types
Required for All Using 2.0 API

43. Bucket Types
Counters from 1.4 API Use Default

44. Not in 2.0

45. Not in 2.0
YZ Extractors

46. Not in 2.0
Type-level Queries

47. Implementation
How do they work underneath?

48. CRDTs
Convergent Replicated Data Types

49. Semilattices
S

Values
Partial Order
Join
Bottom

50. Counters
PN-Counters

51. PN-Counters
[ Node → ( Increments, Decrements ) ]

52. PN-Counters: Inc
Increase Increments for Current Node

53. PN-Counters: Dec
Increase Decrements for Current Node

54. PN-Counters
Merge: Pairwise Maximum per-Node

55. Sets
OR-Set

56. OR-Set
[ Element → ( Tokens , Tombstones ) ]

57. Sets
ORSWOT: Without Tombstones!

58. ORSWOT
VV ⊠ [ Element → [ Dot ] ]

Increment VV

60. ORSWOT: Remove
Leave VV
Erase Element & Dots

61. ORSWOT: Merge
Element In Both:
Merge Dots
Element In Only One:
Drop when Dominated by Other’s VV
Keep Otherwise

62. Maps
OR-Map

63. Maps
VV ⊠ ( Field , Type ) → ( [ Dot ] ,
Value )

64. Map: Update
Apply Update to Value
Increment VV

Like Update, Inserting ⊥

66. Map: Remove
Leave VV
Erase Field & Dots

67. Map: Merge
Field In Both:
Merge Dots
Field In Only One:
Drop when Dominated by Other’s VV
Keep Otherwise

68. Causality
Context & CRDTs

69. Causality
Concurrent Remove

70. Concurrent Remove
{a,b}
{a,b} {a,b}
?
?

71. C. Remove: What We Want
{a,b}
{a,b}
{a,b}
{a,b} {a,b}
Sᵢ₊₁
Sᵢ

72. C. Remove: The Incorrect Solution
{a,b}
{a,b}
{b}
{a,b} {a,b}
Sᵢ₊₁
Sᵢ
Sᵢ₊₁ = op(Sᵢ)
Sᵢ

73. C. Remove: The Correct Solution
{a,b}
{a,b}
{a,b}
{a,b} {a,b}
Sᵢ₊₁
Sᵢ
Sᵢ₊₁ = opᵤ(merge( Sᵢ , opᵣ() ) )

74. Causality
Split Brain

75. Split Brain
remove(a)
{a,b} {b}
?
?
(eventually)
{a,b}

76. Split Brain: What We Want
remove(a)
{a,b} {b}
{b}
{b}
(eventually)
{a,b}

77. Split Brain: The Incorrect Solution
remove(a)
{a,b} {b}
{a,b}
{b}
Error
Sᵢ₊₁ = op(Sᵢ)
(eventually)
{a,b}
Sᵢ₊₁
Sᵢ

78. Split Brain: The Correct Solution
remove(a)
{b}
{a,b}
{b}
(eventually)
{a,b}
{a,b}
Sᵢ₊₁ = opᵤ(merge( Sᵢ , opᵣ() ) )
Sᵢ₊₁
Sᵢ

79. QuickCheck
Is Awesome!

80. QuickCheck
Generalised Checks for CRDTs in riak_dt

81. SyncFree
CRDTs, European Style
Funded by
the European Union

82. SyncFree
Funded by
the European Union

83. SyncFree
Useful Invariants
Funded by
the European Union

84. References

85. http://hal.upmc.fr/inria-00555588

86. http://arxiv.org/abs/1210.3368

87. http://arxiv.org/abs/1011.5808

88. https://github.com/basho/riak_dt

89. Recap
Halp, I forgot something!

90. CRDTs
Data Types

91. Sets & Maps
In 2.0

92. Sets & Maps
“If in doubt, Data is Kept”

93. Operations
Don’t forget Context for Removes