CRDTs: An UPDATE (or just a PUT)

Slide 1

Slide 1 text

CRDTs An UPDATE (or maybe just a PUT)

Slide 2

Slide 2 text

Engineer, Bashtronaut Sam Elliott

Slide 3

Slide 3 text

Not Only Me Russell Brown Sean Cribbs Chris Meiklejohn Carlos Baquero (UMinho)

Slide 4

Slide 4 text

CRDTs Co-what Replicated Data Types?!

Slide 5

Slide 5 text

CRDTs Data Types

Slide 6

Slide 6 text

This Talk Usecases, Interface & Implementation

Slide 7

Slide 7 text

Usecases What can I do with them?

Slide 8

Slide 8 text

Counters Since 1.4

Slide 9

Slide 9 text

Counters Increment & Decrement

Slide 10

Slide 10 text

Counters Events!

Slide 11

Slide 11 text

Burger Counter

Slide 12

Slide 12 text

Sets New in 2.0!

Slide 13

Slide 13 text

Sets Add & Remove Elements

Slide 14

Slide 14 text

Sets Shopping Basket!

Slide 15

Slide 15 text

Shopping Cart

Slide 16

Slide 16 text

Maps New in 2.0!

Slide 17

Slide 17 text

Maps Values are CRDTs

Slide 18

Slide 18 text

Maps Values are Counters Sets

Slide 19

Slide 19 text

Maps Values are Counters Sets Registers Booleans

Slide 20

Slide 20 text

Maps Values are Counters Sets Registers Booleans Maps

Slide 21

Slide 21 text

Maps For all your Big Data™ needs!

Slide 22

Slide 22 text

Address Book

Slide 23

Slide 23 text

Interface How can I do that with them?

Slide 24

Slide 24 text

Counters Updates NOT Idempotent (Sorry Pat)

Slide 25

Slide 25 text

Counters Size: O(Actors)

Slide 26

Slide 26 text

Sets { Add | Remove } ~> Add Wins

Slide 27

Slide 27 text

Sets Size: O(Operations) ?

Slide 28

Slide 28 text

Sets Size: O(Actors + Elements)

Slide 29

Slide 29 text

Maps Entries keyed by Name & Type

Slide 30

Slide 30 text

Maps { Update | Remove } ~> Update Wins

Slide 31

Slide 31 text

Maps Size: O(Actors + Entries)

Slide 32

Slide 32 text

Registers Last Write Wins

Slide 33

Slide 33 text

Registers Some Write Loses

Slide 34

Slide 34 text

Booleans { Enable | Disable } ~> Enable

Slide 35

Slide 35 text

Booleans Size: O(Actors)

Slide 36

Slide 36 text

Operations PB & HTTP

Slide 37

Slide 37 text

Increment(1) Add(“Cuttleﬁsh”) Update(“name”, register, Set(“Sam Elliott”))

Slide 38

Slide 38 text

Operations Atomic

Slide 39

Slide 39 text

Operations Unordered

Slide 40

Slide 40 text

Operations Causality

Slide 41

Slide 41 text

Operations Context Required for Correct Removal

Slide 42

Slide 42 text

Bucket Types Required for All Using 2.0 API

Slide 43

Slide 43 text

Bucket Types Counters from 1.4 API Use Default

Slide 44

Slide 44 text

Not in 2.0 2i, Custom Metadata

Slide 45

Slide 45 text

Not in 2.0 YZ Extractors

Slide 46

Slide 46 text

Not in 2.0 Type-level Queries

Slide 47

Slide 47 text

Implementation How do they work underneath?

Slide 48

Slide 48 text

CRDTs Convergent Replicated Data Types

Slide 49

Slide 49 text

Semilattices S ≤ ∨ ⊥ Values Partial Order Join Bottom

Slide 50

Slide 50 text

Counters PN-Counters

Slide 51

Slide 51 text

PN-Counters [ Node → ( Increments, Decrements ) ]

Slide 52

Slide 52 text

PN-Counters: Inc Increase Increments for Current Node

Slide 53

Slide 53 text

PN-Counters: Dec Increase Decrements for Current Node

Slide 54

Slide 54 text

PN-Counters Merge: Pairwise Maximum per-Node

Slide 55

Slide 55 text

Sets OR-Set

Slide 56

Slide 56 text

OR-Set [ Element → ( Tokens , Tombstones ) ]

Slide 57

Slide 57 text

Sets ORSWOT: Without Tombstones!

Slide 58

Slide 58 text

ORSWOT VV ⊠ [ Element → [ Dot ] ]

Slide 59

Slide 59 text

ORSWOT: Add Increment VV Add Dot to Element’s Dot

Slide 60

Slide 60 text

ORSWOT: Remove Leave VV Erase Element & Dots

Slide 61

Slide 61 text

ORSWOT: Merge Element In Both: Merge Dots Element In Only One: Drop when Dominated by Other’s VV Keep Otherwise

Slide 62

Slide 62 text

Maps OR-Map

Slide 63

Slide 63 text

Maps VV ⊠ ( Field , Type ) → ( [ Dot ] , Value )

Slide 64

Slide 64 text

Map: Update Apply Update to Value Increment VV Add Dot to Field’s Dots

Slide 65

Slide 65 text

Map: Add Like Update, Inserting ⊥

Slide 66

Slide 66 text

Map: Remove Leave VV Erase Field & Dots

Slide 67

Slide 67 text

Map: Merge Field In Both: Merge Dots Field In Only One: Drop when Dominated by Other’s VV Keep Otherwise

Slide 68

Slide 68 text

Causality Context & CRDTs

Slide 69

Slide 69 text

Causality Concurrent Remove

Slide 70

Slide 70 text

Concurrent Remove add(a) remove(a) {a,b} {a,b} {a,b} ? ?

Slide 71

Slide 71 text

C. Remove: What We Want add(a) remove(a) {a,b} {a,b} {a,b} {a,b} {a,b} Sᵢ₊₁ Sᵢ

Slide 72

Slide 72 text

C. Remove: The Incorrect Solution add(a) remove(a) {a,b} {a,b} {b} {a,b} {a,b} Sᵢ₊₁ Sᵢ Sᵢ₊₁ = op(Sᵢ) Sᵢ

Slide 73

Slide 73 text

C. Remove: The Correct Solution add(a) remove(a) {a,b} {a,b} {a,b} {a,b} {a,b} Sᵢ₊₁ Sᵢ Sᵢ₊₁ = opᵤ(merge( Sᵢ , opᵣ() ) )

Slide 74

Slide 74 text

Causality Split Brain

Slide 75

Slide 75 text

Split Brain remove(a) {a,b} {b} ? ? (eventually) {a,b}

Slide 76

Slide 76 text

Split Brain: What We Want remove(a) {a,b} {b} {b} {b} (eventually) {a,b}

Slide 77

Slide 77 text

Split Brain: The Incorrect Solution remove(a) {a,b} {b} {a,b} {b} Error Sᵢ₊₁ = op(Sᵢ) (eventually) {a,b} Sᵢ₊₁ Sᵢ

Slide 78

Slide 78 text

Split Brain: The Correct Solution remove(a) {b} {a,b} {b} (eventually) {a,b} {a,b} Sᵢ₊₁ = opᵤ(merge( Sᵢ , opᵣ() ) ) Sᵢ₊₁ Sᵢ

Slide 79

Slide 79 text

QuickCheck Is Awesome!

Slide 80

Slide 80 text

QuickCheck Generalised Checks for CRDTs in riak_dt

Slide 81

Slide 81 text

SyncFree CRDTs, European Style Funded by the European Union

Slide 82

Slide 82 text

SyncFree Dead Actor Pruning Funded by the European Union

Slide 83

Slide 83 text

SyncFree Useful Invariants Funded by the European Union

Slide 84

Slide 84 text

References I’m a Real Academic!

Slide 85

Slide 85 text

http://hal.upmc.fr/inria-00555588

Slide 86

Slide 86 text

http://arxiv.org/abs/1210.3368

Slide 87

Slide 87 text

http://arxiv.org/abs/1011.5808

Slide 88

Slide 88 text

https://github.com/basho/riak_dt

Slide 89

Slide 89 text

Recap Halp, I forgot something!

Slide 90

Slide 90 text

CRDTs Data Types

Slide 91

Slide 91 text

Sets & Maps In 2.0

Slide 92

Slide 92 text

Sets & Maps “If in doubt, Data is Kept”

Slide 93

Slide 93 text

Operations Don’t forget Context for Removes

Slide 94

Slide 94 text

No 2i, Metadata Yet

Slide 95

Slide 95 text

Bucket Types Except 1.4-Style Counters

Slide 96

Slide 96 text

No More Siblings! !!!!!!!!111111ONEONEONE

Slide 97

Slide 97 text

Magical Unless You Know How They Work (Sorry)

Slide 98

Slide 98 text

Question Time! Sam Elliott: sam.elliott@basho.com riak-users@lists.basho.com riak-core@lists.basho.com