Slide 1

Slide 1 text

Declarative, Distributed, Edge Computation Christopher Meiklejohn // @cmeik RICON 2015, November 6th, 2015 1

Slide 2

Slide 2 text

RA RB

Slide 3

Slide 3 text

RA RB 1 set(1)

Slide 4

Slide 4 text

RA RB 1 set(1) 3 2 set(2) set(3)

Slide 5

Slide 5 text

RA RB 1 set(1) 3 2 set(2) set(3) ? ?

Slide 6

Slide 6 text

Synchronization • To enforce an order
 Makes programming easier 6

Slide 7

Slide 7 text

Synchronization • To enforce an order
 Makes programming easier • Eliminate accidental nondeterminism
 Prevent race conditions 6

Slide 8

Slide 8 text

Synchronization • To enforce an order
 Makes programming easier • Eliminate accidental nondeterminism
 Prevent race conditions • Techniques
 Locks, mutexes, semaphores, monitors, etc. 6

Slide 9

Slide 9 text

Difficult Cases • “Internet of Things”
 Low power, limited memory and connectivity 7

Slide 10

Slide 10 text

Difficult Cases • “Internet of Things”
 Low power, limited memory and connectivity • Mobile Gaming
 Offline operation with replicated, shared state 7

Slide 11

Slide 11 text

Zero Synchronization • We want to have our cake and eat it too. 8

Slide 12

Slide 12 text

Zero Synchronization • We want to have our cake and eat it too. • “Zero synchronization” as a starting point
 Add synchronization only where it is necessary 8

Slide 13

Slide 13 text

Zero Synchronization • We want to have our cake and eat it too. • “Zero synchronization” as a starting point
 Add synchronization only where it is necessary • Explore the limits
 Can we design a model of computation where it’s easy to write coordination-free computations that are free from concurrency anomalies? 8

Slide 14

Slide 14 text

9 Strong Weak Strong (Data Center) Weak (Open Internet) Sharing (Application Property) Coupling (Infrastructure Property)

Slide 15

Slide 15 text

9 Strong Weak Strong (Data Center) Weak (Open Internet) Sharing (Application Property) Coupling (Infrastructure Property) MapReduce SETI@Home

Slide 16

Slide 16 text

9 Strong Weak Strong (Data Center) Weak (Open Internet) Sharing (Application Property) Coupling (Infrastructure Property) Twitter MapReduce SETI@Home

Slide 17

Slide 17 text

9 Strong Weak Strong (Data Center) Weak (Open Internet) Sharing (Application Property) Coupling (Infrastructure Property) Twitter MapReduce SETI@Home Synchronization-Free Computations

Slide 18

Slide 18 text

The Fundamentals of Distributed Computation 10

Slide 19

Slide 19 text

Distributed Computation • Distributed to concurrent programming
 Consistency and now partial failure 11

Slide 20

Slide 20 text

Distributed Computation • Distributed to concurrent programming
 Consistency and now partial failure • Enforcing the “single system” illusion 11

Slide 21

Slide 21 text

Distributed Computation • Distributed to concurrent programming
 Consistency and now partial failure • Enforcing the “single system” illusion • Consistency model as a contract 11

Slide 22

Slide 22 text

Distributed Computation • Distributed to concurrent programming
 Consistency and now partial failure • Enforcing the “single system” illusion • Consistency model as a contract • Enforced through synchronization 11

Slide 23

Slide 23 text

Distributed Computation • Distributed to concurrent programming
 Consistency and now partial failure • Enforcing the “single system” illusion • Consistency model as a contract • Enforced through synchronization • Consistency model
 We can think of a consistency model as analogous to a programming paradigm 11

Slide 24

Slide 24 text

Why Is Synchronization Undesirable? 12

Slide 25

Slide 25 text

The Avatars of Time • The problem of time
 Handling physical time in applications is difficult 13

Slide 26

Slide 26 text

The Avatars of Time • The problem of time
 Handling physical time in applications is difficult • Time has three major avatars in computing 13

Slide 27

Slide 27 text

The Avatars of Time • The problem of time
 Handling physical time in applications is difficult • Time has three major avatars in computing • Mutable state in sequential systems 13

Slide 28

Slide 28 text

The Avatars of Time • The problem of time
 Handling physical time in applications is difficult • Time has three major avatars in computing • Mutable state in sequential systems • Nondeterminism in concurrent systems 13

Slide 29

Slide 29 text

The Avatars of Time • The problem of time
 Handling physical time in applications is difficult • Time has three major avatars in computing • Mutable state in sequential systems • Nondeterminism in concurrent systems ̣ Network latency in distributed systems 13

Slide 30

Slide 30 text

The Avatars of Time • The problem of time
 Handling physical time in applications is difficult • Time has three major avatars in computing • Mutable state in sequential systems • Nondeterminism in concurrent systems ̣ Network latency in distributed systems • Unavoidable
 Time is how users interact with programs 13

Slide 31

Slide 31 text

Parable of the Car • “Car driving on a highway” 14

Slide 32

Slide 32 text

Parable of the Car • “Car driving on a highway” • Friction needed for the car to grip the road 14

Slide 33

Slide 33 text

Parable of the Car • “Car driving on a highway” • Friction needed for the car to grip the road • Motors want as little friction as possible 14

Slide 34

Slide 34 text

Parable of the Car • “Car driving on a highway” • Friction needed for the car to grip the road • Motors want as little friction as possible • Time is like friction
 We can not completely eliminate friction, but aim to reduce it as much as possible 14

Slide 35

Slide 35 text

Physical Time (real world) Parable of the Car 15 • Time only at the interface
 The interface is a small part of the system, and this is where we introduction friction

Slide 36

Slide 36 text

Physical Time (real world) Parable of the Car 15 • Time only at the interface
 The interface is a small part of the system, and this is where we introduction friction • Physical time-free execution
 Internally, avoid synchronization as much as possible and virtualize notion of time

Slide 37

Slide 37 text

No Time In The Box? 16

Slide 38

Slide 38 text

No Time In The Box? 16 You program the box.

Slide 39

Slide 39 text

Weak Synchronization • Can we achieve anything without synchronization?
 Not really. 17

Slide 40

Slide 40 text

Weak Synchronization • Can we achieve anything without synchronization?
 Not really. • Strong Eventual Consistency (SEC)
 “Replicas that deliver the same updates have equivalent state” 17

Slide 41

Slide 41 text

Weak Synchronization • Can we achieve anything without synchronization?
 Not really. • Strong Eventual Consistency (SEC)
 “Replicas that deliver the same updates have equivalent state” • Primary requirement
 Eventual replica-to-replica communication 17

Slide 42

Slide 42 text

Weak Synchronization • Can we achieve anything without synchronization?
 Not really. • Strong Eventual Consistency (SEC)
 “Replicas that deliver the same updates have equivalent state” • Primary requirement
 Eventual replica-to-replica communication • Order insensitive! (Commutativity) 17

Slide 43

Slide 43 text

Weak Synchronization • Can we achieve anything without synchronization?
 Not really. • Strong Eventual Consistency (SEC)
 “Replicas that deliver the same updates have equivalent state” • Primary requirement
 Eventual replica-to-replica communication • Order insensitive! (Commutativity) • Duplicate insensitive! (Idempotent) 17

Slide 44

Slide 44 text

RA RB

Slide 45

Slide 45 text

RA RB 1 set(1)

Slide 46

Slide 46 text

RA RB 1 set(1) 3 2 set(2) set(3)

Slide 47

Slide 47 text

RA RB 1 3 2 3 3 set(1) set(2) set(3) max(2,3) max(2,3)

Slide 48

Slide 48 text

How can we succeed with Strong Eventual Consistency? 22

Slide 49

Slide 49 text

Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic operations monotonically) 23

Slide 50

Slide 50 text

Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic operations monotonically) 2. Retain the properties of functional programming
 (ex. confluence, referential transparency over composition) 23

Slide 51

Slide 51 text

Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic operations monotonically) 2. Retain the properties of functional programming
 (ex. confluence, referential transparency over composition) 3. Distributed, and fault-tolerant runtime
 (ex. replication) 23

Slide 52

Slide 52 text

Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic operations monotonically) 2. Retain the properties of functional programming
 (ex. confluence, referential transparency over composition) 3. Distributed, and fault-tolerant runtime
 (ex. replication) 24

Slide 53

Slide 53 text

Convergent Objects
 Conflict-Free 
 Replicated Data Types 25 SSS 2011

Slide 54

Slide 54 text

Conflict-Free 
 Replicated Data Types • Many types exist with different properties
 Sets, counters, registers, flags, maps, graphs 26

Slide 55

Slide 55 text

Conflict-Free 
 Replicated Data Types • Many types exist with different properties
 Sets, counters, registers, flags, maps, graphs • Strong Eventual Consistency
 Instances satisfy SEC property per-object 26

Slide 56

Slide 56 text

RA RB RC

Slide 57

Slide 57 text

RA RB RC {1} (1, {a}, {}) add(1)

Slide 58

Slide 58 text

RA RB RC {1} (1, {a}, {}) add(1) {1} (1, {c}, {}) add(1)

Slide 59

Slide 59 text

RA RB RC {1} (1, {a}, {}) add(1) {1} (1, {c}, {}) add(1) {} (1, {c}, {c}) remove(1)

Slide 60

Slide 60 text

RA RB RC {1} (1, {a}, {}) add(1) {1} (1, {c}, {}) add(1) {} (1, {c}, {c}) remove(1) {1} {1} {1} (1, {a, c}, {c}) (1, {a, c}, {c}) (1, {a, c}, {c})

Slide 61

Slide 61 text

Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic operations monotonically) 2. Retain the properties of functional programming
 (ex. confluence, referential transparency over composition) 3. Distributed, and fault-tolerant runtime
 (ex. replication) 32

Slide 62

Slide 62 text

Convergent Programs Lattice Processing 33 PPDP 2015

Slide 63

Slide 63 text

Lattice Processing (Lasp) • Distributed, deterministic dataflow
 Distributed, deterministic dataflow programming model for “eventually consistent” computations 34

Slide 64

Slide 64 text

Lattice Processing (Lasp) • Distributed, deterministic dataflow
 Distributed, deterministic dataflow programming model for “eventually consistent” computations • Convergent data structures
 Primary data abstraction is the CRDT 34

Slide 65

Slide 65 text

Lattice Processing (Lasp) • Distributed, deterministic dataflow
 Distributed, deterministic dataflow programming model for “eventually consistent” computations • Convergent data structures
 Primary data abstraction is the CRDT • Enables composition
 Provides functional composition of CRDTs that preserves the SEC property 34

Slide 66

Slide 66 text

35 %% Create initial set. S1 = declare(set), %% Add elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).

Slide 67

Slide 67 text

36 %% Create initial set. S1 = declare(set), %% Add elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).

Slide 68

Slide 68 text

37 %% Create initial set. S1 = declare(set), %% Add elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).

Slide 69

Slide 69 text

38 %% Create initial set. S1 = declare(set), %% Add elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).

Slide 70

Slide 70 text

39 %% Create initial set. S1 = declare(set), %% Add elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).

Slide 71

Slide 71 text

Lattice Processing (Lasp) • Functional and set-theoretic operations on sets
 Product, intersection, union, filter, map, fold 40

Slide 72

Slide 72 text

Lattice Processing (Lasp) • Functional and set-theoretic operations on sets
 Product, intersection, union, filter, map, fold • Metadata computation
 Performs transformation on the internal metadata of CRDTs allowing creation of “composed” CRDTs 40

Slide 73

Slide 73 text

Lasp Processes • Replicas as monotonic streams
 Each replica of a CRDT produces a monotonic stream of states 41

Slide 74

Slide 74 text

Lasp Processes • Replicas as monotonic streams
 Each replica of a CRDT produces a monotonic stream of states • Monotonic processes
 Read from one or more input replica streams and produce a single output replica stream 41

Slide 75

Slide 75 text

Lasp Processes • Replicas as monotonic streams
 Each replica of a CRDT produces a monotonic stream of states • Monotonic processes
 Read from one or more input replica streams and produce a single output replica stream • Inflationary reads
 Read operation ensures that we only read inflationary updates to replicas 41

Slide 76

Slide 76 text

Lattice Processing Monotonic Streams 42

Slide 77

Slide 77 text

RA {} C1 {} C2 {} 43

Slide 78

Slide 78 text

RA {} (1, {a}, {}) C1 (1, {a}, {}) {} C2 {} 44

Slide 79

Slide 79 text

RA {} (1, {a}, {}) (1, {a, c}, {}) C1 (1, {a}, {}) {} C2 (1, {c}, {}) {} 45

Slide 80

Slide 80 text

RA {} (1, {a}, {}) (1, {a, c}, {}) (1, {a, c}, {a}) C1 (1, {a}, {}) {} (1, {a}, {a}) C2 (1, {c}, {}) {} 46

Slide 81

Slide 81 text

RA {} (1, {a}, {}) (1, {a, c}, {}) (1, {a, c}, {a}) C1 (1, {a}, {}) {} (1, {a}, {a}) C2 (1, {c}, {}) {} (1, {a, c}, {a}) (1, {a, c}, {a}) (1, {a, c}, {a}) 47

Slide 82

Slide 82 text

RA {} (1, {a}, {}) (1, {a, c}, {}) (1, {a, c}, {a}) C1 (1, {a}, {}) {} (1, {a}, {a}) C2 (1, {c}, {}) {} (1, {a, c}, {a}) (1, {a, c}, {a}) (1, {a, c}, {a}) 48

Slide 83

Slide 83 text

Lattice Processing Monotonic Processes 49

Slide 84

Slide 84 text

RA {} P1 F(RA) {} 50

Slide 85

Slide 85 text

RA {} P1 F(RA) {} strict_read({}) 51

Slide 86

Slide 86 text

RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) 52

Slide 87

Slide 87 text

RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1, {a}, {})) 53

Slide 88

Slide 88 text

RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1, {a}, {})) (2, {a}, {}) 54

Slide 89

Slide 89 text

RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1, {a}, {})) (2, {a}, {}) strict_read((1, {a}, {}) (1, {a}, {}) 55

Slide 90

Slide 90 text

RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1, {a}, {})) (2, {a}, {}) strict_read((1, {a}, {}) (1, {a}, {}) (1, {a}, {a}) F((1, {a}, {a})) (2, {a}, {a}) strict_read((1, {a}, {a}) (1, {a}, {a}) 56

Slide 91

Slide 91 text

RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) (1, {a}, {}) (1, {a}, {a}) F((1, {a}, {a})) (2, {a}, {a}) strict_read((1, {a}, {a}) (1, {a}, {a}) 57

Slide 92

Slide 92 text

Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic operations monotonically) 2. Retain the properties of functional programming
 (ex. confluence, referential transparency over composition) 3. Distributed, and fault-tolerant runtime
 (ex. replication) 58

Slide 93

Slide 93 text

Distributed Runtime Selective Hearing 59 W-PSDS 2015

Slide 94

Slide 94 text

Selective Hearing • Epidemic broadcast based runtime system
 Provide a runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution 60

Slide 95

Slide 95 text

Selective Hearing • Epidemic broadcast based runtime system
 Provide a runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) 60

Slide 96

Slide 96 text

Selective Hearing • Epidemic broadcast based runtime system
 Provide a runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) • Epidemic broadcast mechanisms provide weak ordering but are resilient and efficient 60

Slide 97

Slide 97 text

Selective Hearing • Epidemic broadcast based runtime system
 Provide a runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) • Epidemic broadcast mechanisms provide weak ordering but are resilient and efficient • Lasp’s programming model is tolerant to message re-ordering, disconnections, and node failures 60

Slide 98

Slide 98 text

Selective Hearing • Epidemic broadcast based runtime system
 Provide a runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) • Epidemic broadcast mechanisms provide weak ordering but are resilient and efficient • Lasp’s programming model is tolerant to message re-ordering, disconnections, and node failures • “Selective Receive”
 Nodes selectively receive and process messages based on interest 60

Slide 99

Slide 99 text

Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic operations monotonically) 2. Retain the properties of functional programming
 (ex. confluence, referential transparency over composition) 3. Distributed, and fault-tolerant runtime
 (ex. replication) 61

Slide 100

Slide 100 text

What can we build? Leaderboard 62

Slide 101

Slide 101 text

Leaderboard • Mobile game platform
 Local leaderboard tracking top-k highest scored games 63

Slide 102

Slide 102 text

Leaderboard • Mobile game platform
 Local leaderboard tracking top-k highest scored games • Clients will go offline
 Clients have limited connectivity and the system still needs to make progress while clients are offline 63

Slide 103

Slide 103 text

Client 1 Leaderboard Client 3 Leaderboard Client 2 Leaderboard Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT 64

Slide 104

Slide 104 text

Leaderboard • Peer-to-peer dissemination
 Nodes periodically “merge” their state with a random peer 65

Slide 105

Slide 105 text

Leaderboard • Peer-to-peer dissemination
 Nodes periodically “merge” their state with a random peer • Complexity in the data type
 Each node tracks a top-k set of its own games in a bounded set 65

Slide 106

Slide 106 text

66 %% Create a leaderboard datatype. L = declare({top_k, [2]}). %% Update leaderboard. update({set, Name, Score}, L).

Slide 107

Slide 107 text

67 %% Create a leaderboard datatype. L = declare({top_k, [2]}). %% Update leaderboard. update({set, Name, Score}, L).

Slide 108

Slide 108 text

68 %% Create a leaderboard datatype. L = declare({top_k, [2]}). %% Update leaderboard. update({set, Name, Score}, L).

Slide 109

Slide 109 text

What if we want to enhance the behavior? 69

Slide 110

Slide 110 text

What if we want to enhance the behavior? 69 Without the creation of a new datatype

Slide 111

Slide 111 text

What can we build? Per-User Leaderboard 70 EdgeCom 2016

Slide 112

Slide 112 text

Per-User Leaderboard • Enhance existing design
 Only the top score for each user at each device 71

Slide 113

Slide 113 text

Per-User Leaderboard • Enhance existing design
 Only the top score for each user at each device • Minimize transmitted state
 Prevent transmission of state that is not necessary to perform the computation 71

Slide 114

Slide 114 text

Per-User Leaderboard • Enhance existing design
 Only the top score for each user at each device • Minimize transmitted state
 Prevent transmission of state that is not necessary to perform the computation • Compose data types
 Build a per-user leaderboard through the composition of existing types 71

Slide 115

Slide 115 text

Client1 Scores Local Top-K Fold Global Top-K Lasp Operation Input User-Maintained CRDT Output Lasp-Maintained CRDT Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold 72

Slide 116

Slide 116 text

Client1 Scores Local Top-K Fold Global Top-K Fold 73 Client1 Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold

Slide 117

Slide 117 text

Client1 Scores Local Top-K Fold Global Top-K Fold 74 Client1 Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold

Slide 118

Slide 118 text

Client1 Scores Local Top-K Fold Global Top-K Fold 75 Client1 Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold

Slide 119

Slide 119 text

Client1 Scores Local Top-K Fold Global Top-K Fold 76 Client1 Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold

Slide 120

Slide 120 text

Client1 Scores Local Top-K Fold Global Top-K Fold 77 Client1 Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold

Slide 121

Slide 121 text

Client1 Scores Local Top-K Fold Global Top-K Fold 78 Client1 Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold

Slide 122

Slide 122 text

Client1 Scores Local Top-K Fold Global Top-K Lasp Operation Input User-Maintained CRDT Output Lasp-Maintained CRDT Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold 79 Client1 Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold

Slide 123

Slide 123 text

Per-User Leaderboard • Dynamically scoped variables
 Variable which take different values depending on where it is executing 80

Slide 124

Slide 124 text

Per-User Leaderboard • Dynamically scoped variables
 Variable which take different values depending on where it is executing • Dynamically scoped fold operation
 Perform a distributed “reduce” operation that combines the state of a dynamically scoped variables across 80

Slide 125

Slide 125 text

81 %% Create a global leaderboard. G = declare({top_k, [10]}). %% Create a local leaderboard. L = declare_dynamic({top_k, [10]}). %% Create a set of scores. S = declare_dynamic(set). %% Compute local top-k list. fold(S, fun max_by_name/2, L). %% Compute global top-k list. fold_dynamic(L, fun max_by_name/2, G).

Slide 126

Slide 126 text

82 %% Create a global leaderboard. G = declare({top_k, [10]}). %% Create a local leaderboard. L = declare_dynamic({top_k, [10]}). %% Create a set of scores. S = declare_dynamic(set). %% Compute local top-k list. fold(S, fun max_by_name/2, L). %% Compute global top-k list. fold_dynamic(L, fun max_by_name/2, G).

Slide 127

Slide 127 text

83 %% Create a global leaderboard. G = declare({top_k, [10]}). %% Create a local leaderboard. L = declare_dynamic({top_k, [10]}). %% Create a set of scores. S = declare_dynamic(set). %% Compute local top-k list. fold(S, fun max_by_name/2, L). %% Compute global top-k list. fold_dynamic(L, fun max_by_name/2, G).

Slide 128

Slide 128 text

84 %% Create a global leaderboard. G = declare({top_k, [10]}). %% Create a local leaderboard. L = declare_dynamic({top_k, [10]}). %% Create a set of scores. S = declare_dynamic(set). %% Compute local top-k list. fold(S, fun max_by_name/2, L). %% Compute global top-k list. fold_dynamic(L, fun max_by_name/2, G).

Slide 129

Slide 129 text

85 %% Create a global leaderboard. G = declare({top_k, [10]}). %% Create a local leaderboard. L = declare_dynamic({top_k, [10]}). %% Create a set of scores. S = declare_dynamic(set). %% Compute local top-k list. fold(S, fun max_by_name/2, L). %% Compute global top-k list. fold_dynamic(L, fun max_by_name/2, G).

Slide 130

Slide 130 text

86 %% Create a global leaderboard. G = declare({top_k, [10]}). %% Create a local leaderboard. L = declare_dynamic({top_k, [10]}). %% Create a set of scores. S = declare_dynamic(set). %% Compute local top-k list. fold(S, fun max_by_name/2, L). %% Compute global top-k list. fold_dynamic(L, fun max_by_name/2, G).

Slide 131

Slide 131 text

Let’s look at a larger application 87

Slide 132

Slide 132 text

Let’s look at a larger application 87 With visible 
 non-monotonicity

Slide 133

Slide 133 text

What can we build? Advertisement Counter 88

Slide 134

Slide 134 text

Advertisement Counter • Mobile game platform selling advertisement space
 Advertisements are paid according to a minimum number of impressions 89

Slide 135

Slide 135 text

Advertisement Counter • Mobile game platform selling advertisement space
 Advertisements are paid according to a minimum number of impressions • Clients will go offline
 Clients have limited connectivity and the system still needs to make progress while clients are offline 89

Slide 136

Slide 136 text

Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 90

Slide 137

Slide 137 text

Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Riot Ads Rovio Ads Product Read 50,000 Remove Increment Union 91 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client

Slide 138

Slide 138 text

Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 1 Client 92 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client

Slide 139

Slide 139 text

Ads ovio Ad ounter 1 ovio Ad ounter 2 iot Ad ounter 1 iot Ad ounter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Client Side, Sing 93 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client

Slide 140

Slide 140 text

Ads Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product move Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 94 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client

Slide 141

Slide 141 text

Ads Contracts Ads Contracts Ads With Contracts Filter Product Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 95 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client

Slide 142

Slide 142 text

Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 96 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client

Slide 143

Slide 143 text

Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Riot Ads Rovio Ads Fil Product Read 50,000 Remove Increment Union 97 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client

Slide 144

Slide 144 text

Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 98 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client

Slide 145

Slide 145 text

Advertisement Counter • Completely monotonic
 Disabling advertisements and contracts are all modeled through monotonic state growth 99

Slide 146

Slide 146 text

Advertisement Counter • Completely monotonic
 Disabling advertisements and contracts are all modeled through monotonic state growth • Arbitrary distribution
 Use of convergent data structures allows computational graph to be arbitrarily distributed 99

Slide 147

Slide 147 text

Advertisement Counter • Completely monotonic
 Disabling advertisements and contracts are all modeled through monotonic state growth • Arbitrary distribution
 Use of convergent data structures allows computational graph to be arbitrarily distributed • Divergence
 Divergence is a factor of synchronization period 99

Slide 148

Slide 148 text

100 %% Generate a series of contracts. Contracts = declare(set), %% Generate advertisements. RiotAds = declare(set), RovioAds = declare(set), create_advertisements_and_contracts(RiotAds, Contracts), create_advertisements_and_contracts(RovioAds, Contracts), %% Union ads. Ads = declare(set), union(RovioAds, RiotAds, Ads), %% Compute the product of both ads and contracts. AdsContracts = declare(set), product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. AdsWithContracts = declare(set), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, filter(AdsContracts, FilterFun, AdsWithContracts).

Slide 149

Slide 149 text

101 %% Generate a series of contracts. Contracts = declare(set), %% Generate advertisements. RiotAds = declare(set), RovioAds = declare(set), create_advertisements_and_contracts(RiotAds, Contracts), create_advertisements_and_contracts(RovioAds, Contracts), %% Union ads. Ads = declare(set), union(RovioAds, RiotAds, Ads), %% Compute the product of both ads and contracts. AdsContracts = declare(set), product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. AdsWithContracts = declare(set), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, filter(AdsContracts, FilterFun, AdsWithContracts).

Slide 150

Slide 150 text

102 %% Generate a series of contracts. Contracts = declare(set), %% Generate advertisements. RiotAds = declare(set), RovioAds = declare(set), create_advertisements_and_contracts(RiotAds, Contracts), create_advertisements_and_contracts(RovioAds, Contracts), %% Union ads. Ads = declare(set), union(RovioAds, RiotAds, Ads), %% Compute the product of both ads and contracts. AdsContracts = declare(set), product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. AdsWithContracts = declare(set), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, filter(AdsContracts, FilterFun, AdsWithContracts).

Slide 151

Slide 151 text

103 %% Generate a series of contracts. Contracts = declare(set), %% Generate advertisements. RiotAds = declare(set), RovioAds = declare(set), create_advertisements_and_contracts(RiotAds, Contracts), create_advertisements_and_contracts(RovioAds, Contracts), %% Union ads. Ads = declare(set), union(RovioAds, RiotAds, Ads), %% Compute the product of both ads and contracts. AdsContracts = declare(set), product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. AdsWithContracts = declare(set), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, filter(AdsContracts, FilterFun, AdsWithContracts).

Slide 152

Slide 152 text

104 %% Generate a series of contracts. Contracts = declare(set), %% Generate advertisements. RiotAds = declare(set), RovioAds = declare(set), create_advertisements_and_contracts(RiotAds, Contracts), create_advertisements_and_contracts(RovioAds, Contracts), %% Union ads. Ads = declare(set), union(RovioAds, RiotAds, Ads), %% Compute the product of both ads and contracts. AdsContracts = declare(set), product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. AdsWithContracts = declare(set), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, filter(AdsContracts, FilterFun, AdsWithContracts).

Slide 153

Slide 153 text

105 %% Generate a series of contracts. Contracts = declare(set), %% Generate advertisements. RiotAds = declare(set), RovioAds = declare(set), create_advertisements_and_contracts(RiotAds, Contracts), create_advertisements_and_contracts(RovioAds, Contracts), %% Union ads. Ads = declare(set), union(RovioAds, RiotAds, Ads), %% Compute the product of both ads and contracts. AdsContracts = declare(set), product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. AdsWithContracts = declare(set), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, filter(AdsContracts, FilterFun, AdsWithContracts).

Slide 154

Slide 154 text

Client3 Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Riot Ad Counter 1 Client1 Client2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Riot Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 1 Ads With Contracts Ads With Contracts Ads With Contracts Server Ads With Contracts Server Computation! 106

Slide 155

Slide 155 text

Client3 Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Riot Ad Counter 1 Client1 Client2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Riot Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 1 Ads With Contracts Ads With Contracts Ads With Contracts Server Ads With Contracts Server Computation! 107

Slide 156

Slide 156 text

Client3 Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Riot Ad Counter 1 Client1 Client2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Riot Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 1 Ads With Contracts Ads With Contracts Ads With Contracts Server Ads With Contracts Server Computation! 108

Slide 157

Slide 157 text

Advertisement Counter • “Servers” as peers to “clients”
 Servers are peers to clients that perform additional computation 109

Slide 158

Slide 158 text

Advertisement Counter • “Servers” as peers to “clients”
 Servers are peers to clients that perform additional computation • Any node can disable an advertisement under this model given enough information 109

Slide 159

Slide 159 text

Advertisement Counter • “Servers” as peers to “clients”
 Servers are peers to clients that perform additional computation • Any node can disable an advertisement under this model given enough information • “Servers” as trusted nodes
 Serve as a location for performing “exactly once” side- effects 109

Slide 160

Slide 160 text

Advertisement Counter • “Servers” as peers to “clients”
 Servers are peers to clients that perform additional computation • Any node can disable an advertisement under this model given enough information • “Servers” as trusted nodes
 Serve as a location for performing “exactly once” side- effects • Billing customers must be done at a central point by a trusted node in the system 109

Slide 161

Slide 161 text

We’ve build up from zero synchronization 110

Slide 162

Slide 162 text

We’ve build up from zero synchronization 110 Instead of working to remove synchronization

Slide 163

Slide 163 text

What have we learned? 111

Slide 164

Slide 164 text

Key Points • Synchronization is expensive
 Locking and other synchronization mechanisms limit scalability to the critical section 112

Slide 165

Slide 165 text

Key Points • Synchronization is expensive
 Locking and other synchronization mechanisms limit scalability to the critical section • Synchronization is sometimes not possible
 Mobile and “Internet of Things” applications make synchronization for replicated state impractical 112

Slide 166

Slide 166 text

Key Points • Synchronization is expensive
 Locking and other synchronization mechanisms limit scalability to the critical section • Synchronization is sometimes not possible
 Mobile and “Internet of Things” applications make synchronization for replicated state impractical • Apply synchronization only where required
 Global invariants, atomic visibility, etc. 112

Slide 167

Slide 167 text

How do I learn more? 113

Slide 168

Slide 168 text

Publications • “Lasp: A Language for Distributed, Coordination-Free Programming” 
 ACM SIGPLAN PPDP 2015 • “Selective Hearing: An Approach to Distributed, Eventually Consistent Edge Computation”
 IEEE W-PSDS 2015 • “The Implementation and Use of a Generic Dataflow Behaviour in Erlang”
 ACM SIGPLAN Erlang Workshop ’15 • “Lasp: A Language for Distributed, Eventually Consistent Computations with CRDTs"
 PaPoC 2015 • “Declarative, Sliding Window Aggregations for Computations at the Edge"
 IEEE EdgeCom 2016 114

Slide 169

Slide 169 text

SyncFree is a European research project taking place for 3 years, staring October 2013, and is funded by the European Union, grant agreement n° 609551. 115

Slide 170

Slide 170 text

Thanks! 116 Christopher Meiklejohn @cmeik