Declarative,
Secure,
Convergent
Edge Computation
Christopher Meiklejohn
Erlang Factory 2016, March 10th, 2016
1
Slide 2
Slide 2 text
Example Application
Hospital Refrigerators
2
Slide 3
Slide 3 text
Hospital Refrigerators
Typical Topology
3
Slide 4
Slide 4 text
4
Slide 5
Slide 5 text
Internet
5
Slide 6
Slide 6 text
Internet HDFS
6
Slide 7
Slide 7 text
Internet HDFS
Hadoop
Client
Client
Internet
7
Slide 8
Slide 8 text
Internet HDFS
Hadoop
Client
Client
Internet
Spark
Client
Client
Internet
8
Slide 9
Slide 9 text
Hospital Refrigerators
Ideal Execution
9
Slide 10
Slide 10 text
Internet HDFS
Spark
10
Slide 11
Slide 11 text
Internet HDFS
Spark
35F
11
Slide 12
Slide 12 text
Internet HDFS
Spark
35F
12
Slide 13
Slide 13 text
Internet HDFS
Spark
35F
40F
13
Slide 14
Slide 14 text
Internet HDFS
Spark
35F
40F
14
Slide 15
Slide 15 text
Client
Internet HDFS
Spark
35F
40F
15
Slide 16
Slide 16 text
Problem
Connectivity
16
Slide 17
Slide 17 text
Internet HDFS
Spark
17
Slide 18
Slide 18 text
Internet HDFS
Spark
35F
18
Slide 19
Slide 19 text
Internet HDFS
Spark
35F
19
Slide 20
Slide 20 text
Internet HDFS
Spark
35F
40F
20
Slide 21
Slide 21 text
Internet HDFS
Spark
35F
40F
21
Slide 22
Slide 22 text
Internet HDFS
Spark
35F
40F
40F
22
Slide 23
Slide 23 text
Internet HDFS
Spark
35F
40F
40F
23
Slide 24
Slide 24 text
Solution
Local Decisions
24
Slide 25
Slide 25 text
Internet HDFS
Spark
25
Slide 26
Slide 26 text
Internet HDFS
Spark
35F
26
Slide 27
Slide 27 text
Internet HDFS
Spark
35F
27
Slide 28
Slide 28 text
Internet HDFS
Spark
35F
40F
28
Slide 29
Slide 29 text
Internet HDFS
Spark
35F
40F
29
Slide 30
Slide 30 text
Solution
Transitive Dissemination
30
Slide 31
Slide 31 text
Internet HDFS
Spark
31
Slide 32
Slide 32 text
Internet HDFS
Spark
35F
32
Slide 33
Slide 33 text
Internet HDFS
Spark
35F
33
Slide 34
Slide 34 text
Internet HDFS
Spark
35F
40F
34
Slide 35
Slide 35 text
Internet HDFS
Spark
35F
40F
35
Slide 36
Slide 36 text
Internet HDFS
Spark
35F
40F
Client
36
Slide 37
Slide 37 text
Problem
State Transmission
37
Slide 38
Slide 38 text
Internet
38
Slide 39
Slide 39 text
Internet
35F
39
Slide 40
Slide 40 text
Internet
35F
35F
35F
35F
40
Slide 41
Slide 41 text
Internet
35F
35F
35F
35F
41
Slide 42
Slide 42 text
Solution
Aggregate Dissemination
42
Slide 43
Slide 43 text
Internet
43
Slide 44
Slide 44 text
Internet
OK
NO
OK
44
Slide 45
Slide 45 text
Internet
OK
OK
OK
OK
OK
NO
OK
OK
OK
45
Slide 46
Slide 46 text
Internet
OK
OK
OK
OK
OK
NO
OK
OK
OK
HDFS
OK
OK
OK
OK
OK
OK
OK
OK
NO
46
Slide 47
Slide 47 text
Internet
OK
OK
OK
OK
OK
NO
OK
OK
OK
OK
OK
OK OK
OK
OK
OK
OK
NO
= =
HDFS
OK
OK
OK
OK
OK
OK
OK
NO
47
Slide 48
Slide 48 text
Internet
OK
OK
OK
OK
OK
NO
OK
OK
OK
?
OK
OK
OK OK
OK
OK
OK
OK
NO
= =
48
Slide 49
Slide 49 text
Internet
OK
OK
OK
OK
OK
NO
OK
OK
OK
?
OK
OK
OK OK
OK
OK
OK
OK
NO
= =
49
Slide 50
Slide 50 text
Local Computation
• Reduce state transmission
Perform some local computation to
reduce transmitted state on the wire
50
Slide 51
Slide 51 text
Local Computation
• Reduce state transmission
Perform some local computation to
reduce transmitted state on the wire
• Make local decisions
Make decisions based on results of
local computation
50
Slide 52
Slide 52 text
Databases
Consistency Models
51
Slide 53
Slide 53 text
Databases
Strong Consistency
52
Slide 54
Slide 54 text
R1
C1 C2
53
Slide 55
Slide 55 text
R1
C1 C2
Value 1
54
Slide 56
Slide 56 text
R1
C1 C2
Read Value 1
Value 1
Value 1
55
Slide 57
Slide 57 text
R1
C1 C2
Read Value 1
Value 1 Value 1
Value 1
56
Slide 58
Slide 58 text
R1
C1 C2
CAS Value 1 with Value 2
Value 2 Value 1
Value 2
57
Slide 59
Slide 59 text
R1
C1 C2
CAS Value 1 with Value 3
Value 2 Value 1
Value 2
58
Slide 60
Slide 60 text
I won’t diagram
the Paxos protocol
59
Slide 61
Slide 61 text
R2
C1 C2
Value 2 Value 1
Value 2
R1 R3
Paxos
60
Slide 62
Slide 62 text
Databases
Eventual Consistency
61
Slide 63
Slide 63 text
R1 R2 R3
C1 C2
62
Slide 64
Slide 64 text
R1 R2 R3
C1 C2
Value 1 @ [1, 0, 0] Value 1 @ [1, 0, 0] Value 1 @ [1, 0, 0]
63
Slide 65
Slide 65 text
R1 R2 R3
C1 C2
Value 1 @ [1, 0, 0] Value 1 @ [1, 0, 0] Value 1 @ [1, 0, 0]
Value 1 @ [1, 0, 0]
64
Slide 66
Slide 66 text
R1 R2 R3
C1 C2
Value 1 @ [1, 0, 0] Value 1 @ [1, 0, 0] Value 1 @ [1, 0, 0]
Value 1 @ [1, 0, 0] Value 1 @ [1, 0, 0]
65
Slide 67
Slide 67 text
R1 R2 R3
C1 C2
Value 1 @ [1, 0, 0] Value 1 @ [1, 0, 0] Value 1 @ [1, 0, 0]
Value 1 @ [1, 0, 0] Value 1 @ [1, 0, 0]
Write 2
66
Slide 68
Slide 68 text
R1 R2 R3
C1 C2
Value 2 @ [2, 0, 0] Value 2 @ [2, 0, 0] Value 2 @ [2, 0, 0]
Value 2 @ [2, 0, 0] Value 1 @ [1, 0, 0]
Write 2
Write Value 2 @ [2, 0, 0]
67
Slide 69
Slide 69 text
R1 R2 R3
C1 C2
Value 2 @ [2, 0, 0]
Value 3 @ [1, 0, 1]
Value 2 @ [2, 0, 0]
Value 3 @ [1, 0, 1]
Value 2 @ [2, 0, 0]
Value 3 @ [1, 0, 1]
Value 2 @ [2, 0, 0] Value 2 @ [2, 0, 0]
Value 3 @ [1, 0, 1]
Write 3
68
Slide 70
Slide 70 text
R1 R2 R3
C1 C2
Value 2 @ [2, 0, 0]
Value 3 @ [1, 0, 1]
Value 2 @ [2, 0, 0]
Value 3 @ [1, 0, 1]
Value 2 @ [2, 0, 0]
Value 3 @ [1, 0, 1]
Value 2 @ [2, 0, 0]
Value 3 @ [1, 0, 1]
Value 2 @ [2, 0, 0]
Value 3 @ [1, 0, 1]
69
Slide 71
Slide 71 text
R1 R2 R3
C1 C2
Value 4 @ [2, 0, 1] Value 4 @ [2, 0, 1] Value 4 @ [2, 0, 1]
Value 4 @ [2, 0, 1] Value 2 @ [2, 0, 0]
Value 3 @ [1, 0, 1]
Write 4
70
Slide 72
Slide 72 text
Eventual Consistency
As The Model
71
Slide 73
Slide 73 text
Clients
Own Their Data
72
Slide 74
Slide 74 text
OK
OK
OK
73
Slide 75
Slide 75 text
OK
OK
OK
OK
OK
OK
OK
OK
OK
74
Slide 76
Slide 76 text
OK
OK
OK
OK
OK
OK
OK
OK
OK
75
Slide 77
Slide 77 text
OK
OK
OK
OK
OK
OK
OK
76
Slide 78
Slide 78 text
OK
OK
OK
OK
OK
OK
OK
OK
OK
77
Slide 79
Slide 79 text
Computations
Mergability & Provenance
78
Slide 80
Slide 80 text
A
A
B
C
79
Slide 81
Slide 81 text
A
A
B
C
D’
D’’
F
F
80
Slide 82
Slide 82 text
A
A
B
C
D’
D’’
D’ D’’ D
F
F
81
Slide 83
Slide 83 text
D’’
A
A
B
C
D’
D’ D’’ D
D
D
Merge
F
F
82
Slide 84
Slide 84 text
Example Application
Preliminary Results
83
Slide 85
Slide 85 text
Preliminary Results
• Conflict-Free Replicated Data Types
Distributed data structures designed for convergence
[Shapiro et al., 2011]
84
Slide 86
Slide 86 text
Preliminary Results
• Conflict-Free Replicated Data Types
Distributed data structures designed for convergence
[Shapiro et al., 2011]
• Lattice Processing
Make decisions based on results of local computation
[Meiklejohn & Van Roy, 2015]
84
Slide 87
Slide 87 text
Preliminary Results
• Conflict-Free Replicated Data Types
Distributed data structures designed for convergence
[Shapiro et al., 2011]
• Lattice Processing
Make decisions based on results of local computation
[Meiklejohn & Van Roy, 2015]
• Selective Hearing
Epidemic broadcast based runtime system
[Meiklejohn & Van Roy, 2015/2016]
84
Slide 88
Slide 88 text
Conflict-Free
Replicated Data Types
• Collection of types
Sets, counters, registers, flags, maps
85
Slide 89
Slide 89 text
Conflict-Free
Replicated Data Types
• Collection of types
Sets, counters, registers, flags, maps
• Strong Eventual Consistency
Objects that receive the same updates,
regardless of order, will reach equivalent
state
85
Lattice Processing
• Distributed, deterministic dataflow
Distributed, dataflow programming model
• Convergent data structures
Data abstraction is the CRDT
91
Slide 97
Slide 97 text
Lattice Processing
• Distributed, deterministic dataflow
Distributed, dataflow programming model
• Convergent data structures
Data abstraction is the CRDT
• Enables composition
Composition preserves SEC
91
Slide 98
Slide 98 text
92
%% Create initial set.
S1 = declare(set),
%% Add elements to initial set and update.
update(S1, {add, [1,2,3]}),
%% Create second set.
S2 = declare(set),
%% Apply map operation between S1 and S2.
map(S1, fun(X) -> X * 2 end, S2).
Slide 99
Slide 99 text
93
%% Create initial set.
S1 = declare(set),
%% Add elements to initial set and update.
update(S1, {add, [1,2,3]}),
%% Create second set.
S2 = declare(set),
%% Apply map operation between S1 and S2.
map(S1, fun(X) -> X * 2 end, S2).
Slide 100
Slide 100 text
94
%% Create initial set.
S1 = declare(set),
%% Add elements to initial set and update.
update(S1, {add, [1,2,3]}),
%% Create second set.
S2 = declare(set),
%% Apply map operation between S1 and S2.
map(S1, fun(X) -> X * 2 end, S2).
Slide 101
Slide 101 text
95
%% Create initial set.
S1 = declare(set),
%% Add elements to initial set and update.
update(S1, {add, [1,2,3]}),
%% Create second set.
S2 = declare(set),
%% Apply map operation between S1 and S2.
map(S1, fun(X) -> X * 2 end, S2).
Slide 102
Slide 102 text
96
%% Create initial set.
S1 = declare(set),
%% Add elements to initial set and update.
update(S1, {add, [1,2,3]}),
%% Create second set.
S2 = declare(set),
%% Apply map operation between S1 and S2.
map(S1, fun(X) -> X * 2 end, S2).
Slide 103
Slide 103 text
Selective Hearing
• Epidemic broadcast protocol
Runtime system for application state & scope
97
Slide 104
Slide 104 text
Selective Hearing
• Epidemic broadcast protocol
Runtime system for application state & scope
• Peer-to-peer dissemination
Pairwise synchronization between peers
without a central coordinator
97
Slide 105
Slide 105 text
Selective Hearing
• Epidemic broadcast protocol
Runtime system for application state & scope
• Peer-to-peer dissemination
Pairwise synchronization between peers
without a central coordinator
• No ordering guarantees on messages
Programming model can tolerate message
reordering and duplication
97
Slide 106
Slide 106 text
What can we build?
Leaderboard
98
Slide 107
Slide 107 text
Leaderboard
• Mobile game platform
Local leaderboard tracking top-k
highest scored games
99
Slide 108
Slide 108 text
Leaderboard
• Mobile game platform
Local leaderboard tracking top-k
highest scored games
• Clients will go offline
Clients have limited connectivity and the
system still needs to make progress
while clients are offline
99
Leaderboard
• Peer-to-peer dissemination
Nodes periodically “merge” their state
with a random peer
101
Slide 111
Slide 111 text
Leaderboard
• Peer-to-peer dissemination
Nodes periodically “merge” their state
with a random peer
• Complexity in the data type
Each node tracks a top-k set of its own
games in a bounded set
101
Slide 112
Slide 112 text
102
%% Create a leaderboard datatype.
L = declare({top_k, [2]}).
%% Update leaderboard.
update({set, Name, Score}, L).
Slide 113
Slide 113 text
103
%% Create a leaderboard datatype.
L = declare({top_k, [2]}).
%% Update leaderboard.
update({set, Name, Score}, L).
Slide 114
Slide 114 text
104
%% Create a leaderboard datatype.
L = declare({top_k, [2]}).
%% Update leaderboard.
update({set, Name, Score}, L).
Slide 115
Slide 115 text
What if we want to enhance
the behavior?
105
Slide 116
Slide 116 text
What if we want to enhance
the behavior?
105
Without the creation of a
new datatype
Slide 117
Slide 117 text
What can we build?
Per-User Leaderboard
106
EdgeCom 2016
Slide 118
Slide 118 text
Per-User Leaderboard
• Enhance existing design
Only the top score for each user at each device
107
Slide 119
Slide 119 text
Per-User Leaderboard
• Enhance existing design
Only the top score for each user at each device
• Minimize transmitted state
Prevent transmission of state that is not
necessary to perform the computation
107
Slide 120
Slide 120 text
Per-User Leaderboard
• Enhance existing design
Only the top score for each user at each device
• Minimize transmitted state
Prevent transmission of state that is not
necessary to perform the computation
• Compose data types
Build a per-user leaderboard through the
composition of existing types
107
Slide 121
Slide 121 text
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Lasp Operation
Input
User-Maintained CRDT
Output
Lasp-Maintained CRDT Fold
Client3
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client2
Scores
Local
Top-K
Fold
Global
Top-K
Fold
108
Slide 122
Slide 122 text
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
109
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client3
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client2
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Slide 123
Slide 123 text
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
110
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client3
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client2
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Slide 124
Slide 124 text
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
111
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client3
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client2
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Slide 125
Slide 125 text
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
112
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client3
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client2
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Slide 126
Slide 126 text
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
113
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client3
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client2
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Slide 127
Slide 127 text
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
114
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client3
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client2
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Slide 128
Slide 128 text
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Lasp Operation
Input
User-Maintained CRDT
Output
Lasp-Maintained CRDT Fold
Client3
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client2
Scores
Local
Top-K
Fold
Global
Top-K
Fold
115
Client1
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client3
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Client2
Scores
Local
Top-K
Fold
Global
Top-K
Fold
Slide 129
Slide 129 text
Per-User Leaderboard
• Dynamically scoped variables
Variable which take different values
depending on where it is executing
116
Slide 130
Slide 130 text
Per-User Leaderboard
• Dynamically scoped variables
Variable which take different values
depending on where it is executing
• Dynamically scoped fold operation
Perform a distributed “reduce”
operation that combines the state of a
dynamically scoped variables across
116
Slide 131
Slide 131 text
117
%% Create a global leaderboard.
G = declare({top_k, [10]}).
%% Create a local leaderboard.
L = declare_dynamic({top_k, [10]}).
%% Create a set of scores.
S = declare_dynamic(set).
%% Compute local top-k list.
fold(S, fun max_by_name/2, L).
%% Compute global top-k list.
fold_dynamic(L, fun max_by_name/2, G).
Slide 132
Slide 132 text
118
%% Create a global leaderboard.
G = declare({top_k, [10]}).
%% Create a local leaderboard.
L = declare_dynamic({top_k, [10]}).
%% Create a set of scores.
S = declare_dynamic(set).
%% Compute local top-k list.
fold(S, fun max_by_name/2, L).
%% Compute global top-k list.
fold_dynamic(L, fun max_by_name/2, G).
Slide 133
Slide 133 text
119
%% Create a global leaderboard.
G = declare({top_k, [10]}).
%% Create a local leaderboard.
L = declare_dynamic({top_k, [10]}).
%% Create a set of scores.
S = declare_dynamic(set).
%% Compute local top-k list.
fold(S, fun max_by_name/2, L).
%% Compute global top-k list.
fold_dynamic(L, fun max_by_name/2, G).
Slide 134
Slide 134 text
120
%% Create a global leaderboard.
G = declare({top_k, [10]}).
%% Create a local leaderboard.
L = declare_dynamic({top_k, [10]}).
%% Create a set of scores.
S = declare_dynamic(set).
%% Compute local top-k list.
fold(S, fun max_by_name/2, L).
%% Compute global top-k list.
fold_dynamic(L, fun max_by_name/2, G).
Slide 135
Slide 135 text
121
%% Create a global leaderboard.
G = declare({top_k, [10]}).
%% Create a local leaderboard.
L = declare_dynamic({top_k, [10]}).
%% Create a set of scores.
S = declare_dynamic(set).
%% Compute local top-k list.
fold(S, fun max_by_name/2, L).
%% Compute global top-k list.
fold_dynamic(L, fun max_by_name/2, G).
Slide 136
Slide 136 text
122
%% Create a global leaderboard.
G = declare({top_k, [10]}).
%% Create a local leaderboard.
L = declare_dynamic({top_k, [10]}).
%% Create a set of scores.
S = declare_dynamic(set).
%% Compute local top-k list.
fold(S, fun max_by_name/2, L).
%% Compute global top-k list.
fold_dynamic(L, fun max_by_name/2, G).
Slide 137
Slide 137 text
Let’s look at a larger
application
123
Slide 138
Slide 138 text
Let’s look at a larger
application
123
With visible
non-monotonicity
Slide 139
Slide 139 text
What can we build?
Advertisement Counter
124
Slide 140
Slide 140 text
Advertisement Counter
• Mobile game platform selling
advertisement space
Advertisements are paid according to a
minimum number of impressions
125
Slide 141
Slide 141 text
Advertisement Counter
• Mobile game platform selling
advertisement space
Advertisements are paid according to a
minimum number of impressions
• Clients will go offline
Clients have limited connectivity and the
system still needs to make progress
while clients are offline
125
Slide 142
Slide 142 text
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Lasp Operation
User-Maintained CRDT
Lasp-Maintained CRDT
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
126
Slide 143
Slide 143 text
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Riot Ads
Rovio
Ads
Product
Read
50,000
Remove
Increment
Union
127
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Lasp Operation
User-Maintained CRDT
Lasp-Maintained CRDT
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
Slide 144
Slide 144 text
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Rovio Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
1
Client
128
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Lasp Operation
User-Maintained CRDT
Lasp-Maintained CRDT
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
Slide 145
Slide 145 text
Ads
ovio Ad
ounter 1
ovio Ad
ounter 2
Riot Ad
ounter 1
Riot Ad
ounter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Rovio Ad
Counter
1
Ro
C
Rovio Ad
Counter
1
Ro
C
Rovio Ad
Counter
1
Ro
C
Rovio Ad
Counter
1
Ro
C
Client Side, Sing
129
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Lasp Operation
User-Maintained CRDT
Lasp-Maintained CRDT
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
Slide 146
Slide 146 text
Ads
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
move Read
Union
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
130
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Lasp Operation
User-Maintained CRDT
Lasp-Maintained CRDT
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
Slide 147
Slide 147 text
Ads
Contracts
Ads
Contracts
Ads
With
Contracts
Filter
Product
Read
Union
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
131
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Lasp Operation
User-Maintained CRDT
Lasp-Maintained CRDT
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
Slide 148
Slide 148 text
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
132
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Lasp Operation
User-Maintained CRDT
Lasp-Maintained CRDT
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
Slide 149
Slide 149 text
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Riot Ads
Rovio
Ads
Fil
Product
Read
50,000
Remove
Increment
Union
133
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Lasp Operation
User-Maintained CRDT
Lasp-Maintained CRDT
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
Slide 150
Slide 150 text
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
134
Ads
Rovio Ad
Counter 1
Rovio Ad
Counter 2
Riot Ad
Counter 1
Riot Ad
Counter 2
Contracts
Ads
Contracts
Ads
With
Contracts
Riot Ads
Rovio
Ads
Filter
Product
Read
50,000
Remove
Increment
Read
Union
Lasp Operation
User-Maintained CRDT
Lasp-Maintained CRDT
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Riot Ad
Counter
1
Client Side, Single Copy at Client
Slide 151
Slide 151 text
Advertisement Counter
• Completely monotonic
Disabling advertisements and contracts are all
modeled through monotonic state growth
135
Slide 152
Slide 152 text
Advertisement Counter
• Completely monotonic
Disabling advertisements and contracts are all
modeled through monotonic state growth
• Arbitrary distribution
Use of convergent data structures allows
computational graph to be arbitrarily distributed
135
Slide 153
Slide 153 text
Advertisement Counter
• Completely monotonic
Disabling advertisements and contracts are all
modeled through monotonic state growth
• Arbitrary distribution
Use of convergent data structures allows
computational graph to be arbitrarily distributed
• Divergence
Divergence is a factor of synchronization period
135
Slide 154
Slide 154 text
Client3
Lasp Operation
User-Maintained CRDT
Lasp-Maintained CRDT
Riot Ad
Counter
1
Client1
Client2
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Rovio Ad
Counter
1
Riot Ad
Counter
2
Riot Ad
Counter
1
Riot Ad
Counter
1
Ads
With
Contracts
Ads
With
Contracts
Ads
With
Contracts
Server
Ads
With
Contracts
Server
Computation!
136
Slide 155
Slide 155 text
Client3
Lasp Operation
User-Maintained CRDT
Lasp-Maintained CRDT
Riot Ad
Counter
1
Client1
Client2
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Rovio Ad
Counter
1
Riot Ad
Counter
2
Riot Ad
Counter
1
Riot Ad
Counter
1
Ads
With
Contracts
Ads
With
Contracts
Ads
With
Contracts
Server
Ads
With
Contracts
Server
Computation!
137
Slide 156
Slide 156 text
Client3
Lasp Operation
User-Maintained CRDT
Lasp-Maintained CRDT
Riot Ad
Counter
1
Client1
Client2
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Rovio Ad
Counter
1
Rovio Ad
Counter
2
Rovio Ad
Counter
1
Riot Ad
Counter
2
Riot Ad
Counter
1
Riot Ad
Counter
1
Ads
With
Contracts
Ads
With
Contracts
Ads
With
Contracts
Server
Ads
With
Contracts
Server
Computation!
138
Slide 157
Slide 157 text
Advertisement Counter
• “Servers” as peers to “clients”
Servers are peers to clients that perform additional
computation
139
Slide 158
Slide 158 text
Advertisement Counter
• “Servers” as peers to “clients”
Servers are peers to clients that perform additional
computation
• Any node can disable an advertisement under this
model given enough information
139
Slide 159
Slide 159 text
Advertisement Counter
• “Servers” as peers to “clients”
Servers are peers to clients that perform additional
computation
• Any node can disable an advertisement under this
model given enough information
• “Servers” as trusted nodes
Serve as a location for performing “exactly once” side-
effects
139
Slide 160
Slide 160 text
Advertisement Counter
• “Servers” as peers to “clients”
Servers are peers to clients that perform additional
computation
• Any node can disable an advertisement under this
model given enough information
• “Servers” as trusted nodes
Serve as a location for performing “exactly once” side-
effects
• Billing customers must be done at a central point by a
trusted node in the system
139
Slide 161
Slide 161 text
We’ve build up from zero
synchronization
140
Slide 162
Slide 162 text
We’ve build up from zero
synchronization
140
Instead of working to
remove synchronization
How restrictive is a programming
model where operations must be
associative, commutative, and
idempotent?
150
Slide 173
Slide 173 text
What’s new?
151
Slide 174
Slide 174 text
Since Erlang Factory 2015
• Erlang 18, rebar3, Common Test
Faster test suite, on Erlang 18 with rebar3
152
Slide 175
Slide 175 text
Since Erlang Factory 2015
• Erlang 18, rebar3, Common Test
Faster test suite, on Erlang 18 with rebar3
• No Riak Core
Plumtree distribution replaces the Riak Core
distribution system
152
Slide 176
Slide 176 text
Since Erlang Factory 2015
• Erlang 18, rebar3, Common Test
Faster test suite, on Erlang 18 with rebar3
• No Riak Core
Plumtree distribution replaces the Riak Core
distribution system
• No NIFs
Allows us to cross-compile to other platforms
and not worry about NIF scheduling
152
Slide 177
Slide 177 text
Semantics Improvements
• Delta-State Based Conflict-Free Replicated Data Types
Optimized state dissemination by shipping deltas
[Almeida et al., 2016]
153
Slide 178
Slide 178 text
Semantics Improvements
• Delta-State Based Conflict-Free Replicated Data Types
Optimized state dissemination by shipping deltas
[Almeida et al., 2016]
• Causal CRDTs
Optimized, garbage-free data structure support
[Almeida et al., 2016; Meiklejohn 2016, in review]
153
Slide 179
Slide 179 text
Semantics Improvements
• Delta-State Based Conflict-Free Replicated Data Types
Optimized state dissemination by shipping deltas
[Almeida et al., 2016]
• Causal CRDTs
Optimized, garbage-free data structure support
[Almeida et al., 2016; Meiklejohn 2016, in review]
• Fold
New semantics for a more expressive fold operation for
arbitrary computation over sets
[Meiklejohn 2016, in review]
153
Slide 180
Slide 180 text
Runtime Improvements
• Delta-State Based Anti-Entropy
Optimized AAE mechanism based on deltas
[Almeida et al., 2016]
154
Slide 181
Slide 181 text
Runtime Improvements
• Delta-State Based Anti-Entropy
Optimized AAE mechanism based on deltas
[Almeida et al., 2016]
• Mesos and Docker Enabled
Run large-scale Lasp clusters on Mesos with Marathon
[Meiklejohn and Yoo 2016, in review]
154
Slide 182
Slide 182 text
Runtime Improvements
• Delta-State Based Anti-Entropy
Optimized AAE mechanism based on deltas
[Almeida et al., 2016]
• Mesos and Docker Enabled
Run large-scale Lasp clusters on Mesos with Marathon
[Meiklejohn and Yoo 2016, in review]
• Loquat
Epidemic broadcast, partially replicated with
Decentralized Information Flow Control
[Meiklejohn 2016, in review]
154
Slide 183
Slide 183 text
Lasp Simulator
• Docker Containers
Docker containers for EPMD and Lasp runtime
system
155
Slide 184
Slide 184 text
Lasp Simulator
• Docker Containers
Docker containers for EPMD and Lasp runtime
system
• Service Discovery
Mechanisms for clustering Erlang nodes based on
either Marathon application definitions of Mesos-DNS
155
Slide 185
Slide 185 text
Lasp Simulator
• Docker Containers
Docker containers for EPMD and Lasp runtime
system
• Service Discovery
Mechanisms for clustering Erlang nodes based on
either Marathon application definitions of Mesos-DNS
• Instrumentation
Transmission instrumentation and divergence
measurement
155
Slide 186
Slide 186 text
Lasp Simulator
• Plumtree VM-to-VM
VM-to-VM communication performed using the Plumtree
epidemic broadcast protocol
156
Slide 187
Slide 187 text
Lasp Simulator
• Plumtree VM-to-VM
VM-to-VM communication performed using the Plumtree
epidemic broadcast protocol
• Clients-as-processes
Multiple clients per virtual machine, acting as mobile/IoT
devices that periodically simulate and can be partitioned
156
Slide 188
Slide 188 text
Lasp Simulator
• Plumtree VM-to-VM
VM-to-VM communication performed using the Plumtree
epidemic broadcast protocol
• Clients-as-processes
Multiple clients per virtual machine, acting as mobile/IoT
devices that periodically simulate and can be partitioned
• “Design in the small, run in the large”
Runtime configuration change for client-to-VM ratio,
allows for single laptop design of multi-machine
evaluations of the programming model
156
Google Summer of Code
(and my Ph.D.!)
• Partial Evaluation
Optimize execution based on analysis and annotations
where we can determine local-vs-remote usage
161
Slide 194
Slide 194 text
Google Summer of Code
(and my Ph.D.!)
• Partial Evaluation
Optimize execution based on analysis and annotations
where we can determine local-vs-remote usage
• Optimizations
General optimizations of the Erlang implementation of
Lasp to improve performance of the runtime system
161
Slide 195
Slide 195 text
Google Summer of Code
(and my Ph.D.!)
• Partial Evaluation
Optimize execution based on analysis and annotations
where we can determine local-vs-remote usage
• Optimizations
General optimizations of the Erlang implementation of
Lasp to improve performance of the runtime system
• Elixir / Macros
Provide a nicer way for working with Lasp, outside of
the current syntax.
161
Slide 196
Slide 196 text
How do I learn
more?
162
Slide 197
Slide 197 text
Publications
• “Lasp: A Language for Distributed, Coordination-Free Programming”
ACM SIGPLAN PPDP 2015
• “Selective Hearing: An Approach to Distributed, Eventually Consistent Edge
Computation”
IEEE W-PSDS 2015
• “The Implementation and Use of a Generic Dataflow Behaviour in Erlang”
ACM SIGPLAN Erlang Workshop ’15
• “Lasp: A Language for Distributed, Eventually Consistent Computations with CRDTs"
PaPoC 2015
• “Declarative, Sliding Window Aggregations for Computations at the Edge"
IEEE EdgeCom 2016
163
Slide 198
Slide 198 text
164
Christopher Meiklejohn
@cmeik
http://www.lasp-lang.org
Thanks!