Declarative, Secure, Convergent Edge Computation

GOTO Chicago 2016, ACM Applicative 2016
Université catholique de Louvain
Christopher Meiklejohn

Transcript

1. Declarative, Secure, Convergent Edge Computation Christopher Meiklejohn Université catholique de Louvain, Belgium 1
2. Internet of Things 2
3. Internet of Things 2 but, more generally…
4. Edge Computation • Logical extremes
 Pushing both computation and data to the logical extremes of the network 3
5. Edge Computation • Logical extremes
 Pushing both computation and data to the logical extremes of the network • Arbitrary computation
 Support arbitrary computations regardless of location of data in the network 3
6. Edge Computation • Logical extremes
 Pushing both computation and data to the logical extremes of the network • Arbitrary computation
 Support arbitrary computations regardless of location of data in the network • Self-organizing, resilient
 Directed diffusion, Cornell circa-1990; self-organizing systems that coordinate to complete computations 3
7. Example Application Hospital Refrigerators 4
8. Hospital Refrigerators Typical Topology 5
10. Internet 7
11. Internet HDFS 8
12. Internet HDFS Hadoop Client Client Internet 9
13. Internet HDFS Spark Client Client Internet 10
14. Hospital Refrigerators Ideal Execution 11
15. Internet HDFS Spark 12
16. Internet HDFS Spark 13
17. Internet HDFS Spark 14
18. Internet HDFS Spark 15
19. Internet HDFS Spark 16
20. Client Internet HDFS Spark 17
21. Problem Connectivity 18
22. Internet HDFS Spark 19
23. Internet HDFS Spark 20
24. Internet HDFS Spark 21
25. Internet HDFS Spark 22
26. Internet HDFS Spark 23
27. Internet HDFS Spark 24
28. Internet HDFS Spark 25
29. Solution Local Decisions 26
30. Internet HDFS Spark 27
31. Internet HDFS Spark 28
32. Internet HDFS Spark 29
33. Internet HDFS Spark 30
34. Internet HDFS Spark 31
35. Local Decisions • Not new for backup (80s-90s)
 Backup communication mechanisms for critical systems; POTS backup for ISDN, etc. 32
36. Local Decisions • Not new for backup (80s-90s)
 Backup communication mechanisms for critical systems; POTS backup for ISDN, etc. • Not new for storage (90s-00s)
 EMC’s “phone home” via POTS when disks failed in NAS devices to signal for replacement unit 32
37. Solution Transitive Dissemination 33
38. Internet HDFS Spark 34
39. Internet HDFS Spark 35
40. Internet HDFS Spark 36
41. Internet HDFS Spark 37
42. Internet HDFS Spark 38
43. Internet HDFS Spark Client 39
44. Problem State Transmission 40
45. Internet 41
46. Internet 42
47. Internet 43
48. Internet 44
49. Solution Aggregate Dissemination 45
50. Internet 46
51. Internet 47
52. Internet 48
53. Internet HDFS 49
54. Internet = = HDFS 50
55. Internet ? = = 51
56. Internet ? = = 52
57. Local Computation • Reduce state transmission
 Perform some local computation to reduce transmitted state on the wire 53
58. Local Computation • Reduce state transmission
 Perform some local computation to reduce transmitted state on the wire • Make local decisions
 Make decisions based on results of local computation 53
59. Databases Consistency Models 54
60. Databases Strong Consistency 55
61. R1 C1 C2 56
62. R1 C1 C2 57
63. R1 C1 C2 Read 58
64. R1 C1 C2 Read 59
65. R1 C1 C2 60
66. R1 C1 C2 CAS 61
67. R1 C1 C2 62
68. R1 C1 C2 CAS 63
69. I won’t diagram the Paxos protocol 64
70. R2 C1 C2 Value 2 Value 1 Value 2 R1 R3 Paxos 65
71. Databases Eventual Consistency 66
72. R1 R2 R3 C1 C2 67
73. R1 R2 R3 C1 C2 68
74. R1 R2 R3 C1 C2 Read 69
75. R1 R2 R3 C1 C2 Read 70
76. R1 R2 R3 C1 C2 Write 71
77. R1 R2 R3 C1 C2 Write 72
78. R1 R2 R3 C1 C2 Write C1 73
79. R1 R2 R3 C1 C2 Read 74
80. R1 R2 R3 C1 C2 Write 75
81. Eventual Consistency As The Model 76
82. Clients Own Their Data 77
86. Computations Mergeability & Provenance 81
87. A A B C 82
88. A A B C D’ D’’ F F 83
89. A A B C D’ D’’ D’ D’’ D   F F 84
90. D’’ A A B C D’ D’ D’’ D   D D Merge F F 85
91. Example Application Preliminary Results 86
92. Preliminary Results • Conflict-Free Replicated Data Types
 Distributed data structures designed for convergence 
 [Shapiro et al., 2011] 87
93. Preliminary Results • Conflict-Free Replicated Data Types
 Distributed data structures designed for convergence 
 [Shapiro et al., 2011] • Lattice Processing
 Make decisions based on results of local computation
 [Meiklejohn & Van Roy, 2015] 87
94. Preliminary Results • Conflict-Free Replicated Data Types
 Distributed data structures designed for convergence 
 [Shapiro et al., 2011] • Lattice Processing
 Make decisions based on results of local computation
 [Meiklejohn & Van Roy, 2015] • Selective Hearing
 Scalable, epidemic broadcast based runtime system
 [Meiklejohn & Van Roy, 2015/2016] 87
95. Conflict-Free 
 Replicated Data Types • Collection of types
 Sets, counters, registers, flags, maps 88
96. Conflict-Free 
 Replicated Data Types • Collection of types
 Sets, counters, registers, flags, maps • Strong Eventual Consistency (SEC)
 Objects that receive the same updates, regardless of order, will reach equivalent state 88
97. RA RB RC
98. RA RB RC {1} (1, {a}, {}) add(1)
99. RA RB RC {1} (1, {a}, {}) add(1) {1} (1, {c}, {}) add(1)
100. RA RB RC {1} (1, {a}, {}) add(1) {1} (1, {c}, {}) add(1) {} (1, {c}, {c}) remove(1)
101. RA RB RC {1} (1, {a}, {}) add(1) {1} (1, {c}, {}) add(1) {} (1, {c}, {c}) remove(1) {1} {1} {1} (1, {a, c}, {c}) (1, {a, c}, {c}) (1, {a, c}, {c})
102. Lattice Processing • Distributed dataflow
 Declarative, functional programming model 94
103. Lattice Processing • Distributed dataflow
 Declarative, functional programming model • Convergent data structures
 Data abstraction is the CRDT 94
104. Lattice Processing • Distributed dataflow
 Declarative, functional programming model • Convergent data structures
 Data abstraction is the CRDT • Enables composition
 Composition preserves SEC 94
105. 95 %% Create initial set. S1 = declare(set), %% Add elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
106. 96 %% Create initial set. S1 = declare(set), %% Add elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
107. 97 %% Create initial set. S1 = declare(set), %% Add elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
108. 98 %% Create initial set. S1 = declare(set), %% Add elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
109. 99 %% Create initial set. S1 = declare(set), %% Add elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
110. Delta-based Dissemination • Delta-state based CRDTs
 Reduces state transmission for clients 100
111. Delta-based Dissemination • Delta-state based CRDTs
 Reduces state transmission for clients • Operate locally
 Objects are mutated locally; deltas buffered locally and periodically gossiped 100
112. Delta-based Dissemination • Delta-state based CRDTs
 Reduces state transmission for clients • Operate locally
 Objects are mutated locally; deltas buffered locally and periodically gossiped • Only fixed number of clients
 Clients resort to full state synchronization when they’ve been partitioned too long 100
121. Selective Hearing • Epidemic broadcast protocol
 Runtime system for application state & scope 109
122. Selective Hearing • Epidemic broadcast protocol
 Runtime system for application state & scope • Peer-to-peer dissemination
 Pairwise synchronization between peers without a central coordinator 109
123. Selective Hearing • Epidemic broadcast protocol
 Runtime system for application state & scope • Peer-to-peer dissemination
 Pairwise synchronization between peers without a central coordinator • No ordering guarantees on messages
 Programming model can tolerate message reordering and duplication 109
124. Membership Overlay
125. Membership Overlay Broadcast Overlay
126. Membership Overlay Broadcast Overlay
127. Membership Overlay Broadcast Overlay Mobile Phone
128. Membership Overlay Broadcast Overlay Mobile Phone Distributed Hash Table
129. Membership Overlay Broadcast Overlay Mobile Phone Distributed Hash Table Lasp Execution
130. What can we build? Advertisement Counter 116
131. Advertisement Counter • Mobile game platform selling advertisement space
 Advertisements are paid according to a minimum number of impressions 117
132. Advertisement Counter • Mobile game platform selling advertisement space
 Advertisements are paid according to a minimum number of impressions • Clients will go offline
 Clients have limited connectivity and the system still needs to make progress while clients are offline 117
133. Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 118
134. Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Riot Ads Rovio Ads Product Read 50,000 Remove Increment Union 119 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
135. Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 1 Client 120 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
136. Ads ovio Ad ounter 1 ovio Ad ounter 2 Riot Ad ounter 1 Riot Ad ounter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Client Side, Sing 121 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
137. Ads Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product move Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 122 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
138. Ads Contracts Ads Contracts Ads With Contracts Filter Product Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 123 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
139. Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 124 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
140. Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Riot Ads Rovio Ads Fil Product Read 50,000 Remove Increment Union 125 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
141. Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 126 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
142. Advertisement Counter • Completely monotonic
 Disabling advertisements and contracts are all modeled through monotonic state growth 127
143. Advertisement Counter • Completely monotonic
 Disabling advertisements and contracts are all modeled through monotonic state growth • Arbitrary distribution
 Use of convergent data structures allows computational graph to be arbitrarily distributed 127
144. Advertisement Counter • Completely monotonic
 Disabling advertisements and contracts are all modeled through monotonic state growth • Arbitrary distribution
 Use of convergent data structures allows computational graph to be arbitrarily distributed • Divergence
 Divergence is a factor of synchronization period 127
145. Client3 Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Riot Ad Counter 1 Client1 Client2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Riot Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 1 Ads With Contracts Ads With Contracts Ads With Contracts Server Ads With Contracts Server Computation! 128
146. Client3 Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Riot Ad Counter 1 Client1 Client2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Riot Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 1 Ads With Contracts Ads With Contracts Ads With Contracts Server Ads With Contracts Server Computation! 129
147. Client3 Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Riot Ad Counter 1 Client1 Client2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Riot Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 1 Ads With Contracts Ads With Contracts Ads With Contracts Server Ads With Contracts Server Computation! 130
148. Advertisement Counter • “Servers” as peers to “clients”
 Servers are peers to clients that perform additional computation 131
149. Advertisement Counter • “Servers” as peers to “clients”
 Servers are peers to clients that perform additional computation • Any node can disable an advertisement under this model given enough information 131
150. Advertisement Counter • “Servers” as peers to “clients”
 Servers are peers to clients that perform additional computation • Any node can disable an advertisement under this model given enough information • “Servers” as trusted nodes
 Serve as a location for performing “exactly once” side- effects 131
151. Advertisement Counter • “Servers” as peers to “clients”
 Servers are peers to clients that perform additional computation • Any node can disable an advertisement under this model given enough information • “Servers” as trusted nodes
 Serve as a location for performing “exactly once” side- effects • Billing customers must be done at a central point by a trusted node in the system 131
152. We’ve built up from zero synchronization 132
153. We’ve built up from zero synchronization 132 Instead of working to remove synchronization
154. Challenges Looking Ahead 133
155. Causality State Explosion 134
156. Set Counter
157. Set Counter {1} (1, {c}, {}) 1 {(c, 1)}, {}
158. Set Counter {1} (1, {c}, {}) 1 {(c, 1)}, {} {} (1, {c}, {c}) {(c, 1)}, {(c, 1)} 0
159. Set Counter {1} (1, {c}, {}) 1 {(c, 1)}, {} {} (1, {c}, {c}) {(c, 1)}, {(c, 1)} 0 {1} (1, {a, c}, {c}) {1} 1 1 {(c, 1), (a, 1}}, {(c, 1)}
160. Security Computing at the Edge 139
162. Computations Expressiveness 141
163. How restrictive is a programming model where operations must be associative, commutative, and idempotent? 142
164. How do I learn more? 143
165. Publications • “Lasp: A Language for Distributed, Coordination-Free Programming” 
 ACM SIGPLAN PPDP 2015 • “Selective Hearing: An Approach to Distributed, Eventually Consistent Edge Computation”
 IEEE W-PSDS 2015 • “The Implementation and Use of a Generic Dataflow Behaviour in Erlang”
 ACM SIGPLAN Erlang Workshop ’15 • “Lasp: A Language for Distributed, Eventually Consistent Computations with CRDTs"
 PaPoC 2015 • “Declarative, Sliding Window Aggregations for Computations at the Edge"
 IEEE EdgeCom 2016 144
166. Three independently successful techniques. 145
167. Can we combine them into a cohesive programming environment for distributed programming? 146
168. 147 Christopher Meiklejohn @cmeik http://www.lasp-lang.org Thanks!