Share
Embed
Selective Hearing: An Approach to Distributed, Eventually Consistent Edge Computation
W-PSDS 2015, with SRDS 2015
Christopher Meiklejohn
September 28, 2015
Transcript
94 slides
-
An Approach to Distributed, Eventually Consistent Edge Computation Selective Hearing
Christopher Meiklejohn, Machine Zone, Inc. Peter Van Roy, Université catholique de Louvain W-PSDS 2015, September 28, 2015 1 -
Edge Computation • Logical extremes of the network Applications, data,
and computation 3 -
Edge Computation • Logical extremes of the network Applications, data,
and computation • Especially important where synchronization is hard 3 -
Edge Computation • Logical extremes of the network Applications, data,
and computation • Especially important where synchronization is hard • “Internet of Things” Low power, limited memory and connectivity 3 -
Edge Computation • Logical extremes of the network Applications, data,
and computation • Especially important where synchronization is hard • “Internet of Things” Low power, limited memory and connectivity • Mobile Applications Offline operation with replicated, shared state 3 -
Edge Computation • Logical extremes of the network Applications, data,
and computation • Especially important where synchronization is hard • “Internet of Things” Low power, limited memory and connectivity • Mobile Applications Offline operation with replicated, shared state • How should we manage state? 3 -
Selective Hearing • Epidemic broadcast based runtime system Provide a
runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution 4 -
Selective Hearing • Epidemic broadcast based runtime system Provide a
runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) 4 -
Selective Hearing • Epidemic broadcast based runtime system Provide a
runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) • Epidemic broadcast mechanisms provide weak ordering but are resilient and efficient 4 -
Selective Hearing • Epidemic broadcast based runtime system Provide a
runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) • Epidemic broadcast mechanisms provide weak ordering but are resilient and efficient • Lasp’s programming model is tolerant to message re-ordering, disconnections, and node failures 4 -
Selective Hearing • Epidemic broadcast based runtime system Provide a
runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) • Epidemic broadcast mechanisms provide weak ordering but are resilient and efficient • Lasp’s programming model is tolerant to message re-ordering, disconnections, and node failures • “Selective Receive” Nodes selectively receive and process messages based on interest. 4 -
Lattice Processing (Lasp) • Distributed, deterministic dataflow Distributed, deterministic dataflow
programming model for “eventually consistent” computations 6 -
Lattice Processing (Lasp) • Distributed, deterministic dataflow Distributed, deterministic dataflow
programming model for “eventually consistent” computations • Convergent data structures Primary data abstraction is the CRDT 6 -
Lattice Processing (Lasp) • Distributed, deterministic dataflow Distributed, deterministic dataflow
programming model for “eventually consistent” computations • Convergent data structures Primary data abstraction is the CRDT • Enables composition Provides functional composition of CRDTs that preserves the SEC property 6 -
7 %% Create initial set. S1 = declare(set), %% Add
elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2). -
8 %% Create initial set. S1 = declare(set), %% Add
elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2). -
9 %% Create initial set. S1 = declare(set), %% Add
elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2). -
10 %% Create initial set. S1 = declare(set), %% Add
elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2). -
11 %% Create initial set. S1 = declare(set), %% Add
elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2). -
Lattice Processing (Lasp) • Functional and set-theoretic operations on sets
Product, intersection, union, filter, map, fold 12 -
Lattice Processing (Lasp) • Functional and set-theoretic operations on sets
Product, intersection, union, filter, map, fold • Metadata computation Performs transformation on the internal metadata of CRDTs allowing creation of “composed” CRDTs 12 -
Lasp Processes • Replicas as monotonic streams Each replica of
a CRDT produces a monotonic stream of states 13 -
Lasp Processes • Replicas as monotonic streams Each replica of
a CRDT produces a monotonic stream of states • Monotonic processes Read from one or more input replica streams and produce a single output replica stream 13 -
Lasp Processes • Replicas as monotonic streams Each replica of
a CRDT produces a monotonic stream of states • Monotonic processes Read from one or more input replica streams and produce a single output replica stream • Inflationary reads Read operation ensures that we only read inflationary updates to replicas 13 -
RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1,
{a}, {})) 18 -
RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1,
{a}, {})) (2, {a}, {}) 19 -
RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1,
{a}, {})) (2, {a}, {}) strict_read((1, {a}, {}) (1, {a}, {}) 20 -
RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1,
{a}, {})) (2, {a}, {}) strict_read((1, {a}, {}) (1, {a}, {}) (1, {a}, {a}) F((1, {a}, {a})) (2, {a}, {a}) strict_read((1, {a}, {a}) (1, {a}, {a}) 21 -
RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) (1,
{a}, {}) (1, {a}, {a}) F((1, {a}, {a})) (2, {a}, {a}) strict_read((1, {a}, {a}) (1, {a}, {a}) 22 -
Advertisement Counter • Mobile game platform selling advertisement space Advertisements
are paid according to a minimum number of impressions 24 -
Advertisement Counter • Mobile game platform selling advertisement space Advertisements
are paid according to a minimum number of impressions • Clients will go offline Clients have limited connectivity and the system still needs to make progress while clients are offline 24 -
Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot
Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 25 -
Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot
Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Riot Ads Rovio Ads Product Read 50,000 Remove Increment Union 26 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client -
Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot
Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 1 Client 27 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client -
Ads ovio Ad ounter 1 ovio Ad ounter 2 Riot
Ad ounter 1 Riot Ad ounter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Client Side, Sing 28 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client -
Ads Contracts Ads Contracts Ads With Contracts Riot Ads Rovio
Ads Filter Product move Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 29 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client -
Ads Contracts Ads Contracts Ads With Contracts Filter Product Read
Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 30 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client -
Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot
Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 31 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client -
Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot
Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Riot Ads Rovio Ads Fil Product Read 50,000 Remove Increment Union 32 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client -
Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot
Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 33 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client -
Advertisement Counter • Completely monotonic Disabling advertisements and contracts are
all modeled through monotonic state growth 34 -
Advertisement Counter • Completely monotonic Disabling advertisements and contracts are
all modeled through monotonic state growth • Divergence Divergence is a factor of synchronization period 34 -
Advertisement Counter • Completely monotonic Disabling advertisements and contracts are
all modeled through monotonic state growth • Divergence Divergence is a factor of synchronization period • Arbitrary distribution Use of convergent data structures allows computational graph to be arbitrarily distributed 34 -
Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot
Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 36 Server/Client Distribution -
What if we want to have multiple replication schemes used
in a single dataflow graph? 37 -
Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot
Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 38 Geo-Replicated Distribution -
Distribution Abstraction • Previous model assumes Dynamo-style Instead, allow nodes
in the graph to represent objects that have their own replication scheme 39 -
Distribution Abstraction • Previous model assumes Dynamo-style Instead, allow nodes
in the graph to represent objects that have their own replication scheme • Each node in the dataflow graph: 39 -
Distribution Abstraction • Previous model assumes Dynamo-style Instead, allow nodes
in the graph to represent objects that have their own replication scheme • Each node in the dataflow graph: • Replicated state machine 39 -
Distribution Abstraction • Previous model assumes Dynamo-style Instead, allow nodes
in the graph to represent objects that have their own replication scheme • Each node in the dataflow graph: • Replicated state machine • Dynamo-style “quorum-intersection” 39 -
Distribution Abstraction • Previous model assumes Dynamo-style Instead, allow nodes
in the graph to represent objects that have their own replication scheme • Each node in the dataflow graph: • Replicated state machine • Dynamo-style “quorum-intersection” • Single value 39 -
Distribution Abstraction • Previous model assumes Dynamo-style Instead, allow nodes
in the graph to represent objects that have their own replication scheme • Each node in the dataflow graph: • Replicated state machine • Dynamo-style “quorum-intersection” • Single value • Epidemic broadcast for data propagation Facilitates distribution of “program state” across nodes in the dataflow graph 39 -
Two Layer Approach • Lasp • Variables and values are
stored on all nodes 41 -
Two Layer Approach • Lasp • Variables and values are
stored on all nodes • Read operations: 41 -
Two Layer Approach • Lasp • Variables and values are
stored on all nodes • Read operations: • Declare “interest” in a value 41 -
Two Layer Approach • Lasp • Variables and values are
stored on all nodes • Read operations: • Declare “interest” in a value • Store a continuation to invoke Predicate function on a lattice position 41 -
Two Layer Approach • Lasp • Variables and values are
stored on all nodes • Read operations: • Declare “interest” in a value • Store a continuation to invoke Predicate function on a lattice position • Bind operations (update operations, as well): 41 -
Two Layer Approach • Lasp • Variables and values are
stored on all nodes • Read operations: • Declare “interest” in a value • Store a continuation to invoke Predicate function on a lattice position • Bind operations (update operations, as well): • Bind a value on each “interested node” This performs a merge with the nodes current value 41 -
Two Layer Approach • Gossip • Weak ordering of broadcast
messages Variable state and metadata for declare and bind operations 42 -
Two Layer Approach • Gossip • Weak ordering of broadcast
messages Variable state and metadata for declare and bind operations • Bind operations: 42 -
Two Layer Approach • Gossip • Weak ordering of broadcast
messages Variable state and metadata for declare and bind operations • Bind operations: • Uniquely identify messages Pair of global message identifier and version vector 42 -
Two Layer Approach • Gossip • Weak ordering of broadcast
messages Variable state and metadata for declare and bind operations • Bind operations: • Uniquely identify messages Pair of global message identifier and version vector • Incremented by coordinating replica When receiving incoming vector, merge with current vector and advance 42 -
Two Layer Approach • Gossip • Weak ordering of broadcast
messages Variable state and metadata for declare and bind operations • Bind operations: • Uniquely identify messages Pair of global message identifier and version vector • Incremented by coordinating replica When receiving incoming vector, merge with current vector and advance • Monotonic state increases per-replica Ignore messages that have been subsumed by later messages 42 -
Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot
Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Riot Ads Rovio Ads Fil Product Read 50,000 Remove Increment Union 43 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client -
44 server({#ad{counter=Counter}=Ad, _}, Ads) -> strict_read(Counter, 50000), update(Ads, {remove, Ad},
Ad). Ad Counter Process • Blocking read of the counter for value 50000 -
44 server({#ad{counter=Counter}=Ad, _}, Ads) -> strict_read(Counter, 50000), update(Ads, {remove, Ad},
Ad). Ad Counter Process • Blocking read of the counter for value 50000 • Records interest in variable Counter -
44 server({#ad{counter=Counter}=Ad, _}, Ads) -> strict_read(Counter, 50000), update(Ads, {remove, Ad},
Ad). Ad Counter Process • Blocking read of the counter for value 50000 • Records interest in variable Counter • Registers a continuation that will invoke when Counter >= 50000 Continuation -
44 server({#ad{counter=Counter}=Ad, _}, Ads) -> strict_read(Counter, 50000), update(Ads, {remove, Ad},
Ad). Ad Counter Process • Blocking read of the counter for value 50000 • Records interest in variable Counter • Registers a continuation that will invoke when Counter >= 50000 • Interest in variable Counter removed Continuation -
45 %% Create initial set. S1 = declare(set), %% Add
elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2). -
46 Map Process map(S1, Function, S2) -> map(S1, Function, Bottom(S1),
S2). map(S1, Function, Last, S2) -> Continuation = fun(Current) -> Result = compute(Function, Current), bind(S2, Result), map(S1, P, Current, S2) end, read(S1, fun(Current) -> Last < Current, Continuation). -
47 Map Process map(S1, Function, S2) -> map(S1, Function, Bottom(S1),
S2). map(S1, Function, Last, S2) -> Continuation = fun(Current) -> Result = compute(Function, Current), bind(S2, Result), map(S1, P, Current, S2) end, read(S1, fun(Current) -> Last < Current, Continuation). Invoke Map -
48 Map Process map(S1, Function, S2) -> map(S1, Function, Bottom(S1),
S2). map(S1, Function, Last, S2) -> Continuation = fun(Current) -> Result = compute(Function, Current), bind(S2, Result), map(S1, P, Current, S2) end, read(S1, fun(Current) -> Last < Current, Continuation). Generate continuation -
49 Map Process map(S1, Function, S2) -> map(S1, Function, Bottom(S1),
S2). map(S1, Function, Last, S2) -> Continuation = fun(Current) -> Result = compute(Function, Current), bind(S2, Result), map(S1, P, Current, S2) end, read(S1, fun(Current) -> Last < Current, Continuation). Invoke continuation on change -
Future Work • Quantitative evaluation Evaluation and optimization of the
prototype implementation in Erlang with Lasp and Plumtree 51 -
Future Work • Quantitative evaluation Evaluation and optimization of the
prototype implementation in Erlang with Lasp and Plumtree • Extensions to increase locality Local variables that do not get distributed 51 -
Future Work • Quantitative evaluation Evaluation and optimization of the
prototype implementation in Erlang with Lasp and Plumtree • Extensions to increase locality Local variables that do not get distributed • Partial evaluation Can we derive optimized distribution models given dataflow execution graphs? 51 -
Conclusion • Successful pairing Pairing of a weak delivery model
with a model that does not rely on ordering guarantees 52 -
Conclusion • Successful pairing Pairing of a weak delivery model
with a model that does not rely on ordering guarantees • Distributed epidemic-based runtime First implementation of a epidemic broadcast based runtime in a general programming model 52 -
SyncFree is a European research project taking place for 3
years, staring October 2013, and is funded by the European Union, grant agreement n° 609551. 53