Selective Hearing: An Approach to Distributed, Eventually Consistent Edge Computation

Selective Hearing: An Approach to Distributed, Eventually Consistent Edge Computation

W-PSDS 2015, with SRDS 2015

3e09fee7b359be847ed5fa48f524a3d3?s=128

Christopher Meiklejohn

September 28, 2015
Tweet

Transcript

  1. 1.

    An Approach to Distributed, Eventually Consistent Edge Computation Selective Hearing

    Christopher Meiklejohn, Machine Zone, Inc. Peter Van Roy, Université catholique de Louvain W-PSDS 2015, September 28, 2015 1
  2. 4.

    Edge Computation • Logical extremes of the network
 Applications, data,

    and computation • Especially important where synchronization is hard 3
  3. 5.

    Edge Computation • Logical extremes of the network
 Applications, data,

    and computation • Especially important where synchronization is hard • “Internet of Things”
 Low power, limited memory and connectivity 3
  4. 6.

    Edge Computation • Logical extremes of the network
 Applications, data,

    and computation • Especially important where synchronization is hard • “Internet of Things”
 Low power, limited memory and connectivity • Mobile Applications
 Offline operation with replicated, shared state 3
  5. 7.

    Edge Computation • Logical extremes of the network
 Applications, data,

    and computation • Especially important where synchronization is hard • “Internet of Things”
 Low power, limited memory and connectivity • Mobile Applications
 Offline operation with replicated, shared state • How should we manage state? 3
  6. 8.

    Selective Hearing • Epidemic broadcast based runtime system
 Provide a

    runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution 4
  7. 9.

    Selective Hearing • Epidemic broadcast based runtime system
 Provide a

    runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) 4
  8. 10.

    Selective Hearing • Epidemic broadcast based runtime system
 Provide a

    runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) • Epidemic broadcast mechanisms provide weak ordering but are resilient and efficient 4
  9. 11.

    Selective Hearing • Epidemic broadcast based runtime system
 Provide a

    runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) • Epidemic broadcast mechanisms provide weak ordering but are resilient and efficient • Lasp’s programming model is tolerant to message re-ordering, disconnections, and node failures 4
  10. 12.

    Selective Hearing • Epidemic broadcast based runtime system
 Provide a

    runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) • Epidemic broadcast mechanisms provide weak ordering but are resilient and efficient • Lasp’s programming model is tolerant to message re-ordering, disconnections, and node failures • “Selective Receive”
 Nodes selectively receive and process messages based on interest. 4
  11. 14.
  12. 15.

    Lattice Processing (Lasp) • Distributed, deterministic dataflow
 Distributed, deterministic dataflow

    programming model for “eventually consistent” computations • Convergent data structures
 Primary data abstraction is the CRDT 6
  13. 16.

    Lattice Processing (Lasp) • Distributed, deterministic dataflow
 Distributed, deterministic dataflow

    programming model for “eventually consistent” computations • Convergent data structures
 Primary data abstraction is the CRDT • Enables composition
 Provides functional composition of CRDTs that preserves the SEC property 6
  14. 17.

    7 %% Create initial set. S1 = declare(set), %% Add

    elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
  15. 18.

    8 %% Create initial set. S1 = declare(set), %% Add

    elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
  16. 19.

    9 %% Create initial set. S1 = declare(set), %% Add

    elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
  17. 20.

    10 %% Create initial set. S1 = declare(set), %% Add

    elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
  18. 21.

    11 %% Create initial set. S1 = declare(set), %% Add

    elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
  19. 22.

    Lattice Processing (Lasp) • Functional and set-theoretic operations on sets


    Product, intersection, union, filter, map, fold 12
  20. 23.

    Lattice Processing (Lasp) • Functional and set-theoretic operations on sets


    Product, intersection, union, filter, map, fold • Metadata computation
 Performs transformation on the internal metadata of CRDTs allowing creation of “composed” CRDTs 12
  21. 24.

    Lasp Processes • Replicas as monotonic streams
 Each replica of

    a CRDT produces a monotonic stream of states 13
  22. 25.

    Lasp Processes • Replicas as monotonic streams
 Each replica of

    a CRDT produces a monotonic stream of states • Monotonic processes
 Read from one or more input replica streams and produce a single output replica stream 13
  23. 26.

    Lasp Processes • Replicas as monotonic streams
 Each replica of

    a CRDT produces a monotonic stream of states • Monotonic processes
 Read from one or more input replica streams and produce a single output replica stream • Inflationary reads
 Read operation ensures that we only read inflationary updates to replicas 13
  24. 33.

    RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1,

    {a}, {})) (2, {a}, {}) strict_read((1, {a}, {}) (1, {a}, {}) 20
  25. 34.

    RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1,

    {a}, {})) (2, {a}, {}) strict_read((1, {a}, {}) (1, {a}, {}) (1, {a}, {a}) F((1, {a}, {a})) (2, {a}, {a}) strict_read((1, {a}, {a}) (1, {a}, {a}) 21
  26. 35.

    RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) (1,

    {a}, {}) (1, {a}, {a}) F((1, {a}, {a})) (2, {a}, {a}) strict_read((1, {a}, {a}) (1, {a}, {a}) 22
  27. 37.
  28. 38.

    Advertisement Counter • Mobile game platform selling advertisement space
 Advertisements

    are paid according to a minimum number of impressions • Clients will go offline
 Clients have limited connectivity and the system still needs to make progress while clients are offline 24
  29. 39.

    Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 25
  30. 40.

    Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Riot Ads Rovio Ads Product Read 50,000 Remove Increment Union 26 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  31. 41.

    Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 1 Client 27 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  32. 42.

    Ads ovio Ad ounter 1 ovio Ad ounter 2 Riot

    Ad ounter 1 Riot Ad ounter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Client Side, Sing 28 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  33. 43.

    Ads Contracts Ads Contracts Ads With Contracts Riot Ads Rovio

    Ads Filter Product move Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 29 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  34. 44.

    Ads Contracts Ads Contracts Ads With Contracts Filter Product Read

    Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 30 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  35. 45.

    Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 31 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  36. 46.

    Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Riot Ads Rovio Ads Fil Product Read 50,000 Remove Increment Union 32 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  37. 47.

    Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 33 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  38. 49.

    Advertisement Counter • Completely monotonic
 Disabling advertisements and contracts are

    all modeled through monotonic state growth • Divergence
 Divergence is a factor of synchronization period 34
  39. 50.

    Advertisement Counter • Completely monotonic
 Disabling advertisements and contracts are

    all modeled through monotonic state growth • Divergence
 Divergence is a factor of synchronization period • Arbitrary distribution
 Use of convergent data structures allows computational graph to be arbitrarily distributed 34
  40. 53.

    Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 36 Server/Client Distribution
  41. 54.
  42. 55.

    Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 38 Geo-Replicated Distribution
  43. 56.

    Distribution Abstraction • Previous model assumes Dynamo-style
 Instead, allow nodes

    in the graph to represent objects that have their own replication scheme 39
  44. 57.

    Distribution Abstraction • Previous model assumes Dynamo-style
 Instead, allow nodes

    in the graph to represent objects that have their own replication scheme • Each node in the dataflow graph: 39
  45. 58.

    Distribution Abstraction • Previous model assumes Dynamo-style
 Instead, allow nodes

    in the graph to represent objects that have their own replication scheme • Each node in the dataflow graph: • Replicated state machine 39
  46. 59.

    Distribution Abstraction • Previous model assumes Dynamo-style
 Instead, allow nodes

    in the graph to represent objects that have their own replication scheme • Each node in the dataflow graph: • Replicated state machine • Dynamo-style “quorum-intersection” 39
  47. 60.

    Distribution Abstraction • Previous model assumes Dynamo-style
 Instead, allow nodes

    in the graph to represent objects that have their own replication scheme • Each node in the dataflow graph: • Replicated state machine • Dynamo-style “quorum-intersection” • Single value 39
  48. 61.

    Distribution Abstraction • Previous model assumes Dynamo-style
 Instead, allow nodes

    in the graph to represent objects that have their own replication scheme • Each node in the dataflow graph: • Replicated state machine • Dynamo-style “quorum-intersection” • Single value • Epidemic broadcast for data propagation
 Facilitates distribution of “program state” across nodes in the dataflow graph 39
  49. 65.

    Two Layer Approach • Lasp • Variables and values are

    stored on all nodes • Read operations: 41
  50. 66.

    Two Layer Approach • Lasp • Variables and values are

    stored on all nodes • Read operations: • Declare “interest” in a value 41
  51. 67.

    Two Layer Approach • Lasp • Variables and values are

    stored on all nodes • Read operations: • Declare “interest” in a value • Store a continuation to invoke 
 Predicate function on a lattice position 41
  52. 68.

    Two Layer Approach • Lasp • Variables and values are

    stored on all nodes • Read operations: • Declare “interest” in a value • Store a continuation to invoke 
 Predicate function on a lattice position • Bind operations (update operations, as well): 41
  53. 69.

    Two Layer Approach • Lasp • Variables and values are

    stored on all nodes • Read operations: • Declare “interest” in a value • Store a continuation to invoke 
 Predicate function on a lattice position • Bind operations (update operations, as well): • Bind a value on each “interested node”
 This performs a merge with the nodes current value 41
  54. 71.

    Two Layer Approach • Gossip • Weak ordering of broadcast

    messages
 Variable state and metadata for declare and bind operations 42
  55. 72.

    Two Layer Approach • Gossip • Weak ordering of broadcast

    messages
 Variable state and metadata for declare and bind operations • Bind operations: 42
  56. 73.

    Two Layer Approach • Gossip • Weak ordering of broadcast

    messages
 Variable state and metadata for declare and bind operations • Bind operations: • Uniquely identify messages
 Pair of global message identifier and version vector 42
  57. 74.

    Two Layer Approach • Gossip • Weak ordering of broadcast

    messages
 Variable state and metadata for declare and bind operations • Bind operations: • Uniquely identify messages
 Pair of global message identifier and version vector • Incremented by coordinating replica
 When receiving incoming vector, merge with current vector and advance 42
  58. 75.

    Two Layer Approach • Gossip • Weak ordering of broadcast

    messages
 Variable state and metadata for declare and bind operations • Bind operations: • Uniquely identify messages
 Pair of global message identifier and version vector • Incremented by coordinating replica
 When receiving incoming vector, merge with current vector and advance • Monotonic state increases per-replica
 Ignore messages that have been subsumed by later messages 42
  59. 76.

    Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Riot Ads Rovio Ads Fil Product Read 50,000 Remove Increment Union 43 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  60. 78.

    44 server({#ad{counter=Counter}=Ad, _}, Ads) -> strict_read(Counter, 50000), update(Ads, {remove, Ad},

    Ad). Ad Counter Process • Blocking read of the counter for value 50000
  61. 79.

    44 server({#ad{counter=Counter}=Ad, _}, Ads) -> strict_read(Counter, 50000), update(Ads, {remove, Ad},

    Ad). Ad Counter Process • Blocking read of the counter for value 50000 • Records interest in variable Counter
  62. 80.

    44 server({#ad{counter=Counter}=Ad, _}, Ads) -> strict_read(Counter, 50000), update(Ads, {remove, Ad},

    Ad). Ad Counter Process • Blocking read of the counter for value 50000 • Records interest in variable Counter • Registers a continuation that will invoke when Counter >= 50000 Continuation
  63. 81.

    44 server({#ad{counter=Counter}=Ad, _}, Ads) -> strict_read(Counter, 50000), update(Ads, {remove, Ad},

    Ad). Ad Counter Process • Blocking read of the counter for value 50000 • Records interest in variable Counter • Registers a continuation that will invoke when Counter >= 50000 • Interest in variable Counter removed Continuation
  64. 82.

    45 %% Create initial set. S1 = declare(set), %% Add

    elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
  65. 83.

    46 Map Process map(S1, Function, S2) -> map(S1, Function, Bottom(S1),

    S2). map(S1, Function, Last, S2) -> Continuation = fun(Current) -> Result = compute(Function, Current), bind(S2, Result), map(S1, P, Current, S2) end, read(S1, fun(Current) -> Last < Current, Continuation).
  66. 84.

    47 Map Process map(S1, Function, S2) -> map(S1, Function, Bottom(S1),

    S2). map(S1, Function, Last, S2) -> Continuation = fun(Current) -> Result = compute(Function, Current), bind(S2, Result), map(S1, P, Current, S2) end, read(S1, fun(Current) -> Last < Current, Continuation). Invoke Map
  67. 85.

    48 Map Process map(S1, Function, S2) -> map(S1, Function, Bottom(S1),

    S2). map(S1, Function, Last, S2) -> Continuation = fun(Current) -> Result = compute(Function, Current), bind(S2, Result), map(S1, P, Current, S2) end, read(S1, fun(Current) -> Last < Current, Continuation). Generate continuation
  68. 86.

    49 Map Process map(S1, Function, S2) -> map(S1, Function, Bottom(S1),

    S2). map(S1, Function, Last, S2) -> Continuation = fun(Current) -> Result = compute(Function, Current), bind(S2, Result), map(S1, P, Current, S2) end, read(S1, fun(Current) -> Last < Current, Continuation). Invoke continuation on change
  69. 88.

    Future Work • Quantitative evaluation
 Evaluation and optimization of the

    prototype implementation in Erlang with Lasp and Plumtree 51
  70. 89.

    Future Work • Quantitative evaluation
 Evaluation and optimization of the

    prototype implementation in Erlang with Lasp and Plumtree • Extensions to increase locality
 Local variables that do not get distributed 51
  71. 90.

    Future Work • Quantitative evaluation
 Evaluation and optimization of the

    prototype implementation in Erlang with Lasp and Plumtree • Extensions to increase locality
 Local variables that do not get distributed • Partial evaluation
 Can we derive optimized distribution models given dataflow execution graphs? 51
  72. 91.

    Conclusion • Successful pairing
 Pairing of a weak delivery model

    with a model that does not rely on ordering guarantees 52
  73. 92.

    Conclusion • Successful pairing
 Pairing of a weak delivery model

    with a model that does not rely on ordering guarantees • Distributed epidemic-based runtime
 First implementation of a epidemic broadcast based runtime in a general programming model 52
  74. 93.

    SyncFree is a European research project taking place for 3

    years, staring October 2013, and is funded by the European Union, grant agreement n° 609551. 53