Declarative, Distributed, Edge Computation

Declarative, Distributed, Edge Computation

RICON 2015

3e09fee7b359be847ed5fa48f524a3d3?s=128

Christopher Meiklejohn

November 06, 2015
Tweet

Transcript

  1. Declarative, Distributed, Edge Computation Christopher Meiklejohn // @cmeik RICON 2015,

    November 6th, 2015 1
  2. RA RB

  3. RA RB 1 set(1)

  4. RA RB 1 set(1) 3 2 set(2) set(3)

  5. RA RB 1 set(1) 3 2 set(2) set(3) ? ?

  6. Synchronization • To enforce an order
 Makes programming easier 6

  7. Synchronization • To enforce an order
 Makes programming easier •

    Eliminate accidental nondeterminism
 Prevent race conditions 6
  8. Synchronization • To enforce an order
 Makes programming easier •

    Eliminate accidental nondeterminism
 Prevent race conditions • Techniques
 Locks, mutexes, semaphores, monitors, etc. 6
  9. Difficult Cases • “Internet of Things”
 Low power, limited memory

    and connectivity 7
  10. Difficult Cases • “Internet of Things”
 Low power, limited memory

    and connectivity • Mobile Gaming
 Offline operation with replicated, shared state 7
  11. Zero Synchronization • We want to have our cake and

    eat it too. 8
  12. Zero Synchronization • We want to have our cake and

    eat it too. • “Zero synchronization” as a starting point
 Add synchronization only where it is necessary 8
  13. Zero Synchronization • We want to have our cake and

    eat it too. • “Zero synchronization” as a starting point
 Add synchronization only where it is necessary • Explore the limits
 Can we design a model of computation where it’s easy to write coordination-free computations that are free from concurrency anomalies? 8
  14. 9 Strong Weak Strong (Data Center) Weak (Open Internet) Sharing

    (Application Property) Coupling (Infrastructure Property)
  15. 9 Strong Weak Strong (Data Center) Weak (Open Internet) Sharing

    (Application Property) Coupling (Infrastructure Property) MapReduce SETI@Home
  16. 9 Strong Weak Strong (Data Center) Weak (Open Internet) Sharing

    (Application Property) Coupling (Infrastructure Property) Twitter MapReduce SETI@Home
  17. 9 Strong Weak Strong (Data Center) Weak (Open Internet) Sharing

    (Application Property) Coupling (Infrastructure Property) Twitter MapReduce SETI@Home Synchronization-Free Computations
  18. The Fundamentals of Distributed Computation 10

  19. Distributed Computation • Distributed to concurrent programming
 Consistency and now

    partial failure 11
  20. Distributed Computation • Distributed to concurrent programming
 Consistency and now

    partial failure • Enforcing the “single system” illusion 11
  21. Distributed Computation • Distributed to concurrent programming
 Consistency and now

    partial failure • Enforcing the “single system” illusion • Consistency model as a contract 11
  22. Distributed Computation • Distributed to concurrent programming
 Consistency and now

    partial failure • Enforcing the “single system” illusion • Consistency model as a contract • Enforced through synchronization 11
  23. Distributed Computation • Distributed to concurrent programming
 Consistency and now

    partial failure • Enforcing the “single system” illusion • Consistency model as a contract • Enforced through synchronization • Consistency model
 We can think of a consistency model as analogous to a programming paradigm 11
  24. Why Is Synchronization Undesirable? 12

  25. The Avatars of Time • The problem of time
 Handling

    physical time in applications is difficult 13
  26. The Avatars of Time • The problem of time
 Handling

    physical time in applications is difficult • Time has three major avatars in computing 13
  27. The Avatars of Time • The problem of time
 Handling

    physical time in applications is difficult • Time has three major avatars in computing • Mutable state in sequential systems 13
  28. The Avatars of Time • The problem of time
 Handling

    physical time in applications is difficult • Time has three major avatars in computing • Mutable state in sequential systems • Nondeterminism in concurrent systems 13
  29. The Avatars of Time • The problem of time
 Handling

    physical time in applications is difficult • Time has three major avatars in computing • Mutable state in sequential systems • Nondeterminism in concurrent systems ̣ Network latency in distributed systems 13
  30. The Avatars of Time • The problem of time
 Handling

    physical time in applications is difficult • Time has three major avatars in computing • Mutable state in sequential systems • Nondeterminism in concurrent systems ̣ Network latency in distributed systems • Unavoidable
 Time is how users interact with programs 13
  31. Parable of the Car • “Car driving on a highway”

    14
  32. Parable of the Car • “Car driving on a highway”

    • Friction needed for the car to grip the road 14
  33. Parable of the Car • “Car driving on a highway”

    • Friction needed for the car to grip the road • Motors want as little friction as possible 14
  34. Parable of the Car • “Car driving on a highway”

    • Friction needed for the car to grip the road • Motors want as little friction as possible • Time is like friction
 We can not completely eliminate friction, but aim to reduce it as much as possible 14
  35. Physical Time (real world) Parable of the Car 15 •

    Time only at the interface
 The interface is a small part of the system, and this is where we introduction friction
  36. Physical Time (real world) Parable of the Car 15 •

    Time only at the interface
 The interface is a small part of the system, and this is where we introduction friction • Physical time-free execution
 Internally, avoid synchronization as much as possible and virtualize notion of time
  37. No Time In The Box? 16

  38. No Time In The Box? 16 You program the box.

  39. Weak Synchronization • Can we achieve anything without synchronization?
 Not

    really. 17
  40. Weak Synchronization • Can we achieve anything without synchronization?
 Not

    really. • Strong Eventual Consistency (SEC)
 “Replicas that deliver the same updates have equivalent state” 17
  41. Weak Synchronization • Can we achieve anything without synchronization?
 Not

    really. • Strong Eventual Consistency (SEC)
 “Replicas that deliver the same updates have equivalent state” • Primary requirement
 Eventual replica-to-replica communication 17
  42. Weak Synchronization • Can we achieve anything without synchronization?
 Not

    really. • Strong Eventual Consistency (SEC)
 “Replicas that deliver the same updates have equivalent state” • Primary requirement
 Eventual replica-to-replica communication • Order insensitive! (Commutativity) 17
  43. Weak Synchronization • Can we achieve anything without synchronization?
 Not

    really. • Strong Eventual Consistency (SEC)
 “Replicas that deliver the same updates have equivalent state” • Primary requirement
 Eventual replica-to-replica communication • Order insensitive! (Commutativity) • Duplicate insensitive! (Idempotent) 17
  44. RA RB

  45. RA RB 1 set(1)

  46. RA RB 1 set(1) 3 2 set(2) set(3)

  47. RA RB 1 3 2 3 3 set(1) set(2) set(3)

    max(2,3) max(2,3)
  48. How can we succeed with Strong Eventual Consistency? 22

  49. Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic

    operations monotonically) 23
  50. Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic

    operations monotonically) 2. Retain the properties of functional programming
 (ex. confluence, referential transparency over composition) 23
  51. Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic

    operations monotonically) 2. Retain the properties of functional programming
 (ex. confluence, referential transparency over composition) 3. Distributed, and fault-tolerant runtime
 (ex. replication) 23
  52. Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic

    operations monotonically) 2. Retain the properties of functional programming
 (ex. confluence, referential transparency over composition) 3. Distributed, and fault-tolerant runtime
 (ex. replication) 24
  53. Convergent Objects
 Conflict-Free 
 Replicated Data Types 25 SSS 2011

  54. Conflict-Free 
 Replicated Data Types • Many types exist with

    different properties
 Sets, counters, registers, flags, maps, graphs 26
  55. Conflict-Free 
 Replicated Data Types • Many types exist with

    different properties
 Sets, counters, registers, flags, maps, graphs • Strong Eventual Consistency
 Instances satisfy SEC property per-object 26
  56. RA RB RC

  57. RA RB RC {1} (1, {a}, {}) add(1)

  58. RA RB RC {1} (1, {a}, {}) add(1) {1} (1,

    {c}, {}) add(1)
  59. RA RB RC {1} (1, {a}, {}) add(1) {1} (1,

    {c}, {}) add(1) {} (1, {c}, {c}) remove(1)
  60. RA RB RC {1} (1, {a}, {}) add(1) {1} (1,

    {c}, {}) add(1) {} (1, {c}, {c}) remove(1) {1} {1} {1} (1, {a, c}, {c}) (1, {a, c}, {c}) (1, {a, c}, {c})
  61. Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic

    operations monotonically) 2. Retain the properties of functional programming
 (ex. confluence, referential transparency over composition) 3. Distributed, and fault-tolerant runtime
 (ex. replication) 32
  62. Convergent Programs Lattice Processing 33 PPDP 2015

  63. Lattice Processing (Lasp) • Distributed, deterministic dataflow
 Distributed, deterministic dataflow

    programming model for “eventually consistent” computations 34
  64. Lattice Processing (Lasp) • Distributed, deterministic dataflow
 Distributed, deterministic dataflow

    programming model for “eventually consistent” computations • Convergent data structures
 Primary data abstraction is the CRDT 34
  65. Lattice Processing (Lasp) • Distributed, deterministic dataflow
 Distributed, deterministic dataflow

    programming model for “eventually consistent” computations • Convergent data structures
 Primary data abstraction is the CRDT • Enables composition
 Provides functional composition of CRDTs that preserves the SEC property 34
  66. 35 %% Create initial set. S1 = declare(set), %% Add

    elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
  67. 36 %% Create initial set. S1 = declare(set), %% Add

    elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
  68. 37 %% Create initial set. S1 = declare(set), %% Add

    elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
  69. 38 %% Create initial set. S1 = declare(set), %% Add

    elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
  70. 39 %% Create initial set. S1 = declare(set), %% Add

    elements to initial set and update. update(S1, {add, [1,2,3]}), %% Create second set. S2 = declare(set), %% Apply map operation between S1 and S2. map(S1, fun(X) -> X * 2 end, S2).
  71. Lattice Processing (Lasp) • Functional and set-theoretic operations on sets


    Product, intersection, union, filter, map, fold 40
  72. Lattice Processing (Lasp) • Functional and set-theoretic operations on sets


    Product, intersection, union, filter, map, fold • Metadata computation
 Performs transformation on the internal metadata of CRDTs allowing creation of “composed” CRDTs 40
  73. Lasp Processes • Replicas as monotonic streams
 Each replica of

    a CRDT produces a monotonic stream of states 41
  74. Lasp Processes • Replicas as monotonic streams
 Each replica of

    a CRDT produces a monotonic stream of states • Monotonic processes
 Read from one or more input replica streams and produce a single output replica stream 41
  75. Lasp Processes • Replicas as monotonic streams
 Each replica of

    a CRDT produces a monotonic stream of states • Monotonic processes
 Read from one or more input replica streams and produce a single output replica stream • Inflationary reads
 Read operation ensures that we only read inflationary updates to replicas 41
  76. Lattice Processing Monotonic Streams 42

  77. RA {} C1 {} C2 {} 43

  78. RA {} (1, {a}, {}) C1 (1, {a}, {}) {}

    C2 {} 44
  79. RA {} (1, {a}, {}) (1, {a, c}, {}) C1

    (1, {a}, {}) {} C2 (1, {c}, {}) {} 45
  80. RA {} (1, {a}, {}) (1, {a, c}, {}) (1,

    {a, c}, {a}) C1 (1, {a}, {}) {} (1, {a}, {a}) C2 (1, {c}, {}) {} 46
  81. RA {} (1, {a}, {}) (1, {a, c}, {}) (1,

    {a, c}, {a}) C1 (1, {a}, {}) {} (1, {a}, {a}) C2 (1, {c}, {}) {} (1, {a, c}, {a}) (1, {a, c}, {a}) (1, {a, c}, {a}) 47
  82. RA {} (1, {a}, {}) (1, {a, c}, {}) (1,

    {a, c}, {a}) C1 (1, {a}, {}) {} (1, {a}, {a}) C2 (1, {c}, {}) {} (1, {a, c}, {a}) (1, {a, c}, {a}) (1, {a, c}, {a}) 48
  83. Lattice Processing Monotonic Processes 49

  84. RA {} P1 F(RA) {} 50

  85. RA {} P1 F(RA) {} strict_read({}) 51

  86. RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) 52

  87. RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1,

    {a}, {})) 53
  88. RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1,

    {a}, {})) (2, {a}, {}) 54
  89. RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1,

    {a}, {})) (2, {a}, {}) strict_read((1, {a}, {}) (1, {a}, {}) 55
  90. RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) F((1,

    {a}, {})) (2, {a}, {}) strict_read((1, {a}, {}) (1, {a}, {}) (1, {a}, {a}) F((1, {a}, {a})) (2, {a}, {a}) strict_read((1, {a}, {a}) (1, {a}, {a}) 56
  91. RA {} P1 F(RA) {} strict_read({}) (1, {a}, {}) (1,

    {a}, {}) (1, {a}, {a}) F((1, {a}, {a})) (2, {a}, {a}) strict_read((1, {a}, {a}) (1, {a}, {a}) 57
  92. Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic

    operations monotonically) 2. Retain the properties of functional programming
 (ex. confluence, referential transparency over composition) 3. Distributed, and fault-tolerant runtime
 (ex. replication) 58
  93. Distributed Runtime Selective Hearing 59 W-PSDS 2015

  94. Selective Hearing • Epidemic broadcast based runtime system
 Provide a

    runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution 60
  95. Selective Hearing • Epidemic broadcast based runtime system
 Provide a

    runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) 60
  96. Selective Hearing • Epidemic broadcast based runtime system
 Provide a

    runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) • Epidemic broadcast mechanisms provide weak ordering but are resilient and efficient 60
  97. Selective Hearing • Epidemic broadcast based runtime system
 Provide a

    runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) • Epidemic broadcast mechanisms provide weak ordering but are resilient and efficient • Lasp’s programming model is tolerant to message re-ordering, disconnections, and node failures 60
  98. Selective Hearing • Epidemic broadcast based runtime system
 Provide a

    runtime system that can scale to large numbers of nodes, that is resilient to failures and provides efficient execution • Well-matched to Lattice Processing (Lasp) • Epidemic broadcast mechanisms provide weak ordering but are resilient and efficient • Lasp’s programming model is tolerant to message re-ordering, disconnections, and node failures • “Selective Receive”
 Nodes selectively receive and process messages based on interest 60
  99. Programming SEC 1. Eliminate accidental nondeterminism
 (ex. deterministic, modeling non-monotonic

    operations monotonically) 2. Retain the properties of functional programming
 (ex. confluence, referential transparency over composition) 3. Distributed, and fault-tolerant runtime
 (ex. replication) 61
  100. What can we build? Leaderboard 62

  101. Leaderboard • Mobile game platform
 Local leaderboard tracking top-k highest

    scored games 63
  102. Leaderboard • Mobile game platform
 Local leaderboard tracking top-k highest

    scored games • Clients will go offline
 Clients have limited connectivity and the system still needs to make progress while clients are offline 63
  103. Client 1 Leaderboard Client 3 Leaderboard Client 2 Leaderboard Lasp

    Operation User-Maintained CRDT Lasp-Maintained CRDT 64
  104. Leaderboard • Peer-to-peer dissemination
 Nodes periodically “merge” their state with

    a random peer 65
  105. Leaderboard • Peer-to-peer dissemination
 Nodes periodically “merge” their state with

    a random peer • Complexity in the data type
 Each node tracks a top-k set of its own games in a bounded set 65
  106. 66 %% Create a leaderboard datatype. L = declare({top_k, [2]}).

    %% Update leaderboard. update({set, Name, Score}, L).
  107. 67 %% Create a leaderboard datatype. L = declare({top_k, [2]}).

    %% Update leaderboard. update({set, Name, Score}, L).
  108. 68 %% Create a leaderboard datatype. L = declare({top_k, [2]}).

    %% Update leaderboard. update({set, Name, Score}, L).
  109. What if we want to enhance the behavior? 69

  110. What if we want to enhance the behavior? 69 Without

    the creation of a new datatype
  111. What can we build? Per-User Leaderboard 70 EdgeCom 2016

  112. Per-User Leaderboard • Enhance existing design
 Only the top score

    for each user at each device 71
  113. Per-User Leaderboard • Enhance existing design
 Only the top score

    for each user at each device • Minimize transmitted state
 Prevent transmission of state that is not necessary to perform the computation 71
  114. Per-User Leaderboard • Enhance existing design
 Only the top score

    for each user at each device • Minimize transmitted state
 Prevent transmission of state that is not necessary to perform the computation • Compose data types
 Build a per-user leaderboard through the composition of existing types 71
  115. Client1 Scores Local Top-K Fold Global Top-K Lasp Operation Input

    User-Maintained CRDT Output Lasp-Maintained CRDT Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold 72
  116. Client1 Scores Local Top-K Fold Global Top-K Fold 73 Client1

    Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold
  117. Client1 Scores Local Top-K Fold Global Top-K Fold 74 Client1

    Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold
  118. Client1 Scores Local Top-K Fold Global Top-K Fold 75 Client1

    Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold
  119. Client1 Scores Local Top-K Fold Global Top-K Fold 76 Client1

    Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold
  120. Client1 Scores Local Top-K Fold Global Top-K Fold 77 Client1

    Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold
  121. Client1 Scores Local Top-K Fold Global Top-K Fold 78 Client1

    Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold
  122. Client1 Scores Local Top-K Fold Global Top-K Lasp Operation Input

    User-Maintained CRDT Output Lasp-Maintained CRDT Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold 79 Client1 Scores Local Top-K Fold Global Top-K Fold Client3 Scores Local Top-K Fold Global Top-K Fold Client2 Scores Local Top-K Fold Global Top-K Fold
  123. Per-User Leaderboard • Dynamically scoped variables
 Variable which take different

    values depending on where it is executing 80
  124. Per-User Leaderboard • Dynamically scoped variables
 Variable which take different

    values depending on where it is executing • Dynamically scoped fold operation
 Perform a distributed “reduce” operation that combines the state of a dynamically scoped variables across 80
  125. 81 %% Create a global leaderboard. G = declare({top_k, [10]}).

    %% Create a local leaderboard. L = declare_dynamic({top_k, [10]}). %% Create a set of scores. S = declare_dynamic(set). %% Compute local top-k list. fold(S, fun max_by_name/2, L). %% Compute global top-k list. fold_dynamic(L, fun max_by_name/2, G).
  126. 82 %% Create a global leaderboard. G = declare({top_k, [10]}).

    %% Create a local leaderboard. L = declare_dynamic({top_k, [10]}). %% Create a set of scores. S = declare_dynamic(set). %% Compute local top-k list. fold(S, fun max_by_name/2, L). %% Compute global top-k list. fold_dynamic(L, fun max_by_name/2, G).
  127. 83 %% Create a global leaderboard. G = declare({top_k, [10]}).

    %% Create a local leaderboard. L = declare_dynamic({top_k, [10]}). %% Create a set of scores. S = declare_dynamic(set). %% Compute local top-k list. fold(S, fun max_by_name/2, L). %% Compute global top-k list. fold_dynamic(L, fun max_by_name/2, G).
  128. 84 %% Create a global leaderboard. G = declare({top_k, [10]}).

    %% Create a local leaderboard. L = declare_dynamic({top_k, [10]}). %% Create a set of scores. S = declare_dynamic(set). %% Compute local top-k list. fold(S, fun max_by_name/2, L). %% Compute global top-k list. fold_dynamic(L, fun max_by_name/2, G).
  129. 85 %% Create a global leaderboard. G = declare({top_k, [10]}).

    %% Create a local leaderboard. L = declare_dynamic({top_k, [10]}). %% Create a set of scores. S = declare_dynamic(set). %% Compute local top-k list. fold(S, fun max_by_name/2, L). %% Compute global top-k list. fold_dynamic(L, fun max_by_name/2, G).
  130. 86 %% Create a global leaderboard. G = declare({top_k, [10]}).

    %% Create a local leaderboard. L = declare_dynamic({top_k, [10]}). %% Create a set of scores. S = declare_dynamic(set). %% Compute local top-k list. fold(S, fun max_by_name/2, L). %% Compute global top-k list. fold_dynamic(L, fun max_by_name/2, G).
  131. Let’s look at a larger application 87

  132. Let’s look at a larger application 87 With visible 


    non-monotonicity
  133. What can we build? Advertisement Counter 88

  134. Advertisement Counter • Mobile game platform selling advertisement space
 Advertisements

    are paid according to a minimum number of impressions 89
  135. Advertisement Counter • Mobile game platform selling advertisement space
 Advertisements

    are paid according to a minimum number of impressions • Clients will go offline
 Clients have limited connectivity and the system still needs to make progress while clients are offline 89
  136. Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 90
  137. Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Riot Ads Rovio Ads Product Read 50,000 Remove Increment Union 91 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  138. Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 1 Client 92 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  139. Ads ovio Ad ounter 1 ovio Ad ounter 2 iot

    Ad ounter 1 iot Ad ounter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Rovio Ad Counter 1 Ro C Client Side, Sing 93 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  140. Ads Contracts Ads Contracts Ads With Contracts Riot Ads Rovio

    Ads Filter Product move Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 94 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  141. Ads Contracts Ads Contracts Ads With Contracts Filter Product Read

    Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 95 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  142. Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 96 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  143. Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Riot Ads Rovio Ads Fil Product Read 50,000 Remove Increment Union 97 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  144. Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot

    Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client 98 Ads Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 2 Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Rovio Ad Counter 1 Rovio Ad Counter 2 Riot Ad Counter 1 Client Side, Single Copy at Client
  145. Advertisement Counter • Completely monotonic
 Disabling advertisements and contracts are

    all modeled through monotonic state growth 99
  146. Advertisement Counter • Completely monotonic
 Disabling advertisements and contracts are

    all modeled through monotonic state growth • Arbitrary distribution
 Use of convergent data structures allows computational graph to be arbitrarily distributed 99
  147. Advertisement Counter • Completely monotonic
 Disabling advertisements and contracts are

    all modeled through monotonic state growth • Arbitrary distribution
 Use of convergent data structures allows computational graph to be arbitrarily distributed • Divergence
 Divergence is a factor of synchronization period 99
  148. 100 %% Generate a series of contracts. Contracts = declare(set),

    %% Generate advertisements. RiotAds = declare(set), RovioAds = declare(set), create_advertisements_and_contracts(RiotAds, Contracts), create_advertisements_and_contracts(RovioAds, Contracts), %% Union ads. Ads = declare(set), union(RovioAds, RiotAds, Ads), %% Compute the product of both ads and contracts. AdsContracts = declare(set), product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. AdsWithContracts = declare(set), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, filter(AdsContracts, FilterFun, AdsWithContracts).
  149. 101 %% Generate a series of contracts. Contracts = declare(set),

    %% Generate advertisements. RiotAds = declare(set), RovioAds = declare(set), create_advertisements_and_contracts(RiotAds, Contracts), create_advertisements_and_contracts(RovioAds, Contracts), %% Union ads. Ads = declare(set), union(RovioAds, RiotAds, Ads), %% Compute the product of both ads and contracts. AdsContracts = declare(set), product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. AdsWithContracts = declare(set), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, filter(AdsContracts, FilterFun, AdsWithContracts).
  150. 102 %% Generate a series of contracts. Contracts = declare(set),

    %% Generate advertisements. RiotAds = declare(set), RovioAds = declare(set), create_advertisements_and_contracts(RiotAds, Contracts), create_advertisements_and_contracts(RovioAds, Contracts), %% Union ads. Ads = declare(set), union(RovioAds, RiotAds, Ads), %% Compute the product of both ads and contracts. AdsContracts = declare(set), product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. AdsWithContracts = declare(set), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, filter(AdsContracts, FilterFun, AdsWithContracts).
  151. 103 %% Generate a series of contracts. Contracts = declare(set),

    %% Generate advertisements. RiotAds = declare(set), RovioAds = declare(set), create_advertisements_and_contracts(RiotAds, Contracts), create_advertisements_and_contracts(RovioAds, Contracts), %% Union ads. Ads = declare(set), union(RovioAds, RiotAds, Ads), %% Compute the product of both ads and contracts. AdsContracts = declare(set), product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. AdsWithContracts = declare(set), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, filter(AdsContracts, FilterFun, AdsWithContracts).
  152. 104 %% Generate a series of contracts. Contracts = declare(set),

    %% Generate advertisements. RiotAds = declare(set), RovioAds = declare(set), create_advertisements_and_contracts(RiotAds, Contracts), create_advertisements_and_contracts(RovioAds, Contracts), %% Union ads. Ads = declare(set), union(RovioAds, RiotAds, Ads), %% Compute the product of both ads and contracts. AdsContracts = declare(set), product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. AdsWithContracts = declare(set), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, filter(AdsContracts, FilterFun, AdsWithContracts).
  153. 105 %% Generate a series of contracts. Contracts = declare(set),

    %% Generate advertisements. RiotAds = declare(set), RovioAds = declare(set), create_advertisements_and_contracts(RiotAds, Contracts), create_advertisements_and_contracts(RovioAds, Contracts), %% Union ads. Ads = declare(set), union(RovioAds, RiotAds, Ads), %% Compute the product of both ads and contracts. AdsContracts = declare(set), product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. AdsWithContracts = declare(set), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, filter(AdsContracts, FilterFun, AdsWithContracts).
  154. Client3 Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Riot Ad Counter

    1 Client1 Client2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Riot Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 1 Ads With Contracts Ads With Contracts Ads With Contracts Server Ads With Contracts Server Computation! 106
  155. Client3 Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Riot Ad Counter

    1 Client1 Client2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Riot Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 1 Ads With Contracts Ads With Contracts Ads With Contracts Server Ads With Contracts Server Computation! 107
  156. Client3 Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Riot Ad Counter

    1 Client1 Client2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Rovio Ad Counter 2 Rovio Ad Counter 1 Riot Ad Counter 2 Riot Ad Counter 1 Riot Ad Counter 1 Ads With Contracts Ads With Contracts Ads With Contracts Server Ads With Contracts Server Computation! 108
  157. Advertisement Counter • “Servers” as peers to “clients”
 Servers are

    peers to clients that perform additional computation 109
  158. Advertisement Counter • “Servers” as peers to “clients”
 Servers are

    peers to clients that perform additional computation • Any node can disable an advertisement under this model given enough information 109
  159. Advertisement Counter • “Servers” as peers to “clients”
 Servers are

    peers to clients that perform additional computation • Any node can disable an advertisement under this model given enough information • “Servers” as trusted nodes
 Serve as a location for performing “exactly once” side- effects 109
  160. Advertisement Counter • “Servers” as peers to “clients”
 Servers are

    peers to clients that perform additional computation • Any node can disable an advertisement under this model given enough information • “Servers” as trusted nodes
 Serve as a location for performing “exactly once” side- effects • Billing customers must be done at a central point by a trusted node in the system 109
  161. We’ve build up from zero synchronization 110

  162. We’ve build up from zero synchronization 110 Instead of working

    to remove synchronization
  163. What have we learned? 111

  164. Key Points • Synchronization is expensive
 Locking and other synchronization

    mechanisms limit scalability to the critical section 112
  165. Key Points • Synchronization is expensive
 Locking and other synchronization

    mechanisms limit scalability to the critical section • Synchronization is sometimes not possible
 Mobile and “Internet of Things” applications make synchronization for replicated state impractical 112
  166. Key Points • Synchronization is expensive
 Locking and other synchronization

    mechanisms limit scalability to the critical section • Synchronization is sometimes not possible
 Mobile and “Internet of Things” applications make synchronization for replicated state impractical • Apply synchronization only where required
 Global invariants, atomic visibility, etc. 112
  167. How do I learn more? 113

  168. Publications • “Lasp: A Language for Distributed, Coordination-Free Programming” 


    ACM SIGPLAN PPDP 2015 • “Selective Hearing: An Approach to Distributed, Eventually Consistent Edge Computation”
 IEEE W-PSDS 2015 • “The Implementation and Use of a Generic Dataflow Behaviour in Erlang”
 ACM SIGPLAN Erlang Workshop ’15 • “Lasp: A Language for Distributed, Eventually Consistent Computations with CRDTs"
 PaPoC 2015 • “Declarative, Sliding Window Aggregations for Computations at the Edge"
 IEEE EdgeCom 2016 114
  169. SyncFree is a European research project taking place for 3

    years, staring October 2013, and is funded by the European Union, grant agreement n° 609551. 115
  170. Thanks! 116 Christopher Meiklejohn @cmeik