Distributed, Eventually Consistent Computations

Distributed, Eventually Consistent Computations

Erlang Factory SF Bay, 2015

3e09fee7b359be847ed5fa48f524a3d3?s=128

Christopher Meiklejohn

March 26, 2015
Tweet

Transcript

  1. None
  2. LASP 2

  3. EVENTUALLY CONSISTENT COMPUTATIONS DISTRIBUTED 3

  4. CHRISTOPHER MEIKLEJOHN EN TAL AV 4

  5. RESEARCH WITH: PETER VAN ROY (UCL) 5

  6. MOTIVATION 6

  7. EXPENSIVE SYNCHRONIZATION IS 7

  8. SOMETIMES IMPRACTICAL SYNCHRONIZATION IS 8

  9. MOBILE GAMES: SHARED STATE BETWEEN CLIENTS CLIENTS GO OFFLINE 9

    http://www.rovio.com/en/news/blog/261/263-million-monthly-active-users-in-december/
  10. DISJOINT STATE AGGREGATED UPSTREAM CLIENTS GO OFFLINE INTERNET OF THINGS:

    10 Gubbi, Jayavardhana, et al. "Internet of Things (IoT): A vision, architectural elements, and future directions." Future Generation Computer Systems 29.7 (2013): 1645-1660.
  11. NO TOTAL ORDER: REPLICATED SHARED STATE WITH OFFLINE CLIENTS CLIENTS

    NEED TO MAKE PROGRESS 11 Gilbert, Seth, and Nancy Lynch. "Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services." ACM SIGACT News 33.2 (2002): 51-59.
  12. WALL CLOCKS: UNRELIABLE AT BEST NON-DETERMINISTIC IF USED IN COMPUTATIONS

    12 Corbett, James C., et al. "Spanner: Google’s globally distributed database." ACM Transactions on Computer Systems (TOCS) 31.3 (2013): 8.
  13. RECONCILED BY USER CONCURRENCY 13

  14. RA RB 1 1 3 2 ? ? 14

  15. RA RB 1 1 3 2 ? ? 15

  16. RA RB 1 1 3 2 ? ? 16

  17. RA RB 1 1 3 2 ? ? 17

  18. CRDTs 18

  19. DETERMINISTIC RESOLUTION CRDTs PROVIDE 19

  20. CRDTs: MAPS, SETS, COUNTERS, REGISTERS, GRAPHS DETERMINISTIC RESOLUTION 20

  21. STRONG EVENTUAL CONSISTENCY CRDTs REALIZE 21

  22. “CORRECT REPLICAS THAT HAVE DELIVERED THE SAME UPDATES HAVE EQUIVALENT

    STATE” 22 Shapiro, Marc, et al. "Conflict-free replicated data types." Stabilization, Safety, and Security of Distributed Systems. Springer Berlin Heidelberg, 2011. 386-400.
  23. ‘MAX’ REGISTER CRDTs EXAMPLE 23

  24. RA RB 1 1 3 2 3 3 24

  25. ‘ORSET’ SET CRDTs EXAMPLE 25

  26. RA RB RC {1} (1, {a}, {}) {1} (1, {b},

    {}) {1} {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {b}, {}) {1} (1, {a, b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) 26
  27. RA RB RC {1} (1, {a}, {}) {1} (1, {b},

    {}) {1} {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {b}, {}) {1} (1, {a, b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) 27
  28. RA RB RC {1} (1, {a}, {}) {1} (1, {b},

    {}) {1} {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {b}, {}) {1} (1, {a, b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) 28
  29. RA RB RC {1} (1, {a}, {}) {1} (1, {b},

    {}) {1} {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {b}, {}) {1} (1, {a, b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) 29
  30. NONTRIVIAL COMPOSITION IS 30

  31. RC {1} (1, {b}, {}) {} (1, {b}, {b}) {1}

    (1, {a, b}, {b}) {1} (1, {a, b}, {b}) fun(X) -> 2 end F(RC) {2} {} {2} {2} 31
  32. RA RB RC {1} (1, {a}, {}) {1} (1, {b},

    {}) {1} {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {b}, {}) {1} (1, {a, b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) 32
  33. RA RC {1} (1, {a}, {}) {1} (1, {b}, {})

    {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {b}, {}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) 33
  34. RA RC {1} (1, {a}, {}) {1} (1, {b}, {})

    {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) {1} {1} (1, {a, b}, {b}) F(RA) {2} {2} {2} (1, {a, b}, {b}) F(RC) {2} {} {2} {2} 34
  35. RA RC {1} (1, {a}, {}) {1} (1, {b}, {})

    {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) {1} {1} (1, {a, b}, {b}) F(RA) {2} {2} {2} (1, {a, b}, {b}) F(RC) {2} {} {2} {2} 35
  36. RA RC {1} (1, {a}, {}) {1} (1, {b}, {})

    {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) {1} {1} (1, {a, b}, {b}) F(RA) {2} {2} {2} (1, {a, b}, {b}) F(RC) {2} {} {2} {2} 36
  37. RA RC {1} (1, {a}, {}) {1} (1, {b}, {})

    {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) {1} {1} (1, {a, b}, {b}) F(RA) {2} {2} {2} (1, {a, b}, {b}) F(RC) {2} {} {2} {2} 37
  38. COMPOSITION: USER OBSERVABLE VALUE VS. STATE METADATA MAPPING IS NONTRIVIAL

    WITHOUT MAPPING METADATA; UNMERGABLE 38 Brown, Russell, et al. "Riak dt map: A composable, convergent replicated dictionary." Proceedings of the First Workshop on Principles and Practice of Eventual Consistency. ACM, 2014. Conway, Neil, et al. "Logic and lattices for distributed programming." Proceedings of the Third ACM Symposium on Cloud Computing. ACM, 2012. Meiklejohn, Christopher. "On the composability of the Riak DT map: expanding from embedded to multi-key structures." Proceedings of the First Workshop on Principles and Practice of Eventual Consistency. ACM, 2014.
  39. LASP 39

  40. LASP WHAT IS 40

  41. LATTICE PROCESSING LASP 41

  42. CENTRALIZED SEMANTICS: DATATYPE COMPOSITION 42

  43. R {1} (1, {b}, {}) {} (1, {b}, {b}) {1}

    (1, {a, b}, {b}) {1} (1, {a, b}, {b}) fun(X) -> 2 end F(R) {2} {} {2} {2} (2, {b}, {}) (2, {b}, {b}) (2, {a, b}, {b}) (2, {a, b}, {b}) 43
  44. DISTRIBUTED SEMANTICS: DATATYPE COMPOSITION 44

  45. RA RB RC {1} (1, {a}, {}) {1} (1, {b},

    {}) {1} {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {b}, {}) {1} (1, {a, b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) fun(X) -> 2 end F(RC) {2} {} {2} {2} (2, {b}, {}) (2, {b}, {b}) (2, {a, b}, {b}) (2, {a, b}, {b}) F(RB) {2} (2, {b}, {}) {2} (2, {a, b}, {b}) {2} (2, {a, b}, {b}) fun(X) -> 2 end F(RA) {2} (2, {a}, {}) {2} (2, {a, b}, {b}) {2} (2, {a, b}, {b}) fun(X) -> 2 end 45
  46. DISTRIBUTED SEMANTICS: SYSTEM COMPOSITION 46

  47. CENTRALIZED RUNTIME: SINGLE NODE MODEL 47

  48. DISTRIBUED RUNTIME: MULTI-NODE MODEL 48

  49. SEMANTICS 49

  50. STREAMS 50

  51. STREAMS: CENTRALIZED EXECUTION 51

  52. RA {1} (1, {a}, {}) {1} (1, {a, b}, {b})

    {1} (1, {a, b}, {b}) 52
  53. STREAMS: DISTRIBUTED EXECUTION 53

  54. RA {1} (1, {a}, {}) {1} (1, {a, b}, {b})

    {1} (1, {a, b}, {b}) RB {1} (1, {a}, {}) {1} (1, {a, b}, {b}) {1} (1, {a, b}, {b}) RC {1} (1, {a}, {}) {1} (1, {a, b}, {b}) {1} (1, {a, b}, {b}) 54
  55. STREAMS: CONCURRENCY; FAILURE; ANTI-ENTROPY 55

  56. RA RB RC {1} (1, {a}, {}) {1} (1, {b},

    {}) {1} {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {b}, {}) {1} (1, {a, b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) {1} (1, {a, b}, {b}) {} 56
  57. STREAMS: DISTRIBUTED EXECUTION DETERMINISTIC REQUIRES EVENTUAL DELIVERY; ANTI-ENTROPY 57

  58. DECLARE: CREATE A STREAM OF A GIVEN CRDT TYPE INITIALIZE

    STATE AT BOTTOM VALUE 58
  59. BIND: ASSIGN A VALUE TO THE STREAM MERGE OF CURRENT

    AND NEW VALUE 59
  60. MONOTONIC READ 60

  61. MONOTONIC READ: ENSURES FORWARD PROGRESS BLOCKS ON INFLATIONS (OR STRICT

    INFLATIONS) 61
  62. COMPOSITIONS ARE PROCESSES: BLOCK FOR CHANGE IN INPUT COMPUTE CHANGE

    PROPAGATE CHANGE TO OUTPUT 62
  63. FUNCTIONAL 63

  64. FUNCTIONAL: MAP; FILTER; FOLD 64

  65. SET-THEORETIC 65

  66. SET-THEORETIC: PRODUCT; INTERSECTION; UNION 66

  67. ARCHITECTURE 67

  68. SHARED VARIABLE STORE LASP STORE 68

  69. LEVELDB, BITCASK, ETS LASP BACKENDS 69

  70. PROVIDED BY RIAK DT LASP CRDTs 70

  71. CENTRALIZED SEMANTICS LASP ARCHITECTURE 71

  72. STORE: SHARED VARIABLE STORE PROCESSESS SYNCHRONIZE ON VARIABLES 72

  73. DISTRIBUTED SEMANTICS LASP ARCHITECTURE 73

  74. STORE: REPLICATED, SHARDED VARIABLE STORE PROCESSESS SYNCHRONIZE ON VARIABLES 74

  75. REPLICATED: DISTRIBUTED WITH RIAK CORE QUORUM REQUESTS; ANTI-ENTROPY PROTOCOL 75

  76. HYBRID: DISTRIBUTE PROGRAMS; R/W WITH LOCAL STORE CENTRALIZED EXECUTION 76

  77. EXAMPLES 77

  78. %% Create initial set. {ok, S1} = lasp:declare(Type), %% Add

    elements to initial set and update. {ok, _} = lasp:update(S1, {add_all, [1,2,3]}, a), %% Create second set. {ok, S2} = lasp:declare(Type), %% Apply map. ok = lasp:map(S1, fun(X) -> X * 2 end, S2),
  79. %% Create initial set. {ok, S1} = lasp_core:declare(Type, Store), %%

    Add elements to initial set and update. {ok, _} = lasp_core:update(S1, {add_all, [1,2,3]}, a, Store), %% Create second set. {ok, S2} = lasp_core:declare(Type, Store), %% Apply map. ok = lasp_core:map(S1, fun(X) -> X * 2 end, S2, Store),
  80. AD COUNTER 80

  81. AD COUNTER: TRACKS AD IMPRESSIONS PUSHES ADVERTISEMENTS TO THE CLIENT

    DISABLES AD AT 50,000+ IMPRESSIONS CLIENTS DISPLAY ADS WHEN OFFLINE 81
  82. INFORMATION FLOW AD COUNTER 82

  83. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 83
  84. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 84
  85. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 85
  86. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 86
  87. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 87
  88. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 88
  89. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 89
  90. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 90
  91. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 91
  92. INFORMATION FLOW: MONOTONIC METADATA TO PREVENT DUPLICATE PROPAGATION 92 Title

    Text
  93. DISTRIBUTION AD COUNTER 93

  94. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 94
  95. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 95
  96. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 96
  97. Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter

    Riot Ad Counter Contracts Ads Contracts Ads With Contracts Clients Clients Clients Clients Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Client Process 97
  98. DISTRIBUTION BOUNDARIES: ARBITRARY ALLOWS COMPOSITION OF ENTIRE SYSTEM 98

  99. EXAMPLE CODE AD COUNTER 99

  100. %% @doc Client process; standard recursive looping server. client(Id, AdsWithContracts,

    PreviousValue) -> receive view_ad -> %% Get current ad list. {ok, {_, _, AdList0}} = lasp:read(AdsWithContracts, PreviousValue), AdList = riak_dt_orset:value(AdList0), case length(AdList) of 0 -> %% No advertisements left to display; ignore %% message. client(Id, AdsWithContracts, AdList0); _ -> %% Select a random advertisement from the list of %% active advertisements. {#ad{counter=Ad}, _} = lists:nth( random:uniform(length(AdList)), AdList), %% Increment it. {ok, _} = lasp:update(Ad, increment, Id), lager:info("Incremented ad counter: ~p", [Ad]), client(Id, AdsWithContracts, AdList0) end end.
  101. %% @doc Server functions for the advertisement counter. After 5

    views, %% disable the advertisement. %% server({#ad{counter=Counter}=Ad, _}, Ads) -> %% Blocking threshold read for 5 advertisement impressions. {ok, _} = lasp:read(Counter, 5), %% Remove the advertisement. {ok, _} = lasp:update(Ads, {remove, Ad}, Ad), lager:info("Removing ad: ~p", [Ad]).
  102. %% Generate a series of unique identifiers. RovioAdIds = lists:map(fun(_)

    -> druuid:v4() end, lists:seq(1, 10)), lager:info("Rovio Ad Identifiers are: ~p", [RovioAdIds]), TriforkAdIds = lists:map(fun(_) -> druuid:v4() end, lists:seq(1, 10)), lager:info("Trifork Ad Identifiers are: ~p", [TriforkAdIds]), Ids = RovioAdIds ++ TriforkAdIds, lager:info("Ad Identifiers are: ~p", [Ids]), %% Generate Rovio's advertisements. {ok, RovioAds} = lasp:declare(?SET), lists:map(fun(Id) -> %% Generate a G-Counter. {ok, CounterId} = lasp:declare(?COUNTER), %% Add it to the advertisement set. {ok, _} = lasp:update(RovioAds, {add, #ad{id=Id, counter=CounterId}}, undefined) end, RovioAdIds), %% Generate Trifork's advertisements. {ok, TriforkAds} = lasp:declare(?SET), lists:map(fun(Id) -> %% Generate a G-Counter. {ok, CounterId} = lasp:declare(?COUNTER), %% Add it to the advertisement set. {ok, _} = lasp:update(TriforkAds, {add, #ad{id=Id, counter=CounterId}}, undefined) end, TriforkAdIds), %% Union ads. {ok, Ads} = lasp:declare(?SET), ok = lasp:union(RovioAds, TriforkAds, Ads), %% For each identifier, generate a contract. {ok, Contracts} = lasp:declare(?SET), lists:map(fun(Id) -> {ok, _} = lasp:update(Contracts, {add, #contract{id=Id}}, undefined) end, Ids), %% Compute the Cartesian product of both ads and contracts. {ok, AdsContracts} = lasp:declare(?SET), ok = lasp:product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. {ok, AdsWithContracts} = lasp:declare(?SET), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, ok = lasp:filter(AdsContracts, FilterFun, AdsWithContracts), %% Launch a series of client processes, each of which is responsible %% for displaying a particular advertisement. %% Generate a OR-set for tracking clients. {ok, Clients} = lasp:declare(?SET), %% Each client takes the full list of ads when it starts, and reads %% from the variable store. lists:map(fun(Id) -> ClientPid = spawn_link(?MODULE, client, [Id, AdsWithContracts, undefined]), {ok, _} = lasp:update(Clients, {add, ClientPid}, undefined) end, lists:seq(1,5)), %% Launch a server process for each advertisement, which will block %% until the advertisement should be disabled. %% Create a OR-set for the server list. {ok, Servers} = lasp:declare(?SET), %% Get the current advertisement list. {ok, {_, _, AdList0}} = lasp:read(AdsWithContracts), AdList = riak_dt_orset:value(AdList0), %% For each advertisement, launch one server for tracking it's %% impressions and wait to disable. lists:map(fun(Ad) -> ServerPid = spawn_link(?MODULE, server, [Ad, Ads]), {ok, _} = lasp:update(Servers, {add, ServerPid}, undefined) end, AdList),
  103. %% Generate a series of unique identifiers. RovioAdIds = lists:map(fun(_)

    -> druuid:v4() end, lists:seq(1, 10)), lager:info("Rovio Ad Identifiers are: ~p", [RovioAdIds]), TriforkAdIds = lists:map(fun(_) -> druuid:v4() end, lists:seq(1, 10)), lager:info("Trifork Ad Identifiers are: ~p", [TriforkAdIds]), Ids = RovioAdIds ++ TriforkAdIds, lager:info("Ad Identifiers are: ~p", [Ids]), %% Generate Rovio's advertisements. {ok, RovioAds} = lasp:declare(?SET), lists:map(fun(Id) -> %% Generate a G-Counter. {ok, CounterId} = lasp:declare(?COUNTER), %% Add it to the advertisement set. {ok, _} = lasp:update(RovioAds, {add, #ad{id=Id, counter=CounterId}}, undefined) end, RovioAdIds), %% Generate Trifork's advertisements. {ok, TriforkAds} = lasp:declare(?SET), lists:map(fun(Id) -> %% Generate a G-Counter. {ok, CounterId} = lasp:declare(?COUNTER), %% Add it to the advertisement set. {ok, _} = lasp:update(TriforkAds, {add, #ad{id=Id, counter=CounterId}}, undefined) end, TriforkAdIds), %% Union ads. {ok, Ads} = lasp:declare(?SET), ok = lasp:union(RovioAds, TriforkAds, Ads), %% For each identifier, generate a contract. {ok, Contracts} = lasp:declare(?SET), lists:map(fun(Id) -> {ok, _} = lasp:update(Contracts, {add, #contract{id=Id}}, undefined) end, Ids), %% Compute the Cartesian product of both ads and contracts. {ok, AdsContracts} = lasp:declare(?SET), ok = lasp:product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. {ok, AdsWithContracts} = lasp:declare(?SET), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, ok = lasp:filter(AdsContracts, FilterFun, AdsWithContracts), %% Launch a series of client processes, each of which is responsible %% for displaying a particular advertisement. %% Generate a OR-set for tracking clients. {ok, Clients} = lasp:declare(?SET), %% Each client takes the full list of ads when it starts, and reads %% from the variable store. lists:map(fun(Id) -> ClientPid = spawn_link(?MODULE, client, [Id, AdsWithContracts, undefined]), {ok, _} = lasp:update(Clients, {add, ClientPid}, undefined) end, lists:seq(1,5)), %% Launch a server process for each advertisement, which will block %% until the advertisement should be disabled. %% Create a OR-set for the server list. {ok, Servers} = lasp:declare(?SET), %% Get the current advertisement list. {ok, {_, _, AdList0}} = lasp:read(AdsWithContracts), AdList = riak_dt_orset:value(AdList0), %% For each advertisement, launch one server for tracking it's %% impressions and wait to disable. lists:map(fun(Ad) -> ServerPid = spawn_link(?MODULE, server, [Ad, Ads]), {ok, _} = lasp:update(Servers, {add, ServerPid}, undefined) end, AdList),
  104. %% Generate a series of unique identifiers. RovioAdIds = lists:map(fun(_)

    -> druuid:v4() end, lists:seq(1, 10)), lager:info("Rovio Ad Identifiers are: ~p", [RovioAdIds]), TriforkAdIds = lists:map(fun(_) -> druuid:v4() end, lists:seq(1, 10)), lager:info("Trifork Ad Identifiers are: ~p", [TriforkAdIds]), Ids = RovioAdIds ++ TriforkAdIds, lager:info("Ad Identifiers are: ~p", [Ids]), %% Generate Rovio's advertisements. {ok, RovioAds} = lasp:declare(?SET), lists:map(fun(Id) -> %% Generate a G-Counter. {ok, CounterId} = lasp:declare(?COUNTER), %% Add it to the advertisement set. {ok, _} = lasp:update(RovioAds, {add, #ad{id=Id, counter=CounterId}}, undefined) end, RovioAdIds), %% Generate Trifork's advertisements. {ok, TriforkAds} = lasp:declare(?SET), lists:map(fun(Id) -> %% Generate a G-Counter. {ok, CounterId} = lasp:declare(?COUNTER), %% Add it to the advertisement set. {ok, _} = lasp:update(TriforkAds, {add, #ad{id=Id, counter=CounterId}}, undefined) end, TriforkAdIds), %% Union ads. {ok, Ads} = lasp:declare(?SET), ok = lasp:union(RovioAds, TriforkAds, Ads), %% For each identifier, generate a contract. {ok, Contracts} = lasp:declare(?SET), lists:map(fun(Id) -> {ok, _} = lasp:update(Contracts, {add, #contract{id=Id}}, undefined) end, Ids), %% Compute the Cartesian product of both ads and contracts. {ok, AdsContracts} = lasp:declare(?SET), ok = lasp:product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. {ok, AdsWithContracts} = lasp:declare(?SET), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, ok = lasp:filter(AdsContracts, FilterFun, AdsWithContracts), %% Launch a series of client processes, each of which is responsible %% for displaying a particular advertisement. %% Generate a OR-set for tracking clients. {ok, Clients} = lasp:declare(?SET), %% Each client takes the full list of ads when it starts, and reads %% from the variable store. lists:map(fun(Id) -> ClientPid = spawn_link(?MODULE, client, [Id, AdsWithContracts, undefined]), {ok, _} = lasp:update(Clients, {add, ClientPid}, undefined) end, lists:seq(1,5)), %% Launch a server process for each advertisement, which will block %% until the advertisement should be disabled. %% Create a OR-set for the server list. {ok, Servers} = lasp:declare(?SET), %% Get the current advertisement list. {ok, {_, _, AdList0}} = lasp:read(AdsWithContracts), AdList = riak_dt_orset:value(AdList0), %% For each advertisement, launch one server for tracking it's %% impressions and wait to disable. lists:map(fun(Ad) -> ServerPid = spawn_link(?MODULE, server, [Ad, Ads]), {ok, _} = lasp:update(Servers, {add, ServerPid}, undefined) end, AdList),
  105. %% Generate a series of unique identifiers. RovioAdIds = lists:map(fun(_)

    -> druuid:v4() end, lists:seq(1, 10)), lager:info("Rovio Ad Identifiers are: ~p", [RovioAdIds]), TriforkAdIds = lists:map(fun(_) -> druuid:v4() end, lists:seq(1, 10)), lager:info("Trifork Ad Identifiers are: ~p", [TriforkAdIds]), Ids = RovioAdIds ++ TriforkAdIds, lager:info("Ad Identifiers are: ~p", [Ids]), %% Generate Rovio's advertisements. {ok, RovioAds} = lasp:declare(?SET), lists:map(fun(Id) -> %% Generate a G-Counter. {ok, CounterId} = lasp:declare(?COUNTER), %% Add it to the advertisement set. {ok, _} = lasp:update(RovioAds, {add, #ad{id=Id, counter=CounterId}}, undefined) end, RovioAdIds), %% Generate Trifork's advertisements. {ok, TriforkAds} = lasp:declare(?SET), lists:map(fun(Id) -> %% Generate a G-Counter. {ok, CounterId} = lasp:declare(?COUNTER), %% Add it to the advertisement set. {ok, _} = lasp:update(TriforkAds, {add, #ad{id=Id, counter=CounterId}}, undefined) end, TriforkAdIds), %% Union ads. {ok, Ads} = lasp:declare(?SET), ok = lasp:union(RovioAds, TriforkAds, Ads), %% For each identifier, generate a contract. {ok, Contracts} = lasp:declare(?SET), lists:map(fun(Id) -> {ok, _} = lasp:update(Contracts, {add, #contract{id=Id}}, undefined) end, Ids), %% Compute the Cartesian product of both ads and contracts. {ok, AdsContracts} = lasp:declare(?SET), ok = lasp:product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. {ok, AdsWithContracts} = lasp:declare(?SET), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, ok = lasp:filter(AdsContracts, FilterFun, AdsWithContracts), %% Launch a series of client processes, each of which is responsible %% for displaying a particular advertisement. %% Generate a OR-set for tracking clients. {ok, Clients} = lasp:declare(?SET), %% Each client takes the full list of ads when it starts, and reads %% from the variable store. lists:map(fun(Id) -> ClientPid = spawn_link(?MODULE, client, [Id, AdsWithContracts, undefined]), {ok, _} = lasp:update(Clients, {add, ClientPid}, undefined) end, lists:seq(1,5)), %% Launch a server process for each advertisement, which will block %% until the advertisement should be disabled. %% Create a OR-set for the server list. {ok, Servers} = lasp:declare(?SET), %% Get the current advertisement list. {ok, {_, _, AdList0}} = lasp:read(AdsWithContracts), AdList = riak_dt_orset:value(AdList0), %% For each advertisement, launch one server for tracking it's %% impressions and wait to disable. lists:map(fun(Ad) -> ServerPid = spawn_link(?MODULE, server, [Ad, Ads]), {ok, _} = lasp:update(Servers, {add, ServerPid}, undefined) end, AdList),
  106. %% Generate a series of unique identifiers. RovioAdIds = lists:map(fun(_)

    -> druuid:v4() end, lists:seq(1, 10)), lager:info("Rovio Ad Identifiers are: ~p", [RovioAdIds]), TriforkAdIds = lists:map(fun(_) -> druuid:v4() end, lists:seq(1, 10)), lager:info("Trifork Ad Identifiers are: ~p", [TriforkAdIds]), Ids = RovioAdIds ++ TriforkAdIds, lager:info("Ad Identifiers are: ~p", [Ids]), %% Generate Rovio's advertisements. {ok, RovioAds} = lasp:declare(?SET), lists:map(fun(Id) -> %% Generate a G-Counter. {ok, CounterId} = lasp:declare(?COUNTER), %% Add it to the advertisement set. {ok, _} = lasp:update(RovioAds, {add, #ad{id=Id, counter=CounterId}}, undefined) end, RovioAdIds), %% Generate Trifork's advertisements. {ok, TriforkAds} = lasp:declare(?SET), lists:map(fun(Id) -> %% Generate a G-Counter. {ok, CounterId} = lasp:declare(?COUNTER), %% Add it to the advertisement set. {ok, _} = lasp:update(TriforkAds, {add, #ad{id=Id, counter=CounterId}}, undefined) end, TriforkAdIds), %% Union ads. {ok, Ads} = lasp:declare(?SET), ok = lasp:union(RovioAds, TriforkAds, Ads), %% For each identifier, generate a contract. {ok, Contracts} = lasp:declare(?SET), lists:map(fun(Id) -> {ok, _} = lasp:update(Contracts, {add, #contract{id=Id}}, undefined) end, Ids), %% Compute the Cartesian product of both ads and contracts. {ok, AdsContracts} = lasp:declare(?SET), ok = lasp:product(Ads, Contracts, AdsContracts), %% Filter items by join on item it. {ok, AdsWithContracts} = lasp:declare(?SET), FilterFun = fun({#ad{id=Id1}, #contract{id=Id2}}) -> Id1 =:= Id2 end, ok = lasp:filter(AdsContracts, FilterFun, AdsWithContracts), %% Launch a series of client processes, each of which is responsible %% for displaying a particular advertisement. %% Generate a OR-set for tracking clients. {ok, Clients} = lasp:declare(?SET), %% Each client takes the full list of ads when it starts, and reads %% from the variable store. lists:map(fun(Id) -> ClientPid = spawn_link(?MODULE, client, [Id, AdsWithContracts, undefined]), {ok, _} = lasp:update(Clients, {add, ClientPid}, undefined) end, lists:seq(1,5)), %% Launch a server process for each advertisement, which will block %% until the advertisement should be disabled. %% Create a OR-set for the server list. {ok, Servers} = lasp:declare(?SET), %% Get the current advertisement list. {ok, {_, _, AdList0}} = lasp:read(AdsWithContracts), AdList = riak_dt_orset:value(AdList0), %% For each advertisement, launch one server for tracking it's %% impressions and wait to disable. lists:map(fun(Ad) -> ServerPid = spawn_link(?MODULE, server, [Ad, Ads]), {ok, _} = lasp:update(Servers, {add, ServerPid}, undefined) end, AdList),
  107. RELATED WORK 107

  108. DISTRIBUTED OZ RELATED WORK 108

  109. DERFLOWL RELATED WORK 109

  110. BLOOML RELATED WORK 110

  111. LVARS RELATED WORK 111

  112. D-STREAMS RELATED WORK 112

  113. SUMMINGBIRD RELATED WORK 113

  114. FUTURE WORK 114

  115. INVARIANT PRESERVATION FUTURE WORK 115

  116. CAUSAL+ CONSISTENCY FUTURE WORK 116

  117. ORSWOT OPTIMIZATION FUTURE WORK 117

  118. DELTA STATE-CRDTs FUTURE WORK 118

  119. OPERATION-BASED CRDTs FUTURE WORK 119

  120. DEFORESTATION FUTURE WORK 120

  121. GITHUB.COM/CMEIKLEJOHN/LASP SOURCE 121

  122. DERFLOW DISTRIBUTED DETERMINISTIC DATAFLOW PROGRAMMING FOR ERLANG ERLANG WORKSHOP 2014

    122
  123. LASP A LANGUAGE FOR DISTRIBUTED, EVENTUALLY CONSISTENT COMPUTATIONS WITH CRDTs

    PAPOC / EUROSYS 2015 123
  124. SYNCFREE IS A EUROPEAN RESEARCH PROJECT TAKING PLACE FOR 3

    YEARS, STARING OCTOBER 2013, AND IS FUNDED BY THE EUROPEAN UNION, GRANT AGREEMENT N°609551. SYNCFREE 124
  125. FIN 125