DISTRIBUTED, EVENTUALLY CONSISTENT COMPUTATIONS LASP 1 CHRISTOPHER MEIKLEJOHN (BASHO TECHNOLOGIES, INC.) PETER VAN ROY (UNIVERSITÉ CATHOLIQUE DE LOUVAIN)

MOTIVATION 2

EXPENSIVE / IMPRACTICAL SYNCHRONIZATION IS 3

MOBILE GAMES: SHARED STATE BETWEEN CLIENTS CLIENTS GO OFFLINE 4 http://www.rovio.com/en/news/blog/261/263-million-monthly-active-users-in-december/

DISJOINT STATE AGGREGATED UPSTREAM CLIENTS GO OFFLINE “INTERNET OF THINGS”: 5 Gubbi, Jayavardhana, et al. "Internet of Things (IoT): A vision, architectural elements, and future directions." Future Generation Computer Systems 29.7 (2013): 1645-1660.

NO TOTAL ORDER: REPLICATED SHARED STATE WITH OFFLINE CLIENTS CLIENTS NEED TO MAKE PROGRESS 6 Gilbert, Seth, and Nancy Lynch. "Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services." ACM SIGACT News 33.2 (2002): 51-59.

WALL CLOCKS: UNRELIABLE AT BEST NON-DETERMINISTIC IF USED IN COMPUTATIONS 7 Corbett, James C., et al. "Spanner: Google’s globally distributed database." ACM Transactions on Computer Systems (TOCS) 31.3 (2013): 8.

RECONCILED BY USER CONCURRENCY 8

RA RB 1 3 2 ? ? set(1) set(2) set(3) 9

RA RB 1 3 2 ? ? set(1) set(2) set(3) 10

RA RB 1 3 2 ? ? set(1) set(2) set(3) 11

RA RB 1 3 2 ? ? set(1) set(2) set(3) 12

CRDTs 13

DETERMINISTIC RESOLUTION CRDTs PROVIDE 14

CRDTs: MAPS, SETS, COUNTERS, REGISTERS, GRAPHS DETERMINISTIC RESOLUTION 15

“CORRECT REPLICAS THAT HAVE DELIVERED THE SAME UPDATES HAVE EQUIVALENT STATE” Shapiro, Marc, et al. "Conflict-free replicated data types." Stabilization, Safety, and Security of Distributed Systems. Springer Berlin Heidelberg, 2011. 386-400. STRONG EVENTUAL CONSISTENCY CRDTs REALIZE 16

‘MAX’ REGISTER CRDTs EXAMPLE 17

RA RB 1 3 2 3 3 set(1) set(2) set(3) max(2,3) max(2,3) 18

‘ORSET’ SET CRDTs EXAMPLE 19

RA RB RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) 20

RA RB RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) 21

RA RB RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) 22

RA RB RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) 23

NONTRIVIAL COMPOSITION IS 24

RA RB RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RC) {2} fun(X) -> 2 end {2} {} 25

RA RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RC) {2} fun(X) -> 2 end {2} {} F(RA) {2} fun(X) -> 2 end {2} {2} {2} 26

RA RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RC) {2} fun(X) -> 2 end {2} {} F(RA) {2} fun(X) -> 2 end {2} {2} {2} 27

RA RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RC) {2} fun(X) -> 2 end {2} {} F(RA) {2} fun(X) -> 2 end {2} {2} {2} 28

RA RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RC) {2} fun(X) -> 2 end {2} {} F(RA) {2} fun(X) -> 2 end {2} {2} {2} 29

COMPOSITION: USER OBSERVABLE VALUE VS. STATE METADATA MAPPING IS NONTRIVIAL WITHOUT MAPPING METADATA; UNMERGABLE 30 Brown, Russell, et al. "Riak dt map: A composable, convergent replicated dictionary." Proceedings of the First Workshop on Principles and Practice of Eventual Consistency. ACM, 2014. Conway, Neil, et al. "Logic and lattices for distributed programming." Proceedings of the Third ACM Symposium on Cloud Computing. ACM, 2012. Meiklejohn, Christopher. "On the composability of the Riak DT map: expanding from embedded to multi-key structures." Proceedings of the First Workshop on Principles and Practice of Eventual Consistency. ACM, 2014.

LASP LANGUAGE 31

LASP: DISTRIBUTED RUNTIME IMPLEMENTED AS ERLANG LIBRARY USES RIAK-CORE 32

LASP: CRDTS AS STREAMS OF STATE CHANGES CRDTS CONNECTED BY MONOTONIC PROCESSES MANY TO ONE MAPPING OF CRDTS 33

PRIMITIVE OPERATIONS: MONOTONIC READ, UPDATE FUNCTIONAL: MAP, FILTER, FOLD SET-THEORETIC: PRODUCT, UNION, INTERSECTION LIFTED TO OPERATE OVER METADATA 34

MAP EXAMPLE 35

RA RB RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RB) {2} fun(X) -> 2 end {2} {} (2, {b}, {}) (2, {b}, {b}) {} (2, {a, b}, {b}) (2, {a, b}, {b}) F(RC) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {} {2} (2, {b}, {}) (2, {b}, {b}) (2, {a, b}, {b}) F(RA) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {2} {2} (2, {a}, {}) (2, {a, b}, {}) (2, {a, b}, {b}) 36

RA RB RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RB) {2} fun(X) -> 2 end {2} {} (2, {b}, {}) (2, {b}, {b}) {} (2, {a, b}, {b}) (2, {a, b}, {b}) F(RC) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {} {2} (2, {b}, {}) (2, {b}, {b}) (2, {a, b}, {b}) F(RA) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {2} {2} (2, {a}, {}) (2, {a, b}, {}) (2, {a, b}, {b}) 37

RA RB RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RB) {2} fun(X) -> 2 end {2} {} (2, {b}, {}) (2, {b}, {b}) {} (2, {a, b}, {b}) (2, {a, b}, {b}) F(RC) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {} {2} (2, {b}, {}) (2, {b}, {b}) (2, {a, b}, {b}) F(RA) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {2} {2} (2, {a}, {}) (2, {a, b}, {}) (2, {a, b}, {b}) 38

RA RB RC {1} (1, {a}, {}) {1} (1, {b}, {}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RB) {2} fun(X) -> 2 end {2} {} (2, {b}, {}) (2, {b}, {b}) {} (2, {a, b}, {b}) (2, {a, b}, {b}) F(RC) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {} {2} (2, {b}, {}) (2, {b}, {b}) (2, {a, b}, {b}) F(RA) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {2} {2} (2, {a}, {}) (2, {a, b}, {}) (2, {a, b}, {b}) 39

AD COUNTER 40

AD COUNTER: TRACKS AD IMPRESSIONS PUSHES ADVERTISEMENTS TO THE CLIENT DISABLES AD AT 50,000+ IMPRESSIONS CLIENTS DISPLAY ADS WHEN OFFLINE 41

INFORMATION FLOW AD COUNTER 42

Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter Riot Ad Counter Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 43

Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter Riot Ad Counter Contracts Riot Ads Rovio Ads Produ Read 50,000 Remove Increment Union 44

Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter Riot Ad Counter Contracts Ads Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT 45

Ads Contracts Ads Contracts Ads With Contracts ds Ads Filter Product Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT 46

Ads ntracts Ads Contracts Ads With Contracts Filter Product Read Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT 47

Ads Contracts Ads With Contracts Filter Read Operation Maintained CRDT Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 48

Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter Riot Ad Counter Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 49

Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter Riot Ad Counter Contracts Ads Contracts Riot Ads Rovio Ads Product Read 50,000 Remove Increment Union Lasp Operation User-Maintained C Lasp-Maintained C 50

Ads Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 51

INFORMATION FLOW: MONOTONIC METADATA TO PREVENT DUPLICATE PROPAGATION 52

DISTRIBUTION AD COUNTER 53

Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter Riot Ad Counter Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 54

Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter Riot Ad Counter Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 55

Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter Riot Ad Counter Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 56

Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter Riot Ad Counter Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 57

DISTRIBUTION BOUNDARIES: ARBITRARY ALLOWS COMPOSITION OF ENTIRE SYSTEM 58

RELATED WORK: DISTRIBUTED OZ DERFLOW, DERFLOW L, BLOOM L LVARS D-STREAMS, SUMMINGBIRD 59

FUTURE WORK: DELTA, OPERATION-BASED CRDTS RESERVATIONS, ESCROW ADDITIONAL CRDTS: ORSWOT, MAP 60

QUESTIONS? 61