Share
Embed
Lasp: Distributed, Eventually Consistent Computations
PaPoC 2015, co-located with EuroSys 2015
Christopher Meiklejohn
Peter Van Roy
Christopher Meiklejohn
April 21, 2015
Transcript
61 slides
-
DISTRIBUTED, EVENTUALLY CONSISTENT COMPUTATIONS LASP 1 CHRISTOPHER MEIKLEJOHN (BASHO TECHNOLOGIES,
INC.) PETER VAN ROY (UNIVERSITÉ CATHOLIQUE DE LOUVAIN) -
MOBILE GAMES: SHARED STATE BETWEEN CLIENTS CLIENTS GO OFFLINE 4
http://www.rovio.com/en/news/blog/261/263-million-monthly-active-users-in-december/ -
DISJOINT STATE AGGREGATED UPSTREAM CLIENTS GO OFFLINE “INTERNET OF THINGS”:
5 Gubbi, Jayavardhana, et al. "Internet of Things (IoT): A vision, architectural elements, and future directions." Future Generation Computer Systems 29.7 (2013): 1645-1660. -
NO TOTAL ORDER: REPLICATED SHARED STATE WITH OFFLINE CLIENTS CLIENTS
NEED TO MAKE PROGRESS 6 Gilbert, Seth, and Nancy Lynch. "Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services." ACM SIGACT News 33.2 (2002): 51-59. -
WALL CLOCKS: UNRELIABLE AT BEST NON-DETERMINISTIC IF USED IN COMPUTATIONS
7 Corbett, James C., et al. "Spanner: Google’s globally distributed database." ACM Transactions on Computer Systems (TOCS) 31.3 (2013): 8. -
“CORRECT REPLICAS THAT HAVE DELIVERED THE SAME UPDATES HAVE EQUIVALENT
STATE” Shapiro, Marc, et al. "Conflict-free replicated data types." Stabilization, Safety, and Security of Distributed Systems. Springer Berlin Heidelberg, 2011. 386-400. STRONG EVENTUAL CONSISTENCY CRDTs REALIZE 16 -
RA RB 1 3 2 3 3 set(1) set(2) set(3)
max(2,3) max(2,3) 18 -
RA RB RC {1} (1, {a}, {}) {1} (1, {b},
{}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) 20 -
RA RB RC {1} (1, {a}, {}) {1} (1, {b},
{}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) 21 -
RA RB RC {1} (1, {a}, {}) {1} (1, {b},
{}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) 22 -
RA RB RC {1} (1, {a}, {}) {1} (1, {b},
{}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) 23 -
RA RB RC {1} (1, {a}, {}) {1} (1, {b},
{}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RC) {2} fun(X) -> 2 end {2} {} 25 -
RA RC {1} (1, {a}, {}) {1} (1, {b}, {})
{} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RC) {2} fun(X) -> 2 end {2} {} F(RA) {2} fun(X) -> 2 end {2} {2} {2} 26 -
RA RC {1} (1, {a}, {}) {1} (1, {b}, {})
{} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RC) {2} fun(X) -> 2 end {2} {} F(RA) {2} fun(X) -> 2 end {2} {2} {2} 27 -
RA RC {1} (1, {a}, {}) {1} (1, {b}, {})
{} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RC) {2} fun(X) -> 2 end {2} {} F(RA) {2} fun(X) -> 2 end {2} {2} {2} 28 -
RA RC {1} (1, {a}, {}) {1} (1, {b}, {})
{} (1, {b}, {b}) {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RC) {2} fun(X) -> 2 end {2} {} F(RA) {2} fun(X) -> 2 end {2} {2} {2} 29 -
COMPOSITION: USER OBSERVABLE VALUE VS. STATE METADATA MAPPING IS NONTRIVIAL
WITHOUT MAPPING METADATA; UNMERGABLE 30 Brown, Russell, et al. "Riak dt map: A composable, convergent replicated dictionary." Proceedings of the First Workshop on Principles and Practice of Eventual Consistency. ACM, 2014. Conway, Neil, et al. "Logic and lattices for distributed programming." Proceedings of the Third ACM Symposium on Cloud Computing. ACM, 2012. Meiklejohn, Christopher. "On the composability of the Riak DT map: expanding from embedded to multi-key structures." Proceedings of the First Workshop on Principles and Practice of Eventual Consistency. ACM, 2014. -
LASP: CRDTS AS STREAMS OF STATE CHANGES CRDTS CONNECTED BY
MONOTONIC PROCESSES MANY TO ONE MAPPING OF CRDTS 33 -
PRIMITIVE OPERATIONS: MONOTONIC READ, UPDATE FUNCTIONAL: MAP, FILTER, FOLD SET-THEORETIC:
PRODUCT, UNION, INTERSECTION LIFTED TO OPERATE OVER METADATA 34 -
RA RB RC {1} (1, {a}, {}) {1} (1, {b},
{}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RB) {2} fun(X) -> 2 end {2} {} (2, {b}, {}) (2, {b}, {b}) {} (2, {a, b}, {b}) (2, {a, b}, {b}) F(RC) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {} {2} (2, {b}, {}) (2, {b}, {b}) (2, {a, b}, {b}) F(RA) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {2} {2} (2, {a}, {}) (2, {a, b}, {}) (2, {a, b}, {b}) 36 -
RA RB RC {1} (1, {a}, {}) {1} (1, {b},
{}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RB) {2} fun(X) -> 2 end {2} {} (2, {b}, {}) (2, {b}, {b}) {} (2, {a, b}, {b}) (2, {a, b}, {b}) F(RC) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {} {2} (2, {b}, {}) (2, {b}, {b}) (2, {a, b}, {b}) F(RA) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {2} {2} (2, {a}, {}) (2, {a, b}, {}) (2, {a, b}, {b}) 37 -
RA RB RC {1} (1, {a}, {}) {1} (1, {b},
{}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RB) {2} fun(X) -> 2 end {2} {} (2, {b}, {}) (2, {b}, {b}) {} (2, {a, b}, {b}) (2, {a, b}, {b}) F(RC) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {} {2} (2, {b}, {}) (2, {b}, {b}) (2, {a, b}, {b}) F(RA) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {2} {2} (2, {a}, {}) (2, {a, b}, {}) (2, {a, b}, {b}) 38 -
RA RB RC {1} (1, {a}, {}) {1} (1, {b},
{}) {} (1, {b}, {b}) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b}) add(1) add(1) remove(1) F(RB) {2} fun(X) -> 2 end {2} {} (2, {b}, {}) (2, {b}, {b}) {} (2, {a, b}, {b}) (2, {a, b}, {b}) F(RC) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {} {2} (2, {b}, {}) (2, {b}, {b}) (2, {a, b}, {b}) F(RA) {2} fun(X) -> 2 end (2, {a, b}, {b}) {2} {2} {2} (2, {a}, {}) (2, {a, b}, {}) (2, {a, b}, {b}) 39 -
AD COUNTER: TRACKS AD IMPRESSIONS PUSHES ADVERTISEMENTS TO THE CLIENT
DISABLES AD AT 50,000+ IMPRESSIONS CLIENTS DISPLAY ADS WHEN OFFLINE 41 -
Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter
Riot Ad Counter Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 43 -
Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter
Riot Ad Counter Contracts Riot Ads Rovio Ads Produ Read 50,000 Remove Increment Union 44 -
Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter
Riot Ad Counter Contracts Ads Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT 45 -
Ads Contracts Ads Contracts Ads With Contracts ds Ads Filter
Product Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT 46 -
Ads ntracts Ads Contracts Ads With Contracts Filter Product Read
Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT 47 -
Ads Contracts Ads With Contracts Filter Read Operation Maintained CRDT
Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 48 -
Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter
Riot Ad Counter Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 49 -
Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter
Riot Ad Counter Contracts Ads Contracts Riot Ads Rovio Ads Product Read 50,000 Remove Increment Union Lasp Operation User-Maintained C Lasp-Maintained C 50 -
Ads Contracts Ads Contracts Ads With Contracts Riot Ads Rovio
Ads Filter Product Read 50,000 Remove Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 51 -
Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter
Riot Ad Counter Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 54 -
Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter
Riot Ad Counter Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 55 -
Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter
Riot Ad Counter Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 56 -
Ads Rovio Ad Counter Rovio Ad Counter Riot Ad Counter
Riot Ad Counter Contracts Ads Contracts Ads With Contracts Riot Ads Rovio Ads Filter Product Read 50,000 Remove Increment Read Union Lasp Operation User-Maintained CRDT Lasp-Maintained CRDT Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 Counter1 Counter2 Counter3 57 -
RELATED WORK: DISTRIBUTED OZ DERFLOW, DERFLOW L, BLOOM L LVARS
D-STREAMS, SUMMINGBIRD 59