Declarative, Convergent, Edge Computation Christopher S. Meiklejohn Université catholique de Louvain, Belgium Instituto Superior Técnico, Portugal LIGHT ONE
Distributed Computation • Concurrent to distributed programming Consistency and now partial failure • Enforcing the “single system” illusion Traditional approach for gaining performance, scalability, fault- tolerance while still “easy to program” 10
Distributed Computation • Concurrent to distributed programming Consistency and now partial failure • Enforcing the “single system” illusion Traditional approach for gaining performance, scalability, fault- tolerance while still “easy to program” • Consistency models Contract between the system and the programmer 10
Distributed Computation • Concurrent to distributed programming Consistency and now partial failure • Enforcing the “single system” illusion Traditional approach for gaining performance, scalability, fault- tolerance while still “easy to program” • Consistency models Contract between the system and the programmer • Correctness criteria For both distributed and concurrent programs 10
Distributed Computation • Concurrent to distributed programming Consistency and now partial failure • Enforcing the “single system” illusion Traditional approach for gaining performance, scalability, fault- tolerance while still “easy to program” • Consistency models Contract between the system and the programmer • Correctness criteria For both distributed and concurrent programs • Synchronization as “knob” Models live along a spectrum requiring more or less synchronization 10
Strong Eventual Consistency (SEC) • Convergent Same updates realize the same state • Primary requirement “Eventual” replica-to-replica communication 33
Strong Eventual Consistency (SEC) • Convergent Same updates realize the same state • Primary requirement “Eventual” replica-to-replica communication • Order insensitive! Operations are commutative 33
Strong Eventual Consistency (SEC) • Convergent Same updates realize the same state • Primary requirement “Eventual” replica-to-replica communication • Order insensitive! Operations are commutative • Duplicate insensitive! Operations are idempotent 33
Conflict-Free Replicated Data Types • Many types exist with different properties Sets, counters, registers, flags, maps • Strong Eventual Consistency Instances satisfy SEC property per-object • Bounded join-semilattices Formalized using bounded join-semilattices where the merge operation is the join 42
Lattice Processing • Asynchronous dataflow with streams Combine and transform streams of inputs into streams of outputs • Convergent data structures Data abstraction (inputs/outputs) is the CRDT 60
Lattice Processing • Asynchronous dataflow with streams Combine and transform streams of inputs into streams of outputs • Convergent data structures Data abstraction (inputs/outputs) is the CRDT • Confluence Provides composition that preserves the SEC property 60
Processes • Replicas as monotonic streams Each replica of a CRDT produces a monotonic stream of states • Monotonic processes Read from one or more input replica streams and produce a single output replica stream 77
Processes • Replicas as monotonic streams Each replica of a CRDT produces a monotonic stream of states • Monotonic processes Read from one or more input replica streams and produce a single output replica stream • Inflationary reads Read operation ensures that we only read inflationary updates to replicas 77
Advertisement Counter • Lower-bound invariant Advertisements are paid according to a minimum number of impressions • Clients will go offline Clients have limited connectivity and the system still needs to make progress while clients are offline 104
Advertisement Counter • Lower-bound invariant Advertisements are paid according to a minimum number of impressions • Clients will go offline Clients have limited connectivity and the system still needs to make progress while clients are offline • No lost updates All displayed advertisements should be accounted for, with no lost updates 104
Advertisement Counter • Completely monotonic Disabling advertisements and contracts are all modeled through monotonic state growth • Arbitrary distribution Use of convergent data structures allows computational graph to be arbitrarily distributed 127
Advertisement Counter • Completely monotonic Disabling advertisements and contracts are all modeled through monotonic state growth • Arbitrary distribution Use of convergent data structures allows computational graph to be arbitrarily distributed • Divergence Divergence is a factor of synchronization period, concurrency, and throughput rate 127
Anabranch • Layered approach Cluster membership and state dissemination for large clusters • Delta-state synchronization Efficient incremental state dissemination and anti-entropy mechanism [Almeida et al. 2016] 130
Anabranch • Layered approach Cluster membership and state dissemination for large clusters • Delta-state synchronization Efficient incremental state dissemination and anti-entropy mechanism [Almeida et al. 2016] • Epsilon-invariants Lower-bound invariants, configurable at runtime 130
Anabranch • Layered approach Cluster membership and state dissemination for large clusters • Delta-state synchronization Efficient incremental state dissemination and anti-entropy mechanism [Almeida et al. 2016] • Epsilon-invariants Lower-bound invariants, configurable at runtime • Scalable Demonstrated high scalability in production Cloud environments 130
Layered Approach • Backend Configurable persistence layer depending on application. • Membership Configurable membership protocol which can operate in a client-server or peer-to-peer mode [Leitao et al. 2007] 132
Layered Approach • Backend Configurable persistence layer depending on application. • Membership Configurable membership protocol which can operate in a client-server or peer-to-peer mode [Leitao et al. 2007] • Broadcast (via Gossip, Tree, etc.) Efficient dissemination of both program state and application state via gossip, broadcast tree, or hybrid mode [Leitao et al. 2007] 132
Mobile Phone Distributed Hash Table Application Language Execution KV Store KV Backend Membership Broadcast Layer Client/Server Peer-to-Peer Language and applications.
Mobile Phone Distributed Hash Table Application Language Execution KV Store KV Backend Membership Broadcast Layer Client/Server Peer-to-Peer State dissemination.
Delta-based Dissemination • Delta-state based CRDTs Reduces state transmission for clients • Operate locally Objects are mutated locally; delta’s buffered locally and periodically gossiped 138
Delta-based Dissemination • Delta-state based CRDTs Reduces state transmission for clients • Operate locally Objects are mutated locally; delta’s buffered locally and periodically gossiped • Only fixed number of clients Clients resort to full state synchronization when they’ve been partitioned too long 138
Scalability • 1024+ nodes Demonstrated scalability to 1024 nodes in Amazon cloud computing environment • Modular approach Many of the components built and can be operated outside of Lasp to improve scalability of Erlang 145
Scalability • 1024+ nodes Demonstrated scalability to 1024 nodes in Amazon cloud computing environment • Modular approach Many of the components built and can be operated outside of Lasp to improve scalability of Erlang • Automated and repeatable Fully automated deployment, scenario execution, log aggregation and archival of experimental results 145
Key Points • Synchronization is sometimes not possible Mobile and “Internet of Things” applications make synchronization for replicated state impractical 150
Key Points • Synchronization is sometimes not possible Mobile and “Internet of Things” applications make synchronization for replicated state impractical • Apply synchronization only where required Global invariants, atomic visibility, etc. 150
Key Points • Synchronization is sometimes not possible Mobile and “Internet of Things” applications make synchronization for replicated state impractical • Apply synchronization only where required Global invariants, atomic visibility, etc. • Holistic approach Taking a holistic approach to the design of distributed systems is important for building higher-availability applications 150
Artifacts • Lasp CRDT based programming model: testbed for Loquat http://github.com/lasp-lang/lasp • Partisan TCP-based pluggable membership service offering client/server, static, and HyParView- based protocols https://github.com/lasp-lang/partisan • Plumtree Epidemic broadcast trees for use with Partisan https://github.com/lasp-lang/plumtree 151
Artifacts • Lasp CRDT based programming model: testbed for Loquat http://github.com/lasp-lang/lasp • Partisan TCP-based pluggable membership service offering client/server, static, and HyParView- based protocols https://github.com/lasp-lang/partisan • Plumtree Epidemic broadcast trees for use with Partisan https://github.com/lasp-lang/plumtree • Sprinter Service discovery and deployment for Mesos and Kubernetes https://github.com/lasp-lang/sprinter 151
Artifacts • Lasp CRDT based programming model: testbed for Loquat http://github.com/lasp-lang/lasp • Partisan TCP-based pluggable membership service offering client/server, static, and HyParView- based protocols https://github.com/lasp-lang/partisan • Plumtree Epidemic broadcast trees for use with Partisan https://github.com/lasp-lang/plumtree • Sprinter Service discovery and deployment for Mesos and Kubernetes https://github.com/lasp-lang/sprinter • Types CRDT implementations for Erlang https://github.com/lasp-lang/types 151
cmeiklejohn
kentaitakura
schulta
unzaih
kinakomoti321
masatoto
yuya4
muraoka7
chokkan
yukinoi
mickey_kubo
eumesy
ryou0634
chriscoyier
kastner
pedronauck
philhawksworth
morganepeng
cherdarchuk
cassininazir
sferik
eileencodes
reverentgeek
maltzj
dotmariusz