SyncFree (FP7) Large-scale computation without synchronization Christopher Meiklejohn Erlang Workshop 2015, ICFP

Background

• Resolving concurrent modifications
 Identify and resolve concurrent operations on replicated data • Geo-replication
 More concurrency; coordination even less viable • Semantic resolution
 Resolution by “user”; not intuitive
 Dynamo (SOSP ‘07)

CRDTs

• “Conflict-Free” Replicated Data Types
 Deterministic resolution under concurrent operations • Types
 Counters, Sets, Maps, Flags, Registers, Graphs • Implementations
 Riak DT (Riak KV, Antidote, Lasp, Plumtree)
 SwiftCloud

• SyncFree (FP7)
 Large scale, synchronization free, computation. • Academic Partners
 Inria, Koç Üniversitesi, Universidade Nova de Lisboa, TU Kaiserslautern, Université catholique de Louvain • Industry Partners
 Basho Technologies, Trifork, Rovio Entertainment, Erlang Solutions

Example

RA RB RC

RA RB RC {1} (1, {a}, {}) add(1)

RA RB RC {1} (1, {a}, {}) add(1) {1} (1, {b}, {}) add(1)

RA RB RC {1} (1, {a}, {}) add(1) {1} (1, {b}, {}) add(1) {} (1, {b}, {b}) remove(1)

RA RB RC {1} (1, {a}, {}) add(1) {1} (1, {b}, {}) add(1) {} (1, {b}, {b}) remove(1) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b})

Challenges

Atomicity • Goal
 How can we guarantee changes to independent objects appear atomically? • Composition (Brown et al., PaPEC ’14)
 Riak DT’s composable map: arbitrary* composition of data types • Transactions (Antidote, SwiftCloud Middleware ’15)
 Weak transactions providing causal consistency

Invariant Preservation • Goal
 How can we enforce global system invariants? • Explicit Consistency (Balegas et al., EuroSys ’15)
 Application-specific invariants maintained through violation-avoidance or invariant-repair • RPB (Najafzadeh et al., DCC ’13)
 Extension to red/blue consistency for transactions that can be executed with “reservations”

Computation • Goal
 How can we write computations with CRDTs that result in correct programs? • Lasp (Meiklejohn et al., PPDP ’15)
 Dataflow programming model that uses CRDTs as the primary data abstraction • C-CRDTs (Navalho et al., PaPoC ’15)
 Computational CRDTs that “compute” results as part of their merge functions

Optimization • Goal
 How can we reduce the state we need to ship and perform metadata reduction? • PO-Log (Baquero et al., DAIS ’14)
 Partially ordered log of operations disseminated through tagged reliable broadcast • Delta-State CRDTs (Almeida et al., PaPEC ’14)
 State reduction techniques for using state-based CRDTs

Verification • Goal
 How can we verify the correctness of CRDTs? • Specification of CRDTs (Zeller et al., IFIP ’14)
 Isabelle/HOL specification of CRDTs from the original Inria report • Specification of applications
 Specifications of industry use cases using TLA+

Evaluation • Goal
 How can we evaluate this at scale? • Use case selection
 Identify industrial partners that can provide realistic use cases (virtual wallet, advertisement counter) • Evaluate at scale
 Using reproducible workloads, run experiments at scale with customers who have deployed Basho’s Riak

Experience

Erlang • Why Erlang?
 Existing deployments of databases in Erlang through Basho’s customer base using Riak • Goal of modularity
 Plug in different modules, share code between Riak and Antidote • In practice, this doesn’t work
 Riak is very complicated, with lots of performance optimizations

Antidote Experience • Great for prototyping
 Distributed Erlang is a huge benefit to prototyping distributed applications • Hard to find experienced developers
 Difficult to find developers with either Erlang or FP experience • Difficulty with cross platform deployment
 Unable to repurpose code for testing in other settings

Lasp Experience • Similar to Antidote
 Same experience with Distributed Erlang and cross platform deployment • Types and Typeclasses
 Implementation would be cleaner; lots of manual type checking by carrying around both type and value.

Questions?