Consistency Types for Distributed Programming Languages Xin Zhao and Philipp Haller KTH Royal Institute of Technology Stockholm, Sweden 32nd Workshop on Languages and Compilers
 for Parallel Computing (LCPC ’19)
 Atlanta, GA, USA, October 22, 2019

Philipp Haller Context: Scalable Distributed Services • Distributed applications – providing scalable services – that run on virtualized cloud infrastructures – in one or more datacenters • Examples:
 Chat services, eCommerce platforms, social media platforms, game servers, etc. 2 Support many concurrent clients!

Philipp Haller Fault-tolerant Distributed Systems Basic problem: multiple clients access shared distributed data concurrently 3 Client 1 Client 2 Client 3 Data Safety goal:
 Never return incorrect result under all non-Byzantine failure conditions, including • network delays, partitions, and • packet loss, duplication, and reordering read write read

Philipp Haller Solution: Replication, Consensus Replicate data, distributed consensus ensures safety 4 Client 1 Client 2 Client 3 Data Safety goal:
 Never return incorrect result under all non-Byzantine failure conditions, including • network delays, partitions, and • packet loss, duplication, and reordering read write read Data Data

Philipp Haller Challenges Replicate data, distributed consensus ensures safety 5 Client 1 Client 2 Client 3 Data • Each update requires distributed consensus • This results in poor performance, poor scalability, and high latency,
 especially in geo-replicated systems read write read Data Data

Philipp Haller Geo-Distribution Challenge • Operating a service in multiple datacenters can improve latency and availability for geographically distributed clients • Geo-distribution directly supported by today's cloud platforms • Challenge: round-trip latency – < 2ms between servers within the same datacenter – up to two orders of magnitude higher between distant datacenters 6 Naive reuse of single-datacenter application architectures and protocols leads to poor performance!

Philipp Haller (Partial) Remedy: Eventual Consistency Eventual consistency promises better availability and performance than strong consistency (= serializing updates in a global total order) • Each update executes at some replica (e.g., geographically closest) without synchronization • Each update is propagated asynchronously to the other replicas • All updates eventually take effect at all replicas, possibly in different orders 7

Philipp Haller Eventual Consistency (2) • Updates are applied without synchronization • Updates/states propagated asynchronously to other replicas • All updates eventually take effect at all replicas, possibly in different orders 8 Image source: Shapiro, Preguica, Baquero, and Zawirski: Conflict-Free Replicated Data Types. SSS 2011

Philipp Haller Strong Eventual Consistency (SEC) • Strong Eventual Consistency (SEC): – leverages mathematical properties that ensure absence of conflict, i.e., commutativity of update merging • A Conflict-Free Replicated Datatype (CRDT)1 provides SEC: – CRDT replicas provably converge to a correct common state – CRDTs remain available and scalable despite high network latency, failures, or network partitioning 9 1 Shapiro, Preguica, Baquero, and Zawirski: Conflict-Free Replicated Data Types. SSS 2011

Philipp Haller Consistency Types: Idea To satisfy a range of performance, scalability, and consistency requirements, provide two different kinds of replicated data types 1. Consistent data types: – Serialize updates in a global total order: sequential consistency – Do not provide availability (in favor of partition tolerance2) 2. Available data types: – Guarantee availability and performance (and partition tolerance) – Weaken consistency: strong eventual consistency 10 2 Gilbert, S., Lynch, N.: Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News 33(2), 51-59 (2002)

Philipp Haller Problem: How to Safely Use Both Consistent and Available Data Within the Same Application? 11 def numberOfSeats(event: ID): Int = { fastRead(event + "num_seats") } def book(event: ID) = { val remaining: Int = numberOfSeats(event) if (remaining >= event.orderNum) paymentProcess(event) else display("Out of seats!") } Result of available but inconsistent operation! Consistent state update! Available read from a single replica Problem: update of consistent state based on inconsistent information!

Philipp Haller Consistency Types in LCD LCD: • A higher-order language with distributed references and consistency types • Values and types annotated with labels indicating their consistency 12 First-class functions Replicated data types • Typed lambda-calculus • ML-style references • Labeled values and types

Philipp Haller Labels and Types • A label l can be . (local), con (consistent), or ava (available) • Labels are attached to both values and types • Labels are ordered: • The label ordering expresses permitted data flow: local → con → ava • Labeled types are covariant in their labels: 13 Subtyping rule for function types: see paper ava → con

Philipp Haller Select Typing Rules • Example 1: t1con := t2ava • Example 2: if xava then tcon := 1con else tcon := 0con 14 Illegal! Illegal!

Philipp Haller Attempted “Fix” 1 15 def numberOfSeats(event: ID): Int@ava = { fastRead(event + "num_seats") } def book(event: ID) = { val remaining: Int@con = numberOfSeats(event) if (remaining >= event.orderNum) paymentProcess(event) else display("Out of seats!") } Int@ava <: Int@con

Philipp Haller Attempted “Fix” 2 16 def numberOfSeats(event: ID): Int@ava = { fastRead(event + "num_seats") } def book(event: ID) = { val remaining: Int@ava = numberOfSeats(event) if (remaining >= event.orderNum) paymentProcess(event) else display("Out of seats!") } Illegal!

Philipp Haller The Real Fix 17 def numberOfSeats(event: ID): Int@con = { consistentRead(event + "num_seats") } def book(event: ID) = { val remaining: Int@con = numberOfSeats(event) if (remaining >= event.orderNum) paymentProcess(event) else display("Out of seats!") } Must strengthen consistency!

Philipp Haller Results: Correctness Properties 1. Proof of type soundness – Reduction of well-typed programs cannot get stuck
 Thus:
 No run-time label violations! 2. Proof of noninterference property – Mutations of available data cannot be observed via consistent data
 Thus:
 Consistent data unaffected by usage of available data! 18

Philipp Haller Conclusion LCD: a higher-order language with replicated types and consistency labels • Consistency types enable safe use of both strongly consistent and available (weakly consistent) data within the same application • Proofs of type soundness and noninterference • Noninterference:
 Cannot observe mutations of available data via consistent data Future work: • Proof of sequential consistency for consistent ops • Practical compiler extension 19 Thanks!