• At the root of several hazards • Example 1: 3 @volatile var x = 0 def m(): Unit = { Future { x = 1 } Future { x = 2 } .. // does not access x } What’s the value of x when an invocation of m returns?
not always a problem! • Example 2: 4 val set = Set.empty[Int] Future { set.put(1) } set.put(2) Eventually, set contains both 1 and 2, always Bottom line: it depends on the datatype
and its operations! • Example 3: 5 val set = Set.empty[Int] Future { set.put(1) } Future { if (set.contains(1)) { .. } } set.put(2) Result depends on schedule!
• Programming model providing static determinism guarantees • Starting from imperative, object-oriented language – pervasive aliasing – global state • Important concerns: expressivity and performance 6
• Extend a future-style programming model with: – Lattice-based datatypes – Quiescence – Resolution of cyclic dependencies • Extend Scala's type system for static safety 7 Crucial for determinism! Increases expressivity!
model Programming model based on two core abstractions: cells and cell completers – Cell = shared “variable” – Cell[K,V]: read-only interface; read values of type V – CellCompleter[K,V]: write values of type V to its associated cell – V must have an instance of a lattice type class 8 Monotonic updates
with user IDs (code simplified) 10 implicit object IntSetLattice extends Lattice[Set[Int]] { val empty = Set() def join(left: Set[Int], right: Set[Int]) = left ++ right } // add a user ID userIDs.putNext(Set(theUserID)) val userIDs = CellCompleter[Set[Int]] Bounded join-semilattice [1] Oliveira, Moors, and Odersky. Type classes as objects and implicits. OOPSLA 2010
results • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities – Manual synchronization (e.g., latches) error-prone 11
• Introduce operation to freeze cells • Attempting to mutate a frozen cell results in a failure • May only read from frozen cells – Ensures only unchangeable values are read – Weakens determinism guarantee 12 "All non-failing executions compute the same cell values." "Quasi- determinism"
alternative approach • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities • Alternative solution: Quiescence • Stronger than quasi-determinism 13
• Intuitively: situation when values of cells are guaranteed not to change any more • Technically: – No concurrent activities ongoing or scheduled which could change values of cells – Detected by the underlying thread pool 14
the example 15 // add a user ID userIDs.putNext(Set(theUserID)) .. val pool = new HandlerPool val userIDs = CellCompleter[Set[Int]](pool) // register handler // upon quiescence: read result value of cell pool.onQuiescent(userIDs.cell) { collectedIDs => .. } Safe to read from cell when pool quiescent!
threshold reads* 16 cell2.whenNext(cell1, Set(v1, v2, v3)) { v => // compute update for cell2 } • When value of a cell "crosses" a threshold set: update another cell – "Crosses" = new value greater than one of the values in the threshold set * non-reactive threshold reads: [2] Kuper, Turon, Krishnaswami, and Newton. Freeze after writing: Quasi-deterministic parallel programming with LVars. POPL 2014
threshold reads 18 • Determinism requires restricting the threshold set • Restriction: Incompatibility of elements of threshold set – v1, v2 incompatible iff LUB(v1, v2) = Top • Concurrent crossings of threshold set due to different elements yield failed executions – Turn potential non-determinism into failure, thus preserving quasi-determinism
of data races: variable capture 19 var x = 0 cell2.whenNext(cell1, Set(1)) { v => NextOutcome(x) } cell3.whenNext(cell1, Set(1)) { v => x = 1 NoOutcome } cell1.putNext(1) [3] Miller, Haller, and Odersky. Spores: a type-based foundation for closures in the age of concurrency and distribution. ECOOP 2014 Solution: use spores [2] to prevent • re-assigning captured variables • capturing mutable, shared data structures
sources of data races 20 class C { def set(x: Int): Unit = Global.f = x def get: Int = Global.f } object Global { var f: Int = 0 } cell2.whenNext(cell1, Set(1)) { v => val c = new C NextOutcome(c.get) } cell3.whenNext(cell1, Set(1)) { v => val c = new C c.set(1) NoOutcome } cell1.putNext(1)
21 • Restrict instantiation within whenNext closures to classes conforming to the object capability model [3] • A class is conformant* ("ocap") iff – its methods only access parameters and this – its methods only instantiate ocap classes – types of fields and method parameters are ocap * simplified [4] Mark S. Miller. Robust Composition: Towards a Unified Approach to Access Control and Concurrency Control. PhD thesis, 2006
practical is the object capability requirement? 22 • How common is object-capability safe code in Scala? • Empirical study of >75,000 SLOC of open-source code
the paper 24 • Formalization: object-oriented core languages • CLC1: type-based notion of object capabilities • CLC2: uniqueness via flow-insensitive permissions • CLC3: concurrent extension • Soundness proof • Isolation theorem for processes with shared heap [5] Haller and Loiko. LaCasa: Lightweight affinity and object capabilities in Scala. OOPSLA 2016
graphs Example: • Static analysis of JVM bytecode • Task: Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 25 A B C “calls”
resolution • What if upon quiescence cells are empty and there is a dependency cycle? • Idea: Pluggable resolution policies • Example: Purity analysis should resolve all cells in cycle to “pure”, since no evidence of impurity 26 A B C
resolution policies • Role of the first type parameter of Cell[K,V] • Example: 27 object PurityKey extends Key[Purity] { def resolve[K <: Key[Purity]](cells: Seq[Cell[K, Purity]]) = cells.map(cell => (cell, Pure)) .. } closed strongly- connected component resolve all cells to Pure
Async: Conclusion • Deterministic concurrent programming model – Extension of imperative, object-oriented base language – Lattices and quiescence for determinism – Resolution of cyclic dependencies – Type system for object capabilities for safety • First experimental results • Ongoing and future work: – Complete formal development – Implement state-of-the-art static analyses 33 Thank you!
phaller
sergeychernyshev
keathley
baasie
stephenakadiri
rakko
iamctodd
chriscoyier
aleyda
minecr
techseoconnect
tanoku
jct
