memory • 2+ incompatible points of view • *x > 0 vs. *x >= 0 • One view may invalidate another’s assumptions • Decades spent on flexible ways to manage this problem • My research: A new approach based on describing interference between aliases • Local, function, and thread interference is via aliases 5
Proving data race freedom • Proving (sequential) data structure invariants • Proving concurrent (lock-free) data structure invariants • Proving functional correctness of data structure operations • Proving data structure invariants for actual Haskell programs 7
Only a few component ideas, all familiar • Flexible precision and rigor • Support serial and concurrent programs • From weak (e.g., readonly) to strong (e.g., functional correctness) properties • From informal (whiteboard) to formal (proof) • Handle gradual verification • Verify pieces of a program in context, not all-or- nothing • Support iterative refinement of program properties 8
assertion P is stable w.r.t relation R if: – P is preserved by any action in R – σ⊨P ∧ (σ,σ’)∈R ⇒ σ’⊨P – x=0⊨x≥0 ∧(x=0,x=1)∈inc(x) ⇒ x=1⊨x≥0 • Stable assertions will not be invalidated by other threads! Interference Summaries Union
• Wickerson et al. verify UNIXv7 malloc (ESOP’10) • This dissertation (portions in PLDI’13) • Recently: Militao et al., Rely-Guarantee Protocols (to appear at ECOOP’14) • Cause: Historical view that sequential and concurrent verification are inherently different
Holding a reference grants certain abilities • Different references: possibly different capabilities! • Successful mental model • Already used; next JS spec adds support (proxies) • Simple cases already work • Reference immutability, ownership types, object capability systems… 13
• My work primarily formalizing and proving • Use Reference Immutability • Per-reference permissions allow or prohibit mutation • Permissions on aliases must be compatible: e.g., no writable alias to immutable data • Exploit mutation and aliasing bounds for race freedom • Results • Data race free language (+ generics, borrowing, & more) • Sequential tool ! concurrency (2 simple type rules) • Concurrent proof structure ! sequential concepts! 15
and Pottier (TLDI’11) • Simple to explain • Represents larger challenges: • Enforce state usage over time • Restrict how state changes, not verify the increment function • Infinite state • Little prior work handles this • Pretty complicated • We can specify and enforce this • Simple specification, no abstraction required 18
Rely R: summary of alias behavior • Guarantee G: permitted local behavior • Unary Relations specify invariants • Predicate P: refinement of data • P preserved by (stable w.r.t.) R • Global alias invariant: For any aliases x and y, x.G => y.R and vice versa • Enforced with every new alias creation! • ref T ≡ ref{T|any}{havoc, havoc} • Can encode reference immutability, typestate, related systems 20 ref{τ|P}[R,G] standard reference (≈ T* in C) Rely (e.g. ==) Guarantee (e.g.≤) Predicate (e.g. >0)
Predicate can talk about heaps… • Nested rely relations? • Interior pointers can’t surprise “root” pointers • Nested guarantees for restricted aliases? • Propagate restrictions at dereference • How much heap can we restrict? • What’s reachable from referent 22 x : ref{ListNode | isSorted } [ updateOK , readonly ] next val … G allows insert … X
So far: small programs • Later: lock-free data structures, Haskell • Correctness (Soundness) • Accepts only correct programs • Requires a proof: • Always, the refinement on each reference is true in the current heap 24
useful • Verifies interesting examples • Interference is the right model for aliasing • Applied concurrency verification ideas to aliasing • Current developer reasoning is subtle • RGrefs roughly formalize what devs already think about • Informal reasoning can be made precise • Informal pitfalls highlighted 25
logic with rely and guarantee for thread actions: R,G⊢{P} C {Q} • To make RGrefs useful for concurrency: 1. Fix atomicity assumptions in first revision 2. Add concurrency primitives • Results • Concurrent reasoning is incremental over sequential • RGrefs strong enough for new general results • Proof structure simplified by considering interference 27
push/pop 1 element • Representation is acyclic • Michael-Scott Queue • Simplified for exposition • Enqueue / dequeue 1 element • Representation is acyclic • Lock-free Union Find • Parent chains acyclic, parent rank decreases • All ops are union or path compression • Elements in the same set remain in same set 29
incremental refinement of sequential case • RGrefs strong enough for new developments • Verifying lock-free union find invariants • Proof by embedding into Views Framework • As in reference immutability work • Synthesize thread interference from alias interference 30
• Interleave local actions and interference • Loop(R!read)!R!increment!R • Simplify abstract trace: R is a rely from a reference! • Loop(R!read)!R!increment!R • Loop(R)!R!increment!R • R!R!increment!R • R!increment!R • This is a spec! • Examples: • Counter • Treiber stack 32
Haskell • Haskell + Refinements + SMT + Rely-Guarantee • Results • RGRef implementation for a real language • Experience with real-world code • Experience with inference • Experience with automation (SMT) • RGRefs nicely complement general refinement types 34
data RGRef a <p :: a -> Prop, r :: a -> a -> Prop > = Wrap (rgref_ref :: IORef a<p>) @-}! RGRef a = Wrap (IORef a) ! " {-@ newRGRef :: forall <p :: a -> Prop, r :: a -> a -> Prop >.! e:a<p> -> ! e2:a<r e> -> ! f:(x:a<p> -> y:a<r x> -> {v:a<p> | (v = y)}) -> IO (RGRef <p, r> a) @-} ! newRGRef :: a -> a -> (a -> a -> a) -> IO (RGRef a)! newRGRef e e2 stabilityPf = do { r <- newIORef e; return (Wrap r) } ! " {-@ assume readRGRef :: forall <p :: a -> Prop, r :: a -> a -> Prop >. ! x:RGRef<p, r> a -> IO (a<p>) @-} ! readRGRef (Wrap x) = readIORef x ! " {-@ assume writeRGRef :: forall <p :: a -> Prop, r :: a -> a -> Prop>. ! x:(RGRef<p,r> a) -> old:a -> new:a<r old> -> IO () @-}! writeRGRef :: RGRef a -> a -> a -> IO () ! writeRGRef (Wrap x) old new = writeIORef x new! " {-@ modifyRGRef :: forall <p :: a -> Prop, r :: a -> a -> Prop >.! rf:(RGRef<p, r> a) -> ! f:(x:a<p> -> a<r x>) -> ! pf:(x:a<p> -> y:a<r x> -> {v:a<p> | (v = y)}) -> IO () @-}! modifyRGRef :: RGRef a -> (a -> a) -> (a -> a -> a) -> IO () ! modifyRGRef (Wrap x) f pf = modifyIORef x (\ v -> pf v (f v)) !
:: x:{v:Int | v > 0 } -> ! y:{v:Int | x <= v } -> {v:Int | ((v = y) && (v > 0)) } @-} stable_monocount :: Int -> Int -> Int! stable_monocount x y = y -- Monotonically increasing counter! {-@ alloc_counter :: ! () -> IO (RGRef<{\x -> x > 0}, {\x y -> x <= y}> Int) @-} alloc_counter :: () -> IO (RGRef Int) alloc_counter _ = newRGRef 1 3 stable_monocount! {-@ inc_counter :: ! RGRef<{\x -> x > 0}, {\x y -> x <= y}> Int -> IO () @-} inc_counter :: RGRef Int -> IO () inc_counter r = modifyRGRef r (\x -> x + 1) stable_monocount! -- the stability proof is used as a cast! 37
fallback for STM Haskell (slightly modified) • Verified undo log is only appended to (on the right end) • Lock-free Linked List • GHC test case • Append, delete, lookup modify correctly • Logical and physical deletion separated, preserve link structure • Observed refinement: A logically deleted node is immutable 39
retaining utility • First order logic • Only local heap access + exploiting frozen cells • RGRefs are amenable to inference • RGRefs are useful for actual Haskell code 40
powerful approach to verifying programs; • 4 prototypes, verified sophisticated examples • It reduces the distance between verifying sequential and concurrent programs • Sequential and concurrent versions nearly the same, for metatheory, code, and proofs 43