Taming Concurrency

Concurrency has assumed a center-stage position in modern systems building -- it is a stubborn source of complexity for many programs, but our tools aren't always equal to the job. Scala provides fertile grounds for new (and old) concurrency constructs--actors, futures, & STM among them--and in this talk I'll look at some of these--where they are valuable, but also where they fall short--and examine how we might reconcile this messy, concurrent world of ours with the ideals espoused by functional programming.

Transcript

1. TAMING CONCURRENCY @marius SVSS ’13 Friday, August 2, 13
2. INTRODUCTION Concurrency is a central theme in the design & construction of modern server software; but, it’s notoriously difficult to construct software which demands concurrency. “State of the art” tools—j.u.c.—are fairly primitive, nuts & bolts affairs. There’s a rich history of constructing and composing concurrent programs; let’s explore (some of) it. Scala is fertile grounds for this. Friday, August 2, 13
3. FIRST THINGS FIRST: CONCURRENCY Briefly, a number of processes that communicate (shared variables, synchronization, conditions, queues, channels). Processes execute independently, at their own rate; they interact with each other in arbitrary ways. Requires nondeterminism: “deterministic concurrency” is really parallelism. Friday, August 2, 13
4. THIS CONCURRENT LIFE (Why do we care?) Most practical server software interfaces with a highly concurrent world. Our environment is nondeterministic! The moment you have a timeout, you have concurrency. We can’t control our environment. (Client request arrival rate, latency distribution of backends, rate of executionm etc.) Friday, August 2, 13
5. THIS CONCURRENT LIFE (II) Another way to view it is this: our world (and also our hardware) is concurrent. But our languages are sequential. Concurrency constructs concern themselves with bridging this gap: How do we express concurrent systems in a sequential language? How do we model the world around us? Without this, we don’t stand a chance of reasoning about our software. Friday, August 2, 13
6. C.A.R. HOARE There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies. Friday, August 2, 13
7. FOUR FLAVORS OF CONCURRENCY 1. Imperative Java Concurrency in Practice 2. Functional Dataflow programming, futures 3. Process-oriented Guarded commands, CSP, Actors 4. Channel-oriented Join calculus, Offer/Broker Friday, August 2, 13
8. 1. IMPERATIVE Friday, August 2, 13
9. JAVA CONCURRENCY Really brought multithreading into the mainstream. Threads; shared memory; shared memory model; java.util.concurrent.*. Imperative; “nuts & bolts.” Friday, August 2, 13
10. NUT, MEET BOLT @volatile var cond: Boolean if (cond) synchronized { if (cond) { cond = doSomething() } } Friday, August 2, 13
11. ATOMIC OPS & EXPLICIT, IMMUTABLE STATE val ref: AtomicReference[..] def activate() { ref.get() match { case Idle => if (!ref.compareAndSet(Idle, Active)) activate() case ... => } } Friday, August 2, 13
12. JAVA CONCURRENCY REDUX Emphasizes sharing, with a propagation model. (happens-before.) This maps nicely to the underlying hardware, but as a model it is flawed. Prone to bugs, and difficult to compose. Imperative — the tools are mechanical (“how do I do this?”), not descriptive. Friday, August 2, 13
13. 2. FUNCTIONAL Friday, August 2, 13
14. “DETERMINISTIC CONCURRENCY” Friday, August 2, 13
15. I-STRUCTURES Friday, August 2, 13
16. OZMA: A SCALA-BASED DATAFLOW LANGUAGE Friday, August 2, 13
17. A CONCURRENT MAP IN OZMA def concMap[A, B]( list: List[A], f: A => B ): List[B] = if (list.isEmpty) Nil else thread(f(list.head)) :: concMap(list.tail, f) Friday, August 2, 13
18. DATAFLOW CONCURRENCY Pure dataflow programming demands timing independence and freedom from errors—determinism. Ozma makes use of ports—channels—that interface with the dirty outside world. These introduce nondeterminacy. Tie together several determinate programs with indeterminate inputs. Friday, August 2, 13
19. FUTURES: A COMPROMISE Futures are a container, representing the result of an asynchronous operation. Principled handling of failures as well as successes. Composed in a declarative manner; simple to reason about. Friday, August 2, 13
20. FUTURES: CRAWL A DAG trait Page { … def links: Seq[String] … } def fetch(url: String): Future[Page] def crawl(url: String): Future[Seq[Page]] = fetch(url) flatMap { page => Future.collect( page.links map { u => crawl(u) } ) map { pps => pps.flatten } } Friday, August 2, 13
21. A CONCRETE PROBLEM val f: Future[..] = http.get(“...”) val g: Future[..] = f.within(1.second) ... printf(“%s\n”, Await.result(g)) Friday, August 2, 13
22. FUTURES REDUX Not a channel: producer and consumers are divorced. Asynchronous: the producer doesn’t know when values are consumed. Can’t be used to synchronize independent processes. e.g. a barrier. (You only get happens-before, one way.) Friday, August 2, 13
23. 3. PROCESS-ORIENTED Friday, August 2, 13
24. A SHORT HISTORY A rich and storied field. (Dating back to ‘60s–‘70s.) Formalizations—used primarily to prove properties about programs—inspire useful ways of thinking about, and constructing, programs. We’ve done this before: remember λ- calculus? Friday, August 2, 13
25. Friday, August 2, 13
26. DIJKSTRA’S GUARDED COMMANDS “Guarded Commands, Nondeterminacy and Formal Derivation of Programs” Introduced two nondeterministic compositions—proto-“choice” operators; gives a flavor of concurrency. A formalization construct—used to prove properties. Not meant as real language. Set the flavor for pretty much everything that follows. (You’ll see.) Friday, August 2, 13
27. IF ... FI. -- Compute the max of x, y -- □ is the “choice” operator if x≥y → m := x □ y≥x → m := y fi. Friday, August 2, 13
28. DO ... OD. -- sort q1<=q2<=q3<=q4 q1,q2,q3,q4 := Q1,Q2,Q3,Q4; do q1>q2 → q1,q2 := q2,q1 □ q2>q3 → q2,q3 := q3,q2 □ q3>q4 → q3,q4 := q4,q3 od. Friday, August 2, 13
29. PROGRAM VS. ENVIRONMENT Dijkstra’s Guarded Commands separated programs from their environment. Nondeterministic choice allows the environment to roll some dice. (Or not!) DO..OD. interleaves execution — a kind of proto-process. Friday, August 2, 13
30. Friday, August 2, 13
31. HOARE’S CSP Extends on the ideas of Guarded Commands. Built from two primitives which compose together: processes are independent, sequential, threads of execution; events represent communications (interactions) between these processes. Composite processes are built on top of these using guards. Friday, August 2, 13
32. CSP: EVENTS ! is the event that proposes to send the value to process . ? is the event that proposes to receive the (unbound) value from process . Friday, August 2, 13
33. CSP: PREFIX COMPOSITION Prefix: → This is the process willing to communicate , then proceed with . Friday, August 2, 13
34. COMBINATORS! (ASIDE) Prefix: → This is what we call composition in Scala: , are values, → is a combinator; in Scala we might write val p: Process val a: Event val q: Process = a.prefix(p) Friday, August 2, 13
35. CSP: PARALLEL COMPOSITION Interleaving: ||| The process that behaves like & at the same time. Friday, August 2, 13
36. CSP: CHOICE COMPOSITION Choice: ( → ) □ ( → ) The process that behaves either like or like , depending on which of or is communicated. If both are possible, the environment chooses one. (Mutually exclusive.) Friday, August 2, 13
37. CSP: EVENT SYNCHRONY Friday, August 2, 13
38. CSP: EVENT SYNCHRONY (II) is unblocked y = 123 Process Process ? !123 is now blocked is now blocked !333 ? is unblocked y = 333 ( ||| ) Friday, August 2, 13
39. CSP: SYNCHRONY UNDER COMPOSITION Friday, August 2, 13
40. SYNCHRONY UNDER COMPOSITION Process Process ! () ! () blocked ( |||*[ ? () → v := v+1 □ v>0; ? () → v := v−1]||| ) ! () ! () ! () Process v=1 v=0 blocked v=0 v=1 blocked unblocked unblocked v=0 Friday, August 2, 13
41. EVENTS ARE A BIG DEAL The big idea here is to think about communication as a set of events. The environment ensures that events rendezvous: a read event is not ready unless its corresponding write is. You can think of this as a form of transaction over a communication itself; it is controlled by the environment. Friday, August 2, 13
42. HOARE& DIJKSTRA& GUARDS& EVENTS Friday, August 2, 13
43. ASIDE: ACTORS Actors are also structured around processes. It’s common to assume that actors share lineage with CSP. They don’t really. Actors are asynchronous, which implies that guards are impossible. CSP wouldn’t be CSP without synchrony and guards. Friday, August 2, 13
44. 4. CHANNEL-ORIENTED Friday, August 2, 13
45. PROCESSES: WHY NOT? If we make channels first class, we gain some flexibility. Channels can be passed around—they’re not tied to a process. This was particularly limiting in CSP where processes are lexical. Friday, August 2, 13
46. JOIN CALCULUS Join calculus combines guards with channels. “Chemical reaction” model; “ingredients” are put into the environment. Asynchronous. Define a set of “reactions”—join patterns— that can occur—they consume “ingredients” and produce new ones. They “react” with the environment. Friday, August 2, 13
47. A CONCURRENT STACK IN JOCAML let new_stack () = def state (s) & push (v) = state (v::s) & reply to push or state (x::s) & pop () = state (s) & reply x to pop in spawn state([]); pop, push Friday, August 2, 13
48. CML&OFFER/BROKER Can we create synchronous, composable channels? Idea: compose over events in a manner similar to how CSP composes over processes. An event isn’t active until it is synchronized. Friday, August 2, 13
49. OFFER trait Offer[T] { def sync(): T def map[U](f: T => U): Offer[U] def orElse[U >: T](other: Offer[U]): Offer[U] } object Offer { def choose[T](ofs: Offer[T]*): Offer[T] def select[T](ofs: Offer[T]*): T = choose(ofs:_*).sync() def timeout(timeout: Duration): Offer[Unit] val never: Offer[Nothing] def const[T](x: T): Offer[T] } Friday, August 2, 13
50. WHENCE OFFERS? BROKERS. class Broker[T] { def send(v: T): Offer[Unit] val recv: Offer[T] } Friday, August 2, 13
51. EXAMPLES val b = new Broker[Int] // does nothing; just a value: b.send(123) // blocks until another process // has received: b.send(123).sync() // receive; blocks until // there’s a sender. b.recv.sync() Friday, August 2, 13
52. A SEMAPHORE // *[?() → v := v+1 // □ v>0; ?() → v := v−1] val P, V: Broker[Unit] def sema(v: Int) { Offer.select( V.recv map { () => sema(v+1) }, if (v <= 0) Offer.never else { P.recv map { () => sema(v-1) } } ) } Friday, August 2, 13
53. A RESOURCE POOL val get, put: Broker[T] def pool(q: Queue[T]) { Offer.select( if (q.isEmpty) Offer.never else { val (head, tail) = q.dequeue get.send(head) map { () => pool(tail) } }, put.recv map { t => pool(q enqueue t) } ) } Friday, August 2, 13
54. USING THE POOL val item = get.recv.sync() val timeout = Offer.timeout(1.second) val maybeItem = Offer.select( pool.get.recv map (Some(_)), timeout map { () => None } ) Friday, August 2, 13
55. A RECENT EXAMPLE Store server lists in ZooKeeper; watches tell us when they change. Sometimes ZK is unstable; reluctant to rely on this entirely. Idea: qualify removals by checking if ZK was healthy at the time of the removal. Problem: we only discover health one RTT from the removal. Friday, August 2, 13
56. HEALTHY ZK ZK events RTT Remove host A Filtered events Remove host A Friday, August 2, 13
57. FLAPPING ZK ZK events RTT Remove host A Filtered events ZK unhealthy ZK healthy RTT Remove host A Friday, August 2, 13
58. CONNECTION EVENTS sealed trait Event case Rem(conn: Conn) extends Event case Add(conn: Conn) extends Event Friday, August 2, 13
59. REPRESENTING EVENTS val evIn: Offer[Event] val evOut: Broker[Event] val health: Offer[Health] val rtt: Duration Friday, August 2, 13
60. LOOP; CASE ANALYSIS def loop(q: Queue[(Conn, Time)], h: Health) Friday, August 2, 13
61. REQUEST TO REMOVE def loop(q: Queue[(Conn, Time)], h: Health) = Offer.select( ... evIn map { case Rem(conn) => loop(q enqueue (conn, Time.now+rtt), h) case ev@Add(conn) => evOut.send(ev).sync() loop(q filter { case (c, _) => c != conn}, h) }, ... ) Friday, August 2, 13
62. HEALTH UPDATE def loop(q: Queue[(Conn, Time)], h: Health) = Offer.select( ... health map { case j if h == j => loop(q, h) case Healthy => loop(q map { case (e, _) => (e, Time.now+rtt) }, Healthy) case Unhealthy => loop(q, Unhealthy) }, ... ) Friday, August 2, 13
63. EXPIRE GRACE PERIOD (MAYBE) def loop(q: Queue[(Conn, Time)], h: Health) = Offer.select( ... if (h == Healthy || q.isEmpty) Offer.never else { val ((conn, when), q1) = q.dequeue Offer.timeout(when-Time.now) map { () => evOut.send(Rem(conn)).sync() loop(q1, h) } ... ) Friday, August 2, 13
64. ALL TOGETHER def loop(q: Queue[(Conn, Time)], h: Health) = Offer.select( evIn map { case Rem(conn) => loop(q enqueue (conn, Time.now+rtt), h) case ev@Add(conn) => evOut.send(ev).sync() loop(q filter { case (c, _) => c != conn}, h) }, health map { case j if h == j => loop(q, h) case Unhealthy => loop(q, h) case Healthy => loop(q map { case (e, _) => (e, Time.now+rtt) }, Healthy) } if (h == Healthy || q.isEmpty) Offer.never else { val ((conn, when), q1) = q.dequeue Offer.timeout(when-Time.now) map { () => evOut.send(Rem(conn)).sync() loop(q1, h) } } ) Friday, August 2, 13
65. SUMMARY Concurrency is more than threads & locks There is a rich history and many interesting ideas for structuring concurrent programs. Synchronicity and composition are especially powerful. (And: everyone should read the CSP paper.) Friday, August 2, 13
66. THANKS! com.twitter.concurrent at https://github.com/twitter/util/ @marius marius@twitter.com Friday, August 2, 13