The Next 700 Asynchronous Programming Models

Slide 1

Slide 1 text

The Next 700 Asynchronous Programming Models Philipp Haller Typesafe, Inc. 1 @philippkhaller

Slide 2

Slide 2 text

What do these companies have in common? • They build systems using the actor model on the JVM • What’s interesting about this? • No special support for actors in Scala • Still, programming with actors in Scala is very natural 2

Slide 3

Slide 3 text

What this talk is about • Scala as a growable language for asynchronous programming • Disclaimer: • I will talk about asynchronous and concurrent programming • In fact, mostly about concurrent programming 3

Slide 4

Slide 4 text

Outline • Why a growable language for asynchronous programming? • Scala as a growable language • Future directions 4

Slide 5

Slide 5 text

Why a growable language for asynchronous programming? • Actors, agents, communicating event-loops • CML • Futures/promises • Reactive Extensions (Rx) • Async/await, async-ﬁnish • STM • ... + Generalizations Which one is going to “win”? 5

Slide 6

Slide 6 text

Why a growable language for asynchronous programming? (cont’d) Enables multiple high-level libraries embedded in the same host language • Richer programming system • Impacts type systems, static analysis + veriﬁcation • Impacts language design • Performance comparisons between diﬀerent libraries more meaningful 6

Slide 7

Slide 7 text

Why a growable language for asynchronous programming? (cont’d) Simpliﬁes research • Library extensions vs. language extensions • Experimental evaluation on real code 7

Slide 8

Slide 8 text

Outline • Why a growable language for asynchronous programming? • Scala as a growable language • Future directions 8

Slide 9

Slide 9 text

Outline • Why a growable language for asynchronous programming? • Lessons learnt • Limits of growability • Actors & futures • Async/await • Future directions • RAY (or, direct-style Rx) 9

Slide 10

Slide 10 text

Scala as a growable language • We (EPFL + Typesafe) have used Scala to build a variety of asynchronous programming models • It has worked surprisingly well • There are limits 10

Slide 11

Slide 11 text

Asynchronous programming landscape 11 Actors/Akka Futures Joins FlowPools Async/await Rasync-await- yield (RAY)

Slide 12

Slide 12 text

Lessons Learnt 12

Slide 13

Slide 13 text

Lesson 1: Scala enables new API designs • Unique integration of features: lots of API designs to be discovered • Objects + functions • Pattern matching, extractors • Implicits • ... • API design requires experience 13

Slide 14

Slide 14 text

Lesson 2: Simplicity • Simplicity critical for success • Semantically and in terms of interface • Complexity creates skepticism • Sophisticated techniques only feasible if purely internal • Zero (known) bugs 14

Slide 15

Slide 15 text

Lesson 3: Combining libraries • Integration of multiple concurrency libraries natural (to some extent) • Developers will do it anyway (see ECOOP’13) • Have to play nicely together 15

Slide 16

Slide 16 text

class ActorWithTasks(tasks: ...) extends Actor { ... def receive = { case TaskFor(workers) => val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => sender ! seq.mkString(",") } } } 16 A ﬁrst attempt

Slide 17

Slide 17 text

class ActorWithTasks(tasks: ...) extends Actor { ... def receive = { case TaskFor(workers) => val from = sender val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => from ! seq.mkString(",") } } } 17 The ﬁxed version

Slide 18

Slide 18 text

Lesson 4: The JVM as a platform • The JVM is a great platform for asynchronous and concurrent programming • Very good performance and scalability • Build upon state-of-the-art libraries and tools (e.g., java.util.concurrent) 18 java.util.concurrent is not a concurrency library, it's a way of life - Doug Lea, Oct 28, 2013 “ ”

Slide 19

Slide 19 text

The JVM as a platform (cont’d) • Knowledge about the JVM invaluable for creating high-performance concurrency libraries • Debugging, proﬁling, benchmarking, tuning, ... • Java Memory Model 19 With great power comes great responsibility... (Know your tools)

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Futures & Composition: Example • Context: Play Framework • Task: Given two web service requests, when both are completed, return response with the results of both: val futureDOY: Future[Response] = WS.url("http://api.day-of-year/today").get val futureDaysLeft: Future[Response] = WS.url("http://api.days-left/today").get 21

Slide 22

Slide 22 text

Example futureDOY.flatMap { doyResponse => val dayOfYear = doyResponse.body futureDaysLeft.map { daysLeftResponse => val daysLeft = daysLeftResponse.body Ok("" + dayOfYear + ": " + daysLeft + " days left!") } } Using plain Scala futures val respFut = async { val dayOfYear = await(futureDOY).body val daysLeft = await(futureDaysLeft).body Ok("" + dayOfYear + ": " + daysLeft + " days left!") } Using Scala Async 22

Slide 23

Slide 23 text

Async/await • The essence of async/await: 1. A way to spawn an asynchronous computation (async), returning a (ﬁrst-class) future 2. A way to suspend an asynchronous computation (await) until a future is completed • Result: a direct-style API for asynchronous futures • Practical relevance: F#, C# 5.0, Scala 2.11 23

Slide 24

Slide 24 text

Implementing async/await • Async/await requires ANF + state machine transform (CPS transform) • Macros of Scala 2.10 essential for transforming async { } • Alternative solution: compiler plugin • Library/language boundary blurred 24

Slide 25

Slide 25 text

Using await • Requires a directly-enclosing async { } • Cannot use await • within closures • within local functions/classes/objects • within an argument to a by-name parameter 25

Slide 26

Slide 26 text

Remedy: Combining push+pull async { list.map(x => await(f(x)).toString ) } Future.sequence( list.map(x => async { await(f(x)).toString })) def f(x: A): Future[B] • Existing combinators in Futures API can help! 26

Slide 27

Slide 27 text

Outline • Why a growable language for asynchronous programming? • Scala as a growable language • Future directions 27

Slide 28

Slide 28 text

Challenge output: 7, 1, 8, 3, 5, 2, ... Two input streams with the following values: stream2: 0, 7, 0, 4, 6, 5, ... stream1: 7, 1, 0, 2, 3, 1, ... Create a new output stream that • yields, for each value of stream1, the sum of the previous 3 values of stream1, • except if the sum is greater than some threshold in which case the next value of stream2 should be subtracted. Task: For a threshold of 5, the output stream has the following values: 28

Slide 29

Slide 29 text

Reactive Extensions (Rx) • Asynchronous event streams and push notiﬁcations: a fundamental abstraction for web and mobile apps • Typically, event streams have to be scalable, robust, and composable • Examples: Netﬂix, Twitter, ... • Most popular framework: Reactive Extensions (Rx) • Based on the duality of iterators and observers (Meijer’12) • Cross-platform framework (RxJava, RxJS, ...) • Composition using higher-order functions 29

Slide 30

Slide 30 text

The Essence of Rx trait Observable[T] { def subscribe(obs: Observer[T]): Closable } trait Observer[T] { def onNext(v: T): Unit def onFailure(t: Throwable): Unit def onDone(): Unit } 30

Slide 31

Slide 31 text

Observer[T]: Interactions Erik Meijer: Your mouse is a database. CACM’12 trait Observer[T] { def onNext(v: T): Unit def onFailure(t: Throwable): Unit def onDone(): Unit } 31

Slide 32

Slide 32 text

The Real Power: Combinators ﬂatMap 32

Slide 33

Slide 33 text

Combinators: Example textChanges(textField) .flatMap(word => completions(word)) .subscribe(observeChanges(output)) Observable[Array[Strin g]] def textChanges(tf: JTextField): Observable[String] 33

Slide 34

Slide 34 text

Challenge: Recap output: 7, 1, 8, 3, 5, 2, ... Two input streams with the following values: stream2: 0, 7, 0, 4, 6, 5, ... stream1: 7, 1, 0, 2, 3, 1, ... Create a new output stream that • yields, for each value of stream1, the sum of the previous 3 values of stream1, • except if the sum is greater than some threshold in which case the next value of stream2 should be subtracted. Task: For a threshold of 5, the output stream has the following values: 34

Slide 35

Slide 35 text

Solution using Rx val three = stream1.window(3).map(w => w.reduce(_ + _)) val withIndex = three.zipWithIndex val big = withIndex.filter(_._1 >= 5).zip(stream2).map { case ((l, i), r) => (l - r, i) } val output = withIndex.filter(_._1 < 5).merge(big) sum previous 3 values Requires “window” and “merge” combinators! 35

Slide 36

Slide 36 text

The Problem • Programming with reactive streams suﬀers from an inversion of control • Purely push-based API • Example: writing stateful combinators is diﬃcult • Hard to use for programmers not comfortable with higher-order functions 36

Slide 37

Slide 37 text

RAY: Idea • Integrate Rx and Async: get the best of both worlds • Introduce variant of async { } to create observables instead of futures => rasync { } • Within rasync { }: enable awaiting events of observables in direct-style • Creating observables means we need a way to yield events from within rasync { } 37

Slide 38

Slide 38 text

RAY: Primitives • rasync[T] { } - create Observable[T] • awaitNextOrDone(obs) - awaits and returns Some(next event of obs), or else if obs has terminated returns None • yieldNext(evt) - yields next event of current observable 38

Slide 39

Slide 39 text

RAY: First Example val forwarder = rasync[Int] { var next: Option[Int] = awaitNextOrDone(stream) while (next.nonEmpty) { yieldNext(next) next = awaitNextOrDone(stream) } } 39

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Solution using RAY val output = rasync[Int] { var window = List(0, 0, 0) var evt = awaitNextOrDone(stream1) while (evt.nonEmpty) { window = window.tail :+ evt.get val next = window.reduce(_ + _) match { case big if big > Threshold => awaitNextOrDone(stream2).map(x => big - x) case small => Some(small) } yieldNext(next) evt = if (next.isEmpty) None else awaitNextOrDone(stream1) } } No additional combinators required! 41

Slide 42

Slide 42 text

RAY: Summary • Generalize async/await from futures to observables • Enables intuitive coordination of streams • Properties: • No need to use higher-order functions • Direct-style API for awaiting stream events • Programmers can leverage their experience with async/await 42

Slide 43

Slide 43 text

Where’s the meat? • Whenever concurrent activities have to wait for events things become tricky • Support from language vs. execution environment • Depending on the waiting pattern suspend +resume can be cheap or expensive (Cilk, X10, ...) • Suspendible computations help! • Exposes limits of growable languages 43

Slide 44

Slide 44 text

Push vs. pull • Thesis: purely push-based programming models will only “get you 80% there” • Fork/join pool vs. simple thread pools • Push-based and pull-based interfaces complement each other • Fundamental property of programming model • Inversion of control 44

Slide 45

Slide 45 text

Isn’t this all way too low-level? • No: • Need a way to implement new programming models eﬃciently • Beneﬁts also higher-level programming systems • Yes: • It’s all about synchronization constraints! • High-level coordination mechanisms needed • Synchronizers, composable events, 45

Slide 46

Slide 46 text

Conclusion • Asynchronous programming a challenge for growable languages • Impact on programming models, languages, libraries, compilers, execution environments, type systems, static analysis, veriﬁcation, ... 46

Slide 47

Slide 47 text

47 Questions? Thank you! Philipp Haller Typesafe, Inc. @philippkhaller