The Next 700 Asynchronous Programming Models

1. The Next 700 Asynchronous Programming Models Philipp Haller Typesafe, Inc. 1 @philippkhaller
2. What do these companies have in common? • They build systems using the actor model on the JVM • What’s interesting about this? • No special support for actors in Scala • Still, programming with actors in Scala is very natural 2
3. What this talk is about • Scala as a growable language for asynchronous programming • Disclaimer: • I will talk about asynchronous and concurrent programming • In fact, mostly about concurrent programming 3
4. Outline • Why a growable language for asynchronous programming? • Scala as a growable language • Future directions 4
5. Why a growable language for asynchronous programming? • Actors, agents, communicating event-loops • CML • Futures/promises • Reactive Extensions (Rx) • Async/await, async-finish • STM • ... + Generalizations Which one is going to “win”? 5
6. Why a growable language for asynchronous programming? (cont’d) Enables multiple high-level libraries embedded in the same host language • Richer programming system • Impacts type systems, static analysis + verification • Impacts language design • Performance comparisons between different libraries more meaningful 6
7. Why a growable language for asynchronous programming? (cont’d) Simplifies research • Library extensions vs. language extensions • Experimental evaluation on real code 7
8. Outline • Why a growable language for asynchronous programming? • Scala as a growable language • Future directions 8
9. Outline • Why a growable language for asynchronous programming? • Lessons learnt • Limits of growability • Actors & futures • Async/await • Future directions • RAY (or, direct-style Rx) 9
10. Scala as a growable language • We (EPFL + Typesafe) have used Scala to build a variety of asynchronous programming models • It has worked surprisingly well • There are limits 10
11. Asynchronous programming landscape 11 Actors/Akka Futures Joins FlowPools Async/await Rasync-await- yield (RAY)
12. Lessons Learnt 12
13. Lesson 1: Scala enables new API designs • Unique integration of features: lots of API designs to be discovered • Objects + functions • Pattern matching, extractors • Implicits • ... • API design requires experience 13
14. Lesson 2: Simplicity • Simplicity critical for success • Semantically and in terms of interface • Complexity creates skepticism • Sophisticated techniques only feasible if purely internal • Zero (known) bugs 14
15. Lesson 3: Combining libraries • Integration of multiple concurrency libraries natural (to some extent) • Developers will do it anyway (see ECOOP’13) • Have to play nicely together 15
16. class ActorWithTasks(tasks: ...) extends Actor { ... def receive = { case TaskFor(workers) => val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => sender ! seq.mkString(",") } } } 16 A first attempt
17. class ActorWithTasks(tasks: ...) extends Actor { ... def receive = { case TaskFor(workers) => val from = sender val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => from ! seq.mkString(",") } } } 17 The fixed version
18. Lesson 4: The JVM as a platform • The JVM is a great platform for asynchronous and concurrent programming • Very good performance and scalability • Build upon state-of-the-art libraries and tools (e.g., java.util.concurrent) 18 java.util.concurrent is not a concurrency library, it's a way of life - Doug Lea, Oct 28, 2013 “ ”
19. The JVM as a platform (cont’d) • Knowledge about the JVM invaluable for creating high-performance concurrency libraries • Debugging, profiling, benchmarking, tuning, ... • Java Memory Model 19 With great power comes great responsibility... (Know your tools)
20. Outline • Why a growable language for asynchronous programming? • Lessons learnt • Limits of growability • Actors & futures • Async/await • Future directions • RAY (or, direct-style Rx) 20
21. Futures & Composition: Example • Context: Play Framework • Task: Given two web service requests, when both are completed, return response with the results of both: val futureDOY: Future[Response] = WS.url("http://api.day-of-year/today").get val futureDaysLeft: Future[Response] = WS.url("http://api.days-left/today").get 21
22. Example futureDOY.flatMap { doyResponse => val dayOfYear = doyResponse.body futureDaysLeft.map { daysLeftResponse => val daysLeft = daysLeftResponse.body Ok("" + dayOfYear + ": " + daysLeft + " days left!") } } Using plain Scala futures val respFut = async { val dayOfYear = await(futureDOY).body val daysLeft = await(futureDaysLeft).body Ok("" + dayOfYear + ": " + daysLeft + " days left!") } Using Scala Async 22
23. Async/await • The essence of async/await: 1. A way to spawn an asynchronous computation (async), returning a (first-class) future 2. A way to suspend an asynchronous computation (await) until a future is completed • Result: a direct-style API for asynchronous futures • Practical relevance: F#, C# 5.0, Scala 2.11 23
24. Implementing async/await • Async/await requires ANF + state machine transform (CPS transform) • Macros of Scala 2.10 essential for transforming async { } • Alternative solution: compiler plugin • Library/language boundary blurred 24
25. Using await • Requires a directly-enclosing async { } • Cannot use await • within closures • within local functions/classes/objects • within an argument to a by-name parameter 25
26. Remedy: Combining push+pull async { list.map(x => await(f(x)).toString ) } Future.sequence( list.map(x => async { await(f(x)).toString })) def f(x: A): Future[B] • Existing combinators in Futures API can help! 26
27. Outline • Why a growable language for asynchronous programming? • Scala as a growable language • Future directions 27
28. Challenge output: 7, 1, 8, 3, 5, 2, ... Two input streams with the following values: stream2: 0, 7, 0, 4, 6, 5, ... stream1: 7, 1, 0, 2, 3, 1, ... Create a new output stream that • yields, for each value of stream1, the sum of the previous 3 values of stream1, • except if the sum is greater than some threshold in which case the next value of stream2 should be subtracted. Task: For a threshold of 5, the output stream has the following values: 28
29. Reactive Extensions (Rx) • Asynchronous event streams and push notifications: a fundamental abstraction for web and mobile apps • Typically, event streams have to be scalable, robust, and composable • Examples: Netflix, Twitter, ... • Most popular framework: Reactive Extensions (Rx) • Based on the duality of iterators and observers (Meijer’12) • Cross-platform framework (RxJava, RxJS, ...) • Composition using higher-order functions 29
30. The Essence of Rx trait Observable[T] { def subscribe(obs: Observer[T]): Closable } trait Observer[T] { def onNext(v: T): Unit def onFailure(t: Throwable): Unit def onDone(): Unit } 30
31. Observer[T]: Interactions Erik Meijer: Your mouse is a database. CACM’12 trait Observer[T] { def onNext(v: T): Unit def onFailure(t: Throwable): Unit def onDone(): Unit } 31
32. The Real Power: Combinators flatMap 32
33. Combinators: Example textChanges(textField) .flatMap(word => completions(word)) .subscribe(observeChanges(output)) Observable[Array[Strin g]] def textChanges(tf: JTextField): Observable[String] 33
34. Challenge: Recap output: 7, 1, 8, 3, 5, 2, ... Two input streams with the following values: stream2: 0, 7, 0, 4, 6, 5, ... stream1: 7, 1, 0, 2, 3, 1, ... Create a new output stream that • yields, for each value of stream1, the sum of the previous 3 values of stream1, • except if the sum is greater than some threshold in which case the next value of stream2 should be subtracted. Task: For a threshold of 5, the output stream has the following values: 34
35. Solution using Rx val three = stream1.window(3).map(w => w.reduce(_ + _)) val withIndex = three.zipWithIndex val big = withIndex.filter(_._1 >= 5).zip(stream2).map { case ((l, i), r) => (l - r, i) } val output = withIndex.filter(_._1 < 5).merge(big) sum previous 3 values Requires “window” and “merge” combinators! 35
36. The Problem • Programming with reactive streams suffers from an inversion of control • Purely push-based API • Example: writing stateful combinators is difficult • Hard to use for programmers not comfortable with higher-order functions 36
37. RAY: Idea • Integrate Rx and Async: get the best of both worlds • Introduce variant of async { } to create observables instead of futures => rasync { } • Within rasync { }: enable awaiting events of observables in direct-style • Creating observables means we need a way to yield events from within rasync { } 37
38. RAY: Primitives • rasync[T] { } - create Observable[T] • awaitNextOrDone(obs) - awaits and returns Some(next event of obs), or else if obs has terminated returns None • yieldNext(evt) - yields next event of current observable 38
39. RAY: First Example val forwarder = rasync[Int] { var next: Option[Int] = awaitNextOrDone(stream) while (next.nonEmpty) { yieldNext(next) next = awaitNextOrDone(stream) } } 39
40. Challenge: Recap output: 7, 1, 8, 3, 5, 2, ... Two input streams with the following values: stream2: 0, 7, 0, 4, 6, 5, ... stream1: 7, 1, 0, 2, 3, 1, ... Create a new output stream that • yields, for each value of stream1, the sum of the previous 3 values of stream1, • except if the sum is greater than some threshold in which case the next value of stream2 should be subtracted. Task: For a threshold of 5, the output stream has the following values: 40
41. Solution using RAY val output = rasync[Int] { var window = List(0, 0, 0) var evt = awaitNextOrDone(stream1) while (evt.nonEmpty) { window = window.tail :+ evt.get val next = window.reduce(_ + _) match { case big if big > Threshold => awaitNextOrDone(stream2).map(x => big - x) case small => Some(small) } yieldNext(next) evt = if (next.isEmpty) None else awaitNextOrDone(stream1) } } No additional combinators required! 41
42. RAY: Summary • Generalize async/await from futures to observables • Enables intuitive coordination of streams • Properties: • No need to use higher-order functions • Direct-style API for awaiting stream events • Programmers can leverage their experience with async/await 42
43. Where’s the meat? • Whenever concurrent activities have to wait for events things become tricky • Support from language vs. execution environment • Depending on the waiting pattern suspend +resume can be cheap or expensive (Cilk, X10, ...) • Suspendible computations help! • Exposes limits of growable languages 43
44. Push vs. pull • Thesis: purely push-based programming models will only “get you 80% there” • Fork/join pool vs. simple thread pools • Push-based and pull-based interfaces complement each other • Fundamental property of programming model • Inversion of control 44
45. Isn’t this all way too low-level? • No: • Need a way to implement new programming models efficiently • Benefits also higher-level programming systems • Yes: • It’s all about synchronization constraints! • High-level coordination mechanisms needed • Synchronizers, composable events, 45
46. Conclusion • Asynchronous programming a challenge for growable languages • Impact on programming models, languages, libraries, compilers, execution environments, type systems, static analysis, verification, ... 46
47. 47 Questions? Thank you! Philipp Haller Typesafe, Inc. @philippkhaller