$30 off During Our Annual Pro Sale. View Details »

The Next 700 Asynchronous Programming Models

Philipp Haller
October 31, 2013
460

The Next 700 Asynchronous Programming Models

Philipp Haller

October 31, 2013
Tweet

Transcript

  1. What do these companies have in common? • They build

    systems using the actor model on the JVM • What’s interesting about this? • No special support for actors in Scala • Still, programming with actors in Scala is very natural 2
  2. What this talk is about • Scala as a growable

    language for asynchronous programming • Disclaimer: • I will talk about asynchronous and concurrent programming • In fact, mostly about concurrent programming 3
  3. Outline • Why a growable language for asynchronous programming? •

    Scala as a growable language • Future directions 4
  4. Why a growable language for asynchronous programming? • Actors, agents,

    communicating event-loops • CML • Futures/promises • Reactive Extensions (Rx) • Async/await, async-finish • STM • ... + Generalizations Which one is going to “win”? 5
  5. Why a growable language for asynchronous programming? (cont’d) Enables multiple

    high-level libraries embedded in the same host language • Richer programming system • Impacts type systems, static analysis + verification • Impacts language design • Performance comparisons between different libraries more meaningful 6
  6. Why a growable language for asynchronous programming? (cont’d) Simplifies research

    • Library extensions vs. language extensions • Experimental evaluation on real code 7
  7. Outline • Why a growable language for asynchronous programming? •

    Scala as a growable language • Future directions 8
  8. Outline • Why a growable language for asynchronous programming? •

    Lessons learnt • Limits of growability • Actors & futures • Async/await • Future directions • RAY (or, direct-style Rx) 9
  9. Scala as a growable language • We (EPFL + Typesafe)

    have used Scala to build a variety of asynchronous programming models • It has worked surprisingly well • There are limits 10
  10. Lesson 1: Scala enables new API designs • Unique integration

    of features: lots of API designs to be discovered • Objects + functions • Pattern matching, extractors • Implicits • ... • API design requires experience 13
  11. Lesson 2: Simplicity • Simplicity critical for success • Semantically

    and in terms of interface • Complexity creates skepticism • Sophisticated techniques only feasible if purely internal • Zero (known) bugs 14
  12. Lesson 3: Combining libraries • Integration of multiple concurrency libraries

    natural (to some extent) • Developers will do it anyway (see ECOOP’13) • Have to play nicely together 15
  13. class ActorWithTasks(tasks: ...) extends Actor { ... def receive =

    { case TaskFor(workers) => val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => sender ! seq.mkString(",") } } } 16 A first attempt
  14. class ActorWithTasks(tasks: ...) extends Actor { ... def receive =

    { case TaskFor(workers) => val from = sender val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => from ! seq.mkString(",") } } } 17 The fixed version
  15. Lesson 4: The JVM as a platform • The JVM

    is a great platform for asynchronous and concurrent programming • Very good performance and scalability • Build upon state-of-the-art libraries and tools (e.g., java.util.concurrent) 18 java.util.concurrent is not a concurrency library, it's a way of life - Doug Lea, Oct 28, 2013 “ ”
  16. The JVM as a platform (cont’d) • Knowledge about the

    JVM invaluable for creating high-performance concurrency libraries • Debugging, profiling, benchmarking, tuning, ... • Java Memory Model 19 With great power comes great responsibility... (Know your tools)
  17. Outline • Why a growable language for asynchronous programming? •

    Lessons learnt • Limits of growability • Actors & futures • Async/await • Future directions • RAY (or, direct-style Rx) 20
  18. Futures & Composition: Example • Context: Play Framework • Task:

    Given two web service requests, when both are completed, return response with the results of both: val futureDOY: Future[Response] = WS.url("http://api.day-of-year/today").get val futureDaysLeft: Future[Response] = WS.url("http://api.days-left/today").get 21
  19. Example futureDOY.flatMap { doyResponse => val dayOfYear = doyResponse.body futureDaysLeft.map

    { daysLeftResponse => val daysLeft = daysLeftResponse.body Ok("" + dayOfYear + ": " + daysLeft + " days left!") } } Using plain Scala futures val respFut = async { val dayOfYear = await(futureDOY).body val daysLeft = await(futureDaysLeft).body Ok("" + dayOfYear + ": " + daysLeft + " days left!") } Using Scala Async 22
  20. Async/await • The essence of async/await: 1. A way to

    spawn an asynchronous computation (async), returning a (first-class) future 2. A way to suspend an asynchronous computation (await) until a future is completed • Result: a direct-style API for asynchronous futures • Practical relevance: F#, C# 5.0, Scala 2.11 23
  21. Implementing async/await • Async/await requires ANF + state machine transform

    (CPS transform) • Macros of Scala 2.10 essential for transforming async { } • Alternative solution: compiler plugin • Library/language boundary blurred 24
  22. Using await • Requires a directly-enclosing async { } •

    Cannot use await • within closures • within local functions/classes/objects • within an argument to a by-name parameter 25
  23. Remedy: Combining push+pull async { list.map(x => await(f(x)).toString ) }

    Future.sequence( list.map(x => async { await(f(x)).toString })) def f(x: A): Future[B] • Existing combinators in Futures API can help! 26
  24. Outline • Why a growable language for asynchronous programming? •

    Scala as a growable language • Future directions 27
  25. Challenge output: 7, 1, 8, 3, 5, 2, ... Two

    input streams with the following values: stream2: 0, 7, 0, 4, 6, 5, ... stream1: 7, 1, 0, 2, 3, 1, ... Create a new output stream that • yields, for each value of stream1, the sum of the previous 3 values of stream1, • except if the sum is greater than some threshold in which case the next value of stream2 should be subtracted. Task: For a threshold of 5, the output stream has the following values: 28
  26. Reactive Extensions (Rx) • Asynchronous event streams and push notifications:

    a fundamental abstraction for web and mobile apps • Typically, event streams have to be scalable, robust, and composable • Examples: Netflix, Twitter, ... • Most popular framework: Reactive Extensions (Rx) • Based on the duality of iterators and observers (Meijer’12) • Cross-platform framework (RxJava, RxJS, ...) • Composition using higher-order functions 29
  27. The Essence of Rx trait Observable[T] { def subscribe(obs: Observer[T]):

    Closable } trait Observer[T] { def onNext(v: T): Unit def onFailure(t: Throwable): Unit def onDone(): Unit } 30
  28. Observer[T]: Interactions Erik Meijer: Your mouse is a database. CACM’12

    trait Observer[T] { def onNext(v: T): Unit def onFailure(t: Throwable): Unit def onDone(): Unit } 31
  29. Challenge: Recap output: 7, 1, 8, 3, 5, 2, ...

    Two input streams with the following values: stream2: 0, 7, 0, 4, 6, 5, ... stream1: 7, 1, 0, 2, 3, 1, ... Create a new output stream that • yields, for each value of stream1, the sum of the previous 3 values of stream1, • except if the sum is greater than some threshold in which case the next value of stream2 should be subtracted. Task: For a threshold of 5, the output stream has the following values: 34
  30. Solution using Rx val three = stream1.window(3).map(w => w.reduce(_ +

    _)) val withIndex = three.zipWithIndex val big = withIndex.filter(_._1 >= 5).zip(stream2).map { case ((l, i), r) => (l - r, i) } val output = withIndex.filter(_._1 < 5).merge(big) sum previous 3 values Requires “window” and “merge” combinators! 35
  31. The Problem • Programming with reactive streams suffers from an

    inversion of control • Purely push-based API • Example: writing stateful combinators is difficult • Hard to use for programmers not comfortable with higher-order functions 36
  32. RAY: Idea • Integrate Rx and Async: get the best

    of both worlds • Introduce variant of async { } to create observables instead of futures => rasync { } • Within rasync { }: enable awaiting events of observables in direct-style • Creating observables means we need a way to yield events from within rasync { } 37
  33. RAY: Primitives • rasync[T] { } - create Observable[T] •

    awaitNextOrDone(obs) - awaits and returns Some(next event of obs), or else if obs has terminated returns None • yieldNext(evt) - yields next event of current observable 38
  34. RAY: First Example val forwarder = rasync[Int] { var next:

    Option[Int] = awaitNextOrDone(stream) while (next.nonEmpty) { yieldNext(next) next = awaitNextOrDone(stream) } } 39
  35. Challenge: Recap output: 7, 1, 8, 3, 5, 2, ...

    Two input streams with the following values: stream2: 0, 7, 0, 4, 6, 5, ... stream1: 7, 1, 0, 2, 3, 1, ... Create a new output stream that • yields, for each value of stream1, the sum of the previous 3 values of stream1, • except if the sum is greater than some threshold in which case the next value of stream2 should be subtracted. Task: For a threshold of 5, the output stream has the following values: 40
  36. Solution using RAY val output = rasync[Int] { var window

    = List(0, 0, 0) var evt = awaitNextOrDone(stream1) while (evt.nonEmpty) { window = window.tail :+ evt.get val next = window.reduce(_ + _) match { case big if big > Threshold => awaitNextOrDone(stream2).map(x => big - x) case small => Some(small) } yieldNext(next) evt = if (next.isEmpty) None else awaitNextOrDone(stream1) } } No additional combinators required! 41
  37. RAY: Summary • Generalize async/await from futures to observables •

    Enables intuitive coordination of streams • Properties: • No need to use higher-order functions • Direct-style API for awaiting stream events • Programmers can leverage their experience with async/await 42
  38. Where’s the meat? • Whenever concurrent activities have to wait

    for events things become tricky • Support from language vs. execution environment • Depending on the waiting pattern suspend +resume can be cheap or expensive (Cilk, X10, ...) • Suspendible computations help! • Exposes limits of growable languages 43
  39. Push vs. pull • Thesis: purely push-based programming models will

    only “get you 80% there” • Fork/join pool vs. simple thread pools • Push-based and pull-based interfaces complement each other • Fundamental property of programming model • Inversion of control 44
  40. Isn’t this all way too low-level? • No: • Need

    a way to implement new programming models efficiently • Benefits also higher-level programming systems • Yes: • It’s all about synchronization constraints! • High-level coordination mechanisms needed • Synchronizers, composable events, 45
  41. Conclusion • Asynchronous programming a challenge for growable languages •

    Impact on programming models, languages, libraries, compilers, execution environments, type systems, static analysis, verification, ... 46