Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Next 700 Asynchronous Programming Models

3b84657fdb075382e3781310ca8a9a70?s=47 Philipp Haller
October 31, 2013
91

The Next 700 Asynchronous Programming Models

3b84657fdb075382e3781310ca8a9a70?s=128

Philipp Haller

October 31, 2013
Tweet

Transcript

  1. The Next 700 Asynchronous Programming Models Philipp Haller Typesafe, Inc.

    1 @philippkhaller
  2. What do these companies have in common? • They build

    systems using the actor model on the JVM • What’s interesting about this? • No special support for actors in Scala • Still, programming with actors in Scala is very natural 2
  3. What this talk is about • Scala as a growable

    language for asynchronous programming • Disclaimer: • I will talk about asynchronous and concurrent programming • In fact, mostly about concurrent programming 3
  4. Outline • Why a growable language for asynchronous programming? •

    Scala as a growable language • Future directions 4
  5. Why a growable language for asynchronous programming? • Actors, agents,

    communicating event-loops • CML • Futures/promises • Reactive Extensions (Rx) • Async/await, async-finish • STM • ... + Generalizations Which one is going to “win”? 5
  6. Why a growable language for asynchronous programming? (cont’d) Enables multiple

    high-level libraries embedded in the same host language • Richer programming system • Impacts type systems, static analysis + verification • Impacts language design • Performance comparisons between different libraries more meaningful 6
  7. Why a growable language for asynchronous programming? (cont’d) Simplifies research

    • Library extensions vs. language extensions • Experimental evaluation on real code 7
  8. Outline • Why a growable language for asynchronous programming? •

    Scala as a growable language • Future directions 8
  9. Outline • Why a growable language for asynchronous programming? •

    Lessons learnt • Limits of growability • Actors & futures • Async/await • Future directions • RAY (or, direct-style Rx) 9
  10. Scala as a growable language • We (EPFL + Typesafe)

    have used Scala to build a variety of asynchronous programming models • It has worked surprisingly well • There are limits 10
  11. Asynchronous programming landscape 11 Actors/Akka Futures Joins FlowPools Async/await Rasync-await-

    yield (RAY)
  12. Lessons Learnt 12

  13. Lesson 1: Scala enables new API designs • Unique integration

    of features: lots of API designs to be discovered • Objects + functions • Pattern matching, extractors • Implicits • ... • API design requires experience 13
  14. Lesson 2: Simplicity • Simplicity critical for success • Semantically

    and in terms of interface • Complexity creates skepticism • Sophisticated techniques only feasible if purely internal • Zero (known) bugs 14
  15. Lesson 3: Combining libraries • Integration of multiple concurrency libraries

    natural (to some extent) • Developers will do it anyway (see ECOOP’13) • Have to play nicely together 15
  16. class ActorWithTasks(tasks: ...) extends Actor { ... def receive =

    { case TaskFor(workers) => val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => sender ! seq.mkString(",") } } } 16 A first attempt
  17. class ActorWithTasks(tasks: ...) extends Actor { ... def receive =

    { case TaskFor(workers) => val from = sender val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => from ! seq.mkString(",") } } } 17 The fixed version
  18. Lesson 4: The JVM as a platform • The JVM

    is a great platform for asynchronous and concurrent programming • Very good performance and scalability • Build upon state-of-the-art libraries and tools (e.g., java.util.concurrent) 18 java.util.concurrent is not a concurrency library, it's a way of life - Doug Lea, Oct 28, 2013 “ ”
  19. The JVM as a platform (cont’d) • Knowledge about the

    JVM invaluable for creating high-performance concurrency libraries • Debugging, profiling, benchmarking, tuning, ... • Java Memory Model 19 With great power comes great responsibility... (Know your tools)
  20. Outline • Why a growable language for asynchronous programming? •

    Lessons learnt • Limits of growability • Actors & futures • Async/await • Future directions • RAY (or, direct-style Rx) 20
  21. Futures & Composition: Example • Context: Play Framework • Task:

    Given two web service requests, when both are completed, return response with the results of both: val futureDOY: Future[Response] = WS.url("http://api.day-of-year/today").get val futureDaysLeft: Future[Response] = WS.url("http://api.days-left/today").get 21
  22. Example futureDOY.flatMap { doyResponse => val dayOfYear = doyResponse.body futureDaysLeft.map

    { daysLeftResponse => val daysLeft = daysLeftResponse.body Ok("" + dayOfYear + ": " + daysLeft + " days left!") } } Using plain Scala futures val respFut = async { val dayOfYear = await(futureDOY).body val daysLeft = await(futureDaysLeft).body Ok("" + dayOfYear + ": " + daysLeft + " days left!") } Using Scala Async 22
  23. Async/await • The essence of async/await: 1. A way to

    spawn an asynchronous computation (async), returning a (first-class) future 2. A way to suspend an asynchronous computation (await) until a future is completed • Result: a direct-style API for asynchronous futures • Practical relevance: F#, C# 5.0, Scala 2.11 23
  24. Implementing async/await • Async/await requires ANF + state machine transform

    (CPS transform) • Macros of Scala 2.10 essential for transforming async { } • Alternative solution: compiler plugin • Library/language boundary blurred 24
  25. Using await • Requires a directly-enclosing async { } •

    Cannot use await • within closures • within local functions/classes/objects • within an argument to a by-name parameter 25
  26. Remedy: Combining push+pull async { list.map(x => await(f(x)).toString ) }

    Future.sequence( list.map(x => async { await(f(x)).toString })) def f(x: A): Future[B] • Existing combinators in Futures API can help! 26
  27. Outline • Why a growable language for asynchronous programming? •

    Scala as a growable language • Future directions 27
  28. Challenge output: 7, 1, 8, 3, 5, 2, ... Two

    input streams with the following values: stream2: 0, 7, 0, 4, 6, 5, ... stream1: 7, 1, 0, 2, 3, 1, ... Create a new output stream that • yields, for each value of stream1, the sum of the previous 3 values of stream1, • except if the sum is greater than some threshold in which case the next value of stream2 should be subtracted. Task: For a threshold of 5, the output stream has the following values: 28
  29. Reactive Extensions (Rx) • Asynchronous event streams and push notifications:

    a fundamental abstraction for web and mobile apps • Typically, event streams have to be scalable, robust, and composable • Examples: Netflix, Twitter, ... • Most popular framework: Reactive Extensions (Rx) • Based on the duality of iterators and observers (Meijer’12) • Cross-platform framework (RxJava, RxJS, ...) • Composition using higher-order functions 29
  30. The Essence of Rx trait Observable[T] { def subscribe(obs: Observer[T]):

    Closable } trait Observer[T] { def onNext(v: T): Unit def onFailure(t: Throwable): Unit def onDone(): Unit } 30
  31. Observer[T]: Interactions Erik Meijer: Your mouse is a database. CACM’12

    trait Observer[T] { def onNext(v: T): Unit def onFailure(t: Throwable): Unit def onDone(): Unit } 31
  32. The Real Power: Combinators flatMap 32

  33. Combinators: Example textChanges(textField) .flatMap(word => completions(word)) .subscribe(observeChanges(output)) Observable[Array[Strin g]] def

    textChanges(tf: JTextField): Observable[String] 33
  34. Challenge: Recap output: 7, 1, 8, 3, 5, 2, ...

    Two input streams with the following values: stream2: 0, 7, 0, 4, 6, 5, ... stream1: 7, 1, 0, 2, 3, 1, ... Create a new output stream that • yields, for each value of stream1, the sum of the previous 3 values of stream1, • except if the sum is greater than some threshold in which case the next value of stream2 should be subtracted. Task: For a threshold of 5, the output stream has the following values: 34
  35. Solution using Rx val three = stream1.window(3).map(w => w.reduce(_ +

    _)) val withIndex = three.zipWithIndex val big = withIndex.filter(_._1 >= 5).zip(stream2).map { case ((l, i), r) => (l - r, i) } val output = withIndex.filter(_._1 < 5).merge(big) sum previous 3 values Requires “window” and “merge” combinators! 35
  36. The Problem • Programming with reactive streams suffers from an

    inversion of control • Purely push-based API • Example: writing stateful combinators is difficult • Hard to use for programmers not comfortable with higher-order functions 36
  37. RAY: Idea • Integrate Rx and Async: get the best

    of both worlds • Introduce variant of async { } to create observables instead of futures => rasync { } • Within rasync { }: enable awaiting events of observables in direct-style • Creating observables means we need a way to yield events from within rasync { } 37
  38. RAY: Primitives • rasync[T] { } - create Observable[T] •

    awaitNextOrDone(obs) - awaits and returns Some(next event of obs), or else if obs has terminated returns None • yieldNext(evt) - yields next event of current observable 38
  39. RAY: First Example val forwarder = rasync[Int] { var next:

    Option[Int] = awaitNextOrDone(stream) while (next.nonEmpty) { yieldNext(next) next = awaitNextOrDone(stream) } } 39
  40. Challenge: Recap output: 7, 1, 8, 3, 5, 2, ...

    Two input streams with the following values: stream2: 0, 7, 0, 4, 6, 5, ... stream1: 7, 1, 0, 2, 3, 1, ... Create a new output stream that • yields, for each value of stream1, the sum of the previous 3 values of stream1, • except if the sum is greater than some threshold in which case the next value of stream2 should be subtracted. Task: For a threshold of 5, the output stream has the following values: 40
  41. Solution using RAY val output = rasync[Int] { var window

    = List(0, 0, 0) var evt = awaitNextOrDone(stream1) while (evt.nonEmpty) { window = window.tail :+ evt.get val next = window.reduce(_ + _) match { case big if big > Threshold => awaitNextOrDone(stream2).map(x => big - x) case small => Some(small) } yieldNext(next) evt = if (next.isEmpty) None else awaitNextOrDone(stream1) } } No additional combinators required! 41
  42. RAY: Summary • Generalize async/await from futures to observables •

    Enables intuitive coordination of streams • Properties: • No need to use higher-order functions • Direct-style API for awaiting stream events • Programmers can leverage their experience with async/await 42
  43. Where’s the meat? • Whenever concurrent activities have to wait

    for events things become tricky • Support from language vs. execution environment • Depending on the waiting pattern suspend +resume can be cheap or expensive (Cilk, X10, ...) • Suspendible computations help! • Exposes limits of growable languages 43
  44. Push vs. pull • Thesis: purely push-based programming models will

    only “get you 80% there” • Fork/join pool vs. simple thread pools • Push-based and pull-based interfaces complement each other • Fundamental property of programming model • Inversion of control 44
  45. Isn’t this all way too low-level? • No: • Need

    a way to implement new programming models efficiently • Benefits also higher-level programming systems • Yes: • It’s all about synchronization constraints! • High-level coordination mechanisms needed • Synchronizers, composable events, 45
  46. Conclusion • Asynchronous programming a challenge for growable languages •

    Impact on programming models, languages, libraries, compilers, execution environments, type systems, static analysis, verification, ... 46
  47. 47 Questions? Thank you! Philipp Haller Typesafe, Inc. @philippkhaller