Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Next 700 Asynchronous Programming Models

Philipp Haller
October 31, 2013
350

The Next 700 Asynchronous Programming Models

Philipp Haller

October 31, 2013
Tweet

Transcript

  1. The Next 700 Asynchronous
    Programming Models
    Philipp Haller
    Typesafe, Inc.
    1
    @philippkhaller

    View full-size slide

  2. What do these companies have in common?
    • They build systems using the actor model on the
    JVM
    • What’s interesting about this?
    • No special support for actors in Scala
    • Still, programming with actors in Scala is very
    natural
    2

    View full-size slide

  3. What this talk is about
    • Scala as a growable language for asynchronous
    programming
    • Disclaimer:
    • I will talk about asynchronous and concurrent
    programming
    • In fact, mostly about concurrent
    programming
    3

    View full-size slide

  4. Outline
    • Why a growable language for asynchronous
    programming?
    • Scala as a growable language
    • Future directions
    4

    View full-size slide

  5. Why a growable language for
    asynchronous programming?
    • Actors, agents, communicating event-loops
    • CML
    • Futures/promises
    • Reactive Extensions (Rx)
    • Async/await, async-finish
    • STM
    • ...
    + Generalizations
    Which one is going to “win”?
    5

    View full-size slide

  6. Why a growable language for
    asynchronous programming? (cont’d)
    Enables multiple high-level libraries embedded in
    the same host language
    • Richer programming system
    • Impacts type systems, static analysis +
    verification
    • Impacts language design
    • Performance comparisons between different
    libraries more meaningful
    6

    View full-size slide

  7. Why a growable language for
    asynchronous programming? (cont’d)
    Simplifies research
    • Library extensions vs. language extensions
    • Experimental evaluation on real code
    7

    View full-size slide

  8. Outline
    • Why a growable language for asynchronous
    programming?
    • Scala as a growable language
    • Future directions
    8

    View full-size slide

  9. Outline
    • Why a growable language for asynchronous
    programming?
    • Lessons learnt
    • Limits of growability
    • Actors & futures
    • Async/await
    • Future directions
    • RAY (or, direct-style Rx)
    9

    View full-size slide

  10. Scala as a growable language
    • We (EPFL + Typesafe) have used Scala to build a
    variety of asynchronous programming models
    • It has worked surprisingly well
    • There are limits
    10

    View full-size slide

  11. Asynchronous programming
    landscape
    11
    Actors/Akka Futures
    Joins FlowPools
    Async/await
    Rasync-await-
    yield (RAY)

    View full-size slide

  12. Lessons Learnt
    12

    View full-size slide

  13. Lesson 1: Scala enables new API
    designs
    • Unique integration of features: lots of API designs to
    be discovered
    • Objects + functions
    • Pattern matching, extractors
    • Implicits
    • ...
    • API design requires experience
    13

    View full-size slide

  14. Lesson 2: Simplicity
    • Simplicity critical for success
    • Semantically and in terms of interface
    • Complexity creates skepticism
    • Sophisticated techniques only feasible if
    purely internal
    • Zero (known) bugs
    14

    View full-size slide

  15. Lesson 3: Combining libraries
    • Integration of multiple concurrency libraries
    natural (to some extent)
    • Developers will do it anyway (see ECOOP’13)
    • Have to play nicely together
    15

    View full-size slide

  16. class ActorWithTasks(tasks: ...) extends Actor {
    ...
    def receive = {
    case TaskFor(workers) =>
    val requests = (tasks zip workers).map {
    case (task, worker) => worker ? task
    }
    val allDone = Future.sequence(requests)
    allDone andThen { seq =>
    sender ! seq.mkString(",")
    }
    }
    }
    16
    A first attempt

    View full-size slide

  17. class ActorWithTasks(tasks: ...) extends Actor {
    ...
    def receive = {
    case TaskFor(workers) =>
    val from = sender
    val requests = (tasks zip workers).map {
    case (task, worker) => worker ? task
    }
    val allDone = Future.sequence(requests)
    allDone andThen { seq =>
    from ! seq.mkString(",")
    }
    }
    }
    17
    The fixed version

    View full-size slide

  18. Lesson 4: The JVM as a platform
    • The JVM is a great platform for asynchronous
    and concurrent programming
    • Very good performance and scalability
    • Build upon state-of-the-art libraries and tools
    (e.g., java.util.concurrent)
    18
    java.util.concurrent is not a
    concurrency library, it's a way of life
    - Doug Lea, Oct 28, 2013


    View full-size slide

  19. The JVM as a platform (cont’d)
    • Knowledge about the JVM invaluable for creating
    high-performance concurrency libraries
    • Debugging, profiling, benchmarking, tuning, ...
    • Java Memory Model
    19
    With great power comes
    great responsibility... (Know your tools)

    View full-size slide

  20. Outline
    • Why a growable language for asynchronous
    programming?
    • Lessons learnt
    • Limits of growability
    • Actors & futures
    • Async/await
    • Future directions
    • RAY (or, direct-style Rx)
    20

    View full-size slide

  21. Futures & Composition: Example
    • Context: Play Framework
    • Task: Given two web service requests, when
    both are completed, return response with the
    results of both:
    val futureDOY: Future[Response] =
    WS.url("http://api.day-of-year/today").get
    val futureDaysLeft: Future[Response] =
    WS.url("http://api.days-left/today").get
    21

    View full-size slide

  22. Example
    futureDOY.flatMap { doyResponse =>
    val dayOfYear = doyResponse.body
    futureDaysLeft.map { daysLeftResponse =>
    val daysLeft = daysLeftResponse.body
    Ok("" + dayOfYear + ": " + daysLeft + " days left!")
    }
    }
    Using plain Scala futures
    val respFut = async {
    val dayOfYear = await(futureDOY).body
    val daysLeft = await(futureDaysLeft).body
    Ok("" + dayOfYear + ": " + daysLeft + " days left!")
    }
    Using Scala Async
    22

    View full-size slide

  23. Async/await
    • The essence of async/await:
    1. A way to spawn an asynchronous computation
    (async), returning a (first-class) future
    2. A way to suspend an asynchronous
    computation (await) until a future is completed
    • Result: a direct-style API for asynchronous futures
    • Practical relevance: F#, C# 5.0, Scala 2.11
    23

    View full-size slide

  24. Implementing async/await
    • Async/await requires ANF + state machine
    transform (CPS transform)
    • Macros of Scala 2.10 essential for transforming
    async { }
    • Alternative solution: compiler plugin
    • Library/language boundary blurred
    24

    View full-size slide

  25. Using await
    • Requires a directly-enclosing async { }
    • Cannot use await
    • within closures
    • within local functions/classes/objects
    • within an argument to a by-name parameter
    25

    View full-size slide

  26. Remedy: Combining push+pull
    async {
    list.map(x =>
    await(f(x)).toString
    )
    }
    Future.sequence(
    list.map(x => async {
    await(f(x)).toString
    }))
    def f(x: A): Future[B]
    • Existing combinators in Futures API can help!
    26

    View full-size slide

  27. Outline
    • Why a growable language for asynchronous
    programming?
    • Scala as a growable language
    • Future directions
    27

    View full-size slide

  28. Challenge
    output: 7, 1, 8, 3, 5, 2, ...
    Two input streams with the following values:
    stream2: 0, 7, 0, 4, 6, 5, ...
    stream1: 7, 1, 0, 2, 3, 1, ...
    Create a new output stream that
    • yields, for each value of stream1, the sum of the previous 3
    values of stream1,
    • except if the sum is greater than some threshold in which
    case the next value of stream2 should be subtracted.
    Task:
    For a threshold of 5, the output stream has
    the following values:
    28

    View full-size slide

  29. Reactive Extensions (Rx)
    • Asynchronous event streams and push notifications:
    a fundamental abstraction for web and mobile apps
    • Typically, event streams have to be scalable, robust,
    and composable
    • Examples: Netflix, Twitter, ...
    • Most popular framework: Reactive Extensions (Rx)
    • Based on the duality of iterators and observers
    (Meijer’12)
    • Cross-platform framework (RxJava, RxJS, ...)
    • Composition using higher-order functions
    29

    View full-size slide

  30. The Essence of Rx
    trait Observable[T] {
    def subscribe(obs: Observer[T]): Closable
    }
    trait Observer[T] {
    def onNext(v: T): Unit
    def onFailure(t: Throwable): Unit
    def onDone(): Unit
    }
    30

    View full-size slide

  31. Observer[T]: Interactions
    Erik Meijer: Your mouse is a database. CACM’12
    trait Observer[T] {
    def onNext(v: T): Unit
    def onFailure(t: Throwable): Unit
    def onDone(): Unit
    }
    31

    View full-size slide

  32. The Real Power: Combinators
    flatMap
    32

    View full-size slide

  33. Combinators: Example
    textChanges(textField)
    .flatMap(word => completions(word))
    .subscribe(observeChanges(output))
    Observable[Array[Strin
    g]]
    def textChanges(tf: JTextField):
    Observable[String]
    33

    View full-size slide

  34. Challenge: Recap
    output: 7, 1, 8, 3, 5, 2, ...
    Two input streams with the following values:
    stream2: 0, 7, 0, 4, 6, 5, ...
    stream1: 7, 1, 0, 2, 3, 1, ...
    Create a new output stream that
    • yields, for each value of stream1, the sum of the previous 3
    values of stream1,
    • except if the sum is greater than some threshold in which
    case the next value of stream2 should be subtracted.
    Task:
    For a threshold of 5, the output stream has
    the following values:
    34

    View full-size slide

  35. Solution using Rx
    val three = stream1.window(3).map(w => w.reduce(_ + _))
    val withIndex = three.zipWithIndex
    val big = withIndex.filter(_._1 >= 5).zip(stream2).map {
    case ((l, i), r) => (l - r, i)
    }
    val output = withIndex.filter(_._1 < 5).merge(big)
    sum previous
    3 values
    Requires “window” and “merge” combinators!
    35

    View full-size slide

  36. The Problem
    • Programming with reactive streams suffers from
    an inversion of control
    • Purely push-based API
    • Example: writing stateful combinators is
    difficult
    • Hard to use for programmers not comfortable
    with higher-order functions
    36

    View full-size slide

  37. RAY: Idea
    • Integrate Rx and Async: get the best of both
    worlds
    • Introduce variant of async { } to create
    observables instead of futures => rasync { }
    • Within rasync { }: enable awaiting events of
    observables in direct-style
    • Creating observables means we need a way to
    yield events from within rasync { }
    37

    View full-size slide

  38. RAY: Primitives
    • rasync[T] { } - create Observable[T]
    • awaitNextOrDone(obs) - awaits and returns
    Some(next event of obs), or else if obs has
    terminated returns None
    • yieldNext(evt) - yields next event of current
    observable
    38

    View full-size slide

  39. RAY: First Example
    val forwarder = rasync[Int] {
    var next: Option[Int] = awaitNextOrDone(stream)
    while (next.nonEmpty) {
    yieldNext(next)
    next = awaitNextOrDone(stream)
    }
    }
    39

    View full-size slide

  40. Challenge: Recap
    output: 7, 1, 8, 3, 5, 2, ...
    Two input streams with the following values:
    stream2: 0, 7, 0, 4, 6, 5, ...
    stream1: 7, 1, 0, 2, 3, 1, ...
    Create a new output stream that
    • yields, for each value of stream1, the sum of the previous 3
    values of stream1,
    • except if the sum is greater than some threshold in which
    case the next value of stream2 should be subtracted.
    Task:
    For a threshold of 5, the output stream has
    the following values:
    40

    View full-size slide

  41. Solution using RAY
    val output = rasync[Int] {
    var window = List(0, 0, 0)
    var evt = awaitNextOrDone(stream1)
    while (evt.nonEmpty) {
    window = window.tail :+ evt.get
    val next = window.reduce(_ + _) match {
    case big if big > Threshold =>
    awaitNextOrDone(stream2).map(x => big - x)
    case small =>
    Some(small)
    }
    yieldNext(next)
    evt =
    if (next.isEmpty) None else awaitNextOrDone(stream1)
    }
    }
    No additional combinators
    required!
    41

    View full-size slide

  42. RAY: Summary
    • Generalize async/await from futures to
    observables
    • Enables intuitive coordination of streams
    • Properties:
    • No need to use higher-order functions
    • Direct-style API for awaiting stream events
    • Programmers can leverage their experience
    with async/await
    42

    View full-size slide

  43. Where’s the meat?
    • Whenever concurrent activities have to wait for
    events things become tricky
    • Support from language vs. execution
    environment
    • Depending on the waiting pattern suspend
    +resume can be cheap or expensive (Cilk,
    X10, ...)
    • Suspendible computations help!
    • Exposes limits of growable languages
    43

    View full-size slide

  44. Push vs. pull
    • Thesis: purely push-based programming models
    will only “get you 80% there”
    • Fork/join pool vs. simple thread pools
    • Push-based and pull-based interfaces
    complement each other
    • Fundamental property of programming model
    • Inversion of control
    44

    View full-size slide

  45. Isn’t this all way too low-level?
    • No:
    • Need a way to implement new programming
    models efficiently
    • Benefits also higher-level programming
    systems
    • Yes:
    • It’s all about synchronization constraints!
    • High-level coordination mechanisms needed
    • Synchronizers, composable events,
    45

    View full-size slide

  46. Conclusion
    • Asynchronous programming a challenge for
    growable languages
    • Impact on programming models, languages,
    libraries, compilers, execution environments,
    type systems, static analysis, verification, ...
    46

    View full-size slide

  47. 47
    Questions?
    Thank you!
    Philipp Haller
    Typesafe, Inc.
    @philippkhaller

    View full-size slide