The Next 700 Asynchronous Programming Models

Transcript

1. The Next 700 Asynchronous
   Programming Models
   Philipp Haller
   Typesafe, Inc.
   1
   @philippkhaller
   

   View Slide

2. What do these companies have in common?
   • They build systems using the actor model on the
   JVM
   • What’s interesting about this?
   • No special support for actors in Scala
   • Still, programming with actors in Scala is very
   natural
   2
   

   View Slide

3. What this talk is about
   • Scala as a growable language for asynchronous
   programming
   • Disclaimer:
   • I will talk about asynchronous and concurrent
   programming
   • In fact, mostly about concurrent
   programming
   3
   

   View Slide

4. Outline
   • Why a growable language for asynchronous
   programming?
   • Scala as a growable language
   • Future directions
   4
   

   View Slide

5. Why a growable language for
   asynchronous programming?
   • Actors, agents, communicating event-loops
   • CML
   • Futures/promises
   • Reactive Extensions (Rx)
   • Async/await, async-finish
   • STM
   • ...
   + Generalizations
   Which one is going to “win”?
   5
   

   View Slide

6. Why a growable language for
   asynchronous programming? (cont’d)
   Enables multiple high-level libraries embedded in
   the same host language
   • Richer programming system
   • Impacts type systems, static analysis +
   verification
   • Impacts language design
   • Performance comparisons between different
   libraries more meaningful
   6
   

   View Slide

7. Why a growable language for
   asynchronous programming? (cont’d)
   Simplifies research
   • Library extensions vs. language extensions
   • Experimental evaluation on real code
   7
   

   View Slide

8. Outline
   • Why a growable language for asynchronous
   programming?
   • Scala as a growable language
   • Future directions
   8
   

   View Slide

9. Outline
   • Why a growable language for asynchronous
   programming?
   • Lessons learnt
   • Limits of growability
   • Actors & futures
   • Async/await
   • Future directions
   • RAY (or, direct-style Rx)
   9
   

   View Slide

10. Scala as a growable language
    • We (EPFL + Typesafe) have used Scala to build a
    variety of asynchronous programming models
    • It has worked surprisingly well
    • There are limits
    10
    

    View Slide

11. Asynchronous programming
    landscape
    11
    Actors/Akka Futures
    Joins FlowPools
    Async/await
    Rasync-await-
    yield (RAY)
    

    View Slide

12. Lessons Learnt
    12
    

    View Slide

13. Lesson 1: Scala enables new API
    designs
    • Unique integration of features: lots of API designs to
    be discovered
    • Objects + functions
    • Pattern matching, extractors
    • Implicits
    • ...
    • API design requires experience
    13
    

    View Slide

14. Lesson 2: Simplicity
    • Simplicity critical for success
    • Semantically and in terms of interface
    • Complexity creates skepticism
    • Sophisticated techniques only feasible if
    purely internal
    • Zero (known) bugs
    14
    

    View Slide

15. Lesson 3: Combining libraries
    • Integration of multiple concurrency libraries
    natural (to some extent)
    • Developers will do it anyway (see ECOOP’13)
    • Have to play nicely together
    15
    

    View Slide

16. class ActorWithTasks(tasks: ...) extends Actor {
    ...
    def receive = {
    case TaskFor(workers) =>
    val requests = (tasks zip workers).map {
    case (task, worker) => worker ? task
    }
    val allDone = Future.sequence(requests)
    allDone andThen { seq =>
    sender ! seq.mkString(",")
    }
    }
    }
    16
    A first attempt
    

    View Slide

17. class ActorWithTasks(tasks: ...) extends Actor {
    ...
    def receive = {
    case TaskFor(workers) =>
    val from = sender
    val requests = (tasks zip workers).map {
    case (task, worker) => worker ? task
    }
    val allDone = Future.sequence(requests)
    allDone andThen { seq =>
    from ! seq.mkString(",")
    }
    }
    }
    17
    The fixed version
    

    View Slide

18. Lesson 4: The JVM as a platform
    • The JVM is a great platform for asynchronous
    and concurrent programming
    • Very good performance and scalability
    • Build upon state-of-the-art libraries and tools
    (e.g., java.util.concurrent)
    18
    java.util.concurrent is not a
    concurrency library, it's a way of life
    - Doug Lea, Oct 28, 2013
    “
    ”
    

    View Slide

19. The JVM as a platform (cont’d)
    • Knowledge about the JVM invaluable for creating
    high-performance concurrency libraries
    • Debugging, profiling, benchmarking, tuning, ...
    • Java Memory Model
    19
    With great power comes
    great responsibility... (Know your tools)
    

    View Slide

20. Outline
    • Why a growable language for asynchronous
    programming?
    • Lessons learnt
    • Limits of growability
    • Actors & futures
    • Async/await
    • Future directions
    • RAY (or, direct-style Rx)
    20
    

    View Slide

21. Futures & Composition: Example
    • Context: Play Framework
    • Task: Given two web service requests, when
    both are completed, return response with the
    results of both:
    val futureDOY: Future[Response] =
    WS.url("http://api.day-of-year/today").get
    val futureDaysLeft: Future[Response] =
    WS.url("http://api.days-left/today").get
    21
    

    View Slide

22. Example
    futureDOY.flatMap { doyResponse =>
    val dayOfYear = doyResponse.body
    futureDaysLeft.map { daysLeftResponse =>
    val daysLeft = daysLeftResponse.body
    Ok("" + dayOfYear + ": " + daysLeft + " days left!")
    }
    }
    Using plain Scala futures
    val respFut = async {
    val dayOfYear = await(futureDOY).body
    val daysLeft = await(futureDaysLeft).body
    Ok("" + dayOfYear + ": " + daysLeft + " days left!")
    }
    Using Scala Async
    22
    

    View Slide

23. Async/await
    • The essence of async/await:
    1. A way to spawn an asynchronous computation
    (async), returning a (first-class) future
    2. A way to suspend an asynchronous
    computation (await) until a future is completed
    • Result: a direct-style API for asynchronous futures
    • Practical relevance: F#, C# 5.0, Scala 2.11
    23
    

    View Slide

24. Implementing async/await
    • Async/await requires ANF + state machine
    transform (CPS transform)
    • Macros of Scala 2.10 essential for transforming
    async { }
    • Alternative solution: compiler plugin
    • Library/language boundary blurred
    24
    

    View Slide

25. Using await
    • Requires a directly-enclosing async { }
    • Cannot use await
    • within closures
    • within local functions/classes/objects
    • within an argument to a by-name parameter
    25
    

    View Slide

26. Remedy: Combining push+pull
    async {
    list.map(x =>
    await(f(x)).toString
    )
    }
    Future.sequence(
    list.map(x => async {
    await(f(x)).toString
    }))
    def f(x: A): Future[B]
    • Existing combinators in Futures API can help!
    26
    

    View Slide

27. Outline
    • Why a growable language for asynchronous
    programming?
    • Scala as a growable language
    • Future directions
    27
    

    View Slide

28. Challenge
    output: 7, 1, 8, 3, 5, 2, ...
    Two input streams with the following values:
    stream2: 0, 7, 0, 4, 6, 5, ...
    stream1: 7, 1, 0, 2, 3, 1, ...
    Create a new output stream that
    • yields, for each value of stream1, the sum of the previous 3
    values of stream1,
    • except if the sum is greater than some threshold in which
    case the next value of stream2 should be subtracted.
    Task:
    For a threshold of 5, the output stream has
    the following values:
    28
    

    View Slide

29. Reactive Extensions (Rx)
    • Asynchronous event streams and push notifications:
    a fundamental abstraction for web and mobile apps
    • Typically, event streams have to be scalable, robust,
    and composable
    • Examples: Netflix, Twitter, ...
    • Most popular framework: Reactive Extensions (Rx)
    • Based on the duality of iterators and observers
    (Meijer’12)
    • Cross-platform framework (RxJava, RxJS, ...)
    • Composition using higher-order functions
    29
    

    View Slide

30. The Essence of Rx
    trait Observable[T] {
    def subscribe(obs: Observer[T]): Closable
    }
    trait Observer[T] {
    def onNext(v: T): Unit
    def onFailure(t: Throwable): Unit
    def onDone(): Unit
    }
    30
    

    View Slide

31. Observer[T]: Interactions
    Erik Meijer: Your mouse is a database. CACM’12
    trait Observer[T] {
    def onNext(v: T): Unit
    def onFailure(t: Throwable): Unit
    def onDone(): Unit
    }
    31
    

    View Slide

32. The Real Power: Combinators
    flatMap
    32
    

    View Slide

33. Combinators: Example
    textChanges(textField)
    .flatMap(word => completions(word))
    .subscribe(observeChanges(output))
    Observable[Array[Strin
    g]]
    def textChanges(tf: JTextField):
    Observable[String]
    33
    

    View Slide

34. Challenge: Recap
    output: 7, 1, 8, 3, 5, 2, ...
    Two input streams with the following values:
    stream2: 0, 7, 0, 4, 6, 5, ...
    stream1: 7, 1, 0, 2, 3, 1, ...
    Create a new output stream that
    • yields, for each value of stream1, the sum of the previous 3
    values of stream1,
    • except if the sum is greater than some threshold in which
    case the next value of stream2 should be subtracted.
    Task:
    For a threshold of 5, the output stream has
    the following values:
    34
    

    View Slide

35. Solution using Rx
    val three = stream1.window(3).map(w => w.reduce(_ + _))
    val withIndex = three.zipWithIndex
    val big = withIndex.filter(_._1 >= 5).zip(stream2).map {
    case ((l, i), r) => (l - r, i)
    }
    val output = withIndex.filter(_._1 < 5).merge(big)
    sum previous
    3 values
    Requires “window” and “merge” combinators!
    35
    

    View Slide

36. The Problem
    • Programming with reactive streams suffers from
    an inversion of control
    • Purely push-based API
    • Example: writing stateful combinators is
    difficult
    • Hard to use for programmers not comfortable
    with higher-order functions
    36
    

    View Slide

37. RAY: Idea
    • Integrate Rx and Async: get the best of both
    worlds
    • Introduce variant of async { } to create
    observables instead of futures => rasync { }
    • Within rasync { }: enable awaiting events of
    observables in direct-style
    • Creating observables means we need a way to
    yield events from within rasync { }
    37
    

    View Slide

38. RAY: Primitives
    • rasync[T] { } - create Observable[T]
    • awaitNextOrDone(obs) - awaits and returns
    Some(next event of obs), or else if obs has
    terminated returns None
    • yieldNext(evt) - yields next event of current
    observable
    38
    

    View Slide

39. RAY: First Example
    val forwarder = rasync[Int] {
    var next: Option[Int] = awaitNextOrDone(stream)
    while (next.nonEmpty) {
    yieldNext(next)
    next = awaitNextOrDone(stream)
    }
    }
    39
    

    View Slide

40. Challenge: Recap
    output: 7, 1, 8, 3, 5, 2, ...
    Two input streams with the following values:
    stream2: 0, 7, 0, 4, 6, 5, ...
    stream1: 7, 1, 0, 2, 3, 1, ...
    Create a new output stream that
    • yields, for each value of stream1, the sum of the previous 3
    values of stream1,
    • except if the sum is greater than some threshold in which
    case the next value of stream2 should be subtracted.
    Task:
    For a threshold of 5, the output stream has
    the following values:
    40
    

    View Slide

41. Solution using RAY
    val output = rasync[Int] {
    var window = List(0, 0, 0)
    var evt = awaitNextOrDone(stream1)
    while (evt.nonEmpty) {
    window = window.tail :+ evt.get
    val next = window.reduce(_ + _) match {
    case big if big > Threshold =>
    awaitNextOrDone(stream2).map(x => big - x)
    case small =>
    Some(small)
    }
    yieldNext(next)
    evt =
    if (next.isEmpty) None else awaitNextOrDone(stream1)
    }
    }
    No additional combinators
    required!
    41
    

    View Slide

42. RAY: Summary
    • Generalize async/await from futures to
    observables
    • Enables intuitive coordination of streams
    • Properties:
    • No need to use higher-order functions
    • Direct-style API for awaiting stream events
    • Programmers can leverage their experience
    with async/await
    42
    

    View Slide

43. Where’s the meat?
    • Whenever concurrent activities have to wait for
    events things become tricky
    • Support from language vs. execution
    environment
    • Depending on the waiting pattern suspend
    +resume can be cheap or expensive (Cilk,
    X10, ...)
    • Suspendible computations help!
    • Exposes limits of growable languages
    43
    

    View Slide

44. Push vs. pull
    • Thesis: purely push-based programming models
    will only “get you 80% there”
    • Fork/join pool vs. simple thread pools
    • Push-based and pull-based interfaces
    complement each other
    • Fundamental property of programming model
    • Inversion of control
    44
    

    View Slide

45. Isn’t this all way too low-level?
    • No:
    • Need a way to implement new programming
    models efficiently
    • Benefits also higher-level programming
    systems
    • Yes:
    • It’s all about synchronization constraints!
    • High-level coordination mechanisms needed
    • Synchronizers, composable events,
    45
    

    View Slide

46. Conclusion
    • Asynchronous programming a challenge for
    growable languages
    • Impact on programming models, languages,
    libraries, compilers, execution environments,
    type systems, static analysis, verification, ...
    46
    

    View Slide

47. 47
    Questions?
    Thank you!
    Philipp Haller
    Typesafe, Inc.
    @philippkhaller
    

    View Slide