Asynchronous streams in direct style with and without macros

1. Asynchronous streams in direct style with and without macros Philipp Haller KTH Royal Institute of Technology Stockholm, Sweden Curry On 2019 London, UK, July 15th, 2019
2. About myself • Associate Professor at KTH Royal Institute of Technology in Stockholm, Sweden • Previous positions at Typesafe Inc., Stanford University, and EPFL • PhD 2010 EPFL • Creator of Scala’s first widely-used actor library, co-author of Scala futures, Scala Async, spores, and others, Akka contributor • Research interests in programming language design and implementation, type systems, concurrency, and distributed programming
3. A simple task • Given an asynchronous method that returns the day of the year as a string wrapped in a future:
 def dayOfYear(): Future[String] • However, sometimes, for unknown reasons, dayOfYear completes the returned future with "", or null, or the string "nope". • Task: create a future which is completed either • with "It's <month>!" where <month> is the textual representation matching the month returned by dayOfYear, or • with "Not a date, mate!" when that's really the best we can do. "04/11" for April 11, or "06/13" for June 13
4. A first solution def solution() = { val date = """(\d+)/(\d+)""".r dayOfYear().flatMap { dayString => dayString match { case date(month, day) => nameOfMonth(month.toInt).map(name => s"It's $name!") case _ => Future.successful("Not a date, mate!") } } }
5. A first solution def solution() = { val date = """(\d+)/(\d+)""".r dayOfYear().flatMap { dayString => dayString match { case date(month, day) => nameOfMonth(month.toInt).map(name => s"It's $name!") case _ => Future.successful("Not a date, mate!") } } } Returns a value of type Future[String]
6. A first solution def solution() = { val date = """(\d+)/(\d+)""".r dayOfYear().flatMap { dayString => dayString match { case date(month, day) => nameOfMonth(month.toInt).map(name => s"It's $name!") case _ => Future.successful("Not a date, mate!") } } } Evaluates to a value of type Future[String] which is completed with “It’s April!” when future returned by nameOfMonth(4) is completed with “April”

7. A first solution def solution() = { val date = """(\d+)/(\d+)""".r dayOfYear().flatMap { dayString => dayString match { case date(month, day) => nameOfMonth(month.toInt).map(name => s"It's $name!") case _ => Future.successful("Not a date, mate!") } } } Evaluates to a value of type Future[String]
8. A first solution def solution() = { val date = """(\d+)/(\d+)""".r dayOfYear().flatMap { dayString => dayString match { case date(month, day) => nameOfMonth(month.toInt).map(name => s"It's $name!") case _ => Future.successful("Not a date, mate!") } } } (a) flatMap returns a future f
 (b) f is completed (asynchronously) as follows:
 once dayOfYear() completes with value dayString, run closure { dayString => .. } which results in a future g;
 when g completes with value v,
 complete f with v
9. Scala Async • Purpose:
 Simplify asynchronous programming using futures. • Created by Jason Zaugg (Lightbend) and yours truly, first released in 2013 • Stable releases for Scala 2.12 and 2.13
10. A solution using Scala Async def solution() = async { val date = """(\d+)/(\d+)""".r await(dayOfYear()) match { case date(month, day) => s"It's ${await(nameOfMonth(month.toInt))}!" case _ => "Not a date, mate!" } } async { <body> } creates a future which is completed asynchronously with result of <body> await(f) suspends current async block until f is completed with value v which is returned A simple, sequential program!
11. A solution using
 Scala for-comprehensions def solution() = { val date = """(\d+)/(\d+)""".r for { dayString <- dayOfYear() response <- dayString match { case date(month, day) => for (name <- nameOfMonth(month.toInt)) yield s"It's $name!" case _ => Future.successful("Not a date, mate!") } } yield response } Readability OK not great
12. A solution using Scala Async def solution() = async { val date = """(\d+)/(\d+)""".r await(dayOfYear()) match { case date(month, day) => s"It's ${await(nameOfMonth(month.toInt))}!" case _ => "Not a date, mate!" } } No need to name all intermediate results No need for explicit Future.successful() Fewer closures allocated: async {} = 1 closure
13. Let’s make our previous task slightly more complex! • Instead of converting a single string "04/11" (or "06/13" etc.), "", null, or "nope", we should consume a stream of multiple such strings • Output should be a publisher which produces a stream of results
14. Intermezzo: Reactive Streams • a.k.a. java.util.concurrent.Flow (since JDK 9) • Interfaces: j.u.c.Flow.{Publisher, Subscriber, Subscription} • Some key prior work: • Observer design pattern (Gamma et al. 1994)1 • Reactive Extensions (Meijer 2012)2 • Key innovation of Reactive Streams: backpressure control 2Erik Meijer, Your mouse is a database, Commun. ACM 55(5) (2012) 66–73. 1Gamma, Erich, Richard Helm, Ralph Johnson, and John Vlissides, Design Patterns:
 Elements of Reusable Object Oriented Software, Addison-Wesley Professional, 1994 Composition of observable streams using higher-order functions
15. Publisher • A Flow.Publisher<T> produces events of type T which are received by subscribers if they have expressed interest. public static interface Flow.Publisher<T> { void subscribe(Flow.Subscriber<? super T> subscriber); }
16. Subscriber • A Flow.Subscriber<T> s consumes events of type T from publishers to which s has subscribed and from which s has requested to receive (a certain number of) events. public static interface Flow.Subscriber<T> { void onSubscribe(Flow.Subscription subscription); void onNext(T item); void onError(Throwable throwable); void onComplete(); }
17. Subscription • A Flow.Subscription enables subscribers to control the flow of events received from publishers. public static interface Flow.Subscription { void request(long n); void cancel(); }
18. Reactive Extensions • Libraries building on basic interfaces for publishers, subscribers etc. • Key: higher-order functions for composing publishers • Implementations for many languages available
19. TextChanges(input) .Select(word => Completions(word)) .Switch() .Subscribe(ObserveChanges(output)); Example from (Meijer 2012) Image source: Erik Meijer, Your mouse is a database, Commun. ACM 55(5) (2012) 66–73.
 https://queue.acm.org/detail.cfm?id=2169076
20. Challenges • Often need diagrams to understand semantics (cf. “marble diagrams” used to document Reactive Extensions) • Creation of combinators challenging • Stateful combinators in particular • Inversion of control
21. Back to our task • Instead of converting a single string "04/11" (or "06/13" etc.), "", null, or "nope", we should consume a stream of multiple such strings • Output should be a publisher which produces a stream of results • Problem: • await only takes futures as arguments, but we need to await stream events! • async creates a future, but we need to create a stream publisher which yields multiple events instead of producing just one result We can’t use async/await! :-(
22. scala-async + Flow = scala-async-flow Extension of async/await model which: • Provides a variant of async { … } which creates a stream publisher:
 rasync { … } • Generalizes await from futures to stream events: next, done, error • Provides additional methods for yielding events:
 yieldNext, yieldError, yieldDone
23. Generalizing await • Previously: await the completion of futures
 def await[T](future: Future[T]): T • Now: await events produced by different asynchronous objects
 (Future[T], Flow.Publisher[T], etc.) • An asynchronous object must provide the following methods: def getCompleted: Try[T] def onComplete[S](handler: Try[T] => S) given (executor: ExecutionContext)
24. Solution using scala-async-flow val stream = rasync[String] { var dateOpt = await(dateStream) while (dateOpt.nonEmpty) { dateOpt.get match { case date(month, day) => yieldNext(s"It's ${await(nameOfMonth(month.toInt))}!") case _ => yieldNext("Not a date, mate!") } dateOpt = await(dateStream) } yieldDone() } Type of stream: Flow.Publisher[String] Here, we are awaiting a future!
25. Implementing scala-async-flow Some alternatives: 1. Extension of Scala Async (macro for Scala 2.12 and 2.13) 2. Direct compiler support 3. Build on top of continuations or fibers
26. Library Component def rasync[T](body: given Flow[T] => T): Flow.Publisher[T] = { delegate for Flow[T] = new Flow[T] ... } def await[T, S](a: Async[T]) given (flow: Flow[S], executor: ExecutionContext): T def yieldNext[T](event: T) given (flow: Flow[T]) = { flow.yieldNext(event) } Implicit function type3 3Odersky, Martin, Olivier Blanvillain, Fengyun Liu, Aggelos Biboudis, Heather Miller, and Sandro Stucki. Simplicitly: Foundations and applications of implicit function types. PACMPL 2(POPL): 42:1-42:29 (2018)
27. Implicit Function Types def rasync[T](body: given Flow[T] => T): Flow.Publisher[T] = .. Usage: rasync[Int] { expr yieldNext(5) 3 } rasync[Int] { given ($f: Flow[Int]) => expr yieldNext(5) given $f 3 } Expands to: def yieldNext[T](event: T) given (flow: Flow[T]) = { flow.yieldNext(event) } yieldNext, yieldDone, etc. have implicit parameters:
28. Library Component def rasync[T](body: given Flow[T] => T): Flow.Publisher[T] = { delegate for Flow[T] = new Flow[T] ... } def await[T, S](a: Async[T]) given (flow: Flow[S], executor: ExecutionContext): T def yieldNext[T](event: T) given (flow: Flow[T]) = { flow.yieldNext(event) }
29. Library Component def rasync[T](body: given Flow[T] => T): Flow.Publisher[T] = { delegate for Flow[T] = new Flow[T] ... } def await[T, S](a: Async[T]) given (flow: Flow[S], executor: ExecutionContext): T def yieldNext[T](event: T) given (flow: Flow[T]) = { flow.yieldNext(event) }
30. Library Component def rasync[T](body: given Flow[T] => T): Flow.Publisher[T] = { delegate for Flow[T] = new Flow[T] ... } def await[T, S](a: Async[T]) given (flow: Flow[S], executor: ExecutionContext): T def yieldNext[T](event: T) given (flow: Flow[T]) = { flow.yieldNext(event) }
31. Macro-based implementation Example program: def fwd(s: Flow.Publisher[Int]) = rasync { var x: Option[Int] = None x = await(s) x.get }
32. class StateMachine(flow: Flow[Int]) { var state = 0 val result = Promise[Int]() var x: Option[Int] = _ def apply(tr: Try[Int]): Unit = state match { case 0 => x = None val sub = flow.pubToSub(s) val completed = sub.getCompleted if (completed == null) { state = 2 sub.onComplete(evt => apply(evt)) } else if (completed.isFailure) { result.complete(completed) } else { x = completed.get state = 1 apply(null) } case 1 => result.complete(x.get) case 2 => if (tr.isFailure) { result.complete(tr) } else { x = tr.get state = 1 apply(null) } }} Generated state machine
33. Alternative: more powerful execution environment • OpenJDK Project Loom • Goal:
 JVM features for supporting lightweight, high-throughput concurrency constructs • Key features: • Delimited continuations • Fibers (“user-mode threads”) • Project under active development • Sponsored by OpenJDK HotSpot Group • Talk by Ron Pressler today at 16:15!
34. Continuations in Project Loom • A continuation turns a task into a suspendable computation • Suspend and resume <body> at yield points • Transfer values into and out of <body> using higher-level abstractions val cont = new Continuation(SCOPE, new Runnable { def run(): Unit = { <body> } })
35. var continue = false val cont = new Continuation(SCOPE, new Runnable { def run(): Unit = { println("hello from continuation") while (!continue) { println("suspending") Continuation.`yield`(SCOPE) println("resuming") } println("all the way to the end") } }) cont.run() println("isDone: " + cont.isDone()) continue = true cont.run() println("isDone: " + cont.isDone()) hello from continuation suspending isDone: false resuming all the way to the end isDone: true Output: Example
36. def rasync[T](body: given Flow[T] => T): Flow.Publisher[T] = { delegate flow for Flow[T] = new Flow[T] ... val cont = new Continuation(SCOPE, new Runnable { def run(): Unit = { try { val v = body flow.emitNext(v) flow.emitComplete() } catch { case NonFatal(error) => flow.emitError(error) } } }) ... flow }
37. def await[T, S](a: Async[T]) given (flow: Flow[S], executor: ExecutionContext): T = { val res = a.getCompleted if (res eq null) { a.onComplete(x => flow.resume(x)) flow.suspend() a.getCompleted.get } else { res.get } }
38. Theoretical foundations Paper:
 P. Haller and H. Miller: A reduction semantics for direct-style asynchronous observables.
 Journal of Logical and Algebraic Methods in Programming 105 (2019) 75–111
 https://doi.org/10.1016/j.jlamp.2019.03.002 • Formalization of programming model • Type soundness proof • Proof of protocol conformance • Describes macro-based implementation Example: a stream created with rasync { … } never emits a next event after emitting a done event
39. Two More Issues • Issue 1: Data races due to variable capture
 
 Idea: constrain variable capturing of closures. See:
 H. Miller, P. Haller, M. Odersky:
 Spores: A Type-Based Foundation for Closures in the Age of
 Concurrency and Distribution. ECOOP 2014 • Issue 2: Data races due to top-level singleton objects
 
 Idea: ensure object-capability safety of objects created within rasync blocks. See:
 P. Haller, A. Loiko:
 LaCasa: lightweight affinity and object capabilities in Scala. OOPSLA 2016 Example: var x = 5 rasync[Int] { x = 10 yieldNext(5) 3 } val y = x + 1
40. Open Issues • “Blue and red functions” problem • Calling a suspendible method (given Flow[T]) requires the caller to be suspendible, too • Need two variants for each higher-order function: regular and suspendible • Loom continuations don’t suffer from that problem; can suspend as long as there is an active continuation • Relaxing need for Flow[T] capabilities may be feasible
41. Resources • Prototype implementation:
 https://github.com/phaller/scala-async-flow
 Branch “cont” makes use of Project Loom continuations and Scala 3 • Papers and more:
 http://www.csc.kth.se/~phaller/ • Twitter: @philippkhaller Thank you!