Asynchronous streams in direct style with and without macros Philipp Haller KTH Royal Institute of Technology Stockholm, Sweden Curry On 2019 London, UK, July 15th, 2019 

About myself • Associate Professor at KTH Royal Institute of Technology in Stockholm, Sweden • Previous positions at Typesafe Inc., Stanford University, and EPFL • PhD 2010 EPFL • Creator of Scala’s first widely-used actor library, co-author of Scala futures, Scala Async, spores, and others, Akka contributor • Research interests in programming language design and implementation, type systems, concurrency, and distributed programming 

A simple task • Given an asynchronous method that returns the day of the year as a string wrapped in a future:
 def dayOfYear(): Future[String] • However, sometimes, for unknown reasons, dayOfYear completes the returned future with "", or null, or the string "nope". • Task: create a future which is completed either • with "It's !" where is the textual representation matching the month returned by dayOfYear, or • with "Not a date, mate!" when that's really the best we can do. "04/11" for April 11, or "06/13" for June 13 

A first solution def solution() = { val date = """(\d+)/(\d+)""".r dayOfYear().flatMap { dayString => dayString match { case date(month, day) => nameOfMonth(month.toInt).map(name => s"It's $name!") case _ => Future.successful("Not a date, mate!") } } } 

A first solution def solution() = { val date = """(\d+)/(\d+)""".r dayOfYear().flatMap { dayString => dayString match { case date(month, day) => nameOfMonth(month.toInt).map(name => s"It's $name!") case _ => Future.successful("Not a date, mate!") } } } Returns a value of type Future[String] 

A first solution def solution() = { val date = """(\d+)/(\d+)""".r dayOfYear().flatMap { dayString => dayString match { case date(month, day) => nameOfMonth(month.toInt).map(name => s"It's $name!") case _ => Future.successful("Not a date, mate!") } } } Evaluates to a value of type Future[String] which is completed with “It’s April!” when future returned by nameOfMonth(4) is completed with “April”
 

A first solution def solution() = { val date = """(\d+)/(\d+)""".r dayOfYear().flatMap { dayString => dayString match { case date(month, day) => nameOfMonth(month.toInt).map(name => s"It's $name!") case _ => Future.successful("Not a date, mate!") } } } Evaluates to a value of type Future[String] 

A first solution def solution() = { val date = """(\d+)/(\d+)""".r dayOfYear().flatMap { dayString => dayString match { case date(month, day) => nameOfMonth(month.toInt).map(name => s"It's $name!") case _ => Future.successful("Not a date, mate!") } } } (a) flatMap returns a future f
 (b) f is completed (asynchronously) as follows:
 once dayOfYear() completes with value dayString, run closure { dayString => .. } which results in a future g;
 when g completes with value v,
 complete f with v 

Scala Async • Purpose:
 Simplify asynchronous programming using futures. • Created by Jason Zaugg (Lightbend) and yours truly, first released in 2013 • Stable releases for Scala 2.12 and 2.13 

A solution using Scala Async def solution() = async { val date = """(\d+)/(\d+)""".r await(dayOfYear()) match { case date(month, day) => s"It's ${await(nameOfMonth(month.toInt))}!" case _ => "Not a date, mate!" } } async { } creates a future which is completed asynchronously with result of await(f) suspends current async block until f is completed with value v which is returned A simple, sequential program! 

A solution using
 Scala for-comprehensions def solution() = { val date = """(\d+)/(\d+)""".r for { dayString <- dayOfYear() response <- dayString match { case date(month, day) => for (name <- nameOfMonth(month.toInt)) yield s"It's $name!" case _ => Future.successful("Not a date, mate!") } } yield response } Readability OK not great 

A solution using Scala Async def solution() = async { val date = """(\d+)/(\d+)""".r await(dayOfYear()) match { case date(month, day) => s"It's ${await(nameOfMonth(month.toInt))}!" case _ => "Not a date, mate!" } } No need to name all intermediate results No need for explicit Future.successful() Fewer closures allocated: async {} = 1 closure 

Let’s make our previous task slightly more complex! • Instead of converting a single string "04/11" (or "06/13" etc.), "", null, or "nope", we should consume a stream of multiple such strings • Output should be a publisher which produces a stream of results 

Intermezzo: Reactive Streams • a.k.a. java.util.concurrent.Flow (since JDK 9) • Interfaces: j.u.c.Flow.{Publisher, Subscriber, Subscription} • Some key prior work: • Observer design pattern (Gamma et al. 1994)1 • Reactive Extensions (Meijer 2012)2 • Key innovation of Reactive Streams: backpressure control 2Erik Meijer, Your mouse is a database, Commun. ACM 55(5) (2012) 66–73. 1Gamma, Erich, Richard Helm, Ralph Johnson, and John Vlissides, Design Patterns:
 Elements of Reusable Object Oriented Software, Addison-Wesley Professional, 1994 Composition of observable streams using higher-order functions 

Publisher • A Flow.Publisher produces events of type T which are received by subscribers if they have expressed interest. public static interface Flow.Publisher { void subscribe(Flow.Subscriber subscriber); } 

Subscriber • A Flow.Subscriber s consumes events of type T from publishers to which s has subscribed and from which s has requested to receive (a certain number of) events. public static interface Flow.Subscriber { void onSubscribe(Flow.Subscription subscription); void onNext(T item); void onError(Throwable throwable); void onComplete(); } 

Subscription • A Flow.Subscription enables subscribers to control the flow of events received from publishers. public static interface Flow.Subscription { void request(long n); void cancel(); } 

Reactive Extensions • Libraries building on basic interfaces for publishers, subscribers etc. • Key: higher-order functions for composing publishers • Implementations for many languages available 

TextChanges(input) .Select(word => Completions(word)) .Switch() .Subscribe(ObserveChanges(output)); Example from (Meijer 2012) Image source: Erik Meijer, Your mouse is a database, Commun. ACM 55(5) (2012) 66–73.
 https://queue.acm.org/detail.cfm?id=2169076 

Challenges • Often need diagrams to understand semantics (cf. “marble diagrams” used to document Reactive Extensions) • Creation of combinators challenging • Stateful combinators in particular • Inversion of control 

Back to our task • Instead of converting a single string "04/11" (or "06/13" etc.), "", null, or "nope", we should consume a stream of multiple such strings • Output should be a publisher which produces a stream of results • Problem: • await only takes futures as arguments, but we need to await stream events! • async creates a future, but we need to create a stream publisher which yields multiple events instead of producing just one result We can’t use async/await! :-( 

scala-async + Flow = scala-async-flow Extension of async/await model which: • Provides a variant of async { … } which creates a stream publisher:
 rasync { … } • Generalizes await from futures to stream events: next, done, error • Provides additional methods for yielding events:
 yieldNext, yieldError, yieldDone 

Generalizing await • Previously: await the completion of futures
 def await[T](future: Future[T]): T • Now: await events produced by different asynchronous objects
 (Future[T], Flow.Publisher[T], etc.) • An asynchronous object must provide the following methods: def getCompleted: Try[T] def onComplete[S](handler: Try[T] => S) given (executor: ExecutionContext) 

Solution using scala-async-flow val stream = rasync[String] { var dateOpt = await(dateStream) while (dateOpt.nonEmpty) { dateOpt.get match { case date(month, day) => yieldNext(s"It's ${await(nameOfMonth(month.toInt))}!") case _ => yieldNext("Not a date, mate!") } dateOpt = await(dateStream) } yieldDone() } Type of stream: Flow.Publisher[String] Here, we are awaiting a future! 

Implementing scala-async-flow Some alternatives: 1. Extension of Scala Async (macro for Scala 2.12 and 2.13) 2. Direct compiler support 3. Build on top of continuations or fibers 

Library Component def rasync[T](body: given Flow[T] => T): Flow.Publisher[T] = { delegate for Flow[T] = new Flow[T] ... } def await[T, S](a: Async[T]) given (flow: Flow[S], executor: ExecutionContext): T def yieldNext[T](event: T) given (flow: Flow[T]) = { flow.yieldNext(event) } Implicit function type3 3Odersky, Martin, Olivier Blanvillain, Fengyun Liu, Aggelos Biboudis, Heather Miller, and Sandro Stucki. Simplicitly: Foundations and applications of implicit function types. PACMPL 2(POPL): 42:1-42:29 (2018) 

Implicit Function Types def rasync[T](body: given Flow[T] => T): Flow.Publisher[T] = .. Usage: rasync[Int] { expr yieldNext(5) 3 } rasync[Int] { given ($f: Flow[Int]) => expr yieldNext(5) given $f 3 } Expands to: def yieldNext[T](event: T) given (flow: Flow[T]) = { flow.yieldNext(event) } yieldNext, yieldDone, etc. have implicit parameters: 

Library Component def rasync[T](body: given Flow[T] => T): Flow.Publisher[T] = { delegate for Flow[T] = new Flow[T] ... } def await[T, S](a: Async[T]) given (flow: Flow[S], executor: ExecutionContext): T def yieldNext[T](event: T) given (flow: Flow[T]) = { flow.yieldNext(event) } 

Library Component def rasync[T](body: given Flow[T] => T): Flow.Publisher[T] = { delegate for Flow[T] = new Flow[T] ... } def await[T, S](a: Async[T]) given (flow: Flow[S], executor: ExecutionContext): T def yieldNext[T](event: T) given (flow: Flow[T]) = { flow.yieldNext(event) } 

Library Component def rasync[T](body: given Flow[T] => T): Flow.Publisher[T] = { delegate for Flow[T] = new Flow[T] ... } def await[T, S](a: Async[T]) given (flow: Flow[S], executor: ExecutionContext): T def yieldNext[T](event: T) given (flow: Flow[T]) = { flow.yieldNext(event) } 

Macro-based implementation Example program: def fwd(s: Flow.Publisher[Int]) = rasync { var x: Option[Int] = None x = await(s) x.get } 

class StateMachine(flow: Flow[Int]) { var state = 0 val result = Promise[Int]() var x: Option[Int] = _ def apply(tr: Try[Int]): Unit = state match { case 0 => x = None val sub = flow.pubToSub(s) val completed = sub.getCompleted if (completed == null) { state = 2 sub.onComplete(evt => apply(evt)) } else if (completed.isFailure) { result.complete(completed) } else { x = completed.get state = 1 apply(null) } case 1 => result.complete(x.get) case 2 => if (tr.isFailure) { result.complete(tr) } else { x = tr.get state = 1 apply(null) } }} Generated state machine 

Alternative: more powerful execution environment • OpenJDK Project Loom • Goal:
 JVM features for supporting lightweight, high-throughput concurrency constructs • Key features: • Delimited continuations • Fibers (“user-mode threads”) • Project under active development • Sponsored by OpenJDK HotSpot Group • Talk by Ron Pressler today at 16:15! 

Continuations in Project Loom • A continuation turns a task into a suspendable computation • Suspend and resume at yield points • Transfer values into and out of using higher-level abstractions val cont = new Continuation(SCOPE, new Runnable { def run(): Unit = { } }) 

var continue = false val cont = new Continuation(SCOPE, new Runnable { def run(): Unit = { println("hello from continuation") while (!continue) { println("suspending") Continuation.`yield`(SCOPE) println("resuming") } println("all the way to the end") } }) cont.run() println("isDone: " + cont.isDone()) continue = true cont.run() println("isDone: " + cont.isDone()) hello from continuation suspending isDone: false resuming all the way to the end isDone: true Output: Example 

def rasync[T](body: given Flow[T] => T): Flow.Publisher[T] = { delegate flow for Flow[T] = new Flow[T] ... val cont = new Continuation(SCOPE, new Runnable { def run(): Unit = { try { val v = body flow.emitNext(v) flow.emitComplete() } catch { case NonFatal(error) => flow.emitError(error) } } }) ... flow } 

def await[T, S](a: Async[T]) given (flow: Flow[S], executor: ExecutionContext): T = { val res = a.getCompleted if (res eq null) { a.onComplete(x => flow.resume(x)) flow.suspend() a.getCompleted.get } else { res.get } } 

Theoretical foundations Paper:
 P. Haller and H. Miller: A reduction semantics for direct-style asynchronous observables.
 Journal of Logical and Algebraic Methods in Programming 105 (2019) 75–111
 https://doi.org/10.1016/j.jlamp.2019.03.002 • Formalization of programming model • Type soundness proof • Proof of protocol conformance • Describes macro-based implementation Example: a stream created with rasync { … } never emits a next event after emitting a done event 

Two More Issues • Issue 1: Data races due to variable capture
 
 Idea: constrain variable capturing of closures. See:
 H. Miller, P. Haller, M. Odersky:
 Spores: A Type-Based Foundation for Closures in the Age of
 Concurrency and Distribution. ECOOP 2014 • Issue 2: Data races due to top-level singleton objects
 
 Idea: ensure object-capability safety of objects created within rasync blocks. See:
 P. Haller, A. Loiko:
 LaCasa: lightweight affinity and object capabilities in Scala. OOPSLA 2016 Example: var x = 5 rasync[Int] { x = 10 yieldNext(5) 3 } val y = x + 1 

Open Issues • “Blue and red functions” problem • Calling a suspendible method (given Flow[T]) requires the caller to be suspendible, too • Need two variants for each higher-order function: regular and suspendible • Loom continuations don’t suffer from that problem; can suspend as long as there is an active continuation • Relaxing need for Flow[T] capabilities may be feasible 

Resources • Prototype implementation:
 https://github.com/phaller/scala-async-flow
 Branch “cont” makes use of Project Loom continuations and Scala 3 • Papers and more:
 http://www.csc.kth.se/~phaller/ • Twitter: @philippkhaller Thank you!