Towards Safer Lightweight Concurrency in Scala

Transcript

1. Towards Safer Lightweight Concurrency in Scala KTH Royal Institute of Technology Stockholm, Sweden Philipp Haller Scala Autumn Meetup @ Wolt Stockholm, Sweden, September 12, 2024
2. Example • Convert a List[Future[Int]] to a Future[List[Int]] which is completed once all futures in the original list are completed val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val wanted: Future[List[Int]] = ???
3. Example • Convert a List[Future[Int]] to a Future[List[Int]] which is completed once all futures in the original list are completed val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val wanted: Future[List[Int]] = Future { futures.map { fut => // await completion of `fut` } } fut.onComplete(...) fut.map(...) fut.onComplete(...) fut.map(...)
4. Example • Convert a List[Future[Int]] to a Future[List[Int]] which is completed once all futures in the original list are completed val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val wanted: Future[List[Int]] = Future { futures.map { fut => // await completion of `fut` } } Await.result(fut, 100.millis) Blocks the calling thread! Best case: 
 Underlying ExecutionContext supports managed blocking 
 → size of thread pool increases dynamically → performance degradation!
5. Example • Convert a List[Future[Int]] to a Future[List[Int]] which is completed once all futures in the original list are completed val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val wanted: Future[List[Int]] = Future { futures.map { fut => // await completion of `fut` } } Await.result(fut, 100.millis) Blocks the calling thread! Worst case: 
 Underlying ExecutionContext does not support managed blocking 
 → fewer threads available in thread pool → possible deadlock!
6. Example • Can we use scala-async to improve the code? • await has the type we need: def await[T](future: Future[T]): T val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val wanted: Future[List[Int]] = Future { futures.map { fut => // await completion of `fut` } } await(fut) Invalid! Limitation of scala-async: 
 await cannot be nested under a lambda! (or a local method, object or class) await(fut)
7. Example • Convert a List[Future[Int]] to a Future[List[Int]] which is completed once all futures in the original list are completed val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val wanted: Future[List[Int]] = Future { futures.map { fut => // await completion of `fut` } } Await.result(fut, 100.millis) Instead of calling List.map …
8. Example • Convert a List[Future[Int]] to a Future[List[Int]] which is completed once all futures in the original list are completed val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val wanted: Future[List[Int]] = Future.sequence(futures) We should be calling Future.sequence! From the ScalaDoc: 
 “Simple version of Future.traverse. Asynchronously and non-blockingly transforms, in essence, a IterableOnce[Future[A]] into a Future[IterableOnce[A]]. Useful for reducing many Futures into a single Future.” Essentially, a specialized method to deal with collections of futures.
9. Example • What if the list in the resulting Future[List[Int]] should only contain numbers less than 500? val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val temp: Future[List[List[Int]]] = Future.sequence(futures.map { fut => fut.map(v => if v < 500 then List(v) else List()) }) val wanted: Future[List[Int]] = temp.map(nested => nested.flatten) Is there a simpler way?? The code uses the methods sequence, map for futures, list methods cannot be reused!
10. Issues • Breaking out of the Future monad is tricky: • Calling Await.result can lead to performance degradation or deadlock • scala-async does not permit calling await nested under a lambda • Result: 
 Instead of using familiar methods on collections, future-specific methods had to be used • Not being able to use collection methods with concurrent code is a problem • Collections provide hundreds of methods… • … duplicate all those methods for concurrent code?!
11. A better way val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val wanted: Future[List[Int]] = Future { futures.flatMap { fut => val v = fut.await if v < 500 then List(v) else List() } } Suspend enclosing future without blocking any underlying OS thread!
12. Direct comparison val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val temp: Future[List[List[Int]]] = Future.sequence(futures.map { fut => fut.map(v => if v < 500 then List(v) else List()) }) val wanted: Future[List[Int]] = temp.map(nested => nested.flatten)
13. A better way, using optional braces in Scala 3 val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val wanted: Future[List[Int]] = Future: futures.flatMap: fut => val v = fut.await if v < 500 then List(v) else List()
14. Benefits val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val wanted: Future[List[Int]] = Future: futures.flatMap: fut => val v = fut.await if v < 500 then List(v) else List() Benefits: • Can simply use List.flatMap, no need for Future.sequence and Future.map • No need for intermediate nested list • Effect safety using capabilities (experimental)
15. How is this possible? • Gears — an experimental concurrency library for Scala 3 • Key features: • Lightweight concurrency • This means: • Concurrent tasks are cheap (memory and context switching) • Concurrent tasks can always suspend without blocking any underlying OS thread • On the JVM: based on virtual threads in JDK 21 (see JEP 444) • On Scala Native: based on delimited continuations • Scala.js not supported • Effects as capabilities • Effect polymorphism enables reusing higher-order functions (like List.map) • In the future: ensuring effect safety using capture checking (experimental)
16. Effects as capabilities • Concurrent programming is effectful! • Starting a concurrent task/computation is an effect, requiring resources such as a thread pool • In fact, this effect has been specified in Scala 2 using an implicit parameter: 
 (implicit executor: ExecutionContext) • Suspending a computation in order to wait for the result of a future is an effect • Message passing (send/receive) is an effect • Etc.
17. Effects in Gears • Concurrency effects expressed as context parameters • “Implicit parameters” in Scala 2 parlance • Later: as true capabilities using capture checking trait Future[+T] extends ..., Cancellable: ... def awaitResult(using ac: Async): Try[T] object Future: ... def apply[T](body: Spawn ?=> T) (using async: Async, spawnable: Spawn & async.type): Future[T] ?=> introduces a context function where Spawn is a context parameter awaitResult can only be called in a context where a given of type Async is in scope
18. Gears: Example • Code in blue: using Gears • Comments in green: 
 inferred by type checker @main def main() = Async.blocking { // (using Spawn) => val futures: List[Future[Int]] = for (i <- List.range(1, 10)) yield Future { i * 101 } val wanted: Future[List[Int]] = Future { // (using Spawn) => futures.map(fut => fut.await) } println(wanted.await) } Spawn is like Async (permitting suspension) but adds the capability to spawn concurrent computations Executes body on current thread which suspends upon await Remarks: • Async.blocking is the only way to obtain a Spawn capability • Therefore, every Gears app needs at least one call to Async.blocking!
19. Effect polymorphism • Suppose a function fun: Int => Int has an effect E • The effect E could be to (potentially) perform I/O or throw an exception • What should be the effect of the following call to map? • The effect of lst.map(fun) is equal to the effect of fun! • This means List.map is polymorphic in the effect of its argument val lst = List.range(1, 10) lst.map(fun) Lukas Rytz, Martin Odersky, Philipp Haller: Lightweight Polymorphic Effects. ECOOP 2012: 258-282 
 https://doi.org/10.1007/978-3-642-31057-7_13
20. Capabilities and effect polymorphism • The function type T -> S represents pure functions which are effect-free (experimental) • The function type T => S represents effectful functions • Higher-order functions are effect-polymorphic by default • Example: the existing type definition of List.map expresses effect polymorphism: • Therefore, it is possible to pass a closure with Async effect to List.map, and the resulting call to map will have Async effect: abstract class List[+A] extends ...: def map[B](f: A => B): List[B] futures.map(fut => fut.await)
21. Capabilities and capture checking • Capture checking: an experimental language feature for checking and ensuring effect safety • In the context of Gears: • Enforce scoping rules for Async/Spawn contexts • Functions taking an Async capability should not capture the capability such that its lifetime exceeds that of the function’s body • Tutorial on capture checking: • See https://docs.scala-lang.org/scala3/reference/experimental/cc.html • Research paper with more details and theoretical foundations: Aleksander Boruch-Gruszecki, Martin Odersky, Edward Lee, Ondrej Lhoták, Jonathan Immanuel Brachthäuser: Capturing Types. ACM Trans. Program. Lang. Syst. 45(4): 21:1-21:52 (2023) https://doi.org/10.1145/3618003
22. What we are working on • Ensuring data-race freedom in Scala • Supporting different concurrency models (actors, X10 async/finish, …) • Ownership and ownership transfer in Scala • Other topics: • Concurrent programming, e.g., deterministic concurrency • Distributed programming • Provable failure transparency (Portals: https://www.portals-project.org/) Haller and Loiko. LaCasa: Lightweight af fi nity and object capabilities in Scala. OOPSLA 2016
 https://doi.org/10.1145/2983990.2984042 Willenbrink. A Type System for Ensuring Safe, Structured Concurrency in Scala. MSc thesis, KTH Royal Institute of Technology, Sweden, 2024. To appear Haller. Enhancing closures in Scala 3 with Spores3. Scala Symposium 2022: 22-27 https://doi.org/10.1145/3550198.3550428
23. Conclusion • Scala 3 and capture checking provide a new approach to effects and effect polymorphism • Gears is an experimental concurrency library which builds on • new runtime support for lightweight threads (virtual threads on JDK 21) • Scala 3 contextual abstractions (context parameters, context functions) • Capture checking (experimental) holds great promise to make effect-polymorphic lightweight concurrency safer • Ensure scoping rules for capabilities (Async, Spawn, etc.) • Ongoing research explores data-race freedom