Reactive Programming in Scala

Transcript

1. Reactive Programming in Scala Philipp Haller KTH Royal Institute of Technology Sweden parallel 2016 Heidelberg, Germany, April 2016
2. Philipp Haller What is Reactive Programming? • Programming of systems with desired properties: • Responsiveness • Scalability • Resiliency • Elasticity
3. Philipp Haller Example Source: https://blog.twitter.com/2009/inauguration-day-twitter “We saw 5x normal tweets-per-second and about 4x tweets-per-minute [..]” Scalability requires resiliency! Core message queue was based on Scala Actors
4. Philipp Haller Simplified • Responsiveness and scalability implies need for resiliency ➟ distribution • Simplified properties of reactive programming: • Responsiveness • Scalability • “Utilization”
5. Philipp Haller Fault Tolerance • What about fault-tolerant systems that are not “planet-scale”? • Need at least 2 machines for fault tolerance! • Result: distributed system that is inherently concurrent Different from data-parallel system!
6. Philipp Haller Programming a Concurrent World • How to compose programs handling • asynchronous events? • streams of asynchronous events? • distributed events? • Callbacks? Probably not. • ➟ Programming abstractions for concurrency!
7. Philipp Haller Reactive Programming • Difference between reactive computing and reactive programming: • Programming is about composition and abstraction • Goal: simplify concurrency and distribution
8. Philipp Haller What is Scala? • Combines object-oriented and functional programming • Statically typed • Lightweight syntax • Fully interoperable with Java • Scala.js: JavaScript backend • As fast as Java
9. Philipp Haller What is Scala? • Scala = “Scalable Language” • Principles of composition and abstraction independent of program size • Cf. Guy Steele, “Growing a Language”, OOPSLA ‘98 Video: https://www.youtube.com/watch?v=_ahvzDzKdB0 Different notion of scalability!
10. Philipp Haller Principles of Scala • Integration of object-oriented and functional programming • Functions are objects • First-class, higher-order, curried, partial, etc. • First-class modules = objects • Abstract type members, path-dependent types • Generic programming • Example: “type classes = objects + implicits”
11. Philipp Haller History of Scala • Created by Martin Odersky (EPFL, Switzerland) • 1997: Pizza • Java extended with generic types, first-class functions, and ADTs/ pattern matching • 1998: GJ (“Generic Java”) • Odersky’s new javac adopted by Sun Microsystems • 1999–2001: Funnel • Since 2001: development of Scala • First public release: 2004-01-20 • First Scala conference 2008, first Scala Days conference 2010, Lausanne
12. Philipp Haller Practice • Express flexible patterns in concise and type-safe way • Scala as a “growable” language • Domain-specific languages (DSLs) • Embedded DSL: DSL = library Programming models as libraries!
13. Philipp Haller Why a Growable Language for Concurrency? • Concurrency not a solved problem ➟ development of new programming models • Futures, promises • async/await • STM • Agents • Actors • Join-calculus • Reactive streams • CSP • Async CML • … Which one is going to “win”?
14. Philipp Haller Background • Authored or co-authored: • Scala Actors (2006) • Scala futures and promises (2011/2012) • Scala Async (2013) • Contributed to Akka • Akka.js project Other proposals and research projects: • Scala Joins (2008) • FlowPools (2012) • Spores (safer closures) • …
15. Philipp Haller Reactive Programming Abstractions • Foundation: futures + promises, actors, reactive streams • New programming models and abstractions in active development
16. Philipp Haller Futures Creating a future: object Future { def apply[T](body: => T): Future[T] } Singleton object Generic type parameter “Code block” with result type T “Unrelated” to the singleton object!
17. Philipp Haller Futures: Example val firstGoodDeal = Future { usedCars.find(car => isGoodDeal(car)) } val firstGoodDeal = Future.apply({ usedCars.find(car => isGoodDeal(car)) }) Short syntax for: Type inference: val firstGoodDeal = Future.apply[Option[Car]]({ usedCars.find(car => isGoodDeal(car)) }) Type Future[Option[Car]]
18. Philipp Haller Reading from a Future • Callback • Combinator • Blocking
19. Philipp Haller Reading from a Future • Callback • Combinator • Blocking val firstGoodDeal = .. firstGoodDeal.foreach { opt => if (opt.nonEmpty) // hurray, found good deal! }
20. Philipp Haller Reading from a Future • Callback • Combinator • Blocking val firstGoodDeal = .. val firstPrice = firstGoodDeal.map { opt => opt.get.price } Transform result of future, returning new future
21. Philipp Haller Type of a Future trait Future[+T] extends Awaitable[T] { def foreach[S](f: T => S): Unit def map[S](f: T => S): Future[S] def flatMap[S](f: T => Future[S]): Future[S] // .. }
22. Philipp Haller Future Pipelining val lowestPrice: Future[Int] = fstPrice.flatMap { p1 => sndPrice.map { p2 => math.min(p1, p2) } }
23. Philipp Haller Collections of Futures val goodDeals: List[Future[Option[Car]]] = .. val bestDeal: Future[Option[Car]] = Future.sequence(goodDeals).map( deals => deals.sorted.head )
24. Philipp Haller Promise Main purpose: create futures for non-lexically- scoped asynchronous code def after[T](delay: Long, value: T): Future[T] Example Function for creating a Future that is completed with value after delay milliseconds
25. Philipp Haller “after”, Version 1 def after1[T](delay: Long, value: T) = Future { Thread.sleep(delay) value }
26. Philipp Haller “after”, Version 1 assert(Runtime.getRuntime() .availableProcessors() == 8) for (_ <- 1 to 8) yield after1(1000, true) val later = after1(1000, true) How does it behave? Quiz: when is “later” completed? Answer: after either ~1 s or ~2 s (most often)
27. Philipp Haller Promise object Promise { def apply[T](): Promise[T] } trait Promise[T] { def success(value: T): this.type def failure(cause: Throwable): this.type def future: Future[T] }
28. Philipp Haller “after”, Version 2 def after2[T](delay: Long, value: T) = { val promise = Promise[T]() timer.schedule(new TimerTask { def run(): Unit = promise.success(value) }, delay) promise.future } Much better behaved!
29. Philipp Haller What is Async? • Scala module • "org.scala-lang.modules" %% "scala-async" • Purpose: simplify non-blocking concurrency • Scala Improvement Proposal SIP-22 • Releases for Scala 2.10 and 2.11
30. Philipp Haller What Async Provides • Future and Promise provide types and operations for managing data flow • There is very little support for control flow • For-comprehensions, ..? • Async complements Future and Promise with constructs to manage control flow
31. Philipp Haller Programming Model Basis: suspendible computations • async { .. } — delimit suspendible computation • await(future) — suspend computation until future is completed
32. Philipp Haller Async object Async { def async[T](body: => T): Future[T] def await[T](future: Future[T]): T }
33. Philipp Haller Example val fstGoodDeal: Future[Option[Car]] = .. val sndGoodDeal: Future[Option[Car]] = .. val goodCar = async { val car1 = await(fstGoodDeal).get val car2 = await(sndGoodDeal).get if (car1.price < car2.price) car1 else car2 }
34. Philipp Haller Futures vs. Async • “Futures and Async: When to Use Which?”, Scala Days 2014, Berlin • Video: https://www.parleys.com/tutorial/futures-async-when-use-which
35. Philipp Haller From Futures to Actors • Limitations of futures: • At most one completion event per future • Overhead when creating many futures • How to model distributed systems?
36. Philipp Haller Actors • Model of concurrent computation whose universal primitive is the “actor” [Hewitt et al. ’73] • Actors = concurrent “processes” communicating via asynchronous messages • Upon reception of a message, an actor may • change its behavior/state • send messages to actors (including itself) • create new actors • Fair scheduling • Decoupling: message sender cannot fail due to receiver Related to active objects
37. Philipp Haller Example class ActorWithTasks(tasks: ...) extends Actor { ... def receive = { case TaskFor(workers) => val from = sender val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => from ! seq.mkString(",") } } } Using Akka (http://akka.io/)
38. Philipp Haller Why Actors? Reason 1: simplified concurrency • “Share nothing”: strong isolation of actors ➟ no race conditions • Actors handle at most one message at a time ➟ sequential reasoning • Asynchronous message handling ➟ less risk of deadlocks • No “inversion of control”: access to own state and messages in safe, direct way “Macro-step semantics”
39. Philipp Haller Why Actors? (cont’d) Reason 2: actors model reality of distributed systems • Message sends truly asynchronous • Message reception not guaranteed • Non-deterministic message ordering • Some implementations preserve message ordering between pairs of actors Therefore, actors well-suited as a foundation for distributed systems
40. Philipp Haller Reactive Streams • Asynchronous event streams and push notifications: a fundamental abstraction for web and mobile apps • Typically, event streams have to be scalable, robust, and composable • Enter reactive streams • Based on the duality of iterators and observers • Composition using higher-order functions • Key feature: asynchronous back-pressure www.reactive-streams.org Erik Meijer. “Your mouse is a database”: http://queue.acm.org/detail.cfm?id=2169076
41. Philipp Haller Issues (1) Composition of actors • Combining messaging protocols requires explicit tagging of messages ➟ not composable • Talk by Aleksandar Prokopec, “Reactors - Road to Composable Distributed Computing”, Scala Days Berlin 2016 See also: http://reactors.io/
42. Philipp Haller Issues (2) class ActorWithTasks(tasks: ...) extends Actor { ... def receive = { case TaskFor(workers) => val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => sender ! seq.mkString(",") } } }
43. Philipp Haller Issues (2) Unsafe futures • Race conditions via shared state • Variable capture • “Spores”: Scala improvement proposal for safer closures (SIP-21) • Environment (captured variables) declared explicitly • Talk by Heather Miller at Scala Days 2014
44. Philipp Haller Issues (3) • Data race safety • cf. Rust’s borrow checker • Haller and Odersky, “Capabilities for Uniqueness and Borrowing”, ECOOP ’10 • Ensure correct messaging protocols between actors • Session types? • Talk by Roland Kuhn, “Akka Typed: Between Session Types and the Actor Model”, Curry On! 2015
45. Philipp Haller Research at KTH (1) • Static prevention of race conditions • Via type and effect systems • Programming with eventually-consistent data • Stronger: deterministic concurrency
46. Philipp Haller Research at KTH (2) • Safe deterministic memory management • Programming with “off-heap” data • Integrated with GC
47. Philipp Haller Conclusion • Programming is about abstraction and composition • Scala enables new ways to do concurrent, reactive, and distributed programming • Lots of exciting developments “[..] the purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise” Edsger W. Dijkstra. “The Humble Programmer”. ACM Turing Lecture 1972 https://www.cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD340.html