Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Reactive Programming in Scala

3b84657fdb075382e3781310ca8a9a70?s=47 Philipp Haller
April 07, 2016
140

Reactive Programming in Scala

3b84657fdb075382e3781310ca8a9a70?s=128

Philipp Haller

April 07, 2016
Tweet

Transcript

  1. Reactive Programming in Scala Philipp Haller KTH Royal Institute of

    Technology Sweden parallel 2016 Heidelberg, Germany, April 2016
  2. Philipp Haller What is Reactive Programming? • Programming of systems

    with desired properties: • Responsiveness • Scalability • Resiliency • Elasticity
  3. Philipp Haller Example Source: https://blog.twitter.com/2009/inauguration-day-twitter “We saw 5x normal tweets-per-second

    and about 4x tweets-per-minute [..]” Scalability requires resiliency! Core message queue was based on Scala Actors
  4. Philipp Haller Simplified • Responsiveness and scalability implies need for

    resiliency ➟ distribution • Simplified properties of reactive programming: • Responsiveness • Scalability • “Utilization”
  5. Philipp Haller Fault Tolerance • What about fault-tolerant systems that

    are not “planet-scale”? • Need at least 2 machines for fault tolerance! • Result: distributed system that is inherently concurrent Different from data-parallel system!
  6. Philipp Haller Programming a Concurrent World • How to compose

    programs handling • asynchronous events? • streams of asynchronous events? • distributed events? • Callbacks? Probably not. • ➟ Programming abstractions for concurrency!
  7. Philipp Haller Reactive Programming • Difference between reactive computing and

    reactive programming: • Programming is about composition and abstraction • Goal: simplify concurrency and distribution
  8. Philipp Haller What is Scala? • Combines object-oriented and functional

    programming • Statically typed • Lightweight syntax • Fully interoperable with Java • Scala.js: JavaScript backend • As fast as Java
  9. Philipp Haller What is Scala? • Scala = “Scalable Language”

    • Principles of composition and abstraction independent of program size • Cf. Guy Steele, “Growing a Language”, OOPSLA ‘98 Video: https://www.youtube.com/watch?v=_ahvzDzKdB0 Different notion of scalability!
  10. Philipp Haller Principles of Scala • Integration of object-oriented and

    functional programming • Functions are objects • First-class, higher-order, curried, partial, etc. • First-class modules = objects • Abstract type members, path-dependent types • Generic programming • Example: “type classes = objects + implicits”
  11. Philipp Haller History of Scala • Created by Martin Odersky

    (EPFL, Switzerland) • 1997: Pizza • Java extended with generic types, first-class functions, and ADTs/ pattern matching • 1998: GJ (“Generic Java”) • Odersky’s new javac adopted by Sun Microsystems • 1999–2001: Funnel • Since 2001: development of Scala • First public release: 2004-01-20 • First Scala conference 2008, first Scala Days conference 2010, Lausanne
  12. Philipp Haller Practice • Express flexible patterns in concise and

    type-safe way • Scala as a “growable” language • Domain-specific languages (DSLs) • Embedded DSL: DSL = library Programming models as libraries!
  13. Philipp Haller Why a Growable Language for Concurrency? • Concurrency

    not a solved problem ➟ development of new programming models • Futures, promises • async/await • STM • Agents • Actors • Join-calculus • Reactive streams • CSP • Async CML • … Which one is going to “win”?
  14. Philipp Haller Background • Authored or co-authored: • Scala Actors

    (2006) • Scala futures and promises (2011/2012) • Scala Async (2013) • Contributed to Akka • Akka.js project Other proposals and research projects: • Scala Joins (2008) • FlowPools (2012) • Spores (safer closures) • …
  15. Philipp Haller Reactive Programming Abstractions • Foundation: futures + promises,

    actors, reactive streams • New programming models and abstractions in active development
  16. Philipp Haller Futures Creating a future: object Future { def

    apply[T](body: => T): Future[T] } Singleton object Generic type parameter “Code block” with result type T “Unrelated” to the singleton object!
  17. Philipp Haller Futures: Example val firstGoodDeal = Future { usedCars.find(car

    => isGoodDeal(car)) } val firstGoodDeal = Future.apply({ usedCars.find(car => isGoodDeal(car)) }) Short syntax for: Type inference: val firstGoodDeal = Future.apply[Option[Car]]({ usedCars.find(car => isGoodDeal(car)) }) Type Future[Option[Car]]
  18. Philipp Haller Reading from a Future • Callback • Combinator

    • Blocking
  19. Philipp Haller Reading from a Future • Callback • Combinator

    • Blocking val firstGoodDeal = .. firstGoodDeal.foreach { opt => if (opt.nonEmpty) // hurray, found good deal! }
  20. Philipp Haller Reading from a Future • Callback • Combinator

    • Blocking val firstGoodDeal = .. val firstPrice = firstGoodDeal.map { opt => opt.get.price } Transform result of future, returning new future
  21. Philipp Haller Type of a Future trait Future[+T] extends Awaitable[T]

    { def foreach[S](f: T => S): Unit def map[S](f: T => S): Future[S] def flatMap[S](f: T => Future[S]): Future[S] // .. }
  22. Philipp Haller Future Pipelining val lowestPrice: Future[Int] = fstPrice.flatMap {

    p1 => sndPrice.map { p2 => math.min(p1, p2) } }
  23. Philipp Haller Collections of Futures val goodDeals: List[Future[Option[Car]]] = ..

    val bestDeal: Future[Option[Car]] = Future.sequence(goodDeals).map( deals => deals.sorted.head )
  24. Philipp Haller Promise Main purpose: create futures for non-lexically- scoped

    asynchronous code def after[T](delay: Long, value: T): Future[T] Example Function for creating a Future that is completed with value after delay milliseconds
  25. Philipp Haller “after”, Version 1 def after1[T](delay: Long, value: T)

    = Future { Thread.sleep(delay) value }
  26. Philipp Haller “after”, Version 1 assert(Runtime.getRuntime() .availableProcessors() == 8) for

    (_ <- 1 to 8) yield after1(1000, true) val later = after1(1000, true) How does it behave? Quiz: when is “later” completed? Answer: after either ~1 s or ~2 s (most often)
  27. Philipp Haller Promise object Promise { def apply[T](): Promise[T] }

    trait Promise[T] { def success(value: T): this.type def failure(cause: Throwable): this.type def future: Future[T] }
  28. Philipp Haller “after”, Version 2 def after2[T](delay: Long, value: T)

    = { val promise = Promise[T]() timer.schedule(new TimerTask { def run(): Unit = promise.success(value) }, delay) promise.future } Much better behaved!
  29. Philipp Haller What is Async? • Scala module • "org.scala-lang.modules"

    %% "scala-async" • Purpose: simplify non-blocking concurrency • Scala Improvement Proposal SIP-22 • Releases for Scala 2.10 and 2.11
  30. Philipp Haller What Async Provides • Future and Promise provide

    types and operations for managing data flow • There is very little support for control flow • For-comprehensions, ..? • Async complements Future and Promise with constructs to manage control flow
  31. Philipp Haller Programming Model Basis: suspendible computations • async {

    .. } — delimit suspendible computation • await(future) — suspend computation until future is completed
  32. Philipp Haller Async object Async { def async[T](body: => T):

    Future[T] def await[T](future: Future[T]): T }
  33. Philipp Haller Example val fstGoodDeal: Future[Option[Car]] = .. val sndGoodDeal:

    Future[Option[Car]] = .. val goodCar = async { val car1 = await(fstGoodDeal).get val car2 = await(sndGoodDeal).get if (car1.price < car2.price) car1 else car2 }
  34. Philipp Haller Futures vs. Async • “Futures and Async: When

    to Use Which?”, Scala Days 2014, Berlin • Video: https://www.parleys.com/tutorial/futures-async-when-use-which
  35. Philipp Haller From Futures to Actors • Limitations of futures:

    • At most one completion event per future • Overhead when creating many futures • How to model distributed systems?
  36. Philipp Haller Actors • Model of concurrent computation whose universal

    primitive is the “actor” [Hewitt et al. ’73] • Actors = concurrent “processes” communicating via asynchronous messages • Upon reception of a message, an actor may • change its behavior/state • send messages to actors (including itself) • create new actors • Fair scheduling • Decoupling: message sender cannot fail due to receiver Related to active objects
  37. Philipp Haller Example class ActorWithTasks(tasks: ...) extends Actor { ...

    def receive = { case TaskFor(workers) => val from = sender val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => from ! seq.mkString(",") } } } Using Akka (http://akka.io/)
  38. Philipp Haller Why Actors? Reason 1: simplified concurrency • “Share

    nothing”: strong isolation of actors ➟ no race conditions • Actors handle at most one message at a time ➟ sequential reasoning • Asynchronous message handling ➟ less risk of deadlocks • No “inversion of control”: access to own state and messages in safe, direct way “Macro-step semantics”
  39. Philipp Haller Why Actors? (cont’d) Reason 2: actors model reality

    of distributed systems • Message sends truly asynchronous • Message reception not guaranteed • Non-deterministic message ordering • Some implementations preserve message ordering between pairs of actors Therefore, actors well-suited as a foundation for distributed systems
  40. Philipp Haller Reactive Streams • Asynchronous event streams and push

    notifications: a fundamental abstraction for web and mobile apps • Typically, event streams have to be scalable, robust, and composable • Enter reactive streams • Based on the duality of iterators and observers • Composition using higher-order functions • Key feature: asynchronous back-pressure www.reactive-streams.org Erik Meijer. “Your mouse is a database”: http://queue.acm.org/detail.cfm?id=2169076
  41. Philipp Haller Issues (1) Composition of actors • Combining messaging

    protocols requires explicit tagging of messages ➟ not composable • Talk by Aleksandar Prokopec, “Reactors - Road to Composable Distributed Computing”, Scala Days Berlin 2016 See also: http://reactors.io/
  42. Philipp Haller Issues (2) class ActorWithTasks(tasks: ...) extends Actor {

    ... def receive = { case TaskFor(workers) => val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => sender ! seq.mkString(",") } } }
  43. Philipp Haller Issues (2) Unsafe futures • Race conditions via

    shared state • Variable capture • “Spores”: Scala improvement proposal for safer closures (SIP-21) • Environment (captured variables) declared explicitly • Talk by Heather Miller at Scala Days 2014
  44. Philipp Haller Issues (3) • Data race safety • cf.

    Rust’s borrow checker • Haller and Odersky, “Capabilities for Uniqueness and Borrowing”, ECOOP ’10 • Ensure correct messaging protocols between actors • Session types? • Talk by Roland Kuhn, “Akka Typed: Between Session Types and the Actor Model”, Curry On! 2015
  45. Philipp Haller Research at KTH (1) • Static prevention of

    race conditions • Via type and effect systems • Programming with eventually-consistent data • Stronger: deterministic concurrency
  46. Philipp Haller Research at KTH (2) • Safe deterministic memory

    management • Programming with “off-heap” data • Integrated with GC
  47. Philipp Haller Conclusion • Programming is about abstraction and composition

    • Scala enables new ways to do concurrent, reactive, and distributed programming • Lots of exciting developments “[..] the purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise” Edsger W. Dijkstra. “The Humble Programmer”. ACM Turing Lecture 1972 https://www.cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD340.html