Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Reactive Programming in Scala

Philipp Haller
April 07, 2016
280

Reactive Programming in Scala

Philipp Haller

April 07, 2016
Tweet

Transcript

  1. Reactive Programming in Scala
    Philipp Haller
    KTH Royal Institute of Technology
    Sweden
    parallel 2016
    Heidelberg, Germany, April 2016

    View Slide

  2. Philipp Haller
    What is Reactive
    Programming?
    • Programming of systems with desired properties:
    • Responsiveness
    • Scalability
    • Resiliency
    • Elasticity

    View Slide

  3. Philipp Haller
    Example
    Source: https://blog.twitter.com/2009/inauguration-day-twitter
    “We saw 5x normal tweets-per-second
    and about 4x tweets-per-minute [..]”
    Scalability
    requires resiliency!
    Core message queue was
    based on Scala Actors

    View Slide

  4. Philipp Haller
    Simplified
    • Responsiveness and scalability implies need for
    resiliency ➟ distribution
    • Simplified properties of reactive programming:
    • Responsiveness
    • Scalability
    • “Utilization”

    View Slide

  5. Philipp Haller
    Fault Tolerance
    • What about fault-tolerant systems that are not
    “planet-scale”?
    • Need at least 2 machines for fault tolerance!
    • Result: distributed system that is inherently
    concurrent
    Different from
    data-parallel system!

    View Slide

  6. Philipp Haller
    Programming a Concurrent World
    • How to compose programs handling
    • asynchronous events?
    • streams of asynchronous events?
    • distributed events?
    • Callbacks? Probably not.
    • ➟ Programming abstractions for concurrency!

    View Slide

  7. Philipp Haller
    Reactive Programming
    • Difference between reactive computing and reactive
    programming:
    • Programming is about composition and
    abstraction
    • Goal: simplify concurrency and distribution

    View Slide

  8. Philipp Haller
    What is Scala?
    • Combines object-oriented and functional programming
    • Statically typed
    • Lightweight syntax
    • Fully interoperable with Java
    • Scala.js: JavaScript backend
    • As fast as Java

    View Slide

  9. Philipp Haller
    What is Scala?
    • Scala = “Scalable Language”
    • Principles of composition and abstraction
    independent of program size
    • Cf. Guy Steele, “Growing a Language”, OOPSLA ‘98
    Video: https://www.youtube.com/watch?v=_ahvzDzKdB0
    Different notion
    of scalability!

    View Slide

  10. Philipp Haller
    Principles of Scala
    • Integration of object-oriented and functional
    programming
    • Functions are objects
    • First-class, higher-order, curried, partial, etc.
    • First-class modules = objects
    • Abstract type members, path-dependent types
    • Generic programming
    • Example: “type classes = objects + implicits”

    View Slide

  11. Philipp Haller
    History of Scala
    • Created by Martin Odersky (EPFL, Switzerland)
    • 1997: Pizza
    • Java extended with generic types, first-class functions, and ADTs/
    pattern matching
    • 1998: GJ (“Generic Java”)
    • Odersky’s new javac adopted by Sun Microsystems
    • 1999–2001: Funnel
    • Since 2001: development of Scala
    • First public release: 2004-01-20
    • First Scala conference 2008, first Scala Days conference 2010, Lausanne

    View Slide

  12. Philipp Haller
    Practice
    • Express flexible patterns in concise and type-safe way
    • Scala as a “growable” language
    • Domain-specific languages (DSLs)
    • Embedded DSL: DSL = library
    Programming models as libraries!

    View Slide

  13. Philipp Haller
    Why a Growable Language
    for Concurrency?
    • Concurrency not a solved problem ➟ development of
    new programming models
    • Futures, promises
    • async/await
    • STM
    • Agents
    • Actors
    • Join-calculus
    • Reactive streams
    • CSP
    • Async CML
    • …
    Which one is going to “win”?

    View Slide

  14. Philipp Haller
    Background
    • Authored or co-authored:
    • Scala Actors (2006)
    • Scala futures and promises (2011/2012)
    • Scala Async (2013)
    • Contributed to Akka
    • Akka.js project
    Other proposals and research
    projects:
    • Scala Joins (2008)
    • FlowPools (2012)
    • Spores (safer closures)
    • …

    View Slide

  15. Philipp Haller
    Reactive Programming
    Abstractions
    • Foundation: futures + promises, actors, reactive
    streams
    • New programming models and abstractions in active
    development

    View Slide

  16. Philipp Haller
    Futures
    Creating a future:
    object Future {
    def apply[T](body: => T): Future[T]
    }
    Singleton object
    Generic type
    parameter
    “Code block”
    with result type T
    “Unrelated”
    to the singleton
    object!

    View Slide

  17. Philipp Haller
    Futures: Example
    val firstGoodDeal = Future {
    usedCars.find(car => isGoodDeal(car))
    }
    val firstGoodDeal = Future.apply({
    usedCars.find(car => isGoodDeal(car))
    })
    Short syntax for:
    Type inference:
    val firstGoodDeal = Future.apply[Option[Car]]({
    usedCars.find(car => isGoodDeal(car))
    })
    Type
    Future[Option[Car]]

    View Slide

  18. Philipp Haller
    Reading from a Future
    • Callback
    • Combinator
    • Blocking

    View Slide

  19. Philipp Haller
    Reading from a Future
    • Callback
    • Combinator
    • Blocking
    val firstGoodDeal = ..
    firstGoodDeal.foreach { opt =>
    if (opt.nonEmpty) // hurray, found good deal!
    }

    View Slide

  20. Philipp Haller
    Reading from a Future
    • Callback
    • Combinator
    • Blocking
    val firstGoodDeal = ..
    val firstPrice = firstGoodDeal.map { opt =>
    opt.get.price
    }
    Transform result of
    future, returning new future

    View Slide

  21. Philipp Haller
    Type of a Future
    trait Future[+T] extends Awaitable[T] {
    def foreach[S](f: T => S): Unit
    def map[S](f: T => S): Future[S]
    def flatMap[S](f: T => Future[S]): Future[S]
    // ..
    }

    View Slide

  22. Philipp Haller
    Future Pipelining
    val lowestPrice: Future[Int] =
    fstPrice.flatMap { p1 =>
    sndPrice.map { p2 =>
    math.min(p1, p2)
    }
    }

    View Slide

  23. Philipp Haller
    Collections of Futures
    val goodDeals: List[Future[Option[Car]]] = ..
    val bestDeal: Future[Option[Car]] =
    Future.sequence(goodDeals).map(
    deals => deals.sorted.head
    )

    View Slide

  24. Philipp Haller
    Promise
    Main purpose: create futures for non-lexically-
    scoped asynchronous code
    def after[T](delay: Long, value: T): Future[T]
    Example
    Function for creating a Future that is completed
    with value after delay milliseconds

    View Slide

  25. Philipp Haller
    “after”, Version 1
    def after1[T](delay: Long, value: T) =
    Future {
    Thread.sleep(delay)
    value
    }

    View Slide

  26. Philipp Haller
    “after”, Version 1
    assert(Runtime.getRuntime()
    .availableProcessors() == 8)
    for (_ after1(1000, true)
    val later = after1(1000, true)
    How does it behave?
    Quiz: when is “later” completed?
    Answer: after either ~1 s or ~2 s (most often)

    View Slide

  27. Philipp Haller
    Promise
    object Promise {
    def apply[T](): Promise[T]
    }
    trait Promise[T] {
    def success(value: T): this.type
    def failure(cause: Throwable): this.type
    def future: Future[T]
    }

    View Slide

  28. Philipp Haller
    “after”, Version 2
    def after2[T](delay: Long, value: T) = {
    val promise = Promise[T]()
    timer.schedule(new TimerTask {
    def run(): Unit = promise.success(value)
    }, delay)
    promise.future
    }
    Much better behaved!

    View Slide

  29. Philipp Haller
    What is Async?
    • Scala module
    • "org.scala-lang.modules" %% "scala-async"
    • Purpose: simplify non-blocking concurrency
    • Scala Improvement Proposal SIP-22
    • Releases for Scala 2.10 and 2.11

    View Slide

  30. Philipp Haller
    What Async Provides
    • Future and Promise provide types and operations for
    managing data flow
    • There is very little support for control flow
    • For-comprehensions, ..?
    • Async complements Future and Promise with
    constructs to manage control flow

    View Slide

  31. Philipp Haller
    Programming Model
    Basis: suspendible computations
    • async { .. } — delimit suspendible computation
    • await(future) — suspend computation until
    future is completed

    View Slide

  32. Philipp Haller
    Async
    object Async {
    def async[T](body: => T): Future[T]
    def await[T](future: Future[T]): T
    }

    View Slide

  33. Philipp Haller
    Example
    val fstGoodDeal: Future[Option[Car]] = ..
    val sndGoodDeal: Future[Option[Car]] = ..
    val goodCar = async {
    val car1 = await(fstGoodDeal).get
    val car2 = await(sndGoodDeal).get
    if (car1.price < car2.price) car1
    else car2
    }

    View Slide

  34. Philipp Haller
    Futures vs. Async
    • “Futures and Async: When to Use Which?”, Scala Days
    2014, Berlin
    • Video: https://www.parleys.com/tutorial/futures-async-when-use-which

    View Slide

  35. Philipp Haller
    From Futures to Actors
    • Limitations of futures:
    • At most one completion event per future
    • Overhead when creating many futures
    • How to model distributed systems?

    View Slide

  36. Philipp Haller
    Actors
    • Model of concurrent computation whose universal primitive is
    the “actor” [Hewitt et al. ’73]
    • Actors = concurrent “processes” communicating via asynchronous
    messages
    • Upon reception of a message, an actor may
    • change its behavior/state
    • send messages to actors (including itself)
    • create new actors
    • Fair scheduling
    • Decoupling: message sender cannot fail due to receiver
    Related to active objects

    View Slide

  37. Philipp Haller
    Example
    class ActorWithTasks(tasks: ...) extends Actor {
    ...
    def receive = {
    case TaskFor(workers) =>
    val from = sender
    val requests = (tasks zip workers).map {
    case (task, worker) => worker ? task
    }
    val allDone = Future.sequence(requests)
    allDone andThen { seq =>
    from ! seq.mkString(",")
    }
    }
    } Using Akka (http://akka.io/)

    View Slide

  38. Philipp Haller
    Why Actors?
    Reason 1: simplified concurrency
    • “Share nothing”: strong isolation of actors ➟ no
    race conditions
    • Actors handle at most one message at a time ➟
    sequential reasoning
    • Asynchronous message handling ➟ less risk of
    deadlocks
    • No “inversion of control”: access to own state and
    messages in safe, direct way
    “Macro-step semantics”

    View Slide

  39. Philipp Haller
    Why Actors? (cont’d)
    Reason 2: actors model reality of distributed systems
    • Message sends truly asynchronous
    • Message reception not guaranteed
    • Non-deterministic message ordering
    • Some implementations preserve message ordering
    between pairs of actors
    Therefore, actors well-suited as a foundation for
    distributed systems

    View Slide

  40. Philipp Haller
    Reactive Streams
    • Asynchronous event streams and push notifications: a
    fundamental abstraction for web and mobile apps
    • Typically, event streams have to be scalable, robust, and
    composable
    • Enter reactive streams
    • Based on the duality of iterators and observers
    • Composition using higher-order functions
    • Key feature: asynchronous back-pressure
    www.reactive-streams.org
    Erik Meijer. “Your mouse is a database”: http://queue.acm.org/detail.cfm?id=2169076

    View Slide

  41. Philipp Haller
    Issues (1)
    Composition of actors
    • Combining messaging protocols requires explicit
    tagging of messages ➟ not composable
    • Talk by Aleksandar Prokopec, “Reactors - Road to
    Composable Distributed Computing”, Scala Days
    Berlin 2016
    See also: http://reactors.io/

    View Slide

  42. Philipp Haller
    Issues (2)
    class ActorWithTasks(tasks: ...) extends Actor {
    ...
    def receive = {
    case TaskFor(workers) =>
    val requests = (tasks zip workers).map {
    case (task, worker) => worker ? task
    }
    val allDone = Future.sequence(requests)
    allDone andThen { seq =>
    sender ! seq.mkString(",")
    }
    }
    }

    View Slide

  43. Philipp Haller
    Issues (2)
    Unsafe futures
    • Race conditions via shared state
    • Variable capture
    • “Spores”: Scala improvement proposal for safer
    closures (SIP-21)
    • Environment (captured variables) declared
    explicitly
    • Talk by Heather Miller at Scala Days 2014

    View Slide

  44. Philipp Haller
    Issues (3)
    • Data race safety
    • cf. Rust’s borrow checker
    • Haller and Odersky, “Capabilities for Uniqueness
    and Borrowing”, ECOOP ’10
    • Ensure correct messaging protocols between actors
    • Session types?
    • Talk by Roland Kuhn, “Akka Typed: Between
    Session Types and the Actor Model”, Curry On! 2015

    View Slide

  45. Philipp Haller
    Research at KTH (1)
    • Static prevention of race conditions
    • Via type and effect systems
    • Programming with eventually-consistent data
    • Stronger: deterministic concurrency

    View Slide

  46. Philipp Haller
    Research at KTH (2)
    • Safe deterministic memory management
    • Programming with “off-heap” data
    • Integrated with GC

    View Slide

  47. Philipp Haller
    Conclusion
    • Programming is about abstraction and composition
    • Scala enables new ways to do concurrent, reactive,
    and distributed programming
    • Lots of exciting developments
    “[..] the purpose of abstracting is not to be
    vague, but to create a new semantic level in
    which one can be absolutely precise”
    Edsger W. Dijkstra. “The Humble Programmer”. ACM Turing Lecture 1972
    https://www.cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD340.html

    View Slide