Concurrency with Cats Effect

Concurrency with Cats Effect

The Cats Effect library recently reached its 1.0 release, providing powerful data types and type classes for purely functional effectful programming. In this talk, we’ll focus on building programs using fiber based concurrency and the synchronization primitives provided by Cats Effect. We’ll see how FS2 (Functional Streams for Scala) uses such primitives to build more advanced concurrent data types like bounded and unbounded queues. Finally, we’ll see how to apply these techniques to application design.

C9ab1175a6981a2f67ce8d08aa17c15a?s=128

Michael Pilquist

November 13, 2018
Tweet

Transcript

  1. Polymorphic Nonsense Concurrency with Cats Effect Michael Pilquist // @mpilquist

    Scale By The Bay November 2018
  2. Cats Effect Evolution in 2018 2 MonadError Sync Async Effect

    LiftIO MonadError Bracket Sync Async LiftIO Concurrent Effect ConcurrentEffect Defer 0.8 (Jan 18) 1.0 (Sept 18)
  3. Cats Effect 1.0 3 MonadError Sync Async Effect LiftIO MonadError

    Bracket Sync Async LiftIO Concurrent Effect ConcurrentEffect Defer 1.0 (Sept 18) Principled & polymorphic type classes: • resource management (Bracket) • concurrency (Concurrent) Principled & polymorphic utilities: • concurrent data structures • timers & clocks • thread pool management Effect types: • Much faster IO • SyncIO (guaranteed sync execution)
  4. Fibers 4 trait Concurrent[F[_]] extends Async[F] { def start[A](fa: F[A]):

    F[Fiber[F, A]] } trait Fiber[F[_], A] { def join: F[A] def cancel: CancelToken[F] } type CancelToken[F[_]] = F[Unit]
  5. Fibers 5 trait Concurrent[F[_]] extends Async[F] { def start[A](fa: F[A]):

    F[Fiber[F, A]] } trait Fiber[F[_], A] { def join: F[A] def cancel: CancelToken[F] } type CancelToken[F[_]] = F[Unit] • Example of "semantic blocking" • Semantic blocking involves NO thread blocking • Similar to calling map or flatMap on a Future
  6. Example Fiber Use Case 6 import cats.effect.Concurrent import cats.effect.implicits._ def

    parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] =
  7. Example Fiber Use Case 7 import cats.effect.Concurrent import cats.effect.implicits._ def

    parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = for { fiberA  fa.start fiberB  fb.start a  fiberA.join b  fiberB.join } yield (a, b)
  8. Example Fiber Use Case 8 import cats.effect.Concurrent import cats.effect.implicits._ def

    parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = for { fiberA  fa.start fiberB  fb.start a  fiberA.join b  fiberB.join } yield (a, b) parTupled(F.delay(throw new RuntimeException), fb)
  9. Example Fiber Use Case 8 import cats.effect.Concurrent import cats.effect.implicits._ def

    parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = for { fiberA  fa.start fiberB  fb.start a  fiberA.join b  fiberB.join } yield (a, b) parTupled(F.delay(throw new RuntimeException), fb) Fiber Leak! • Leaks fibers on errors • Leaks fibers on cancelation
  10. Example Fiber Use Case Corrected 9 import cats.effect.Concurrent import cats.effect.implicits._

    import cats.implicits._ def parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = fa.start.bracket(fiberA  fb.start.bracket(fiberB  (fiberA.join, fiberB.join).tupled )(_.cancel) )(_.cancel) Replaced flatMap with bracket
  11. Ref 10 package cats.effect.concurrent trait Ref[F[_], A] { def get:

    F[A] def set(a: A): F[Unit] def access: F[(A, A  F[Boolean])] def update(f: A  A): F[Unit] def modify(f: A  (A, B)): F[B] } object Ref { def of[F[_]: Sync, A](a: A): F[Ref[F, A]] } • Effectful version of j.u.c.AtomicReference • Always has a value (never "unset") • Low level APIs: get, set, access • High level APIs: update, modify • Constructing a Ref is effectful
  12. Ref 11 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))

     r.flatMap(_.get) ??? unsafeRunSync
  13. Ref 12 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))

     r.flatMap(_.get) 2 unsafeRunSync
  14. Ref 13 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))

     r.flatMap(_.get) •Create a ref initialized to 2 •Update its value to 5 times its current value
  15. Ref 14 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))

     r.flatMap(_.get) •Create a ref initialized to 2 •Get its current value
  16. Ref 14 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))

     r.flatMap(_.get) •Create a ref initialized to 2 •Get its current value Misuse of Ref constructor
  17. Ref 15 val mk = Ref.of[IO, Int](2) mk.flatMap(r  r.update(_

    * 5)  r.get)
  18. Ref & Fiber & Bracket Use Case: Debouncing UI Events

    16 class Debouncer[F[_], A] { def apply(t: F[A]): F[A] = ??? def cancel: F[Unit] = ??? } • Delay evaluation of t • If while delayed, another t is supplied, cancel original t and delay new t
  19. Ref & Fiber & Bracket Use Case: Debouncing UI Events

    17 class Debouncer[F[_], A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = ??? def cancel: F[Unit] = ??? } Current holds the state in a Ref containing a cancelation action for the last t, if any
  20. Ref & Fiber & Bracket Use Case: Debouncing UI Events

    18 class Debouncer[F[_]: FlatMap, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = ??? def cancel: F[Unit] = current.modify(c  None  c).flatMap(_.sequence_) } Set ref to None and return current cancelation token
  21. Ref & Fiber & Bracket Use Case: Debouncing UI Events

    19 class Debouncer[F[_]: FlatMap, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = ??? def cancel: F[Unit] = current.modify(c  None  c).flatMap(_.sequence_) } Cancelation token is an Option[F[Unit]] so sequence and void it to convert to F[Unit]
  22. Ref & Fiber & Bracket Use Case: Debouncing UI Events

    20 class Debouncer[F[_]: FlatMap, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  ??? } } Access gives the current value of the ref and a setter that may be called once and may fail if the ref’s value has changed
  23. Ref & Fiber & Bracket Use Case: Debouncing UI Events

    21 class Debouncer[F[_]: FlatMap, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_ } } Cancel the old task, if any
  24. Ref & Fiber & Bracket Use Case: Debouncing UI Events

    22 class Debouncer[F[_]: FlatMap: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t) } } Delay the new task
  25. Ref & Fiber & Bracket Use Case: Debouncing UI Events

    23 class Debouncer[F[_]: Concurrent: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t).start.bracket { delayed  ??? }(_.cancel) } } Start the new task as a resource
  26. Ref & Fiber & Bracket Use Case: Debouncing UI Events

    24 class Debouncer[F[_]: Concurrent: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t).start.bracket { delayed  trySet(Some(delayed.cancel)) }(_.cancel) } } Update the ref with the cancelation token for the new fiber
  27. Ref & Fiber & Bracket Use Case: Debouncing UI Events

    25 class Debouncer[F[_]: Concurrent: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t).start.bracket { delayed  trySet(Some(delayed.cancel)).flatMap { successful  if (successful) delayed.join else ??? } }(_.cancel) } } If the set succeeds, join the fiber the overall result F[A] represents the successful completion of t
  28. Ref & Fiber & Bracket Use Case: Debouncing UI Events

    26 class Debouncer[F[_]: Concurrent: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t).start.bracket { delayed  trySet(Some(delayed.cancel)).flatMap { successful  if (successful) delayed.join else (delayed.cancel  Concurrent[F].never[A]) } }(_.cancel) } } If the set fails, another task must have been concurrently run, so cancel the new fiber and return a F[A] that never terminates
  29. Deferred 27 package cats.effect.concurrent trait Deferred[F[_], A] { def get:

    F[A] def complete(a: A): F[Unit] } object Deferred { def apply[F[_]: Concurrent, A]( a: A): F[Deferred[F, A]] = … } • Effectful promise-like data type (get doesn’t fail though) • Initially unset and may be set once • Get (semantically) blocks until the the value has been set via complete • Constructing a Deferred is effectful
  30. Naive Effectful Unbounded Queue 28 class EQueue[F[_], A] { def

    enqueue(a: A): F[Unit] def dequeue: F[A] } Let’s build an effectful queue which: • Supports enqueuing and dequeuing individual elements • Is unbounded
  31. Naive Effectful Unbounded Queue 29 import scala.collection.immutable.Queue class EQueue[F[_], A](

    state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def enqueue(a: A): F[Unit] def dequeue: F[A] } Queue state is either: • One or more dequeues are outstanding • Zero or more elements available for dequeuing
  32. Naive Effectful Unbounded Queue 30 class EQueue[F[_]: Concurrent, A]( state:

    Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { … } object EQueue { def apply[F[_]: Concurrent, A]: F[EQueue[F, A]] = Ref.of[F, Either[Queue[Deferred[F, A]], Queue[A]]] (Right(Queue.empty)).map(new EQueue[F, A](_)) } Construction is effectful
  33. Naive Effectful Unbounded Queue 31 class EQueue[F[_]: Monad, A]( state:

    Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def enqueue(a: A): F[Unit] = state.modify { case Right(q)  (Right(q :+ a), ().pure[F]) case Left(ws)  ??? }.flatten If state is a right, enqueue the new element to the underlying queue and perform no subsequent effect
  34. Naive Effectful Unbounded Queue 32 class EQueue[F[_]: FlatMap, A]( state:

    Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def enqueue(a: A): F[Unit] = state.modify { case Right(q)  (Right(q :+ a), ().pure[F]) case Left(ws)  val rem = ws.tail val next = if (rem.isEmpty) Right(Queue.empty[A]) else Left(rem) (next, ws.head.complete(a)) }.flatten If state is a left, return an effect that completes the head waiter and remove that waiter from queue state
  35. Naive Effectful Unbounded Queue 33 class EQueue[F[_]: Monad, A]( state:

    Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = ??? }
  36. Naive Effectful Unbounded Queue 34 class EQueue[F[_]: Concurrent, A]( state:

    Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].flatMap { w  state.modify { case Right(q)  ??? case Left(ws)  ??? }.flatten  w.get } } Create a waiter, add it to the queue state, and await on it
  37. Naive Effectful Unbounded Queue 35 class EQueue[F[_]: Concurrent, A]( state:

    Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].flatMap { w  state.modify { case Right(q)  if (q.isEmpty) (Left(Queue(w)), ().pure[F]) else (Right(q.tail), w.complete(q.head)) case Left(ws)  (Left(ws :+ w), ().pure[F]) }.flatten  w.get } } If state is a right, dequeue an element from underlying queue and immediately complete the waiter with it
  38. Naive Effectful Unbounded Queue 36 class EQueue[F[_]: Concurrent, A]( state:

    Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].flatMap { w  state.modify { case Right(q)  if (q.isEmpty) (Left(Queue(w)), ().pure[F]) else (Right(q.tail), w.complete(q.head)) case Left(ws)  (Left(ws :+ w), ().pure[F]) }.flatten  w.get } }
  39. Naive Effectful Unbounded Queue 36 class EQueue[F[_]: Concurrent, A]( state:

    Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].flatMap { w  state.modify { case Right(q)  if (q.isEmpty) (Left(Queue(w)), ().pure[F]) else (Right(q.tail), w.complete(q.head)) case Left(ws)  (Left(ws :+ w), ().pure[F]) }.flatten  w.get } } Memory Leak
  40. Naive Effectful Unbounded Queue 37 class EQueue[F[_]: Concurrent, A]( state:

    Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].bracket { w  state.modify { case Right(q)  if (q.isEmpty) (Left(Queue(w)), ().pure[F]) else (Right(q.tail), w.complete(q.head)) case Left(ws)  (Left(ws :+ w), ().pure[F]) }.flatten  w.get } { w  state.update(_.leftMap(_.filterNot(_  w))) } } Remove the waiter from the queue state upon finalization
  41. Naive Effectful Unbounded Queue 38 class EQueue[F[_]: Concurrent, A]( state:

    Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].bracketCase { w  …  w.get } { case (w, ExitCase.Canceled)  state.update(_.leftMap(_.filterNot(_  w))) case _  ().pure[F] } } Optimize finalization to only do a linear search when task is canceled
  42. Semaphore 39 package cats.effect.concurrent trait Semaphore[F[_]] { def acquire: F[Unit]

    def release: F[Unit] def withPermit[A](fa: F[A]): F[A] } object Semaphore { def apply[F[_]: Concurrent]( n: Long): F[Semaphore[F]] = … } • Effectful semaphore • Initialized with N permits • Permits may be acquired & released • Constructing a Semaphore is effectful
  43. Naive Effectful Bounded Queue 40 class BQueue[F[_], A] { def

    enqueue(a: A): F[Unit] def dequeue: F[A] } Let’s build an effectful queue which: • Is bounded • Builds on our unbounded queue
  44. Naive Effectful Bounded Queue 41 class BQueue[F[_]: Concurrent, A](q: EQueue[F,

    A], s: Semaphore[F]) { … } object BQueue { def apply[F[_]: Concurrent, A](n: Long): F[BQueue[F, A]] = (EQueue[F, A], Semaphore[F](n)).mapN(new BQueue(_, _)) } Construction is effectful
  45. Naive Effectful Bounded Queue 42 class BQueue[F[_]: Concurrent, A](q: EQueue[F,

    A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = ??? def dequeue: F[A] = ??? } Bound is represented by a semaphore
  46. Naive Effectful Bounded Queue 43 class BQueue[F[_]: Concurrent, A](q: EQueue[F,

    A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire  q.enqueue(a) def dequeue: F[A] = ??? } Before enqueuing an element, acquire a permit
  47. Naive Effectful Bounded Queue 44 class BQueue[F[_]: Concurrent, A](q: EQueue[F,

    A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire  q.enqueue(a) def dequeue: F[A] = q.dequeue  s.release } After dequeuing an element, release a permit
  48. Naive Effectful Bounded Queue 44 class BQueue[F[_]: Concurrent, A](q: EQueue[F,

    A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire  q.enqueue(a) def dequeue: F[A] = q.dequeue  s.release } After dequeuing an element, release a permit Interruption?
  49. Naive Effectful Bounded Queue 45 class BQueue[F[_]: Concurrent, A](q: EQueue[F,

    A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire.bracketCase(_  q.enqueue(a)) { case (_, ExitCase.Completed)  ().pure[F] case _  s.release } def dequeue: F[A] = ??? }
  50. Naive Effectful Bounded Queue 45 class BQueue[F[_]: Concurrent, A](q: EQueue[F,

    A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire.bracketCase(_  q.enqueue(a)) { case (_, ExitCase.Completed)  ().pure[F] case _  s.release } def dequeue: F[A] = ??? } Memory leak
  51. Naive Effectful Bounded Queue 46 class BQueue[F[_]: Concurrent, A](q: EQueue[F,

    A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquireInterruptibly.bracketCase { case (g, _)  g  q.enqueue(a) } { case ((_, c), ExitCase.Completed)  ().pure[F] case ((_, c), _)  c } def dequeue: F[A] = ??? } • Separate state changes and async awaits, bracket on state changes • Similar refactoring possible for dequeue • Open area of research for future releases
  52. Concurrency with Cats Effect • Cats Effect 1.0 • Rich,

    principled, polymorphic effects • Works with many effect types and transformer stacks • cats.effect.IO • Monix • ZIO • Foundation of FS2, Doobie, HTTP4S, and dozens of other libraries • Lots more concurrency support: • MVar • Clock • ContextShift • TestContext 47