Concurrency with Cats Effect

The Cats Effect library recently reached its 1.0 release, providing powerful data types and type classes for purely functional effectful programming. In this talk, we’ll focus on building programs using fiber based concurrency and the synchronization primitives provided by Cats Effect. We’ll see how FS2 (Functional Streams for Scala) uses such primitives to build more advanced concurrent data types like bounded and unbounded queues. Finally, we’ll see how to apply these techniques to application design.

Transcript

1. Polymorphic Nonsense Concurrency with Cats Effect Michael Pilquist // @mpilquist Scale By The Bay November 2018
2. Cats Effect Evolution in 2018 2 MonadError Sync Async Effect LiftIO MonadError Bracket Sync Async LiftIO Concurrent Effect ConcurrentEffect Defer 0.8 (Jan 18) 1.0 (Sept 18)
3. Cats Effect 1.0 3 MonadError Sync Async Effect LiftIO MonadError Bracket Sync Async LiftIO Concurrent Effect ConcurrentEffect Defer 1.0 (Sept 18) Principled & polymorphic type classes: • resource management (Bracket) • concurrency (Concurrent) Principled & polymorphic utilities: • concurrent data structures • timers & clocks • thread pool management Effect types: • Much faster IO • SyncIO (guaranteed sync execution)
4. Fibers 4 trait Concurrent[F[_]] extends Async[F] { def start[A](fa: F[A]): F[Fiber[F, A]] } trait Fiber[F[_], A] { def join: F[A] def cancel: CancelToken[F] } type CancelToken[F[_]] = F[Unit]
5. Fibers 5 trait Concurrent[F[_]] extends Async[F] { def start[A](fa: F[A]): F[Fiber[F, A]] } trait Fiber[F[_], A] { def join: F[A] def cancel: CancelToken[F] } type CancelToken[F[_]] = F[Unit] • Example of "semantic blocking" • Semantic blocking involves NO thread blocking • Similar to calling map or flatMap on a Future
6. Example Fiber Use Case 6 import cats.effect.Concurrent import cats.effect.implicits._ def parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] =
7. Example Fiber Use Case 7 import cats.effect.Concurrent import cats.effect.implicits._ def parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = for { fiberA  fa.start fiberB  fb.start a  fiberA.join b  fiberB.join } yield (a, b)
8. Example Fiber Use Case 8 import cats.effect.Concurrent import cats.effect.implicits._ def parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = for { fiberA  fa.start fiberB  fb.start a  fiberA.join b  fiberB.join } yield (a, b) parTupled(F.delay(throw new RuntimeException), fb)
9. Example Fiber Use Case 8 import cats.effect.Concurrent import cats.effect.implicits._ def parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = for { fiberA  fa.start fiberB  fb.start a  fiberA.join b  fiberB.join } yield (a, b) parTupled(F.delay(throw new RuntimeException), fb) Fiber Leak! • Leaks fibers on errors • Leaks fibers on cancelation
10. Example Fiber Use Case Corrected 9 import cats.effect.Concurrent import cats.effect.implicits._ import cats.implicits._ def parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = fa.start.bracket(fiberA  fb.start.bracket(fiberB  (fiberA.join, fiberB.join).tupled )(_.cancel) )(_.cancel) Replaced flatMap with bracket
11. Ref 10 package cats.effect.concurrent trait Ref[F[_], A] { def get: F[A] def set(a: A): F[Unit] def access: F[(A, A  F[Boolean])] def update(f: A  A): F[Unit] def modify(f: A  (A, B)): F[B] } object Ref { def of[F[_]: Sync, A](a: A): F[Ref[F, A]] } • Effectful version of j.u.c.AtomicReference • Always has a value (never "unset") • Low level APIs: get, set, access • High level APIs: update, modify • Constructing a Ref is effectful
12. Ref 11 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))  r.flatMap(_.get) ??? unsafeRunSync
13. Ref 12 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))  r.flatMap(_.get) 2 unsafeRunSync
14. Ref 13 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))  r.flatMap(_.get) •Create a ref initialized to 2 •Update its value to 5 times its current value
15. Ref 14 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))  r.flatMap(_.get) •Create a ref initialized to 2 •Get its current value
16. Ref 14 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))  r.flatMap(_.get) •Create a ref initialized to 2 •Get its current value Misuse of Ref constructor
17. Ref 15 val mk = Ref.of[IO, Int](2) mk.flatMap(r  r.update(_ * 5)  r.get)
18. Ref & Fiber & Bracket Use Case: Debouncing UI Events 16 class Debouncer[F[_], A] { def apply(t: F[A]): F[A] = ??? def cancel: F[Unit] = ??? } • Delay evaluation of t • If while delayed, another t is supplied, cancel original t and delay new t
19. Ref & Fiber & Bracket Use Case: Debouncing UI Events 17 class Debouncer[F[_], A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = ??? def cancel: F[Unit] = ??? } Current holds the state in a Ref containing a cancelation action for the last t, if any
20. Ref & Fiber & Bracket Use Case: Debouncing UI Events 18 class Debouncer[F[_]: FlatMap, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = ??? def cancel: F[Unit] = current.modify(c  None  c).flatMap(_.sequence_) } Set ref to None and return current cancelation token
21. Ref & Fiber & Bracket Use Case: Debouncing UI Events 19 class Debouncer[F[_]: FlatMap, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = ??? def cancel: F[Unit] = current.modify(c  None  c).flatMap(_.sequence_) } Cancelation token is an Option[F[Unit]] so sequence and void it to convert to F[Unit]
22. Ref & Fiber & Bracket Use Case: Debouncing UI Events 20 class Debouncer[F[_]: FlatMap, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  ??? } } Access gives the current value of the ref and a setter that may be called once and may fail if the ref’s value has changed
23. Ref & Fiber & Bracket Use Case: Debouncing UI Events 21 class Debouncer[F[_]: FlatMap, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_ } } Cancel the old task, if any
24. Ref & Fiber & Bracket Use Case: Debouncing UI Events 22 class Debouncer[F[_]: FlatMap: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t) } } Delay the new task
25. Ref & Fiber & Bracket Use Case: Debouncing UI Events 23 class Debouncer[F[_]: Concurrent: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t).start.bracket { delayed  ??? }(_.cancel) } } Start the new task as a resource
26. Ref & Fiber & Bracket Use Case: Debouncing UI Events 24 class Debouncer[F[_]: Concurrent: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t).start.bracket { delayed  trySet(Some(delayed.cancel)) }(_.cancel) } } Update the ref with the cancelation token for the new fiber
27. Ref & Fiber & Bracket Use Case: Debouncing UI Events 25 class Debouncer[F[_]: Concurrent: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t).start.bracket { delayed  trySet(Some(delayed.cancel)).flatMap { successful  if (successful) delayed.join else ??? } }(_.cancel) } } If the set succeeds, join the fiber the overall result F[A] represents the successful completion of t
28. Ref & Fiber & Bracket Use Case: Debouncing UI Events 26 class Debouncer[F[_]: Concurrent: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t).start.bracket { delayed  trySet(Some(delayed.cancel)).flatMap { successful  if (successful) delayed.join else (delayed.cancel  Concurrent[F].never[A]) } }(_.cancel) } } If the set fails, another task must have been concurrently run, so cancel the new fiber and return a F[A] that never terminates
29. Deferred 27 package cats.effect.concurrent trait Deferred[F[_], A] { def get: F[A] def complete(a: A): F[Unit] } object Deferred { def apply[F[_]: Concurrent, A]( a: A): F[Deferred[F, A]] = … } • Effectful promise-like data type (get doesn’t fail though) • Initially unset and may be set once • Get (semantically) blocks until the the value has been set via complete • Constructing a Deferred is effectful
30. Naive Effectful Unbounded Queue 28 class EQueue[F[_], A] { def enqueue(a: A): F[Unit] def dequeue: F[A] } Let’s build an effectful queue which: • Supports enqueuing and dequeuing individual elements • Is unbounded
31. Naive Effectful Unbounded Queue 29 import scala.collection.immutable.Queue class EQueue[F[_], A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def enqueue(a: A): F[Unit] def dequeue: F[A] } Queue state is either: • One or more dequeues are outstanding • Zero or more elements available for dequeuing
32. Naive Effectful Unbounded Queue 30 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { … } object EQueue { def apply[F[_]: Concurrent, A]: F[EQueue[F, A]] = Ref.of[F, Either[Queue[Deferred[F, A]], Queue[A]]] (Right(Queue.empty)).map(new EQueue[F, A](_)) } Construction is effectful
33. Naive Effectful Unbounded Queue 31 class EQueue[F[_]: Monad, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def enqueue(a: A): F[Unit] = state.modify { case Right(q)  (Right(q :+ a), ().pure[F]) case Left(ws)  ??? }.flatten If state is a right, enqueue the new element to the underlying queue and perform no subsequent effect
34. Naive Effectful Unbounded Queue 32 class EQueue[F[_]: FlatMap, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def enqueue(a: A): F[Unit] = state.modify { case Right(q)  (Right(q :+ a), ().pure[F]) case Left(ws)  val rem = ws.tail val next = if (rem.isEmpty) Right(Queue.empty[A]) else Left(rem) (next, ws.head.complete(a)) }.flatten If state is a left, return an effect that completes the head waiter and remove that waiter from queue state
35. Naive Effectful Unbounded Queue 33 class EQueue[F[_]: Monad, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = ??? }
36. Naive Effectful Unbounded Queue 34 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].flatMap { w  state.modify { case Right(q)  ??? case Left(ws)  ??? }.flatten  w.get } } Create a waiter, add it to the queue state, and await on it
37. Naive Effectful Unbounded Queue 35 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].flatMap { w  state.modify { case Right(q)  if (q.isEmpty) (Left(Queue(w)), ().pure[F]) else (Right(q.tail), w.complete(q.head)) case Left(ws)  (Left(ws :+ w), ().pure[F]) }.flatten  w.get } } If state is a right, dequeue an element from underlying queue and immediately complete the waiter with it
38. Naive Effectful Unbounded Queue 36 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].flatMap { w  state.modify { case Right(q)  if (q.isEmpty) (Left(Queue(w)), ().pure[F]) else (Right(q.tail), w.complete(q.head)) case Left(ws)  (Left(ws :+ w), ().pure[F]) }.flatten  w.get } }
39. Naive Effectful Unbounded Queue 36 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].flatMap { w  state.modify { case Right(q)  if (q.isEmpty) (Left(Queue(w)), ().pure[F]) else (Right(q.tail), w.complete(q.head)) case Left(ws)  (Left(ws :+ w), ().pure[F]) }.flatten  w.get } } Memory Leak
40. Naive Effectful Unbounded Queue 37 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].bracket { w  state.modify { case Right(q)  if (q.isEmpty) (Left(Queue(w)), ().pure[F]) else (Right(q.tail), w.complete(q.head)) case Left(ws)  (Left(ws :+ w), ().pure[F]) }.flatten  w.get } { w  state.update(_.leftMap(_.filterNot(_  w))) } } Remove the waiter from the queue state upon finalization
41. Naive Effectful Unbounded Queue 38 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].bracketCase { w  …  w.get } { case (w, ExitCase.Canceled)  state.update(_.leftMap(_.filterNot(_  w))) case _  ().pure[F] } } Optimize finalization to only do a linear search when task is canceled
42. Semaphore 39 package cats.effect.concurrent trait Semaphore[F[_]] { def acquire: F[Unit] def release: F[Unit] def withPermit[A](fa: F[A]): F[A] } object Semaphore { def apply[F[_]: Concurrent]( n: Long): F[Semaphore[F]] = … } • Effectful semaphore • Initialized with N permits • Permits may be acquired & released • Constructing a Semaphore is effectful
43. Naive Effectful Bounded Queue 40 class BQueue[F[_], A] { def enqueue(a: A): F[Unit] def dequeue: F[A] } Let’s build an effectful queue which: • Is bounded • Builds on our unbounded queue
44. Naive Effectful Bounded Queue 41 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { … } object BQueue { def apply[F[_]: Concurrent, A](n: Long): F[BQueue[F, A]] = (EQueue[F, A], Semaphore[F](n)).mapN(new BQueue(_, _)) } Construction is effectful
45. Naive Effectful Bounded Queue 42 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = ??? def dequeue: F[A] = ??? } Bound is represented by a semaphore
46. Naive Effectful Bounded Queue 43 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire  q.enqueue(a) def dequeue: F[A] = ??? } Before enqueuing an element, acquire a permit
47. Naive Effectful Bounded Queue 44 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire  q.enqueue(a) def dequeue: F[A] = q.dequeue  s.release } After dequeuing an element, release a permit
48. Naive Effectful Bounded Queue 44 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire  q.enqueue(a) def dequeue: F[A] = q.dequeue  s.release } After dequeuing an element, release a permit Interruption?
49. Naive Effectful Bounded Queue 45 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire.bracketCase(_  q.enqueue(a)) { case (_, ExitCase.Completed)  ().pure[F] case _  s.release } def dequeue: F[A] = ??? }
50. Naive Effectful Bounded Queue 45 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire.bracketCase(_  q.enqueue(a)) { case (_, ExitCase.Completed)  ().pure[F] case _  s.release } def dequeue: F[A] = ??? } Memory leak
51. Naive Effectful Bounded Queue 46 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquireInterruptibly.bracketCase { case (g, _)  g  q.enqueue(a) } { case ((_, c), ExitCase.Completed)  ().pure[F] case ((_, c), _)  c } def dequeue: F[A] = ??? } • Separate state changes and async awaits, bracket on state changes • Similar refactoring possible for dequeue • Open area of research for future releases
52. Concurrency with Cats Effect • Cats Effect 1.0 • Rich, principled, polymorphic effects • Works with many effect types and transformer stacks • cats.effect.IO • Monix • ZIO • Foundation of FS2, Doobie, HTTP4S, and dozens of other libraries • Lots more concurrency support: • MVar • Clock • ContextShift • TestContext 47