Polymorphic Nonsense Concurrency with Cats Effect Michael Pilquist // @mpilquist Scale By The Bay November 2018 

Cats Effect Evolution in 2018 2 MonadError Sync Async Effect LiftIO MonadError Bracket Sync Async LiftIO Concurrent Effect ConcurrentEffect Defer 0.8 (Jan 18) 1.0 (Sept 18) 

Cats Effect 1.0 3 MonadError Sync Async Effect LiftIO MonadError Bracket Sync Async LiftIO Concurrent Effect ConcurrentEffect Defer 1.0 (Sept 18) Principled & polymorphic type classes: • resource management (Bracket) • concurrency (Concurrent) Principled & polymorphic utilities: • concurrent data structures • timers & clocks • thread pool management Effect types: • Much faster IO • SyncIO (guaranteed sync execution) 

Fibers 4 trait Concurrent[F[_]] extends Async[F] { def start[A](fa: F[A]): F[Fiber[F, A]] } trait Fiber[F[_], A] { def join: F[A] def cancel: CancelToken[F] } type CancelToken[F[_]] = F[Unit] 

Fibers 5 trait Concurrent[F[_]] extends Async[F] { def start[A](fa: F[A]): F[Fiber[F, A]] } trait Fiber[F[_], A] { def join: F[A] def cancel: CancelToken[F] } type CancelToken[F[_]] = F[Unit] • Example of "semantic blocking" • Semantic blocking involves NO thread blocking • Similar to calling map or flatMap on a Future 

Example Fiber Use Case 6 import cats.effect.Concurrent import cats.effect.implicits._ def parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = 

Example Fiber Use Case 7 import cats.effect.Concurrent import cats.effect.implicits._ def parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = for { fiberA  fa.start fiberB  fb.start a  fiberA.join b  fiberB.join } yield (a, b) 

Example Fiber Use Case 8 import cats.effect.Concurrent import cats.effect.implicits._ def parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = for { fiberA  fa.start fiberB  fb.start a  fiberA.join b  fiberB.join } yield (a, b) parTupled(F.delay(throw new RuntimeException), fb) 

Example Fiber Use Case 8 import cats.effect.Concurrent import cats.effect.implicits._ def parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = for { fiberA  fa.start fiberB  fb.start a  fiberA.join b  fiberB.join } yield (a, b) parTupled(F.delay(throw new RuntimeException), fb) Fiber Leak! • Leaks fibers on errors • Leaks fibers on cancelation 

Example Fiber Use Case Corrected 9 import cats.effect.Concurrent import cats.effect.implicits._ import cats.implicits._ def parTupled[F[_]: Concurrent, A, B](fa: F[A], fb: F[B]): F[(A, B)] = fa.start.bracket(fiberA  fb.start.bracket(fiberB  (fiberA.join, fiberB.join).tupled )(_.cancel) )(_.cancel) Replaced flatMap with bracket 

Ref 10 package cats.effect.concurrent trait Ref[F[_], A] { def get: F[A] def set(a: A): F[Unit] def access: F[(A, A  F[Boolean])] def update(f: A  A): F[Unit] def modify(f: A  (A, B)): F[B] } object Ref { def of[F[_]: Sync, A](a: A): F[Ref[F, A]] } • Effectful version of j.u.c.AtomicReference • Always has a value (never "unset") • Low level APIs: get, set, access • High level APIs: update, modify • Constructing a Ref is effectful 

Ref 11 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))  r.flatMap(_.get) ??? unsafeRunSync 

Ref 12 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))  r.flatMap(_.get) 2 unsafeRunSync 

Ref 13 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))  r.flatMap(_.get) •Create a ref initialized to 2 •Update its value to 5 times its current value 

Ref 14 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))  r.flatMap(_.get) •Create a ref initialized to 2 •Get its current value 

Ref 14 val r = Ref.of[IO, Int](2) r.flatMap(_.update(_ * 5))  r.flatMap(_.get) •Create a ref initialized to 2 •Get its current value Misuse of Ref constructor 

Ref 15 val mk = Ref.of[IO, Int](2) mk.flatMap(r  r.update(_ * 5)  r.get) 

Ref & Fiber & Bracket Use Case: Debouncing UI Events 16 class Debouncer[F[_], A] { def apply(t: F[A]): F[A] = ??? def cancel: F[Unit] = ??? } • Delay evaluation of t • If while delayed, another t is supplied, cancel original t and delay new t 

Ref & Fiber & Bracket Use Case: Debouncing UI Events 17 class Debouncer[F[_], A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = ??? def cancel: F[Unit] = ??? } Current holds the state in a Ref containing a cancelation action for the last t, if any 

Ref & Fiber & Bracket Use Case: Debouncing UI Events 18 class Debouncer[F[_]: FlatMap, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = ??? def cancel: F[Unit] = current.modify(c  None  c).flatMap(_.sequence_) } Set ref to None and return current cancelation token 

Ref & Fiber & Bracket Use Case: Debouncing UI Events 19 class Debouncer[F[_]: FlatMap, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = ??? def cancel: F[Unit] = current.modify(c  None  c).flatMap(_.sequence_) } Cancelation token is an Option[F[Unit]] so sequence and void it to convert to F[Unit] 

Ref & Fiber & Bracket Use Case: Debouncing UI Events 20 class Debouncer[F[_]: FlatMap, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  ??? } } Access gives the current value of the ref and a setter that may be called once and may fail if the ref’s value has changed 

Ref & Fiber & Bracket Use Case: Debouncing UI Events 21 class Debouncer[F[_]: FlatMap, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_ } } Cancel the old task, if any 

Ref & Fiber & Bracket Use Case: Debouncing UI Events 22 class Debouncer[F[_]: FlatMap: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t) } } Delay the new task 

Ref & Fiber & Bracket Use Case: Debouncing UI Events 23 class Debouncer[F[_]: Concurrent: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t).start.bracket { delayed  ??? }(_.cancel) } } Start the new task as a resource 

Ref & Fiber & Bracket Use Case: Debouncing UI Events 24 class Debouncer[F[_]: Concurrent: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t).start.bracket { delayed  trySet(Some(delayed.cancel)) }(_.cancel) } } Update the ref with the cancelation token for the new fiber 

Ref & Fiber & Bracket Use Case: Debouncing UI Events 25 class Debouncer[F[_]: Concurrent: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t).start.bracket { delayed  trySet(Some(delayed.cancel)).flatMap { successful  if (successful) delayed.join else ??? } }(_.cancel) } } If the set succeeds, join the fiber the overall result F[A] represents the successful completion of t 

Ref & Fiber & Bracket Use Case: Debouncing UI Events 26 class Debouncer[F[_]: Concurrent: Timer, A]( debounceOver: FiniteDuration, current: Ref[F, Option[F[Unit]]] ) { private val delay: F[Unit] = implicitly[Timer[F]].sleep(debounceOver) def apply(t: F[A]): F[A] = current.access.flatMap { case (cancel, trySet)  cancel.sequence_  (delay  t).start.bracket { delayed  trySet(Some(delayed.cancel)).flatMap { successful  if (successful) delayed.join else (delayed.cancel  Concurrent[F].never[A]) } }(_.cancel) } } If the set fails, another task must have been concurrently run, so cancel the new fiber and return a F[A] that never terminates 

Deferred 27 package cats.effect.concurrent trait Deferred[F[_], A] { def get: F[A] def complete(a: A): F[Unit] } object Deferred { def apply[F[_]: Concurrent, A]( a: A): F[Deferred[F, A]] = … } • Effectful promise-like data type (get doesn’t fail though) • Initially unset and may be set once • Get (semantically) blocks until the the value has been set via complete • Constructing a Deferred is effectful 

Naive Effectful Unbounded Queue 28 class EQueue[F[_], A] { def enqueue(a: A): F[Unit] def dequeue: F[A] } Let’s build an effectful queue which: • Supports enqueuing and dequeuing individual elements • Is unbounded 

Naive Effectful Unbounded Queue 29 import scala.collection.immutable.Queue class EQueue[F[_], A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def enqueue(a: A): F[Unit] def dequeue: F[A] } Queue state is either: • One or more dequeues are outstanding • Zero or more elements available for dequeuing 

Naive Effectful Unbounded Queue 30 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { … } object EQueue { def apply[F[_]: Concurrent, A]: F[EQueue[F, A]] = Ref.of[F, Either[Queue[Deferred[F, A]], Queue[A]]] (Right(Queue.empty)).map(new EQueue[F, A](_)) } Construction is effectful 

Naive Effectful Unbounded Queue 31 class EQueue[F[_]: Monad, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def enqueue(a: A): F[Unit] = state.modify { case Right(q)  (Right(q :+ a), ().pure[F]) case Left(ws)  ??? }.flatten If state is a right, enqueue the new element to the underlying queue and perform no subsequent effect 

Naive Effectful Unbounded Queue 32 class EQueue[F[_]: FlatMap, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def enqueue(a: A): F[Unit] = state.modify { case Right(q)  (Right(q :+ a), ().pure[F]) case Left(ws)  val rem = ws.tail val next = if (rem.isEmpty) Right(Queue.empty[A]) else Left(rem) (next, ws.head.complete(a)) }.flatten If state is a left, return an effect that completes the head waiter and remove that waiter from queue state 

Naive Effectful Unbounded Queue 33 class EQueue[F[_]: Monad, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = ??? } 

Naive Effectful Unbounded Queue 34 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].flatMap { w  state.modify { case Right(q)  ??? case Left(ws)  ??? }.flatten  w.get } } Create a waiter, add it to the queue state, and await on it 

Naive Effectful Unbounded Queue 35 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].flatMap { w  state.modify { case Right(q)  if (q.isEmpty) (Left(Queue(w)), ().pure[F]) else (Right(q.tail), w.complete(q.head)) case Left(ws)  (Left(ws :+ w), ().pure[F]) }.flatten  w.get } } If state is a right, dequeue an element from underlying queue and immediately complete the waiter with it 

Naive Effectful Unbounded Queue 36 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].flatMap { w  state.modify { case Right(q)  if (q.isEmpty) (Left(Queue(w)), ().pure[F]) else (Right(q.tail), w.complete(q.head)) case Left(ws)  (Left(ws :+ w), ().pure[F]) }.flatten  w.get } } 

Naive Effectful Unbounded Queue 36 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].flatMap { w  state.modify { case Right(q)  if (q.isEmpty) (Left(Queue(w)), ().pure[F]) else (Right(q.tail), w.complete(q.head)) case Left(ws)  (Left(ws :+ w), ().pure[F]) }.flatten  w.get } } Memory Leak 

Naive Effectful Unbounded Queue 37 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].bracket { w  state.modify { case Right(q)  if (q.isEmpty) (Left(Queue(w)), ().pure[F]) else (Right(q.tail), w.complete(q.head)) case Left(ws)  (Left(ws :+ w), ().pure[F]) }.flatten  w.get } { w  state.update(_.leftMap(_.filterNot(_  w))) } } Remove the waiter from the queue state upon finalization 

Naive Effectful Unbounded Queue 38 class EQueue[F[_]: Concurrent, A]( state: Ref[F, Either[Queue[Deferred[F, A]], Queue[A]]] ) { def dequeue: F[A] = Deferred[F, A].bracketCase { w  …  w.get } { case (w, ExitCase.Canceled)  state.update(_.leftMap(_.filterNot(_  w))) case _  ().pure[F] } } Optimize finalization to only do a linear search when task is canceled 

Semaphore 39 package cats.effect.concurrent trait Semaphore[F[_]] { def acquire: F[Unit] def release: F[Unit] def withPermit[A](fa: F[A]): F[A] } object Semaphore { def apply[F[_]: Concurrent]( n: Long): F[Semaphore[F]] = … } • Effectful semaphore • Initialized with N permits • Permits may be acquired & released • Constructing a Semaphore is effectful 

Naive Effectful Bounded Queue 40 class BQueue[F[_], A] { def enqueue(a: A): F[Unit] def dequeue: F[A] } Let’s build an effectful queue which: • Is bounded • Builds on our unbounded queue 

Naive Effectful Bounded Queue 41 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { … } object BQueue { def apply[F[_]: Concurrent, A](n: Long): F[BQueue[F, A]] = (EQueue[F, A], Semaphore[F](n)).mapN(new BQueue(_, _)) } Construction is effectful 

Naive Effectful Bounded Queue 42 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = ??? def dequeue: F[A] = ??? } Bound is represented by a semaphore 

Naive Effectful Bounded Queue 43 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire  q.enqueue(a) def dequeue: F[A] = ??? } Before enqueuing an element, acquire a permit 

Naive Effectful Bounded Queue 44 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire  q.enqueue(a) def dequeue: F[A] = q.dequeue  s.release } After dequeuing an element, release a permit 

Naive Effectful Bounded Queue 44 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire  q.enqueue(a) def dequeue: F[A] = q.dequeue  s.release } After dequeuing an element, release a permit Interruption? 

Naive Effectful Bounded Queue 45 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire.bracketCase(_  q.enqueue(a)) { case (_, ExitCase.Completed)  ().pure[F] case _  s.release } def dequeue: F[A] = ??? } 

Naive Effectful Bounded Queue 45 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquire.bracketCase(_  q.enqueue(a)) { case (_, ExitCase.Completed)  ().pure[F] case _  s.release } def dequeue: F[A] = ??? } Memory leak 

Naive Effectful Bounded Queue 46 class BQueue[F[_]: Concurrent, A](q: EQueue[F, A], s: Semaphore[F]) { def enqueue(a: A): F[Unit] = s.acquireInterruptibly.bracketCase { case (g, _)  g  q.enqueue(a) } { case ((_, c), ExitCase.Completed)  ().pure[F] case ((_, c), _)  c } def dequeue: F[A] = ??? } • Separate state changes and async awaits, bracket on state changes • Similar refactoring possible for dequeue • Open area of research for future releases 

Concurrency with Cats Effect • Cats Effect 1.0 • Rich, principled, polymorphic effects • Works with many effect types and transformer stacks • cats.effect.IO • Monix • ZIO • Foundation of FS2, Doobie, HTTP4S, and dozens of other libraries • Lots more concurrency support: • MVar • Clock • ContextShift • TestContext 47