Extensible Effects: beyond the Monad Transformers

Ca4df28501e4c9cfbceb91f367afa784?s=47 fuzyco
October 31, 2019

Extensible Effects: beyond the Monad Transformers

This presentation includes these following three things:
1. Monad
2. Monad Transformers
3. Extensible-Effects
Main topics are how to use atnos-eff and comparison between mtl and eff.

Ca4df28501e4c9cfbceb91f367afa784?s=128

fuzyco

October 31, 2019
Tweet

Transcript

  1. Extensible-Effects: beyond the Monad Transformers Hiroki Fujino Unipos GmbH 1

  2. Introduction 2 • Hiroki Fujino • Head of Engineer in

    Unipos • Came from Japan since this February • My interests: Scala, Architecture Design, Concurrency Programming, etc.
  3. Agenda • Monad • Monad Transformers • Extensible Effects 3

    Prerequisites: Basic knowledge of Scala(includes Experience with using Monad) https://github.com/Hiroki6/SampleAtnosEff Sample Code:
  4. Monad 4 Allows composition of dependent effectful functions. Option Future

    Either Writer Reader State IO https://typelevel.org/cats/typeclasses/monad.html
  5. Monad 5 trait Monad[M[_]] { def pure[A](x: A): M[A] def

    map[A, B](x: M[A])(f: A => B): M[B] def flatMap[A, B](x: M[A])(f: A => M[B]): M[B] }
  6. Option Monad 6 sealed trait Option[+A] case object None extends

    Option[Nothing] case class Some[A](value: A) extends Option[A] new Monad[Option] { def pure[A](x: A) = Some(x) def flatMap[A, B](x: Option[A])(f: A => Option[B]): Option[B] = x match { case Some(x) => f(x) case None => None } }
  7. Option Monad 7 def hogeOption(a: Int): Option[Int] = Some(a) hogeOption(1).flatMap(a

    => hogeOption(a).map(b => a+b)) // Some(2) for { a <- hogeOption(1) b <- hogeOption(a) } yield (a + b) // Some(2)
  8. 8 How to use multi monads?

  9. Two Monads in program 9 def findUser(userId: UserId): IO[Option[User]] def

    getTeam(userId: UserId): IO[Option[Team]] def createResponse(user: User, team: Team): Response def program(userId: UserId): IO[Option[Response]] = for { userOpt <- findUser(userId) teamOpt <- getTeam(userId) user <- userOpt team <- teamOpt } yield createResponse(user, team)
  10. 10 [error] found : Option[Response] [error] required: cats.effect.IO[Option[Response]] [error] user

    <- userOpt def findUser(userId: UserId): IO[Option[User]] def getTeam(userId: UserId): IO[Option[Team]] def createResponse(user: User, team: Team): Response def program(userId: UserId): IO[Option[Response]] = for { userOpt <- findUser(userId) teamOpt <- getTeam(userId) user <- userOpt team <- teamOpt } yield createResponse(user, team) } Composition of IO } Composition of Option Two Monads with For-statement
  11. Two Monads with For-statement 11 def program(userId: UserId): IO[Option[Response]] =

    for { userOpt <- findUser(userId) teamOpt <- getTeam(userId) } yield { (userOpt, teamOpt) match { case (Some(user), Some(team)) => Some(createResponse(user, team)) case (_, _) => None } } def program(userId: UserId): IO[Option[Response]] = for { userOpt <- findUser(userId) teamOpt <- getTeam(userId) } yield for { user <- userOpt team <- teamOpt } yield createResponse(user, team)
  12. Monad Transformers 12

  13. Monad Transformers 13 G F •Popular solution to create monad

    stack •Monads do not compose(G[_], F[_] => F[G[_]]) •But, if G[_] is implemented monad(Option, Either, etc.), F[G[_]] can be monad •Option •Either •Reader etc.
  14. Monad Transformers 14 case class OptionT[F[_], A](value: F[Option[A]]) { def

    map[B](f: A => B)(implicit F: Functor[F]): OptionT[F, B] = OptionT(F.map(value)(_ map f)) def flatMap[B](f: A => OptionT[F, B])(implicit F: Monad[F]): OptionT[F, B] = OptionT(F.flatMap(value)(_.fold(F.pure[Option[B]](None))(f(_).value))) } Option F
  15. Two Monads with OptionT 15 def program(userId: UserId): OptionT[IO, Response]

    = for { user <- OptionT(findUser(userId)) team <- OptionT(getTeam(userId)) } yield createResponse(user, team) program(userId).value // IO[Option[Response]] Option IO def flatMap[B](f: A => OptionT[F, B])(implicit F: Monad[F]): OptionT[F, B] } Composition of OptionT
  16. Implementation of Monad Transformers 16 OptionT EitherT WriterT ReaderT StateT

    These are implemented in cats and Scalaz.
  17. 17 Transformer can wrap another transformer type EitherIO[A] = EitherT[IO,

    Error, A] OptionT[EitherIO, A] // OptionT[EitherT[IO, Error, A]] IO Either Option
  18. Three Monad in Program 18 type EitherIO[A] = EitherT[IO, Error,

    A] def program(userId: String, teamName: String): OptionT[EitherIO, Response] = { for { user <- OptionT[EitherIO, User] { EitherT.liftF[IO, Error, Option[User]](findUser(userId)) } team <- OptionT.liftF[EitherIO, Team] { EitherT.fromEither[IO](createTeam(userId, teamName)) } _ <- OptionT.liftF[EitherIO, Unit] { EitherT.liftF[IO, Error, Unit](storeTeam(team)) } } yield createResponse(user, team) } program(userId, teamName) // OptionT[EitherIO, Response] .value // EitherIO[Option[Response]] == EitherT[IO, Error, Option[Response]] .value // IO[Either[Error, Option[Response]]] IO[Option[User]] Either[Error, Team] IO[Unit]
  19. Three Monad in Program 19 type EitherIO[A] = EitherT[IO, Error,

    A] def program(userId: String, teamName: String): OptionT[EitherIO, Response] = { for { user <- OptionT[EitherIO, User] { EitherT.liftF[IO, Error, Option[User]](findUser(userId)) } team <- OptionT.liftF[EitherIO, Team] { EitherT.fromEither[IO](createTeam(userId, teamName)) } _ <- OptionT.liftF[EitherIO, Unit] { EitherT.liftF[IO, Error, Unit](storeTeam(team)) } } yield createResponse(user, team) } program(userId, teamName) // OptionT[EitherIO, Response] .value // EitherIO[Option[Response]] == EitherT[IO, Error, Option[Response]] .value // IO[Either[Error, Option[Response]]] IO[Option[User]] Either[Error, Team] IO[Unit]
  20. Issues of Monad Transformers • Cumbersome lift • Overhead of

    composition • Fixed order of Monad Stack 20
  21. Cumbersome lift 21 Option Either IO def storeTeam(team: Team): IO[Unit]

    OptionT.liftF[EitherIO, Unit] { EitherT.liftF[IO, Error, Unit](storeTeam(team)).map(_.some) } IO Either IO lift lift
  22. Overhead of composition 22 OptionT { Monad[EitherIO].flatMap(value)(_.fold(F.pure[Option[User]](None))(f(_).value)) } OptionT[EitherIO, User](EitherT.liftF[IO,

    Error, Option[User]](findUser(userId))).flatMap(f) EitherT(Monad[IO].flatMap(value) { case l @ Left(_) => F.pure(l.rightCast) case Right(b) => f(b).value })
  23. Fixed order of Monad Stack 23 program.value // Option[Either[String, Int]]

    Definition Execution Either Option Option Either ❌ def program: EitherT[Option, String, Int] = for { t <- EitherT.fromEither[Option](Right(1)) result <- EitherT.liftF[Option, String, Int](Some(t)) } yield result
  24. 24 Extensible Effects

  25. 25 “Extensible Effects: an alternative to monad transformers” –Oleg Kiselyov

    in 2013
  26. Extensible Effects(Eff) Library in Scala 26 •atnos-eff •emm

  27. Today’s Topic about Eff 27 atnos-eff •How to use atnos-eff

    •Advantage and Disadvantage •How to define your own Eff monad •Detail of atons-eff(Union Type, Interpreter) Not covered: Explain these below in this presentation:
  28. Eff 28 Option State •Eff is monad. •Eff has effect

    stack in a flat state. •List of effects. order of effects is not important. Either Reader ex. Fx.fx4[Reader[Int, ?], Either[String, ?], Option, State[Int, ?]]
  29. Monad Transformers Eff Comparison of concept 29 Definition Execution Option

    Either Either Option Option Either Either Option Either Option
  30. Monad Transformers Eff Comparison of concept 30 Definition Execution Option

    Either Either Option Either Option Either Option No layer of Monad Stack No need to lift Option Either
  31. MTL vs atnos-eff 31 Monad Transformers Eff Definition Execution program.value

    // Option[Either[String, Int]] def program: EitherT[Option, String, Int] = for { t <- EitherT.fromEither[Option](Right(1)) result <- EitherT.liftF[Option, String, Int](Some(t)) } yield result def program[R :_stringEither :_option]: Eff[R, Int] = for { t <- fromEither[R, String, Int](Right(1)) result <- fromOption[R, Int](Some(t)) } yield result type Stack = Fx.fx2[Option, StringEither] // Either[String, Option[Int]] program[Stack].runOption.runEither.run // Option[Either[String, Int]] program[Stack].runEither.runOption.run
  32. atnos-eff 32 Definition Execution def program[R :_stringEither :_option]: Eff[R, Int]

    = for { t <- fromEither[R, String, Int](Right(1)) result <- fromOption[R, Int](Some(t)) } yield result Smart Constructor Eff Monad type Stack = Fx.fx2[Option, StringEither] program[Stack] // Eff[Fx2[Option, StringEither], Int] .runOption // Eff[Fx1[StringEither], Option[Int]] .runEither // Eff[NoFx, Either[String, Option[Int]]] .run // Either[String, Option[Int]] Interpreter
  33. Eff Monad 33 import org.atnos.eff.{Eff, Member} import org.atnos.eff.either._ import org.atnos.eff.option._

    type StringEither[A] = String Either A type _stringEither[R] = StringEither |= R def program[R :_stringEither :_option]: Eff[R, Int] type Stack = Fx.fx2[Option, StringEither] program[Stack] Eff sealed trait Eff[R, A] { def map[B](f: A => B): Eff[R, B] def flatMap[B](f: A => Eff[R, B]): Eff[R, B] } Effect Stack Option Either https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/Eff.scala
  34. atnos-eff 34 https://atnos-org.github.io/eff/org.atnos.site.OutOfTheBox.html

  35. Smart Constructor 35 trait OptionCreation { type _option[R] = Option

    |= R /** create an Option effect from a single Option value */ def fromOption[R :_option, A](o: Option[A]): Eff[R, A] = send[Option, R, A](o) } Option fromOption Eff Transforms effect into Eff Option Either https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/ OptionEffect.scala
  36. atnos-eff 36 Definition import org.atnos.eff.either._ import org.atnos.eff.option._ def program[R :_stringEither

    :_option]: Eff[R, Int] = for { t <- fromEither[R, String, Int](Right(1)) result <- fromOption[R, Int](Some(t)) } yield result Smart Constructor
  37. Interpreter 37 trait OptionInterpretation { def runOption[R, U, A](effect: Eff[R,

    A])(implicit m: Member.Aux[Option, R, U]): Eff[U, Option[A]] } implicit class OptionEffectOps[R, A](e: Eff[R, A]) { def runOption(implicit member: Member[Option, R]): Eff[member.Out, Option[A]] = OptionInterpretation.runOption(e)(member.aux) } runOption Eff Can execute the effect Option Either https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/ OptionEffect.scala
  38. Interpreter 38 import org.atnos.eff.syntax.option._ import org.atnos.eff.Fx /** Stack declaration **/

    type Stack = Fx.fx2[Option, StringEither] program[Stack] // Eff[Fx2[Option, StringEither], Int] .runOption // Eff[Fx1[StringEither], Option[Int]] Option Can execute the effect runOption Eff Either
  39. atnos-eff 39 Definition Execution def program[R :_stringEither :_option]: Eff[R, Int]

    = for { t <- fromEither[R, String, Int](Right(1)) result <- fromOption[R, Int](Some(t)) } yield result import org.atnos.eff.syntax.either._ import org.atnos.eff.syntax.option._ import org.atnos.eff.syntax.eff._ type Stack = Fx.fx2[Option, StringEither] program[Stack] // Eff[Fx2[Option, StringEither], Int] .runOption // Eff[Fx1[StringEither], Option[Int]] .runEither // Eff[NoFx, Either[String, Option[Int]]] .run // Either[String, Option[Int]] Interpreter
  40. Three effects in eff 40 type ErrorOr[A] = Either[Error, A]

    type _option[R] = Option |= R type _io[R] = |=[IO, R] type _errorOr[R] = ErrorOr |= R def program[R :_io :_option :_errorOr] (userId: UserId, teamName: TeamName): Eff[R, Response] = for { userOpt <- fromIO(findUser(userId)) user <- fromOption(userOpt) team <- fromEither(createTeam(userId, teamName)) _ <- fromIO(storeTeam(team)) } yield createResponse(user, team) type Stack = Fx.fx3[Option, ErrorOr, IO] program[Stack](userId, teamName) .runOption // Eff[Fx2[Either, IO], Option[Response]] .runEither[Error] // Eff[Fx1[IO], Either[Error, Option[Response]] .to[IO] // IO[Either[Error, Option[Response]]]
  41. type ErrorOr[A] = Either[Error, A] type _option[R] = Option |=

    R type _io[R] = |=[IO, R] type _errorOr[R] = ErrorOr |= R def program[R :_io :_option :_errorOr] (userId: String, teamName: String): Eff[R, Response] = for { userOpt <- fromIO(findUser(userId)) user <- fromOption(userOpt) team <- fromEither(createTeam(userId, teamName)) _ <- fromIO(storeTeam(team)) } yield createResponse(user, team) MTL Eff type Stack = Fx.fx3[Option, ErrorOr, IO] program[Stack](userId, teamName) .runOption // Eff[Fx2[Either, IO], Option[Response]] .runEither[Error] // Eff[Fx1[IO], Either[Error, Option[Response]] .to[IO] // IO[Either[Error, Option[Response]]] program(userId, teamName) // OptionT[EitherIO, Response] .value // EitherT[IO, Error, Option[Response]] .value // IO[Either[Error, Option[Response]]] type EitherIO[A] = EitherT[IO, Error, A] def program(userId: String, teamName: String): OptionT[EitherIO, Response] = { for { user <- OptionT[EitherIO, User] (EitherT.liftF[IO, Error, Option[User]](findUser(userId))) team <- OptionT.liftF[EitherIO, Team] (EitherT.fromEither[IO](createTeam(userId, teamName))) _ <- OptionT.liftF[EitherIO, Unit] (EitherT.liftF[IO, Error, Unit](storeTeam(team))) } yield createResponse(user, team) } MTL vs atnos-eff 41
  42. Apply Eff to entire program 42 class CreatePostService[R: _io: _connectionIO:

    _errorOr] (…) { override def execute(in: CreatePostParam): Eff[R, CreatePostDTO] = { for { _ <- userRepository.resolve[R](Id(in.userId)).validate("user is not exist.") post <- Post.create[R](in.userId, in.text, None) _ <- postRepository.store[R](post) } yield CreatePostDTO(post.id.value) } } def resolve[R: _connectionIO](id: Id[E]): Eff[R, Option[E]] def store[R: _connectionIO](entity: E): Eff[R, Unit] Service Repository Domain Model def create[R: _errorOr](…): Eff[R, Post]
  43. Apply Eff to entire program 43 trait CreatePostController { type

    R = Fx.fx3[IO, ConnectionIO, Validated] def run(createPostParam: CreatePostParam): IO[ThrowableEither[ErrorOr[CreatePostDTO]]] = CreatePostService[R] .execute(createPostParam) .runValidatedNel[String] .runConnectionIO(Database.xa) .ioAttempt .to[IO] } Controller
  44. Advantage of Eff 44 • Readability • Easy to compose

    • No match type game for { userOpt <- fromIO(findUser(userId)) user <- fromOption(userOpt) team <- fromEither(createTeam(userId, teamName)) _ <- fromIO(storeTeam(team)) } yield createResponse(user, team)
  45. Disadvantage of Eff 45 • Dependency of syntax • Confused

    to multiple run
  46. Dependency of syntax 46 type R = Fx.fx3[IO, ConnectionIO, Validated]

    CreatePostService[R].execute(createPostParam) def resolve[R: _connectionIO](id: Id[E]): Eff[R, Option[E]] def create[R: _errorOr](userId: String, text: String, parentPostId: Option[String]): Eff[R, Post] class CreatePostService[R: _io: _connectionIO: _errorOr] Controller Service Repository Domain Model
  47. Confused to multiple run 47 type Stack = Fx.fx3[Option, ErrorOr,

    IO] program[Stack](userId, teamName) .runOption // Eff[Fx2[Either, IO], Option[Response]] .runEither[Error] // Eff[Fx1[IO], Either[Error, Option[Response]] .to[IO] // IO[Either[Error, Option[Response]] program[Stack](userId, teamName) .runEither[Error] // Eff[Fx2[Option, IO], Either[Error, Response]] .runOption // Eff[Fx1[IO], Option[Either[Error, Response]] .to[IO] // IO[Option[Either[Error, Response]]])
  48. Summary 48 •Compose two monads •Compose more than three monads

    •Entire Architecture Monad Transformer is suit. Eff is better. Eff is good, but there are some challenges.
  49. References 49 •Emm A Sane Alternative to Monad Transformers in

    Scala •Monad transformers down to earth •A Journey into Extensible Effects in Scala •Extensible Effects vs Monad Transformers
  50. Appendix 50 type Stack = Fx.fx2[Option, StringEither] def program: Eff[Stack,

    Int] = for { t <- fromEither[Stack, String, Int](Right(1)) result <- fromOption[Stack, Int](Some(t)) } yield result program.runOption.runEither.run = def program[R :_stringEither :_option]: Eff[R, Int] = for { t <- fromEither[R, String, Int](Right(1)) result <- fromOption[R, Int](Some(t)) } yield result type Stack = Fx.fx2[Option, StringEither] program[Stack].runOption.runEither.run
  51. Appendix 51 def program[R: _stateInt :_option]: Eff[R, Int] def program[R](implicit

    stateInt: _stateInt[R], option: _option[R]): Eff[R, Int] =
  52. Appendix 52 /** create an Either effect from a single

    Either value */ def fromEither[R, E, A](Either: E Either A)(implicit member: (E Either ?) |= R): Eff[R, A] = Either.fold[Eff[R, A]](left[R, E, A], right[R, E, A]) /** create an Either effect from a value possibly throwing an exception */ def fromCatchNonFatal[R, E, A](a: =>A)(onThrowable: Throwable => E)(implicit member: (E Either ?) |= R): Eff[R, A] = fromEither(Either.catchNonFatal(a).leftMap(onThrowable)) /** create an Either effect from a value possibly throwing a Throwable */ def catchNonFatalThrowable[R, A](a: =>A)(implicit member: (Throwable Either ?) |= R): Eff[R, A] = fromCatchNonFatal(a)(identity) SmartConstructor is not only one each Effect. https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/ EitherEffect.scala
  53. Appendix 53 Interpreter is not only one each Effect. https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/

    EitherEffect.scala /** catch possible left values */ def attemptEither[R, E, A](effect: Eff[R, A])(implicit member: (E Either ?) /= R): Eff[R, E Either A] = catchLeft[R, E, E Either A](effect.map(a => Either.right(a)))(e => pure(Either.left(e))) /** catch and handle a possible left value */ def catchLeft[R, E, A](effect: Eff[R, A])(handle: E => Eff[R, A])(implicit member: (E Either ?) /= R): Eff[R, A] = catchLeftEither[R, E, A](effect)(handle)(cats.instances.either.catsStdInstancesForEither[E]) /** run the Either effect, handling E (with effects) and yielding A */ def runEitherCatchLeft[R, U, E, A](r: Eff[R, A])(handle: E => Eff[U, A])(implicit m: Member.Aux[(E Either ?), R, U]): Eff[U, A] = runEither(r).flatMap(_.fold(handle, pure))