Extensible Effects: beyond the Monad Transformers

This presentation includes these following three things:
1. Monad
2. Monad Transformers
3. Extensible-Effects
Main topics are how to use atnos-eff and comparison between mtl and eff.

Transcript

1. Extensible-Effects:
   beyond the Monad Transformers
   Hiroki Fujino
   Unipos GmbH
   1
   

   View Slide

2. Introduction
   2
   • Hiroki Fujino
   • Head of Engineer in Unipos
   • Came from Japan since this February
   • My interests: Scala, Architecture Design, Concurrency
   Programming, etc.
   

   View Slide

3. Agenda
   • Monad
   • Monad Transformers
   • Extensible Effects
   3
   Prerequisites:
   Basic knowledge of Scala(includes Experience with using Monad)
   https://github.com/Hiroki6/SampleAtnosEff
   Sample Code:
   

   View Slide

4. Monad
   4
   Allows composition of dependent effectful functions.
   Option
   Future
   Either
   Writer
   Reader
   State
   IO
   https://typelevel.org/cats/typeclasses/monad.html
   

   View Slide

5. Monad
   5
   trait Monad[M[_]] {
   def pure[A](x: A): M[A]
   def map[A, B](x: M[A])(f: A => B): M[B]
   def flatMap[A, B](x: M[A])(f: A => M[B]): M[B]
   }
   

   View Slide

6. Option Monad
   6
   sealed trait Option[+A]
   case object None extends Option[Nothing]
   case class Some[A](value: A) extends Option[A]
   new Monad[Option] {
   def pure[A](x: A) = Some(x)
   def flatMap[A, B](x: Option[A])(f: A => Option[B]): Option[B] =
   x match {
   case Some(x) => f(x)
   case None => None
   }
   }
   

   View Slide

7. Option Monad
   7
   def hogeOption(a: Int): Option[Int] = Some(a)
   hogeOption(1).flatMap(a => hogeOption(a).map(b => a+b))
   // Some(2)
   for {
   a b } yield (a + b)
   // Some(2)
   

   View Slide

8. 8
   How to use multi monads?
   

   View Slide

9. Two Monads in program
   9
   def findUser(userId: UserId): IO[Option[User]]
   def getTeam(userId: UserId): IO[Option[Team]]
   def createResponse(user: User, team: Team): Response
   def program(userId: UserId): IO[Option[Response]] =
   for {
   userOpt teamOpt user team } yield createResponse(user, team)
   

   View Slide

10. 10
    [error] found : Option[Response]
    [error] required: cats.effect.IO[Option[Response]]
    [error] user def findUser(userId: UserId): IO[Option[User]]
    def getTeam(userId: UserId): IO[Option[Team]]
    def createResponse(user: User, team: Team): Response
    def program(userId: UserId): IO[Option[Response]] =
    for {
    userOpt teamOpt user team } yield createResponse(user, team)
    } Composition of IO
    } Composition of Option
    Two Monads with
    For-statement
    

    View Slide

11. Two Monads with
    For-statement
    11
    def program(userId: UserId): IO[Option[Response]] =
    for {
    userOpt teamOpt } yield {
    (userOpt, teamOpt) match {
    case (Some(user), Some(team)) =>
    Some(createResponse(user, team))
    case (_, _) =>
    None
    }
    }
    def program(userId: UserId): IO[Option[Response]] =
    for {
    userOpt teamOpt } yield for {
    user team } yield createResponse(user, team)
    

    View Slide

12. Monad Transformers
    12
    

    View Slide

13. Monad Transformers
    13
    G
    F
    •Popular solution to create monad stack
    •Monads do not compose(G[_], F[_] => F[G[_]])
    •But, if G[_] is implemented monad(Option, Either, etc.),
    F[G[_]] can be monad
    •Option
    •Either
    •Reader
    etc.
    

    View Slide

14. Monad Transformers
    14
    case class OptionT[F[_], A](value: F[Option[A]]) {
    def map[B](f: A => B)(implicit F: Functor[F]): OptionT[F, B] =
    OptionT(F.map(value)(_ map f))
    def flatMap[B](f: A => OptionT[F, B])(implicit F: Monad[F]): OptionT[F, B] =
    OptionT(F.flatMap(value)(_.fold(F.pure[Option[B]](None))(f(_).value)))
    }
    Option
    F
    

    View Slide

15. Two Monads with OptionT
    15
    def program(userId: UserId): OptionT[IO, Response] =
    for {
    user team } yield createResponse(user, team)
    program(userId).value // IO[Option[Response]]
    Option
    IO
    def flatMap[B](f: A => OptionT[F, B])(implicit F: Monad[F]): OptionT[F, B]
    } Composition of OptionT
    

    View Slide

16. Implementation of
    Monad Transformers
    16
    OptionT
    EitherT
    WriterT
    ReaderT
    StateT
    These are implemented in cats and Scalaz.
    

    View Slide

17. 17
    Transformer can wrap another transformer
    type EitherIO[A] = EitherT[IO, Error, A]
    OptionT[EitherIO, A] // OptionT[EitherT[IO, Error, A]]
    IO
    Either
    Option
    

    View Slide

18. Three Monad in Program
    18
    type EitherIO[A] = EitherT[IO, Error, A]
    def program(userId: String, teamName: String): OptionT[EitherIO, Response] = {
    for {
    user EitherT.liftF[IO, Error, Option[User]](findUser(userId))
    }
    team EitherT.fromEither[IO](createTeam(userId, teamName))
    }
    _ EitherT.liftF[IO, Error, Unit](storeTeam(team))
    }
    } yield createResponse(user, team)
    }
    program(userId, teamName) // OptionT[EitherIO, Response]
    .value // EitherIO[Option[Response]] == EitherT[IO, Error, Option[Response]]
    .value // IO[Either[Error, Option[Response]]]
    IO[Option[User]]
    Either[Error, Team]
    IO[Unit]
    

    View Slide

19. Three Monad in Program
    19
    type EitherIO[A] = EitherT[IO, Error, A]
    def program(userId: String, teamName: String): OptionT[EitherIO, Response] = {
    for {
    user EitherT.liftF[IO, Error, Option[User]](findUser(userId))
    }
    team EitherT.fromEither[IO](createTeam(userId, teamName))
    }
    _ EitherT.liftF[IO, Error, Unit](storeTeam(team))
    }
    } yield createResponse(user, team)
    }
    program(userId, teamName) // OptionT[EitherIO, Response]
    .value // EitherIO[Option[Response]] == EitherT[IO, Error, Option[Response]]
    .value // IO[Either[Error, Option[Response]]]
    IO[Option[User]]
    Either[Error, Team]
    IO[Unit]
    

    View Slide

20. Issues of
    Monad Transformers
    • Cumbersome lift
    • Overhead of composition
    • Fixed order of Monad Stack
    20
    

    View Slide

21. Cumbersome lift
    21
    Option
    Either
    IO
    def storeTeam(team: Team): IO[Unit]
    OptionT.liftF[EitherIO, Unit] {
    EitherT.liftF[IO, Error, Unit](storeTeam(team)).map(_.some)
    }
    IO
    Either
    IO
    lift lift
    

    View Slide

22. Overhead of composition
    22
    OptionT {
    Monad[EitherIO].flatMap(value)(_.fold(F.pure[Option[User]](None))(f(_).value))
    }
    OptionT[EitherIO, User](EitherT.liftF[IO, Error, Option[User]](findUser(userId))).flatMap(f)
    EitherT(Monad[IO].flatMap(value) {
    case l @ Left(_) => F.pure(l.rightCast)
    case Right(b) => f(b).value
    })
    

    View Slide

23. Fixed order of Monad Stack
    23
    program.value // Option[Either[String, Int]]
    Definition
    Execution
    Either
    Option
    Option
    Either
    ❌
    def program: EitherT[Option, String, Int] = for {
    t result } yield result
    

    View Slide

24. 24
    Extensible Effects
    

    View Slide

25. 25
    “Extensible Effects:
    an alternative to monad transformers”
    –Oleg Kiselyov in 2013
    

    View Slide

26. Extensible Effects(Eff)
    Library in Scala
    26
    •atnos-eff
    •emm
    

    View Slide

27. Today’s Topic about Eff
    27
    atnos-eff
    •How to use atnos-eff
    •Advantage and Disadvantage
    •How to define your own Eff monad
    •Detail of atons-eff(Union Type, Interpreter)
    Not covered:
    Explain these below in this presentation:
    

    View Slide

28. Eff
    28
    Option State
    •Eff is monad.
    •Eff has effect stack in a flat state.
    •List of effects. order of effects is not important.
    Either
    Reader
    ex. Fx.fx4[Reader[Int, ?], Either[String, ?], Option, State[Int, ?]]
    

    View Slide

29. Monad Transformers Eff
    Comparison of concept
    29
    Definition
    Execution
    Option
    Either
    Either
    Option
    Option Either
    Either
    Option
    Either
    Option
    

    View Slide

30. Monad Transformers Eff
    Comparison of concept
    30
    Definition
    Execution
    Option
    Either
    Either
    Option
    Either
    Option
    Either
    Option
    No layer of Monad Stack
    No need to lift Option Either
    

    View Slide

31. MTL vs atnos-eff
    31
    Monad Transformers Eff
    Definition
    Execution
    program.value // Option[Either[String, Int]]
    def program: EitherT[Option, String, Int] = for {
    t result } yield result
    def program[R :_stringEither :_option]: Eff[R, Int] = for {
    t result } yield result
    type Stack = Fx.fx2[Option, StringEither]
    // Either[String, Option[Int]]
    program[Stack].runOption.runEither.run
    // Option[Either[String, Int]]
    program[Stack].runEither.runOption.run
    

    View Slide

32. atnos-eff
    32
    Definition
    Execution
    def program[R :_stringEither :_option]: Eff[R, Int] = for {
    t result } yield result
    Smart Constructor
    Eff Monad
    type Stack = Fx.fx2[Option, StringEither]
    program[Stack] // Eff[Fx2[Option, StringEither], Int]
    .runOption // Eff[Fx1[StringEither], Option[Int]]
    .runEither // Eff[NoFx, Either[String, Option[Int]]]
    .run // Either[String, Option[Int]]
    Interpreter
    

    View Slide

33. Eff Monad
    33
    import org.atnos.eff.{Eff, Member}
    import org.atnos.eff.either._
    import org.atnos.eff.option._
    type StringEither[A] = String Either A
    type _stringEither[R] = StringEither |= R
    def program[R :_stringEither :_option]: Eff[R, Int]
    type Stack = Fx.fx2[Option, StringEither]
    program[Stack]
    Eff
    sealed trait Eff[R, A] {
    def map[B](f: A => B): Eff[R, B]
    def flatMap[B](f: A => Eff[R, B]): Eff[R, B]
    }
    Effect Stack
    Option Either
    https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/Eff.scala
    

    View Slide

34. atnos-eff
    34
    https://atnos-org.github.io/eff/org.atnos.site.OutOfTheBox.html
    

    View Slide

35. Smart Constructor
    35
    trait OptionCreation {
    type _option[R] = Option |= R
    /** create an Option effect from a single Option value */
    def fromOption[R :_option, A](o: Option[A]): Eff[R, A] =
    send[Option, R, A](o)
    }
    Option
    fromOption
    Eff
    Transforms effect into Eff
    Option Either
    https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/
    OptionEffect.scala
    

    View Slide

36. atnos-eff
    36
    Definition
    import org.atnos.eff.either._
    import org.atnos.eff.option._
    def program[R :_stringEither :_option]: Eff[R, Int] = for {
    t result } yield result
    Smart Constructor
    

    View Slide

37. Interpreter
    37
    trait OptionInterpretation {
    def runOption[R, U, A](effect: Eff[R, A])(implicit m: Member.Aux[Option, R, U]): Eff[U, Option[A]]
    }
    implicit class OptionEffectOps[R, A](e: Eff[R, A]) {
    def runOption(implicit member: Member[Option, R]): Eff[member.Out, Option[A]] =
    OptionInterpretation.runOption(e)(member.aux)
    }
    runOption
    Eff
    Can execute the effect
    Option Either
    https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/
    OptionEffect.scala
    

    View Slide

38. Interpreter
    38
    import org.atnos.eff.syntax.option._
    import org.atnos.eff.Fx
    /** Stack declaration **/
    type Stack = Fx.fx2[Option, StringEither]
    program[Stack] // Eff[Fx2[Option, StringEither], Int]
    .runOption // Eff[Fx1[StringEither], Option[Int]]
    Option
    Can execute the effect
    runOption
    Eff
    Either
    

    View Slide

39. atnos-eff
    39
    Definition
    Execution
    def program[R :_stringEither :_option]: Eff[R, Int] = for {
    t result } yield result
    import org.atnos.eff.syntax.either._
    import org.atnos.eff.syntax.option._
    import org.atnos.eff.syntax.eff._
    type Stack = Fx.fx2[Option, StringEither]
    program[Stack] // Eff[Fx2[Option, StringEither], Int]
    .runOption // Eff[Fx1[StringEither], Option[Int]]
    .runEither // Eff[NoFx, Either[String, Option[Int]]]
    .run // Either[String, Option[Int]]
    Interpreter
    

    View Slide

40. Three effects in eff
    40
    type ErrorOr[A] = Either[Error, A]
    type _option[R] = Option |= R
    type _io[R] = |=[IO, R]
    type _errorOr[R] = ErrorOr |= R
    def program[R :_io :_option :_errorOr]
    (userId: UserId, teamName: TeamName): Eff[R, Response] =
    for {
    userOpt user team _ } yield createResponse(user, team)
    type Stack = Fx.fx3[Option, ErrorOr, IO]
    program[Stack](userId, teamName)
    .runOption // Eff[Fx2[Either, IO], Option[Response]]
    .runEither[Error] // Eff[Fx1[IO], Either[Error, Option[Response]]
    .to[IO] // IO[Either[Error, Option[Response]]]
    

    View Slide

41. type ErrorOr[A] = Either[Error, A]
    type _option[R] = Option |= R
    type _io[R] = |=[IO, R]
    type _errorOr[R] = ErrorOr |= R
    def program[R :_io :_option :_errorOr]
    (userId: String, teamName: String): Eff[R, Response] =
    for {
    userOpt user team _ } yield createResponse(user, team)
    MTL Eff
    type Stack = Fx.fx3[Option, ErrorOr, IO]
    program[Stack](userId, teamName)
    .runOption // Eff[Fx2[Either, IO], Option[Response]]
    .runEither[Error] // Eff[Fx1[IO], Either[Error, Option[Response]]
    .to[IO] // IO[Either[Error, Option[Response]]]
    program(userId, teamName) // OptionT[EitherIO, Response]
    .value // EitherT[IO, Error, Option[Response]]
    .value // IO[Either[Error, Option[Response]]]
    type EitherIO[A] = EitherT[IO, Error, A]
    def program(userId: String, teamName: String):
    OptionT[EitherIO, Response] = {
    for {
    user (EitherT.liftF[IO, Error, Option[User]](findUser(userId)))
    team (EitherT.fromEither[IO](createTeam(userId, teamName)))
    _ (EitherT.liftF[IO, Error, Unit](storeTeam(team)))
    } yield createResponse(user, team)
    }
    MTL vs atnos-eff
    41
    

    View Slide

42. Apply Eff to entire program
    42
    class CreatePostService[R: _io: _connectionIO: _errorOr] (…) {
    override def execute(in: CreatePostParam): Eff[R, CreatePostDTO] = {
    for {
    _ post _ } yield CreatePostDTO(post.id.value)
    }
    }
    def resolve[R: _connectionIO](id: Id[E]): Eff[R, Option[E]]
    def store[R: _connectionIO](entity: E): Eff[R, Unit]
    Service
    Repository
    Domain Model
    def create[R: _errorOr](…): Eff[R, Post]
    

    View Slide

43. Apply Eff to entire program
    43
    trait CreatePostController {
    type R = Fx.fx3[IO, ConnectionIO, Validated]
    def run(createPostParam: CreatePostParam):
    IO[ThrowableEither[ErrorOr[CreatePostDTO]]] =
    CreatePostService[R]
    .execute(createPostParam)
    .runValidatedNel[String]
    .runConnectionIO(Database.xa)
    .ioAttempt
    .to[IO]
    }
    Controller
    

    View Slide

44. Advantage of Eff
    44
    • Readability
    • Easy to compose
    • No match type game
    for {
    userOpt user team _ } yield createResponse(user, team)
    

    View Slide

45. Disadvantage of Eff
    45
    • Dependency of syntax
    • Confused to multiple run
    

    View Slide

46. Dependency of syntax
    46
    type R = Fx.fx3[IO, ConnectionIO, Validated]
    CreatePostService[R].execute(createPostParam)
    def resolve[R: _connectionIO](id: Id[E]): Eff[R, Option[E]]
    def create[R: _errorOr](userId: String,
    text: String,
    parentPostId: Option[String]): Eff[R, Post]
    class CreatePostService[R: _io: _connectionIO: _errorOr]
    Controller
    Service
    Repository
    Domain Model
    

    View Slide

47. Confused to multiple run
    47
    type Stack = Fx.fx3[Option, ErrorOr, IO]
    program[Stack](userId, teamName)
    .runOption // Eff[Fx2[Either, IO], Option[Response]]
    .runEither[Error] // Eff[Fx1[IO], Either[Error, Option[Response]]
    .to[IO] // IO[Either[Error, Option[Response]]
    program[Stack](userId, teamName)
    .runEither[Error] // Eff[Fx2[Option, IO], Either[Error, Response]]
    .runOption // Eff[Fx1[IO], Option[Either[Error, Response]]
    .to[IO] // IO[Option[Either[Error, Response]]])
    

    View Slide

48. Summary
    48
    •Compose two monads
    •Compose more than three monads
    •Entire Architecture
    Monad Transformer is suit.
    Eff is better.
    Eff is good, but there are some challenges.
    

    View Slide

49. References
    49
    •Emm A Sane Alternative to Monad Transformers in Scala
    •Monad transformers down to earth
    •A Journey into Extensible Effects in Scala
    •Extensible Effects vs Monad Transformers
    

    View Slide

50. Appendix
    50
    type Stack = Fx.fx2[Option, StringEither]
    def program: Eff[Stack, Int] = for {
    t result } yield result
    program.runOption.runEither.run
    =
    def program[R :_stringEither :_option]: Eff[R, Int] = for {
    t result } yield result
    type Stack = Fx.fx2[Option, StringEither]
    program[Stack].runOption.runEither.run
    

    View Slide

51. Appendix
    51
    def program[R: _stateInt :_option]: Eff[R, Int]
    def program[R](implicit stateInt: _stateInt[R], option: _option[R]): Eff[R, Int]
    =
    

    View Slide

52. Appendix
    52
    /** create an Either effect from a single Either value */
    def fromEither[R, E, A](Either: E Either A)(implicit member: (E Either ?) |= R): Eff[R, A] =
    Either.fold[Eff[R, A]](left[R, E, A], right[R, E, A])
    /** create an Either effect from a value possibly throwing an exception */
    def fromCatchNonFatal[R, E, A](a: =>A)(onThrowable: Throwable => E)(implicit member: (E Either ?) |= R): Eff[R, A] =
    fromEither(Either.catchNonFatal(a).leftMap(onThrowable))
    /** create an Either effect from a value possibly throwing a Throwable */
    def catchNonFatalThrowable[R, A](a: =>A)(implicit member: (Throwable Either ?) |= R): Eff[R, A] =
    fromCatchNonFatal(a)(identity)
    SmartConstructor is not only one each Effect.
    https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/
    EitherEffect.scala
    

    View Slide

53. Appendix
    53
    Interpreter is not only one each Effect.
    https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/
    EitherEffect.scala
    /** catch possible left values */
    def attemptEither[R, E, A](effect: Eff[R, A])(implicit member: (E Either ?) /= R): Eff[R, E Either A] =
    catchLeft[R, E, E Either A](effect.map(a => Either.right(a)))(e => pure(Either.left(e)))
    /** catch and handle a possible left value */
    def catchLeft[R, E, A](effect: Eff[R, A])(handle: E => Eff[R, A])(implicit member: (E Either ?) /= R): Eff[R, A] =
    catchLeftEither[R, E, A](effect)(handle)(cats.instances.either.catsStdInstancesForEither[E])
    /** run the Either effect, handling E (with effects) and yielding A */
    def runEitherCatchLeft[R, U, E, A](r: Eff[R, A])(handle: E => Eff[U, A])(implicit m: Member.Aux[(E Either ?), R, U]): Eff[U, A] =
    runEither(r).flatMap(_.fold(handle, pure))
    

    View Slide