Monad Transformers - Speaker Deck

Slide 1

Slide 1 text

MONAD TRANSFORMERS In The ld

Slide 2

Slide 2 text

speakerdeck.com/u/jrwest/p/monad-transformers

Slide 3

Slide 3 text

TWITTER: @_JRWEST GITHUB.COM/JRWEST BLOG.LOOPEDSTRANGE.COM

Slide 4

Slide 4 text

SF SCALA May 2012 * http://marakana.com/s/scala_typeclassopedia_with_john_kodumal_of_atlassian_video,1198/index.html

Slide 5

Slide 5 text

trait Monad[F[_]] extends Applicative[F] { def ﬂatMap[A, B](fa: F[A])(f :A=>F[B]):F[B] } * monad type class * ﬂatMap also called bind, >>=

Slide 6

Slide 6 text

def point[A](a: => A): M[A] def map[A,B](ma: M[A])(f: A => B): M[B] def ﬂatMap[A,B](ma: M[A])(f: A => M[B]): M[B] * the functions we care about * lift pure value, lift pure function, chain “operations”

Slide 7

Slide 7 text

scala> import scalaz.Monad scala> import scalaz.std.option._ scala> val a = Monad[Option].point(1) a: Option[Int] = Some(1) scala> Monad[Option].map(a)(_.toString + "hi") res2: Option[java.lang.String] = Some(1hi) scala> Monad[Option].bind(a)(i => if (i < 0) None else Some(i + 1)) res4: Option[Int] = Some(2) * explicit type class usage in scalaz seven

Slide 8

Slide 8 text

scala> import scalaz.syntax.monad._ import scalaz.syntax.monad._ scala> Option(1).ﬂatMap(i => if (i < 0) None else Some(i+1)) res6: Option[Int] = Some(2) scala> 1.point[Option].ﬂatMap(...) res7: Option[Int] = Some(2) * implicit type class usage in scalaz7 using syntax extensions

Slide 9

Slide 9 text

“A MONADIC FOR COMPREHENSION IS AN EMBEDDED PROGRAMMING LANGUAGE WITH SEMANTICS DEFINED BY THE MONAD” * “one intuition of monads” - john

Slide 10

Slide 10 text

MULTIPLE EFFECTS C position

Slide 11

Slide 11 text

Option[A] * it may not exist

Slide 12

Slide 12 text

SEMANTICS SIDE NOTE: * to an extent, you can “choose” the meaning of a monad * Option -- anon. exceptions -- more narrowly, the exception that something is not there. Validation - monad/not monad - can mean different things in different contexts

Slide 13

Slide 13 text

IO[Option[A]] * but side-effects are needed to even look for that value

Slide 14

Slide 14 text

IO[Validation[Throwable,Option[A]] * and looking for that value may throw exceptions (or fail in some way)

Slide 15

Slide 15 text

IO[(List[String], Validation[Throwable,Option[A])] * and logging what is going on is necessary

Slide 16

Slide 16 text

MULTIPLE EFFECTS A P oblem

Slide 17

Slide 17 text

MONADS DO NOT COMPOSE * the problem in theory (core issue)

Slide 18

Slide 18 text

“COMPOSE”?

Slide 19

Slide 19 text

FUNCTORS DO COMPOSE * as well as applicatives

Slide 20

Slide 20 text

trait Functor[F[_]] { def map[A, B](fa: F[A])(f :A=>B):F[B] }

Slide 21

Slide 21 text

def composeFunctor[M[_],N[_]](implicit m: Functor[M], n: Functor[N]) = new Functor[({type MN[A]=[M[N[A]]]})#MN] { def map[A,B](mna: M[N[A]])(f: A => B): M[N[B]] = ... } * generic function that composes any two functors M[_] and N[_]

Slide 22

Slide 22 text

def composeFunctor[M[_],N[_]](implicit m: Functor[M], n: Functor[N]) = new Functor[({type MN[A]=[M[N[A]]]})#MN] { def map[A,B](mna: M[N[A]])(f: A => B): M[N[B]] = { M.map(mna)(na => N.map(na)(f)) } }

Slide 23

Slide 23 text

scala> Option("abc").map(f) res1: Option[Int] = Some(3) scala> List(Option("abc"), Option("d"), Option("ef")).map2(f) res2: List[Option[Int]] = List(Some(3), Some(1), Some(2)) * can compose functors inﬁnitely deep but... * scalaz provides method to compose 2, with nice syntatic sugar, easily (map2)

Slide 24

Slide 24 text

def notPossible[M[_],N[_]](implicit m: Monad[M], n: Monad[N]) = new Monad[({type MN[A]=[M[N[A]]]})#MN] { def ﬂatMap[A,B](mna: M[N[A]])(f: A => M[N[B]]): M[N[B]] = ... } * cannot write the same function for any two monads M[_], N[_]

Slide 25

Slide 25 text

def notPossible[M[_],N[_]](implicit m: Monad[M], n: Monad[N]) = new Monad[({type MN[A]=[M[N[A]]]})#MN] { def ﬂatMap[A,B](mna: M[N[A]])(f: A => M[N[B]]): M[N[B]] = ... } TRY IT! * best way to understand this is attempt to write it yourself * it won’t compile

Slide 26

Slide 26 text

http://blog.tmorris.net/monads-do-not-compose/ * good resource to dive into this in more detail * some of previous slides based on above * provides template, in the form of a gist, for trying this stuff out

Slide 27

Slide 27 text

STAIR STEPPING * the problem in practice *http://www.ﬂickr.com/photos/caliperstudio/2667302181/

Slide 28

Slide 28 text

val a: IO[Option[MyData]] = ... val b: IO[Option[MyData]] = ... * have two values that require we communicate w/ outside world to fetch * those values may not exist (alternative meaning, fetching may result in exceptions that are anonymous)

Slide 29

Slide 29 text

for { data1 <- a data2 <- b } yield { data1 merge data2 // fail } * want to merge the two pieces of data if they both exist

Slide 30

Slide 30 text

for { // we've escaped IO, fail d1 <- a.unsafePerformIO d2 <- b.unsafePerformIO } yield d1 merge d2 * don’t want to perform the actions until later (don’t escape the IO monad)

Slide 31

Slide 31 text

for { od1 <- a od2 <- b } yield (od1,od2) match { case (Some(d1),Some(d2) => Option(d1 merge d2) case (a@Some(d1),_)) => a case (_,a@Some(d2)) => a case _ => None } for { od1 <- a od2 <- b } yield for { d1 <- od1 d2 <- od2 } yield d1 merge d2 * may notice the semi-group here * can also write it w/ an applicative * this is a contrived example

Slide 32

Slide 32 text

def b(data: MyData): IO[Option[MyData] BUT WHAT IF... * even w/ simple example, this minor change throws a monkey wrench in things

Slide 33

Slide 33 text

for { readRes <- readIO(domain) res <- readRes.fold( success = _.cata( some = meta => if (meta.enabledStatus /== status) { writeIO(meta.copy(enabledStatus = status)) } else meta.successNel[BarneyException].pure[IO], none = new ReadFailure(domain).failNel[AppMetadata].pure[IO] ), failure = errors => errors.fail[AppMetadata].pure[IO] ) } yield res ): * example of what not to do from something I wrote a while back

Slide 34

Slide 34 text

MULTIPLE EFFECTS A Solution

Slide 35

Slide 35 text

case class IOOption[A](run: IO[Option[A]]) deﬁne type that boxes box the value, doesn’t need to be a case class, similar to haskell newtype.

Slide 36

Slide 36 text

new Monad[IOOption] { def point[A](a: => A): IOOption[A] = IOOption(a.point[Option].point[IO]) def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] = IOOption(fa.run.map(opt => opt.map(f))) def flatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]):IOOption[B] = IOOption(fa.run.flatMap((o: Option[A]) => o match { case Some(a) => f(a).run case None => (None : Option[B]).point[IO] })) } * can define a Monad instance for new type

Slide 37

Slide 37 text

val a: IOOption[MyData] = ... val b: IOOption[MyData] = ... val c: IOOption[MyData] = for { data1 <- a data2 <- b } yield { data1 merge data2 } val d: IO[Option[MyData]] = c.run can use new type to improve previous contrived example

Slide 38

Slide 38 text

type MyState[A] = State[StateData,A] case class MyStateOption[A](run: MyState[Option[A]]) * what if we don’t need effects, but state we can read and write to produce a final optional value and some new state * State[S,A] where S is fixed is a monad * can define a new type for that as well

Slide 39

Slide 39 text

new Monad[MyStateOption] { def map[A,B](fa: MyStateOption[A])(f: A => B): MyStateOption[B] = MyStateOption(Functor[MyState].map(fa)(opt => opt.map(f))) def flatMap[A, B](fa: MyStateOption[A])(f :A=>IOOption[B]) = MyStateOption(Monad[MyState]].bind(fa)((o: Option[A]) => o match { case Some(a) => f(a).run case None => (None : Option[B]).point[MyState] })) } new Monad[IOOption] { def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] = IOOption(Functor[IO].map(fa)(opt => opt.map(f))) def flatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]) = IOOption(Monad[IO]].bind(fa)((o: Option[A]) => o match { case Some(a) => f(a).run case None => (None : Option[B]).point[IO] })) } * opportunity for more abstraction * if you were going to do this, not exactly the way you would define these in real code, cheated a bit using {Functor,Monad}.apply

Slide 40

Slide 40 text

case class OptionT[M[_], A](run: M[Option[A]]) deﬁne a new type parameterized * -> * and *.

Slide 41

Slide 41 text

case class OptionT[M[_], A](run: M[Option[A]]) { def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] def flatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]): OptionT[M,B] } * define map/flatMap a little differently, can be done like previous as typeclass instance but convention is to define the interface on the transformer and later define typeclass instance using the interface

Slide 42

Slide 42 text

case class OptionT[M[_], A](run: M[Option[A]]) { def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] = OptionT[M,B](F.map(run)((o: Option[A]) => o map f)) def ﬂatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]): OptionT[M,B] = OptionT[M,B](M.bind(run)((o: Option[A]) => o match { case Some(a) => f(a).run case None => M.point((None: Option[B])) })) } * implementations resemble what has already been shown

Slide 43

Slide 43 text

new Monad[IOOption] { def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] = IOOption(Functor[IO].map(fa)(opt => opt.map(f))) def ﬂatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]) = IOOption(Monad[IO]].bind(fa)((o: Option[A]) => o match { case Some(a) => f(a).run case None => (None : Option[B]).point[IO] })) } case class OptionT[M[_], A](run: M[Option[A]]) { def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] = OptionT[M,B](F.map(run)((o: Option[A]) => o map f)) def ﬂatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]) = OptionT[M,B](M.bind(run)((o: Option[A]) => o match { case Some(a) => f(a).run case None => M.point((None: Option[B])) })) } * it the generalization of what was written before

Slide 44

Slide 44 text

type FlowState[A] = State[ReqRespData, A] val f: Option[String] => FlowState[Boolean] = (etag: Option[String]) => { val a: OptionT[FlowState, Boolean] = for { // string <- OptionT[FlowState,String] e <- optionT[FlowState](etag.point[FlowState]) // wrap FlowState[Option[String]] in OptionT matches <- optionT[FlowState]((requestHeadersL member IfMatch)) } yield matches.split(",").map(_.trim).toList.contains(e) a getOrElse false // FlowState[Boolean] } * check existence of etag in an http request, data lives in state * has minor bug, doesn’t deal w/ double quotes as written * https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/ WebmachineDecisions.scala#L282-285

Slide 45

Slide 45 text

val reqCType: OptionT[FlowState,ContentType] = for { contentType <- optionT[FlowState]( (requestHeadersL member ContentTypeHeader) ) mediaInfo <- optionT[FlowState]( parseMediaTypes(contentType).headOption.point[FlowState] ) } yield mediaInfo.mediaRange * determine content type of the request, data lives in state, may not be speciﬁed * https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/ WebmachineDecisions.scala#L772-775

Slide 46

Slide 46 text

scala> type EitherTString[M[_],A] = EitherT[M,String,A] deﬁned type alias EitherTString scala> val items = eitherT[List,String,Int](List(1,2,3,4,5,6).map(Right(_))) items: scalaz.EitherT[List,String,Int] = ... * adding features to a “embedded language”

Slide 47

Slide 47 text

for { i <- items } yield print(i) // 123456 for { i <- items _ <- if (i > 4) leftT[List,String,Unit]("fail") else rightT[List,String,Unit](()) } yield print(i) // 1234 * adding error handling, and early termination to non-deterministic computation

Slide 48

Slide 48 text

MONAD TRANSFORMERS In General

Slide 49

Slide 49 text

MyMonad[A]

Slide 50

Slide 50 text

NAMING CONVENTION MyMonadT[M[_], A] * transformer name ends in T

Slide 51

Slide 51 text

BOXES A VALUE run: M[MyMonad[A] * value is typically called “run” in scalaz7 * often called “value” in scalaz6 (because of NewType)

Slide 52

Slide 52 text

A MONAD TRANSFORMER IS A MONAD TOO * i mean, its thats kinda the point of this whole exercise isn’t it :)

Slide 53

Slide 53 text

def optTMonad[M[_] : Monad] = new Monad[({type O[X]=OptionT[M,X]]})#O) { def point[A](a: => A): OptionT[M,A] = OptionT(a.point[Option].point[M]) def map[A,B](fa: OptionT[M,A])(f: A => B): OptionT[M,B] = fa map f def flatMap[A, B](fa: OptionT[M,A])(f :A=> OptionT[M,B]): OptionT[M, B] = fa flatMap f } * monad instance definition for OptionT

Slide 54

Slide 54 text

HAS INTERFACE RESEMBLING UNDERLYING MONAD’S INTERFACE * can interact with the monad transformer in a manner similar to working with the actual monad * same methods, slightly different type signatures * different from haskell, “feature” of scala, since we can deﬁne methods on a type

Slide 55

Slide 55 text

case class OptionT[M[_], A](run: M[Option[A]]) { def getOrElse[AA >: A](d: => AA)(implicit F: Functor[M]): M[AA] = F.map(run)((_: Option[A]) getOrElse default) def orElse[AA >: A](o: OptionT[M,AA])(implicit M: Monad[M]): OptionT[M,AA] = OptionT[M,AA](M.bind(run) { case x@Some(_) => M.point(x) case None => o.run } }

Slide 56

Slide 56 text

MONAD TRANSFORMERS Stacked Effects

Slide 57

Slide 57 text

TRANSFORMER IS A MONAD 㱺 TRANSFORMER CAN WRAP ANOTHER TRANSFORMER * at the start, the goal was to stack effects (not just stack 2 effects) * this makes it possible

Slide 58

Slide 58 text

type VIO[A] = ValidationT[IO,Throwable,A] def doWork(): VIO[Option[Int]] = ... val r: OptionT[VIO,Int] = optionT[VIO](doWork()) * wrap the ValidationT with success type Option[A] in an OptionT * deﬁne type alias for connivence -- avoids nasty type lambda syntax inline

Slide 59

Slide 59 text

val action: OptionT[VIO, Boolean] = for { devDomain <- optionT[VIO] { validationT( bucket.fetch[CName]("%s.%s".format(devPreﬁx,hostname)) ).mapFailure(CNameServiceException(_)) } _ <- optionT[VIO] { validationT(deleteDomains(devDomain)).map(_.point[Option]) } } yield true * code (slightly modiﬁed) from one of stackmob’s internal services * uses Scaliak to fetch hostname data from riak and then remove them * possible to clean this code up a bit, will discuss shortly (monadtrans)

Slide 60

Slide 60 text

KEEP ON STACKIN’ ON * don’t have to stop at 2 levels deep, our new stack is monad too * each monad/transformer we add to the stack compose more types of effects

Slide 61

Slide 61 text

“ORDER” MATTERS * how stack is built, which transformers wrap which monads, determines the overall semantics of the entire stack * changing that order can, and usually does, change semantics

Slide 62

Slide 62 text

OptionT[FlowState, A] vs. StateT[Option,ReqRespData,A] * what is the difference in semantics between the two? * type FlowState[A] = State[ReqRespData,A]

Slide 63

Slide 63 text

FlowState[Option[A]] vs. Option[State[ReqRespData,A] * unboxing makes things easier to see * a state action that returns an optional value vs a state action that may not exist * the latter probably doesn’t make as much sense in the majority of cases

Slide 64

Slide 64 text

MONADTRANS The Type Class * type classes beget more type classes

Slide 65

Slide 65 text

REMOVING REPETITION === MORE ABSTRACTION * previous examples have had a repetitive, annoying, & verbose task * can be abstracted away...by a type class of course

Slide 66

Slide 66 text

optionT[VIO](validationT(deleteDomains(devDomain)).map(_.point[Option])) eitherT[List,String,Int](List(1,2,3,4,5,6).map(Right(_))) resT[FlowState](encodeBodyIfSet(resource).map(_.point[Res])) * some cases require lifting the value into the monad and then wrap it in the transformer * from previous examples

Slide 67

Slide 67 text

M[A] -> M[N[A]] -> NT[M[N[_]], A] * this is basically what we are doing every time * taking some monad M[A], lifting A into N, a monad we have a transformer for, and then wrapping all of that in N’s monad transformer

Slide 68

Slide 68 text

trait MonadTrans[F[_[_], _]] { def liftM[G[_] : Monad, A](a: G[A]): F[G, A] } * liftM will do this for any transformer F[_[_],_] and any monad G[_] provided an instance of it is deﬁned for F[_[_],_]

Slide 69

Slide 69 text

def liftM[G[_], A](a: G[A])(implicit G: Monad[G]): OptionT[G, A] = OptionT[G, A](G.map[A, Option[A]](a)((a: A) => a.point[Option])) * full deﬁnition requires some type ceremony * https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/scalaz/OptionT.scala#L155-156

Slide 70

Slide 70 text

def liftM[G[_], A](ga: G[A])(implicit G: Monad[G]): ResT[G,A] = ResT[G,A](G.map(ga)(_.point[Res])) * implementation for scalamachine’s Res monad * https://github.com/stackmob/scalamachine/blob/master/scalaz7/src/main/scala/scalamachine/scalaz/res/ ResT.scala#L75-76

Slide 71

Slide 71 text

encodeBodyIfSet(resource).liftM[OptionT] List(1,2,3).liftM[EitherTString] validationT(deleteDomains(devDomain)).liftM[OptionT] * cleanup of previous examples * method-like syntax requires a bit more work: https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/ scalaz/syntax/MonadSyntax.scala#L9

Slide 72

Slide 72 text

for { media <- (metadataL >=> contentTypeL).map(_ | ContentType("text/plain")).liftM[ResT] charset <- (metadataL >=> chosenCharsetL).map2(";charset=" + _).getOrElse("")).liftM[ResT] _ <- (responseHeadersL += (ContentTypeHeader, media.toHeader + charset)).liftM[ResT] mbHeader <- (requestHeadersL member AcceptEncoding).liftM[ResT] decision <- mbHeader >| f7.point[ResTFlow] | chooseEncoding(resource, "identity;q=1.0,*;q=0.5") } yield decision * https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/ WebmachineDecisions.scala#L199-205

Slide 73

Slide 73 text

MONAD TRANSFORMERS In Review

Slide 74

Slide 74 text

STACKING MONADS COMPOSES EFFECTS * when monads are stacked an embedded language is being built with multiple effects * this is not the only intuition of monads/transformers

Slide 75

Slide 75 text

CAN NOT COMPOSE MONADS GENERICALLY * cannot write generic function to compose any two monads M[_], N[_] like we can for any two functors

Slide 76

Slide 76 text

MONAD TRANSFORMERS COMPOSE M[_] : MONAD WITH ANY N[_] : MONAD * can’t compose any two, but can compose a given one with any other

Slide 77

Slide 77 text

MONAD TRANSFORMERS WRAP OTHER MONAD TRANSFORMERS * monad transformers are monads * so they can be the N[_] : Monad that the transformer composes with its underlying monad

Slide 78

Slide 78 text

MONADTRANS REDUCES REPETITION * often need to take a value that is not entirely lifted into a monad transformer stack and do just that

Slide 79

Slide 79 text

STACK MONADS DON’T STAIR-STEP * monad transformers reduce ugly, stair-stepping or nested code and focuses on core task * focuses on intuition of mutiple effects instead of handling things haphazardly