Using scala Futures the functional way

Slide 1

Slide 1 text

Using Futures the functional way    OC Scala meetup 2015

Slide 2

Slide 2 text

Who am I?   ! Vincent Marquez" ! Twitter: @runT1ME" ! Github: /vmarquez

Slide 3

Slide 3 text

I write code for   ! www.JazelAuto.com   (automotive advertising platform)   ! www.10thdegree.com  (digital marketing agency)    We consume and process a lot of data.   Necessitates concurrency, speciﬁcally Futures.

Slide 4

Slide 4 text

We need a ﬂexible way to sprinkle in concurrency that doesn’t affect our program logic. We need to utilize concurrency(Futures) with all sorts of different data structures.

Slide 5

Slide 5 text

! def futureBasics(): Unit = {! val f1 = Future { 1 + 2 } //happening asynchronously! val f2 = Future { 3 + 5 } ! ! val f3 = for {! a <- f1! b <- f2! } yield ! a + b! ! //we only want to Await at the end of the world ! val result = Await.result(f3, Duration(5, SECONDS))! ! println(result)! }!

Slide 6

Slide 6 text

! //What if we want parallelism but for a function returning option?! def doStuff[A](a1: A, a2: A)(f: A => Option[Int]): Future[Option[Int]] = {! val fi1 = Future { f(a1) }! val fi2 = Future { f(a2) }! for {! o1 <- fi1 //call to map! o2 <- fi2! } yield! for {! i1 <- o1! i2 <- o2! } yield! i1 + i2! }!

Slide 7

Slide 7 text

This is very common, we need to work with Futures over some other context (Option/Either/Kleisli(Reader) etc.) To solve that problem we need a detour on abstraction…

Slide 8

Slide 8 text

//Can we generalize this?! ! val o1 = Some(2) // or 1.some! ! val l1 = List(1,2,3)! ! val f1 = Future { 1+6 }! ! ! val r1 = o1.map(i => i*i)! ! val r2 = l1.map(i => i*i)! ! val r3 = f1.map(i => i*i)!

Slide 9

Slide 9 text

//can we abstract over that idea? Subclass polymorphism?! def squareInContext(s: Seq[Int]): Seq[Int] = ! s.map(i => i*i)! ! //No, not quite right. ! ! def squareInContextF[A <: Seq[Int]](s: A)! ! (implicit ev: Seq[Int] =:= A): A = ! ! s.map(i => i*i)! ! //Ugly, and doesn’t work with things that don’t extend Seq!

Slide 10

Slide 10 text

What if we could abstract over anything that had a ‘map like’! method, as long as we provided evidence that the type could be ‘mapped’?

Slide 11

Slide 11 text

We would call it…!

Slide 12

Slide 12 text

//F[_] is called a higher kinded type! trait Mapable[F[_]] {! def map[A,B](fa: F[A])(mf: A => B): F[B]! }! ! implicit def listMapable: Mapable[List] = new Mapable[List] {! def map[A,B](fa: List[A])(mf: A => B): List[B] = fa.map(mf)! }! ! implicit def futureMapable: Mapable[Future] = new Mapable[Future] {! def map[A,B](fa: Future[A])(mf: A => B): Future[B] = fa.map(mf)! }!

Slide 13

Slide 13 text

What about for something that doesn’t have a map method?

Slide 14

Slide 14 text

//Function0[A] is the type of () => A! implicit def functionMappable: Mapable[Function0] = ! ! new Mapable[Function0] {! def map[A,B](f: Function0[A])(mf: A => B): Function0[B] = ! ! ! ! () => mf(f())! } !

Slide 15

Slide 15 text

! ! def squareInCtxtT[F[_]](s: F[Int])(implicit M: Mapable[F]): F[Int] = ! M.map(s)(i => i*i)!

Slide 16

Slide 16 text

! ! What do we gain? Considering we can call a method in the map, not much. Yet…

Slide 17

Slide 17 text

! trait FlatMapable[F[_]] extends Mapable[F] {! ! //notice => A, not A. ! def point[A](a: => A): F[A]! ! def flatMap[A,B](fa: F[A])(fm: A => F[B]): F[B]! ! //we can derive map from the previous two methods! ! override def map[A,B](fa: F[A])(m: A => B): F[B] = ! ! ! flatMap(fa)(a => point(m(a)))! }! !

Slide 18

Slide 18 text

implicit def optionFlatMapable: FlatMapable[Option] = ! ! new FlatMapable[Option] {! def point[A](a: => A): Option[A] = Option(a)! ! ! def flatMap[A,B](fa: Option[A])(f: A => Option[B]): Option[B] = ! ! ! fa.flatMap(a => ! f(a)) ! }! ! implicit def futureFlatMapable: FlatMapable[Future] = ! ! new FlatMapable[Future] {! !def point[A](a: => A): Future[A] = Future { a }! ! ! ! def flatMap[A,B](fa: Future[A])(f: A => Future[B]): Future[B] = ! ! ! ! fa.flatMap(a => f(a))! }! !

Slide 19

Slide 19 text

! object UnsafeJavaWebApi { ! def unsafeApi(i: Int): String = {! if (i % 2 == 0)! null! else {! //long computation might occur here! i+5.toString! }! }! ! def unsafeApi2(s: String): String = ! try {! (s.toInt+1).toString! } catch {! case ex: Exception => null ! }! }!

Slide 20

Slide 20 text

// Let’s make use of our flatMapable trait. We’ll create a pipeline ! def pipeline[F[_]](fa: F[Int])(implicit M: FlatMapable[F]): F[String] = {! val newm = M.point(unsafeApi(2))! M.flatMap(fa)(i => ! M.flatMap(M.point(unsafeApi(i)))(s => ! M.flatMap(M.point(unsafeApi2(s+"blah")))(s2 => ! M.map(newm)(s3 => s3 + s2)! )! )! )! }!

Slide 21

Slide 21 text

! So why all this trouble? Why abstract over the flatMapable? I thought this was a talk on Futures?!

Slide 22

Slide 22 text

//we get short circuiting. if null is passed to point, we return none! def doPipeLine(i: Int): Option[String] = ! pipeline(Option(i))! ! ! // we get parallelism (because of the call to point in the pipeline)! def doPipelineF(i: Int): Future[String] = ! pipeline(Future { i })!

Slide 23

Slide 23 text

! OK that’s cool, but is it really worth the trouble?!

Slide 24

Slide 24 text

! What if we want both short circuiting and parallelism? We could make the method take an F[G[A]] and require FlatMapable’s for both kind parameters, but it’s not scalable. Remember the nested ‘fors’?

Slide 25

Slide 25 text

! ! class OptionHelper[F[_],A](val foa: F[Option[A]]) {! ! ! def flatMap[B](f: A => OptionHelper[F,B])! ! ! ! ! ! ! (implicit M:FlatMapable[F]): OptionHelper[F,B] = ! ! ! ! ! ! new OptionHelper[F,B](M.flatMap(foa)(o => o match {! case Some(a) => f(a).foa! case None => M.point(None: Option[B])! }))! }! ! ! /*! we are able to access what’s inside the F[Option[A]] no matter what the F is now, as long as there exists an implicit FlatMapable for F!! */ !

Slide 26

Slide 26 text

! Ok now we want a ‘FlatMapable’ trait for OptionHelper. But there’s a problem. The type parameter for FlatMapable needs to have a single kind parameter.     The types for OptionHelper are [_],A. [_],A doesn’t fit in something that takes just an [_].! !

Slide 27

Slide 27 text

! //Option 1: Type Aliasing. (The one your coworkers will prefer).??! ! type OptionFutureHelper[A] = OptionHelper[Future,A]! ! def OptionHelperFutureFlatMapable(implicit M: FlatMapable[Future]): ! ! FlatMapable[OptionFutureHelper] = ! ! new FlatMapable[OptionFutureHelper] {! def point[A](a: => A): OptionFutureHelper[A] =! ! ! new OptionHelper(M.point(Option(a)))! ! def flatMap[A,B](fa: OptionFutureHelper[A])! ! ! (f: A => OptionFutureHelper[B]): OptionFutureHelper[B] =! ! ! ! fa.flatMap(f)(M)! ! }!

Slide 28

Slide 28 text

! //Option 2: no type alias necessary…but uglier and you will have to explain this to all your new team members.! ! implicit def OptionHelperFlatMapable[F[_]](implicit M: FlatMapable[F]): ! ! FlatMapable[({ type l[a] = OptionHelper[F,a]})#l] = ! ! new FlatMapable[({ type l[a] = OptionHelper[F,a]})#l] { ! ! ! def point[A](a: => A): OptionHelper[F,A] = ! ! ! ! new OptionHelper(M.point(Option(a)))! ! !! def flatMap[A,B](fa: OptionHelper[F,A])(f: A => OptionHelper[F,B]):! ! ! ! OptionHelper[F,B] = fa.flatMap(f)! }!

Slide 29

Slide 29 text

! wtf?!

Slide 30

Slide 30 text

! ! FlatMapable[({ type l[a] = OptionHelper[F,a]})#l]! ! ! All we are doing is declaring ‘on the fly’ a type alias called L that takes a single parameter (a), that is equal to the OptionHelper[F,a]. The F comes from the class declaration.

Slide 31

Slide 31 text

//NOW WE HAVE BOTH! ! ! def runPipelineOptionHelper(): Unit = {! val oh: OptionFutureHelper[Int] = ! ! ! new OptionHelper[Future,Int](Future { Some(1) })! pipeline(oh) ! }! ! // We get BOTH parallelism and we short circuit on any None value. !

Slide 32

Slide 32 text

Ok that is super cool but that pipeline method is so ugly. !

Slide 33

Slide 33 text

class FlatMapableOps[F[_],A](fa: F[A])(implicit M: FlatMapable[F]) {! def flatMap[B](f: A => F[B]): F[B] = M.flatMap(fa)(f)! def map[B](f: A => B): F[B] = M.map(fa)(f)! }! ! implicit def toFlatMapableOps[F[_],A](fa: F[A])! ! (implicit M: ! FlatMapable[F]) = new FlatMapableOps(fa)! ! ! //remember, anything with flatMap and Map methods get nice syntactic sugar from scala ala for comprehension. !

Slide 34

Slide 34 text

//Ugly version one more time! ! def pipeline[F[_]](fa: F[Int])(implicit M: FlatMapable[F]): F[String] = {! val newm = M.point(unsafeApi(2))! M.flatMap(fa)(i => ! M.flatMap(M.point(unsafeApi(i)))(s => ! M.flatMap(M.point(unsafeApi2(s+"blah")))(s2 => ! M.map(newm)(s3 => s3 + s2)! )! )! )! }!

Slide 35

Slide 35 text

//Rewritten pipeline method:! ! def pipeline2[F[_]](fa: F[Int])(implicit M: FlatMapable[F]): F[String] = {! val nm = M.point(unsafeApi(2))! for {! i <- fa! s <- M.point(unsafeApi(i))! s2 <- M.point(unsafeApi2(s+"blah"))! s3 <- nm! } yield! s3 + s2 ! }!  

Slide 36

Slide 36 text

We are writing code that abstracts away everything but the necessary methods. We still exploit the benefit of whatever F is passed in. We can nest multiple F’s to get multiple benefits just as we did with our OptionHelper with a Future.

Slide 37

Slide 37 text

Awesome, but do I have to bootstrap this kind of code every time?

Slide 38

Slide 38 text

No, someone did this for us! https://github.com/scalaz/scalaz    Except, Mapable is called Functor, and FlatMapable is called…

Slide 39

Slide 39 text

!!!!!!!!!!. . .!

Slide 40

Slide 40 text

Monad! Along with defining the two aforementioned type classes, we get a plethora of instances of them for free too. Future, Option, List, Stream, Tuple, Either,\/ (right biased Either), Nel. ! ! OptionHelper is already written too, it is called OptionT (because it is the Option Monad Transformer). ! ! Mapable (Functor) and FlatMapable (Monad) are not the only ways to abstract over things!    

Slide 41

Slide 41 text

Let’s write a pipeline method that’s using the Scalaz type classes.

Slide 42

Slide 42 text

//Exactly the same but with different type class names! def pipeline3[F[_]](fa: F[Int])(implicit M: Monad[F]): F[String] = {! val nm = M.point(unsafeApi(2))! for {! i <- fa! s <- M.point(unsafeApi(i))! s2 <- M.point(unsafeApi2(s+"blah"))! s3 <- nm! } yield! s3 + s2 ! }!

Slide 43

Slide 43 text

! //need the type lambda to help out scalac! def doPipelineScalaz(i: Int): OptionT[Future,String] = {! val ot= OptionT( Future { 1.some } )! pipeline3[({ type l[a] = OptionT[Future,a]})#l](ot)! }!

Slide 44

Slide 44 text

Ok cool but what else can we do?

Slide 45

Slide 45 text

//\/.fromTryCatchNonFatal just wraps an expression in \/[Exception, A]! ! def tryUnsafeApi(i: Int): \/[String, Int] = ! ! \/.fromTryCatchNonFatal( i + 2).leftMap(ex => ex.toString) ! ! def tryUnsafeApi2(s: String): \/[String, String] = ! \/.fromTryCatchNonFatal(s + "asdf" ).leftMap(ex => ex.toString)!

Slide 46

Slide 46 text

def pipeline4[F[_]](fa: EitherT[F, String, Int])! ! ! ! ! ! ! ! (implicit M: Monad[F]): EitherT[F, String, String] = {! val nm = EitherT(M.point(tryUnsafeApi(2)))! for {! i <- fa! s <- EitherT(M.point(tryUnsafeApi(i)))! s2 <- EitherT(M.point(tryUnsafeApi2(s + "blah")))! s3 <- nm! } yield ! s3 + s2! }! ! /*We can now short circuit getting an error message, and abstract over whatever F is (Future for parallelism for instance).*/!

Slide 47

Slide 47 text

Can we run multiple things through a pipeline, then only sequence one it’s all done?

Slide 48

Slide 48 text

//Run every element in the list through the pipeline, separate right vs. left! ! def transformPipeline[F[_], A](l: List[Int], f: EitherT[F, String, Int] => EitherT[F, String, A])! ! (implicit M: Monad[F]): F[(List[String], List[A])] = ???!

Slide 49

Slide 49 text

def separateSequence[F[_], A, B](list: List[EitherT[F,A,B]]) (implicit M: Monad[F]): F[(List[A],List[B])] = {! list.foldRight(M.point((List[A](), List[B]())))( (et, ftup) => {! for {! e <- et.run! t <- ftup! } yield e.fold(a => (a :: t._1,t._2), b => (t._1, b :: t._2)) ! })! }!

Slide 50

Slide 50 text

! def pipelineList[F[_]](l: List[Int])! ! (implicit M: Monad[F]): F[(List[String], List[String])] = {! ! //List[EitherT[Future,String,int]] ! ! val eithers = l.map(i => ! ! ! Monad[({ type l[a] = EitherT[F, String, a]})#l].point(i)) ! val pipelineResults = eithers.map(et => pipeline4(et))! separateSequence(pipelineResults)! }!

Slide 51

Slide 51 text

This is just the beginning. We could actually abstract away the List to a Foldable type class so it wouldn’t matter if we had a List, Stream, DList, NonEmptyList, IList, FingerTree, etc

Slide 52

Slide 52 text

Any time you’re forced to sequence expressions with a flatMap, you need ordering. If not, there is potential for concurrency. The more you abstract functions, often times the simpler it is to spot potential for concurrency.

Slide 53

Slide 53 text

Thanks to:    @arosien! @dainkaplan! @dibblego! @puffnfresh! @tpolecat! @apocalisp! @s11001001! ! and everyone else in #scalaz on Freenode! ! (come say hi!)!