Using scala Futures the functional way

Get more code reuse from your Futures and add concurrency to your programs in a functional way. Hint: a word that beings with 'M'...

Transcript

1. Using Futures the functional way
 
 OC Scala meetup 2015
2. Who am I? 
 ! Vincent Marquez" ! Twitter: @runT1ME" ! Github: /vmarquez
3. I write code for 
 ! www.JazelAuto.com 
 (automotive advertising platform) 
 ! www.10thdegree.com
 (digital marketing agency)
 
 We consume and process a lot of data. 
 Necessitates concurrency, specifically Futures.
4. We need a flexible way to sprinkle in concurrency that doesn’t affect our program logic. We need to utilize concurrency(Futures) with all sorts of different data structures.
5. ! def futureBasics(): Unit = {! val f1 = Future { 1 + 2 } //happening asynchronously! val f2 = Future { 3 + 5 } ! ! val f3 = for {! a <- f1! b <- f2! } yield ! a + b! ! //we only want to Await at the end of the world ! val result = Await.result(f3, Duration(5, SECONDS))! ! println(result)! }!
6. ! //What if we want parallelism but for a function returning option?! def doStuff[A](a1: A, a2: A)(f: A => Option[Int]): Future[Option[Int]] = {! val fi1 = Future { f(a1) }! val fi2 = Future { f(a2) }! for {! o1 <- fi1 //call to map! o2 <- fi2! } yield! for {! i1 <- o1! i2 <- o2! } yield! i1 + i2! }!
7. This is very common, we need to work with Futures over some other context (Option/Either/Kleisli(Reader) etc.) To solve that problem we need a detour on abstraction…
8. //Can we generalize this?! ! val o1 = Some(2) // or 1.some! ! val l1 = List(1,2,3)! ! val f1 = Future { 1+6 }! ! ! val r1 = o1.map(i => i*i)! ! val r2 = l1.map(i => i*i)! ! val r3 = f1.map(i => i*i)!
9. //can we abstract over that idea? Subclass polymorphism?! def squareInContext(s: Seq[Int]): Seq[Int] = ! s.map(i => i*i)! ! //No, not quite right. ! ! def squareInContextF[A <: Seq[Int]](s: A)! ! (implicit ev: Seq[Int] =:= A): A = ! ! s.map(i => i*i)! ! //Ugly, and doesn’t work with things that don’t extend Seq!
10. What if we could abstract over anything that had a ‘map like’! method, as long as we provided evidence that the type could be ‘mapped’?
11. We would call it…!
12. //F[_] is called a higher kinded type! trait Mapable[F[_]] {! def map[A,B](fa: F[A])(mf: A => B): F[B]! }! ! implicit def listMapable: Mapable[List] = new Mapable[List] {! def map[A,B](fa: List[A])(mf: A => B): List[B] = fa.map(mf)! }! ! implicit def futureMapable: Mapable[Future] = new Mapable[Future] {! def map[A,B](fa: Future[A])(mf: A => B): Future[B] = fa.map(mf)! }!
13. What about for something that doesn’t have a map method?
14. //Function0[A] is the type of () => A! implicit def functionMappable: Mapable[Function0] = ! ! new Mapable[Function0] {! def map[A,B](f: Function0[A])(mf: A => B): Function0[B] = ! ! ! ! () => mf(f())! } !
15. ! ! def squareInCtxtT[F[_]](s: F[Int])(implicit M: Mapable[F]): F[Int] = ! M.map(s)(i => i*i)!
16. ! ! What do we gain? Considering we can call a method in the map, not much. Yet…
17. ! trait FlatMapable[F[_]] extends Mapable[F] {! ! //notice => A, not A. ! def point[A](a: => A): F[A]! ! def flatMap[A,B](fa: F[A])(fm: A => F[B]): F[B]! ! //we can derive map from the previous two methods! ! override def map[A,B](fa: F[A])(m: A => B): F[B] = ! ! ! flatMap(fa)(a => point(m(a)))! }! !
18. implicit def optionFlatMapable: FlatMapable[Option] = ! ! new FlatMapable[Option] {! def point[A](a: => A): Option[A] = Option(a)! ! ! def flatMap[A,B](fa: Option[A])(f: A => Option[B]): Option[B] = ! ! ! fa.flatMap(a => ! f(a)) ! }! ! implicit def futureFlatMapable: FlatMapable[Future] = ! ! new FlatMapable[Future] {! !def point[A](a: => A): Future[A] = Future { a }! ! ! ! def flatMap[A,B](fa: Future[A])(f: A => Future[B]): Future[B] = ! ! ! ! fa.flatMap(a => f(a))! }! !
19. ! object UnsafeJavaWebApi { ! def unsafeApi(i: Int): String = {! if (i % 2 == 0)! null! else {! //long computation might occur here! i+5.toString! }! }! ! def unsafeApi2(s: String): String = ! try {! (s.toInt+1).toString! } catch {! case ex: Exception => null ! }! }!
20. // Let’s make use of our flatMapable trait. We’ll create a pipeline ! def pipeline[F[_]](fa: F[Int])(implicit M: FlatMapable[F]): F[String] = {! val newm = M.point(unsafeApi(2))! M.flatMap(fa)(i => ! M.flatMap(M.point(unsafeApi(i)))(s => ! M.flatMap(M.point(unsafeApi2(s+"blah")))(s2 => ! M.map(newm)(s3 => s3 + s2)! )! )! )! }!
21. ! So why all this trouble? Why abstract over the flatMapable? I thought this was a talk on Futures?!
22. //we get short circuiting. if null is passed to point, we return none! def doPipeLine(i: Int): Option[String] = ! pipeline(Option(i))! ! ! // we get parallelism (because of the call to point in the pipeline)! def doPipelineF(i: Int): Future[String] = ! pipeline(Future { i })!
23. ! OK that’s cool, but is it really worth the trouble?!
24. ! What if we want both short circuiting and parallelism? We could make the method take an F[G[A]] and require FlatMapable’s for both kind parameters, but it’s not scalable. Remember the nested ‘fors’?
25. ! ! class OptionHelper[F[_],A](val foa: F[Option[A]]) {! ! ! def flatMap[B](f: A => OptionHelper[F,B])! ! ! ! ! ! ! (implicit M:FlatMapable[F]): OptionHelper[F,B] = ! ! ! ! ! ! new OptionHelper[F,B](M.flatMap(foa)(o => o match {! case Some(a) => f(a).foa! case None => M.point(None: Option[B])! }))! }! ! ! /*! we are able to access what’s inside the F[Option[A]] no matter what the F is now, as long as there exists an implicit FlatMapable for F!! */ !
26. ! Ok now we want a ‘FlatMapable’ trait for OptionHelper. But there’s a problem. The type parameter for FlatMapable needs to have a single kind parameter. 
 
 The types for OptionHelper are [_],A. [_],A doesn’t fit in something that takes just an [_].! !
27. ! //Option 1: Type Aliasing. (The one your coworkers will prefer).??! ! type OptionFutureHelper[A] = OptionHelper[Future,A]! ! def OptionHelperFutureFlatMapable(implicit M: FlatMapable[Future]): ! ! FlatMapable[OptionFutureHelper] = ! ! new FlatMapable[OptionFutureHelper] {! def point[A](a: => A): OptionFutureHelper[A] =! ! ! new OptionHelper(M.point(Option(a)))! ! def flatMap[A,B](fa: OptionFutureHelper[A])! ! ! (f: A => OptionFutureHelper[B]): OptionFutureHelper[B] =! ! ! ! fa.flatMap(f)(M)! ! }!
28. ! //Option 2: no type alias necessary…but uglier and you will have to explain this to all your new team members.! ! implicit def OptionHelperFlatMapable[F[_]](implicit M: FlatMapable[F]): ! ! FlatMapable[({ type l[a] = OptionHelper[F,a]})#l] = ! ! new FlatMapable[({ type l[a] = OptionHelper[F,a]})#l] { ! ! ! def point[A](a: => A): OptionHelper[F,A] = ! ! ! ! new OptionHelper(M.point(Option(a)))! ! !! def flatMap[A,B](fa: OptionHelper[F,A])(f: A => OptionHelper[F,B]):! ! ! ! OptionHelper[F,B] = fa.flatMap(f)! }!
29. ! wtf?!
30. ! ! FlatMapable[({ type l[a] = OptionHelper[F,a]})#l]! ! ! All we are doing is declaring ‘on the fly’ a type alias called L that takes a single parameter (a), that is equal to the OptionHelper[F,a]. The F comes from the class declaration.
31. //NOW WE HAVE BOTH! ! ! def runPipelineOptionHelper(): Unit = {! val oh: OptionFutureHelper[Int] = ! ! ! new OptionHelper[Future,Int](Future { Some(1) })! pipeline(oh) ! }! ! // We get BOTH parallelism and we short circuit on any None value. !
32. Ok that is super cool but that pipeline method is so ugly. !
33. class FlatMapableOps[F[_],A](fa: F[A])(implicit M: FlatMapable[F]) {! def flatMap[B](f: A => F[B]): F[B] = M.flatMap(fa)(f)! def map[B](f: A => B): F[B] = M.map(fa)(f)! }! ! implicit def toFlatMapableOps[F[_],A](fa: F[A])! ! (implicit M: ! FlatMapable[F]) = new FlatMapableOps(fa)! ! ! //remember, anything with flatMap and Map methods get nice syntactic sugar from scala ala for comprehension. !
34. //Ugly version one more time! ! def pipeline[F[_]](fa: F[Int])(implicit M: FlatMapable[F]): F[String] = {! val newm = M.point(unsafeApi(2))! M.flatMap(fa)(i => ! M.flatMap(M.point(unsafeApi(i)))(s => ! M.flatMap(M.point(unsafeApi2(s+"blah")))(s2 => ! M.map(newm)(s3 => s3 + s2)! )! )! )! }!
35. //Rewritten pipeline method:! ! def pipeline2[F[_]](fa: F[Int])(implicit M: FlatMapable[F]): F[String] = {! val nm = M.point(unsafeApi(2))! for {! i <- fa! s <- M.point(unsafeApi(i))! s2 <- M.point(unsafeApi2(s+"blah"))! s3 <- nm! } yield! s3 + s2 ! }! 

36. We are writing code that abstracts away everything but the necessary methods. We still exploit the benefit of whatever F is passed in. We can nest multiple F’s to get multiple benefits just as we did with our OptionHelper with a Future.
37. Awesome, but do I have to bootstrap this kind of code every time?
38. No, someone did this for us! https://github.com/scalaz/scalaz
 
 Except, Mapable is called Functor, and FlatMapable is called…
39. !!!!!!!!!!. . .!
40. Monad! Along with defining the two aforementioned type classes, we get a plethora of instances of them for free too. Future, Option, List, Stream, Tuple, Either,\/ (right biased Either), Nel. ! ! OptionHelper is already written too, it is called OptionT (because it is the Option Monad Transformer). ! ! Mapable (Functor) and FlatMapable (Monad) are not the only ways to abstract over things! 
 

41. Let’s write a pipeline method that’s using the Scalaz type classes.
42. //Exactly the same but with different type class names! def pipeline3[F[_]](fa: F[Int])(implicit M: Monad[F]): F[String] = {! val nm = M.point(unsafeApi(2))! for {! i <- fa! s <- M.point(unsafeApi(i))! s2 <- M.point(unsafeApi2(s+"blah"))! s3 <- nm! } yield! s3 + s2 ! }!
43. ! //need the type lambda to help out scalac! def doPipelineScalaz(i: Int): OptionT[Future,String] = {! val ot= OptionT( Future { 1.some } )! pipeline3[({ type l[a] = OptionT[Future,a]})#l](ot)! }!
44. Ok cool but what else can we do?
45. //\/.fromTryCatchNonFatal just wraps an expression in \/[Exception, A]! ! def tryUnsafeApi(i: Int): \/[String, Int] = ! ! \/.fromTryCatchNonFatal( i + 2).leftMap(ex => ex.toString) ! ! def tryUnsafeApi2(s: String): \/[String, String] = ! \/.fromTryCatchNonFatal(s + "asdf" ).leftMap(ex => ex.toString)!
46. def pipeline4[F[_]](fa: EitherT[F, String, Int])! ! ! ! ! ! ! ! (implicit M: Monad[F]): EitherT[F, String, String] = {! val nm = EitherT(M.point(tryUnsafeApi(2)))! for {! i <- fa! s <- EitherT(M.point(tryUnsafeApi(i)))! s2 <- EitherT(M.point(tryUnsafeApi2(s + "blah")))! s3 <- nm! } yield ! s3 + s2! }! ! /*We can now short circuit getting an error message, and abstract over whatever F is (Future for parallelism for instance).*/!
47. Can we run multiple things through a pipeline, then only sequence one it’s all done?
48. //Run every element in the list through the pipeline, separate right vs. left! ! def transformPipeline[F[_], A](l: List[Int], f: EitherT[F, String, Int] => EitherT[F, String, A])! ! (implicit M: Monad[F]): F[(List[String], List[A])] = ???!
49. def separateSequence[F[_], A, B](list: List[EitherT[F,A,B]]) (implicit M: Monad[F]): F[(List[A],List[B])] = {! list.foldRight(M.point((List[A](), List[B]())))( (et, ftup) => {! for {! e <- et.run! t <- ftup! } yield e.fold(a => (a :: t._1,t._2), b => (t._1, b :: t._2)) ! })! }!
50. ! def pipelineList[F[_]](l: List[Int])! ! (implicit M: Monad[F]): F[(List[String], List[String])] = {! ! //List[EitherT[Future,String,int]] ! ! val eithers = l.map(i => ! ! ! Monad[({ type l[a] = EitherT[F, String, a]})#l].point(i)) ! val pipelineResults = eithers.map(et => pipeline4(et))! separateSequence(pipelineResults)! }!
51. This is just the beginning. We could actually abstract away the List to a Foldable type class so it wouldn’t matter if we had a List, Stream, DList, NonEmptyList, IList, FingerTree, etc
52. Any time you’re forced to sequence expressions with a flatMap, you need ordering. If not, there is potential for concurrency. The more you abstract functions, often times the simpler it is to spot potential for concurrency.
53. Thanks to:
 
 @arosien! @dainkaplan! @dibblego! @puffnfresh! @tpolecat! @apocalisp! @s11001001! ! and everyone else in #scalaz on Freenode! ! (come say hi!)!