Boring Usecases for Exciting Types

Functional programming, in Scala and in general, is very jargon-y. Words and co-Words such as indexed state transformers, codensity monads and divisible functors are impressive, but without making effort to place them in our day-to-day context, they remain an academic interest. That's a shame, though - these data types and type classes can really make our life easier!

In this talk, I will take 3 plain and business-y use cases and show how they can benefit from some lesser known constructs from functional programming. Boilerplate will be slain, elegance will ensue and hopefully - you will find them useful enough to incorporate in your day-to-day work!

Transcript

1. Boring Usecases for Exciting Types Itamar Ravid - Lambda World Cádiz, 2018 Itamar Ravid - @itrvd 1
2. About me • Freelance software developer • Doing functional programming, microservices, DevOps • Co-organize the Underscore Scala meetup • Hit me up if you want to work together! Itamar Ravid - @itrvd 2
3. Agenda • Why I wrote this talk • Three "boring" use-cases and types that can help Itamar Ravid - @itrvd 3
4. Itamar Ravid - @itrvd 4
5. Itamar Ravid - @itrvd 5
6. Itamar Ravid - @itrvd 6
7. Itamar Ravid - @itrvd 7
8. How do we grow as functional programmers Itamar Ravid - @itrvd 8
9. Classic "draw an owl" problem How do we go from this: quicksort [] = [] quicksort (p:xs) = (quicksort lesser) ++ [p] ++ (quicksort greater) where lesser = filter (< p) xs greater = filter ( >= p) xs Itamar Ravid - @itrvd 9
10. Classic "draw an owl" problem To this: -- | Fokkinga's postpromorphism postpro :: Recursive t => (forall b. Base t b -> Base t b) -- natural transformation -> (a -> Base t a) -- a (Base t)-coalgebra -> a -- seed -> t postpro e g = a where a = embed . fmap (hoist e . a) . g Itamar Ravid - @itrvd 10
11. Examples Itamar Ravid - @itrvd 11
12. Examples We are sorely lacking! Itamar Ravid - @itrvd 12
13. Actual agenda Purely functional and type-safe: • Time measurement and instrumentation • State machines • Resource allocation and release • Let's go! Itamar Ravid - @itrvd 13
14. Examples ahead assume: // build.sbt scalacOptions += "-Ypartial-unification" addCompilerPlugin("com.olegpy" %% "better-monadic-for" % "0.2.4") // *.scala import cats._, cats.data._, cats.implicits._ import cats.effect._ Itamar Ravid - @itrvd 14
15. Time measurement and instrumentation Let's start with a warm-up! Problem: I have a bunch of computations, and I want to measure their execution time. def computeCv(data: Data): IO[Double] = for { sample <- computeSample(data) mean <- computeMean(sample) variance <- computeVariance(sample) } yield mean / variance Itamar Ravid - @itrvd 15
16. Time measurement and instrumentation Non-solution: manually instrument. def computeCv(data: Data): IO[Double] = for { startTime <- time() sample <- computeSample(data) sampleTime <- time() mean <- computeMean(sample) meanTime <- time() variance <- computeVariance(sample) varTime <- time() } yield mean / variance Itamar Ravid - @itrvd 16
17. Itamar Ravid - @itrvd 17
18. Time measurement and instrumentation To start, we define a timed combinator: def timed[A](fa: IO[A]): IO[(FiniteDuration, A)] = for { startTime <- time() a <- fa endTime <- time() } yield ((endTime - startTime).millis, a) fa is pure, so we can just pass it around. Itamar Ravid - @itrvd 18
19. Time measurement and instrumentation We can also name this computation: def timed[A](fa: IO[A], name: String): IO[((String, FiniteDuration), A)] = for { startTime <- time() a <- fa endTime <- time() } yield (name -> (endTime - startTime).millis, a) Itamar Ravid - @itrvd 19
20. Time measurement and instrumentation We can now use our new timed function: def computeCv(data: Data): IO[Double] = for { (sampleTime, sample) <- timed(computeSample(data), "sample") (meanTime, mean) <- timed(computeMean(sample), "mean") (varTime, variance) <- timed(computeVariance(sample), "var") } yield mean / variance Kinda tedious though! Itamar Ravid - @itrvd 20
21. Time measurement and instrumentation To save us from that bookeeping, comes WriterT: case class WriterT[F[_], Log, Value]( run: F[(Log, Value)] ) Itamar Ravid - @itrvd 21
22. Time measurement and instrumentation To save us from that bookeeping, comes WriterT: case class WriterT[F[_], Log, Value]( run: F[(Log, Value)] ) To be honest, WriterT is nothing more than a tuple. Itamar Ravid - @itrvd 22
23. Tuples are magical! We can map them: type Metadata = List[String] type Annotated[A] = (Metadata, A) case class Person(name: String, age: Int) val person: Annotated[Person] = (List(), Person("Bob", 23)) val name: Annotated[String] = person.map(_.name) Itamar Ravid - @itrvd 23
24. Tuples are magical! We can traverse them: def persistPerson(p: Person): IO[Int] val persistedId: IO[Annotated[Int]] = person.traverse(persistPerson) Itamar Ravid - @itrvd 24
25. Tuples are magical! And if we have a Semigroup for the Metadata, we can flatMap too! val person: Annotated[Person] = (List("log1"), Person("Bob", 23)) val flatMapped: Annotated[Person] = for { p <- person newP <- (List("log2"), p.copy(age = 24)) } yield newP Itamar Ravid - @itrvd 25
26. Tuples are magical! And if we have a Semigroup for the Metadata, we can flatMap too! val person: Annotated[Person] = (List("log1"), Person("Bob", 23)) val flatMapped: Annotated[Person] = for { p <- person newP <- (List("log2"), p.copy(age = 24)) } yield newP flatMapped = (List("log1", "log2"), Person("Bob", 24)) Itamar Ravid - @itrvd 26
27. Back to WriterT WriterT is identical, apart from the additional effect: case class WriterT[F[_], Log, Value]( run: F[(Log, Value)] ) Itamar Ravid - @itrvd 27
28. Time measurement and instrumentation Here's the type of our timed program: timedProgram: IO[((String, FiniteDuration), A)] Itamar Ravid - @itrvd 28
29. Time measurement and instrumentation Here's the type of our timed program: timedProgram: IO[((String, FiniteDuration), A)] Adapted to WriterT, this is: timedProgram: WriterT[IO, (String, FiniteDuration), A] Itamar Ravid - @itrvd 29
30. Time measurement and instrumentation Here's the type of our timed program: timedProgram: IO[((String, FiniteDuration), A)] Adapted to WriterT, this is: timedProgram: WriterT[IO, (String, FiniteDuration), A] This won't (sensibly) work! Itamar Ravid - @itrvd 30
31. Time measurement and instrumentation Let's just use a map. type Timed[A] = WriterT[IO, Map[String, FiniteDuration], A] def timed(name: String, fa: IO[A]): Timed[A] = WriterT { for { startTime <- time() a <- fa endTime <- time() } yield (Map(name -> (endTime - startTime).millis), a) } Itamar Ravid - @itrvd 31
32. Time measurement and instrumentation Our original program is only slightly changed: def computeCv(data: Data): Timed[Double] = for { sample <- timed(computeSample(data), "sample") mean <- timed(computeMean(sample), "mean") variance <- timed(computeVariance(sample), "var") } yield mean / variance Itamar Ravid - @itrvd 32
33. Time measurement and instrumentation If you fancy infix syntax, that works too (implicit class): def computeCv(data: Data): Timed[Double] = for { sample <- computeSample(data).timed("sample") mean <- computeMean(sample).timed("mean") variance <- computeVariance(sample).timed("var") } yield mean / variance Itamar Ravid - @itrvd 33
34. Time measurement and instrumentation To unpack the timing data, we can define another function: implicit class TimedIOOps[A](fa: Timed[A]) { def logTimings: IO[A] = for { (timing, a) <- timed.run _ <- IO(println(timing)) } yield a } Itamar Ravid - @itrvd 34
35. Time measurement and instrumentation And use it: def computeCv(data: Data): IO[Double] = (for { sample <- computeSample(data).timed("sample") mean <- computeMean(sample).timed("mean") variance <- computeVariance(sample).timed("var") } yield mean / variance).logTimings Itamar Ravid - @itrvd 35
36. Summary • WriterT is a good example of making auxillary details ambient • Other things you can do with it: • Accumulate events to emit when doing event sourcing; • Accumulate errors when emitting partial results; • and more and more! Itamar Ravid - @itrvd 36
37. Safe state machines Problem: I want to safely model an authentication protocol: case class Connection(socket: Socket) trait Authentication { def connect(): IO[Connection] def handshake(connection: Connection): IO[Unit] = for { _ <- IO(connection.socket.write(HandshakeMagic)) response <- IO(connection.socket.read(HandshakeResponseLen)) _ <- verifyHandshake(response) } yield () def sendCreds(connection: Connection): IO[Unit] def authenticate(connection: Connection): IO[Unit] } Itamar Ravid - @itrvd 37
38. Safe state machines Sure, I can just use the functions: val api: Authentication import api._ val auth: IO[Unit] = for { conn <- connect() _ <- handshake(conn) _ <- authenticate(conn) _ <- sendCreds(conn) } yield () Itamar Ravid - @itrvd 38
39. Safe state machines But there's a bug here! val api: Authentication import api._ val auth: IO[Unit] = for { conn <- connect() _ <- handshake(conn) _ <- authenticate(conn) _ <- sendCreds(conn) } yield () Itamar Ravid - @itrvd 39
40. Safe state machines Static types to the rescue! We will modify Connection: case class Handshake() case class Credentials() case class Auth() case class Done() case class Connection[S](socket: Socket) Itamar Ravid - @itrvd 40
41. Safe state machines And the API: trait Authentication { def connect(): IO[Connection[Handshake]] def handshake(connection: Connection[Handshake]): IO[Connection[Credentials]] def sendCreds(connection: Connection[Credentials]): IO[Connection[Auth]] IO(println("Sending credentials!")) *> IO.pure(connection.copy()) def authenticate(connection: Connection[Auth]): IO[Connection[Done]] } Itamar Ravid - @itrvd 41
42. Safe state machines Now we need to carry even more connections around: val api: Authentication import api._ val done: IO[Connection[Done]] = for { connHandshake <- connect() connCreds <- handshake(connHandshake) connAuth <- sendCreds(connCreds) connDone <- authenticate(connAuth) } yield connDone Safe, but ugly. Itamar Ravid - @itrvd 42
43. Itamar Ravid - @itrvd 43
44. Safe state machines FP saves the day! This is IndexedStateT: case class IndexedStateT[F, Start, End, Result]( run: Start => F[(End, Result)] ) Itamar Ravid - @itrvd 44
45. Safe state machines FP saves the day! This is IndexedStateT: case class IndexedStateT[F, Start, End, Result]( run: Start => F[(End, Result)] ) For example: Start = Connection[Handshake], End = Connection[Credentials] F = IO, Result = Unit Itamar Ravid - @itrvd 45
46. Safe state machines val handshake: Connection[Handshake] => IO[(Connection[Credentials], Unit)] Itamar Ravid - @itrvd 46
47. Safe state machines val handshake: Connection[Handshake] => IO[(Connection[Credentials], Unit)] type Protocol[A, B] = IndexedStateT[ IO, Connection[A], Connection[B], Unit ] val nicerHandshake: Protocol[Handshake, Credentials] Itamar Ravid - @itrvd 47
48. Safe state machines We can modify our API to use it: trait Authentication { def connect: Protocol[Unit, Handshake] def handshake: Protocol[Handshake, Credentials] def sendCreds: Protocol[Credentials, Auth] = IndexedStateT.modifyF { conn => IO(println("Sending credentials!")) *> IO(conn.copy()) } def authenticate: Protocol[Auth, Done] } Itamar Ravid - @itrvd 48
49. Safe state machines And this is how the usage looks like: val api: Authentication import api._ val authenticate: Protocol[Unit, Done] = for { _ <- connect _ <- handshake _ <- sendCreds _ <- authenticate } yield () Itamar Ravid - @itrvd 49
50. Safe state machines The original example, by the way, errors out: val api: Authentication import api._ val authenticate: Protocol[Unit, Done] = for { _ <- connect _ <- handshake _ <- authenticate _ <- sendCreds } yield () Itamar Ravid - @itrvd 50
51. Safe state machines cmd6.sc:6: polymorphic expression cannot be instantiated to expected type; found : [B, SC]IndexedStateT[IO,Connection[Auth],SC,B] required: IndexedStateT[IO,Connection[Credentials],?,?] _ <- authenticate ^ Itamar Ravid - @itrvd 51
52. Summary • IndexedStateT is very cool for type-safe transitions • It is a Monad and an Applicative when SA = SB • For non-linear transitions: shapeless.Coproduct and shapeless.HList • IxStateT in Haskell is in the indexed-extras package Itamar Ravid - @itrvd 52
53. Resource allocation and release Problem: I have many resources allocated in my app. I need to manage their acquisition and release. def createKafkaConsumer: IO[KafkaConsumer] trait KafkaConsumer { def close: IO[Unit] } def createDBConnection: IO[DBConnection] trait DBConnection { def close: IO[Unit] } Itamar Ravid - @itrvd 53
54. Resource allocation and release How should my initialization look like? Itamar Ravid - @itrvd 54
55. Resource allocation and release How should my initialization look like? def runApp(): IO[Unit] = for { consumer <- createKafkaConsumer conn <- createDBConnection _ <- doSomething(consumer, conn) _ <- conn.close _ <- consumer.close } yield () Itamar Ravid - @itrvd 55
56. Resource allocation and release How should my initialization look like? def runApp(): IO[Unit] = for { consumer <- createKafkaConsumer conn <- createDBConnection _ <- doSomething(consumer, conn) _ <- ignoreFailures(conn.close) _ <- ignoreFailures(consumer.close) } yield () Itamar Ravid - @itrvd 56
57. The smell is strong with you! Itamar Ravid - @itrvd 57
58. Resource allocation and release If you know Java, you're probably aware of try-with- resources: try (BufferedReader br = new BufferedReader(new FileReader(path))) { return br.readLine(); } This works for any T <: AutoCloseable. Itamar Ravid - @itrvd 58
59. Resource allocation and release Our example would look like this: try (Consumer consumer = new Consumer()) { try (DBConnection conn = new DBConnection()) { doSomething(consumer, conn); } } Itamar Ravid - @itrvd 59
60. Resource allocation and release These are not the abstractions you're looking for! Itamar Ravid - @itrvd 60
61. Resource allocation and release IO monads offer us the bracket operation: object IO { def bracket[R, A](acquire: IO[R]) (release: R => IO[Unit]) (use: R => IO[A]): IO[A] } Itamar Ravid - @itrvd 61
62. Resource allocation and release But this gets us to the same place: def createConsumer: IO[Consumer] def createDBConnection: IO[DBConnection] bracket(createConsumer)(_.close) { consumer => // use consumer } Itamar Ravid - @itrvd 62
63. Resource allocation and release But this gets us to the same place: def createConsumer: IO[Consumer] def createDBConnection: IO[DBConnection] bracket(createConsumer)(_.close) { consumer => bracket(createDBConnection)(_.close) { conn => // use both } } Itamar Ravid - @itrvd 63
64. Resource allocation and release There's an interesting data type called Codensity in Scalaz: abstract class Codensity[F[_], A] { def apply[B](f: A => F[B]): F[B] } Itamar Ravid - @itrvd 64
65. Resource allocation and release There's an interesting data type called Codensity in Scalaz: abstract class Codensity[F[_], A] { def apply[B](f: A => F[B]): F[B] } When we set F = IO, A = R, B = A, apply looks like this: def apply(f: R => IO[A]): IO[A] Itamar Ravid - @itrvd 65
66. Resource allocation and release This is remarkably similar to part of bracket! def bracket[R, A](acquire: IO[R]) (release: R => IO[Unit]) (use: R => IO[A]): IO[A] def apply (f: R => IO[A]): IO[A] Remember: type signatures are never a coincidence ;-) Itamar Ravid - @itrvd 66
67. Resource allocation and release Let's write a combinator using Codensity: def resource[R](acquire: IO[R])(release: R => IO[Unit]) = new Codensity[IO, R] { def apply[A](use: R => IO[A]): IO[A] = IO.bracket(acquire)(release)(use) } Itamar Ravid - @itrvd 67
68. Resource allocation and release So now, when we write a resource, we get back a value: val consumer: Codensity[IO, Consumer] = resource(createKafkaConsumer)(_.close) Itamar Ravid - @itrvd 68
69. Resource allocation and release So now, when we write a resource, we get back a value: val consumer: Codensity[IO, Consumer] = resource(createKafkaConsumer)(_.close) And we can use it: val things: IO[List[Thing]] = consumer { c => c.consumeSomeThings } Itamar Ravid - @itrvd 69
70. Composition of resources, using Codensity Itamar Ravid - @itrvd 70
71. Resource allocation and release It just so happens that Codensity is a Monad. So I can do this: val resources: Codensity[IO, (Consumer, DBConnection)] = for { consumer <- resource(createKafkaConsumer)(_.close) dbConn <- resource(createDBConnection)(_.close) } yield (consumer, dbConn) What's happening here though? Itamar Ravid - @itrvd 71
72. Resource allocation and release Codensity's flatMap: class Codensity[F[_], A] { self => def flatMap[B](f: A => Codensity[F, B]): Codensity[F, B] = new Codensity[F, B] { def apply[R](use: B => F[R]): F[R] = self.apply { a => f(a).apply { b => use(b) } } } } Itamar Ravid - @itrvd 72
73. Resource allocation and release If we substitue the bracketing in: self.apply { a => f(a).apply { b => use(b) } } Itamar Ravid - @itrvd 73
74. Resource allocation and release If we substitue the bracketing in: IO.bracket(acquireA)(releaseA) { a => IO.bracket(acquireB)(releaseB) { b => use(b) } } Itamar Ravid - @itrvd 74
75. Resource allocation and release Finally, I can do something with these! val something: IO[Something] = resources { case (consumer, conn) => doSomething(consumer, dbConn) } Finalizers will run in reverse order of acquisition. Itamar Ravid - @itrvd 75
76. Itamar Ravid - @itrvd 76
77. Itamar Ravid - @itrvd 77
78. Itamar Ravid - @itrvd 78
79. Itamar Ravid - @itrvd 79
80. Itamar Ravid - @itrvd 80
81. Itamar Ravid - @itrvd 81
82. Resource allocation and release • Codensity is 18 lines of code in total • cats-effect includes Resource[F, R] • scalaz-zio includes Managed[E, R] • In Haskell, you can use ResourceT or Managed Itamar Ravid - @itrvd 82
83. Summary • Don't be afraid of the exotic data types! • Be on the lookout for examples Itamar Ravid - @itrvd 83