Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Boring Usecases for Exciting Types

Itamar Ravid
October 26, 2018

Boring Usecases for Exciting Types

Functional programming, in Scala and in general, is very jargon-y. Words and co-Words such as indexed state transformers, codensity monads and divisible functors are impressive, but without making effort to place them in our day-to-day context, they remain an academic interest. That's a shame, though - these data types and type classes can really make our life easier!

In this talk, I will take 3 plain and business-y use cases and show how they can benefit from some lesser known constructs from functional programming. Boilerplate will be slain, elegance will ensue and hopefully - you will find them useful enough to incorporate in your day-to-day work!

Itamar Ravid

October 26, 2018
Tweet

More Decks by Itamar Ravid

Other Decks in Programming

Transcript

  1. Boring Usecases
    for Exciting Types
    Itamar Ravid - Lambda World Cádiz, 2018
    Itamar Ravid - @itrvd 1

    View full-size slide

  2. About me
    • Freelance software developer
    • Doing functional programming, microservices, DevOps
    • Co-organize the Underscore Scala meetup
    • Hit me up if you want to work together!
    Itamar Ravid - @itrvd 2

    View full-size slide

  3. Agenda
    • Why I wrote this talk
    • Three "boring" use-cases and types that can help
    Itamar Ravid - @itrvd 3

    View full-size slide

  4. Itamar Ravid - @itrvd 4

    View full-size slide

  5. Itamar Ravid - @itrvd 5

    View full-size slide

  6. Itamar Ravid - @itrvd 6

    View full-size slide

  7. Itamar Ravid - @itrvd 7

    View full-size slide

  8. How do we grow
    as functional programmers
    Itamar Ravid - @itrvd 8

    View full-size slide

  9. Classic "draw an owl" problem
    How do we go from this:
    quicksort [] = []
    quicksort (p:xs) = (quicksort lesser) ++ [p] ++ (quicksort greater)
    where
    lesser = filter (< p) xs
    greater = filter ( >= p) xs
    Itamar Ravid - @itrvd 9

    View full-size slide

  10. Classic "draw an owl" problem
    To this:
    -- | Fokkinga's postpromorphism
    postpro
    :: Recursive t
    => (forall b. Base t b -> Base t b) -- natural transformation
    -> (a -> Base t a) -- a (Base t)-coalgebra
    -> a -- seed
    -> t
    postpro e g = a where a = embed . fmap (hoist e . a) . g
    Itamar Ravid - @itrvd 10

    View full-size slide

  11. Examples
    Itamar Ravid - @itrvd 11

    View full-size slide

  12. Examples
    We are sorely lacking!
    Itamar Ravid - @itrvd 12

    View full-size slide

  13. Actual agenda
    Purely functional and type-safe:
    • Time measurement and instrumentation
    • State machines
    • Resource allocation and release
    • Let's go!
    Itamar Ravid - @itrvd 13

    View full-size slide

  14. Examples ahead assume:
    // build.sbt
    scalacOptions += "-Ypartial-unification"
    addCompilerPlugin("com.olegpy" %% "better-monadic-for" % "0.2.4")
    // *.scala
    import cats._, cats.data._, cats.implicits._
    import cats.effect._
    Itamar Ravid - @itrvd 14

    View full-size slide

  15. Time measurement and instrumentation
    Let's start with a warm-up!
    Problem: I have a bunch of computations, and I want to measure
    their execution time.
    def computeCv(data: Data): IO[Double] =
    for {
    sample <- computeSample(data)
    mean <- computeMean(sample)
    variance <- computeVariance(sample)
    } yield mean / variance
    Itamar Ravid - @itrvd 15

    View full-size slide

  16. Time measurement and instrumentation
    Non-solution: manually instrument.
    def computeCv(data: Data): IO[Double] =
    for {
    startTime <- time()
    sample <- computeSample(data)
    sampleTime <- time()
    mean <- computeMean(sample)
    meanTime <- time()
    variance <- computeVariance(sample)
    varTime <- time()
    } yield mean / variance
    Itamar Ravid - @itrvd 16

    View full-size slide

  17. Itamar Ravid - @itrvd 17

    View full-size slide

  18. Time measurement and instrumentation
    To start, we define a timed combinator:
    def timed[A](fa: IO[A]): IO[(FiniteDuration, A)] =
    for {
    startTime <- time()
    a <- fa
    endTime <- time()
    } yield ((endTime - startTime).millis, a)
    fa is pure, so we can just pass it around.
    Itamar Ravid - @itrvd 18

    View full-size slide

  19. Time measurement and instrumentation
    We can also name this computation:
    def timed[A](fa: IO[A],
    name: String): IO[((String, FiniteDuration), A)] =
    for {
    startTime <- time()
    a <- fa
    endTime <- time()
    } yield (name -> (endTime - startTime).millis, a)
    Itamar Ravid - @itrvd 19

    View full-size slide

  20. Time measurement and instrumentation
    We can now use our new timed function:
    def computeCv(data: Data): IO[Double] =
    for {
    (sampleTime, sample) <- timed(computeSample(data), "sample")
    (meanTime, mean) <- timed(computeMean(sample), "mean")
    (varTime, variance) <- timed(computeVariance(sample), "var")
    } yield mean / variance
    Kinda tedious though!
    Itamar Ravid - @itrvd 20

    View full-size slide

  21. Time measurement and instrumentation
    To save us from that bookeeping, comes WriterT:
    case class WriterT[F[_], Log, Value](
    run: F[(Log, Value)]
    )
    Itamar Ravid - @itrvd 21

    View full-size slide

  22. Time measurement and instrumentation
    To save us from that bookeeping, comes WriterT:
    case class WriterT[F[_], Log, Value](
    run: F[(Log, Value)]
    )
    To be honest, WriterT is nothing more than a tuple.
    Itamar Ravid - @itrvd 22

    View full-size slide

  23. Tuples are magical!
    We can map them:
    type Metadata = List[String]
    type Annotated[A] = (Metadata, A)
    case class Person(name: String, age: Int)
    val person: Annotated[Person] = (List(), Person("Bob", 23))
    val name: Annotated[String] = person.map(_.name)
    Itamar Ravid - @itrvd 23

    View full-size slide

  24. Tuples are magical!
    We can traverse them:
    def persistPerson(p: Person): IO[Int]
    val persistedId: IO[Annotated[Int]] = person.traverse(persistPerson)
    Itamar Ravid - @itrvd 24

    View full-size slide

  25. Tuples are magical!
    And if we have a Semigroup for the Metadata, we can flatMap
    too!
    val person: Annotated[Person] = (List("log1"), Person("Bob", 23))
    val flatMapped: Annotated[Person] =
    for {
    p <- person
    newP <- (List("log2"), p.copy(age = 24))
    } yield newP
    Itamar Ravid - @itrvd 25

    View full-size slide

  26. Tuples are magical!
    And if we have a Semigroup for the Metadata, we can flatMap
    too!
    val person: Annotated[Person] = (List("log1"), Person("Bob", 23))
    val flatMapped: Annotated[Person] =
    for {
    p <- person
    newP <- (List("log2"), p.copy(age = 24))
    } yield newP
    flatMapped = (List("log1", "log2"), Person("Bob", 24))
    Itamar Ravid - @itrvd 26

    View full-size slide

  27. Back to WriterT
    WriterT is identical, apart from the additional effect:
    case class WriterT[F[_], Log, Value](
    run: F[(Log, Value)]
    )
    Itamar Ravid - @itrvd 27

    View full-size slide

  28. Time measurement and instrumentation
    Here's the type of our timed program:
    timedProgram: IO[((String, FiniteDuration), A)]
    Itamar Ravid - @itrvd 28

    View full-size slide

  29. Time measurement and instrumentation
    Here's the type of our timed program:
    timedProgram: IO[((String, FiniteDuration), A)]
    Adapted to WriterT, this is:
    timedProgram: WriterT[IO, (String, FiniteDuration), A]
    Itamar Ravid - @itrvd 29

    View full-size slide

  30. Time measurement and instrumentation
    Here's the type of our timed program:
    timedProgram: IO[((String, FiniteDuration), A)]
    Adapted to WriterT, this is:
    timedProgram: WriterT[IO, (String, FiniteDuration), A]
    This won't (sensibly) work!
    Itamar Ravid - @itrvd 30

    View full-size slide

  31. Time measurement and instrumentation
    Let's just use a map.
    type Timed[A] = WriterT[IO, Map[String, FiniteDuration], A]
    def timed(name: String, fa: IO[A]): Timed[A] =
    WriterT {
    for {
    startTime <- time()
    a <- fa
    endTime <- time()
    } yield (Map(name -> (endTime - startTime).millis), a)
    }
    Itamar Ravid - @itrvd 31

    View full-size slide

  32. Time measurement and instrumentation
    Our original program is only slightly changed:
    def computeCv(data: Data): Timed[Double] =
    for {
    sample <- timed(computeSample(data), "sample")
    mean <- timed(computeMean(sample), "mean")
    variance <- timed(computeVariance(sample), "var")
    } yield mean / variance
    Itamar Ravid - @itrvd 32

    View full-size slide

  33. Time measurement and instrumentation
    If you fancy infix syntax, that works too (implicit class):
    def computeCv(data: Data): Timed[Double] =
    for {
    sample <- computeSample(data).timed("sample")
    mean <- computeMean(sample).timed("mean")
    variance <- computeVariance(sample).timed("var")
    } yield mean / variance
    Itamar Ravid - @itrvd 33

    View full-size slide

  34. Time measurement and instrumentation
    To unpack the timing data, we can define another function:
    implicit class TimedIOOps[A](fa: Timed[A]) {
    def logTimings: IO[A] =
    for {
    (timing, a) <- timed.run
    _ <- IO(println(timing))
    } yield a
    }
    Itamar Ravid - @itrvd 34

    View full-size slide

  35. Time measurement and instrumentation
    And use it:
    def computeCv(data: Data): IO[Double] =
    (for {
    sample <- computeSample(data).timed("sample")
    mean <- computeMean(sample).timed("mean")
    variance <- computeVariance(sample).timed("var")
    } yield mean / variance).logTimings
    Itamar Ravid - @itrvd 35

    View full-size slide

  36. Summary
    • WriterT is a good example of making auxillary details
    ambient
    • Other things you can do with it:
    • Accumulate events to emit when doing event sourcing;
    • Accumulate errors when emitting partial results;
    • and more and more!
    Itamar Ravid - @itrvd 36

    View full-size slide

  37. Safe state machines
    Problem: I want to safely model an authentication protocol:
    case class Connection(socket: Socket)
    trait Authentication {
    def connect(): IO[Connection]
    def handshake(connection: Connection): IO[Unit] =
    for {
    _ <- IO(connection.socket.write(HandshakeMagic))
    response <- IO(connection.socket.read(HandshakeResponseLen))
    _ <- verifyHandshake(response)
    } yield ()
    def sendCreds(connection: Connection): IO[Unit]
    def authenticate(connection: Connection): IO[Unit]
    }
    Itamar Ravid - @itrvd 37

    View full-size slide

  38. Safe state machines
    Sure, I can just use the functions:
    val api: Authentication
    import api._
    val auth: IO[Unit] = for {
    conn <- connect()
    _ <- handshake(conn)
    _ <- authenticate(conn)
    _ <- sendCreds(conn)
    } yield ()
    Itamar Ravid - @itrvd 38

    View full-size slide

  39. Safe state machines
    But there's a bug here!
    val api: Authentication
    import api._
    val auth: IO[Unit] = for {
    conn <- connect()
    _ <- handshake(conn)
    _ <- authenticate(conn)
    _ <- sendCreds(conn)
    } yield ()
    Itamar Ravid - @itrvd 39

    View full-size slide

  40. Safe state machines
    Static types to the rescue! We will modify Connection:
    case class Handshake()
    case class Credentials()
    case class Auth()
    case class Done()
    case class Connection[S](socket: Socket)
    Itamar Ravid - @itrvd 40

    View full-size slide

  41. Safe state machines
    And the API:
    trait Authentication {
    def connect(): IO[Connection[Handshake]]
    def handshake(connection: Connection[Handshake]): IO[Connection[Credentials]]
    def sendCreds(connection: Connection[Credentials]): IO[Connection[Auth]]
    IO(println("Sending credentials!")) *>
    IO.pure(connection.copy())
    def authenticate(connection: Connection[Auth]): IO[Connection[Done]]
    }
    Itamar Ravid - @itrvd 41

    View full-size slide

  42. Safe state machines
    Now we need to carry even more connections around:
    val api: Authentication
    import api._
    val done: IO[Connection[Done]] = for {
    connHandshake <- connect()
    connCreds <- handshake(connHandshake)
    connAuth <- sendCreds(connCreds)
    connDone <- authenticate(connAuth)
    } yield connDone
    Safe, but ugly.
    Itamar Ravid - @itrvd 42

    View full-size slide

  43. Itamar Ravid - @itrvd 43

    View full-size slide

  44. Safe state machines
    FP saves the day! This is IndexedStateT:
    case class IndexedStateT[F, Start, End, Result](
    run: Start => F[(End, Result)]
    )
    Itamar Ravid - @itrvd 44

    View full-size slide

  45. Safe state machines
    FP saves the day! This is IndexedStateT:
    case class IndexedStateT[F, Start, End, Result](
    run: Start => F[(End, Result)]
    )
    For example:
    Start = Connection[Handshake], End = Connection[Credentials]
    F = IO, Result = Unit
    Itamar Ravid - @itrvd 45

    View full-size slide

  46. Safe state machines
    val handshake: Connection[Handshake] =>
    IO[(Connection[Credentials], Unit)]
    Itamar Ravid - @itrvd 46

    View full-size slide

  47. Safe state machines
    val handshake: Connection[Handshake] =>
    IO[(Connection[Credentials], Unit)]
    type Protocol[A, B] = IndexedStateT[
    IO,
    Connection[A],
    Connection[B],
    Unit
    ]
    val nicerHandshake: Protocol[Handshake, Credentials]
    Itamar Ravid - @itrvd 47

    View full-size slide

  48. Safe state machines
    We can modify our API to use it:
    trait Authentication {
    def connect: Protocol[Unit, Handshake]
    def handshake: Protocol[Handshake, Credentials]
    def sendCreds: Protocol[Credentials, Auth] =
    IndexedStateT.modifyF { conn =>
    IO(println("Sending credentials!")) *>
    IO(conn.copy())
    }
    def authenticate: Protocol[Auth, Done]
    }
    Itamar Ravid - @itrvd 48

    View full-size slide

  49. Safe state machines
    And this is how the usage looks like:
    val api: Authentication
    import api._
    val authenticate: Protocol[Unit, Done] =
    for {
    _ <- connect
    _ <- handshake
    _ <- sendCreds
    _ <- authenticate
    } yield ()
    Itamar Ravid - @itrvd 49

    View full-size slide

  50. Safe state machines
    The original example, by the way, errors out:
    val api: Authentication
    import api._
    val authenticate: Protocol[Unit, Done] =
    for {
    _ <- connect
    _ <- handshake
    _ <- authenticate
    _ <- sendCreds
    } yield ()
    Itamar Ravid - @itrvd 50

    View full-size slide

  51. Safe state machines
    cmd6.sc:6: polymorphic expression cannot be instantiated to expected type;
    found : [B, SC]IndexedStateT[IO,Connection[Auth],SC,B]
    required: IndexedStateT[IO,Connection[Credentials],?,?]
    _ <- authenticate
    ^
    Itamar Ravid - @itrvd 51

    View full-size slide

  52. Summary
    • IndexedStateT is very cool for type-safe transitions
    • It is a Monad and an Applicative when SA = SB
    • For non-linear transitions: shapeless.Coproduct and
    shapeless.HList
    • IxStateT in Haskell is in the indexed-extras package
    Itamar Ravid - @itrvd 52

    View full-size slide

  53. Resource allocation and release
    Problem: I have many resources allocated in my app.
    I need to manage their acquisition and release.
    def createKafkaConsumer: IO[KafkaConsumer]
    trait KafkaConsumer {
    def close: IO[Unit]
    }
    def createDBConnection: IO[DBConnection]
    trait DBConnection {
    def close: IO[Unit]
    }
    Itamar Ravid - @itrvd 53

    View full-size slide

  54. Resource allocation and release
    How should my initialization look like?
    Itamar Ravid - @itrvd 54

    View full-size slide

  55. Resource allocation and release
    How should my initialization look like?
    def runApp(): IO[Unit] =
    for {
    consumer <- createKafkaConsumer
    conn <- createDBConnection
    _ <- doSomething(consumer, conn)
    _ <- conn.close
    _ <- consumer.close
    } yield ()
    Itamar Ravid - @itrvd 55

    View full-size slide

  56. Resource allocation and release
    How should my initialization look like?
    def runApp(): IO[Unit] =
    for {
    consumer <- createKafkaConsumer
    conn <- createDBConnection
    _ <- doSomething(consumer, conn)
    _ <- ignoreFailures(conn.close)
    _ <- ignoreFailures(consumer.close)
    } yield ()
    Itamar Ravid - @itrvd 56

    View full-size slide

  57. The smell is strong with you!
    Itamar Ravid - @itrvd 57

    View full-size slide

  58. Resource allocation and release
    If you know Java, you're probably aware of try-with-
    resources:
    try (BufferedReader br =
    new BufferedReader(new FileReader(path))) {
    return br.readLine();
    }
    This works for any T <: AutoCloseable.
    Itamar Ravid - @itrvd 58

    View full-size slide

  59. Resource allocation and release
    Our example would look like this:
    try (Consumer consumer = new Consumer()) {
    try (DBConnection conn = new DBConnection()) {
    doSomething(consumer, conn);
    }
    }
    Itamar Ravid - @itrvd 59

    View full-size slide

  60. Resource allocation and release
    These are not the abstractions you're looking for!
    Itamar Ravid - @itrvd 60

    View full-size slide

  61. Resource allocation and release
    IO monads offer us the bracket operation:
    object IO {
    def bracket[R, A](acquire: IO[R])
    (release: R => IO[Unit])
    (use: R => IO[A]): IO[A]
    }
    Itamar Ravid - @itrvd 61

    View full-size slide

  62. Resource allocation and release
    But this gets us to the same place:
    def createConsumer: IO[Consumer]
    def createDBConnection: IO[DBConnection]
    bracket(createConsumer)(_.close) { consumer =>
    // use consumer
    }
    Itamar Ravid - @itrvd 62

    View full-size slide

  63. Resource allocation and release
    But this gets us to the same place:
    def createConsumer: IO[Consumer]
    def createDBConnection: IO[DBConnection]
    bracket(createConsumer)(_.close) { consumer =>
    bracket(createDBConnection)(_.close) { conn =>
    // use both
    }
    }
    Itamar Ravid - @itrvd 63

    View full-size slide

  64. Resource allocation and release
    There's an interesting data type called Codensity in Scalaz:
    abstract class Codensity[F[_], A] {
    def apply[B](f: A => F[B]): F[B]
    }
    Itamar Ravid - @itrvd 64

    View full-size slide

  65. Resource allocation and release
    There's an interesting data type called Codensity in Scalaz:
    abstract class Codensity[F[_], A] {
    def apply[B](f: A => F[B]): F[B]
    }
    When we set F = IO, A = R, B = A, apply looks like this:
    def apply(f: R => IO[A]): IO[A]
    Itamar Ravid - @itrvd 65

    View full-size slide

  66. Resource allocation and release
    This is remarkably similar to part of bracket!
    def bracket[R, A](acquire: IO[R])
    (release: R => IO[Unit])
    (use: R => IO[A]): IO[A]
    def apply (f: R => IO[A]): IO[A]
    Remember: type signatures are never a coincidence ;-)
    Itamar Ravid - @itrvd 66

    View full-size slide

  67. Resource allocation and release
    Let's write a combinator using Codensity:
    def resource[R](acquire: IO[R])(release: R => IO[Unit]) =
    new Codensity[IO, R] {
    def apply[A](use: R => IO[A]): IO[A] =
    IO.bracket(acquire)(release)(use)
    }
    Itamar Ravid - @itrvd 67

    View full-size slide

  68. Resource allocation and release
    So now, when we write a resource, we get back a value:
    val consumer: Codensity[IO, Consumer] =
    resource(createKafkaConsumer)(_.close)
    Itamar Ravid - @itrvd 68

    View full-size slide

  69. Resource allocation and release
    So now, when we write a resource, we get back a value:
    val consumer: Codensity[IO, Consumer] =
    resource(createKafkaConsumer)(_.close)
    And we can use it:
    val things: IO[List[Thing]] =
    consumer { c =>
    c.consumeSomeThings
    }
    Itamar Ravid - @itrvd 69

    View full-size slide

  70. Composition
    of resources, using Codensity
    Itamar Ravid - @itrvd 70

    View full-size slide

  71. Resource allocation and release
    It just so happens that Codensity is a Monad.
    So I can do this:
    val resources: Codensity[IO, (Consumer, DBConnection)] =
    for {
    consumer <- resource(createKafkaConsumer)(_.close)
    dbConn <- resource(createDBConnection)(_.close)
    } yield (consumer, dbConn)
    What's happening here though?
    Itamar Ravid - @itrvd 71

    View full-size slide

  72. Resource allocation and release
    Codensity's flatMap:
    class Codensity[F[_], A] { self =>
    def flatMap[B](f: A => Codensity[F, B]): Codensity[F, B] =
    new Codensity[F, B] {
    def apply[R](use: B => F[R]): F[R] =
    self.apply { a =>
    f(a).apply { b =>
    use(b)
    }
    }
    }
    }
    Itamar Ravid - @itrvd 72

    View full-size slide

  73. Resource allocation and release
    If we substitue the bracketing in:
    self.apply { a =>
    f(a).apply { b =>
    use(b)
    }
    }
    Itamar Ravid - @itrvd 73

    View full-size slide

  74. Resource allocation and release
    If we substitue the bracketing in:
    IO.bracket(acquireA)(releaseA) { a =>
    IO.bracket(acquireB)(releaseB) { b =>
    use(b)
    }
    }
    Itamar Ravid - @itrvd 74

    View full-size slide

  75. Resource allocation and release
    Finally, I can do something with these!
    val something: IO[Something] =
    resources { case (consumer, conn) =>
    doSomething(consumer, dbConn)
    }
    Finalizers will run in reverse order of acquisition.
    Itamar Ravid - @itrvd 75

    View full-size slide

  76. Itamar Ravid - @itrvd 76

    View full-size slide

  77. Itamar Ravid - @itrvd 77

    View full-size slide

  78. Itamar Ravid - @itrvd 78

    View full-size slide

  79. Itamar Ravid - @itrvd 79

    View full-size slide

  80. Itamar Ravid - @itrvd 80

    View full-size slide

  81. Itamar Ravid - @itrvd 81

    View full-size slide

  82. Resource allocation and release
    • Codensity is 18 lines of code in total
    • cats-effect includes Resource[F, R]
    • scalaz-zio includes Managed[E, R]
    • In Haskell, you can use ResourceT or Managed
    Itamar Ravid - @itrvd 82

    View full-size slide

  83. Summary
    • Don't be afraid of the exotic data types!
    • Be on the lookout for examples
    Itamar Ravid - @itrvd 83

    View full-size slide