A server is just a function: introduction to http4s

A server is just a function: introduction to http4s

If you're going to write a modern web service, you have a wide range of frameworks and libraries to choose from. One of them is http4s, a functional library for writing HTTP services and clients in Scala. I'm going to show you why you should consider it for your next application, for reasons including but not limited to:
- type-safe request/response models that are uniform between client and server
- resource-safe servers, response/request bodies, and streaming
- framework-agnostic testing.

At the end of this talk, you should have a clear understanding of how http4s can make it easier to build modern, runtime-safe services in Scala.

08f642741fba006656cb86fb61c160b3?s=128

Jakub Kozłowski

April 25, 2019
Tweet

Transcript

  1. A SERVER IS JUST A FUNCTION A N I N

    T R O D U C T I O N TO H T T P 4 S J A K U B KO Z Ł O W S K I
  2. WHAT MAKES A SERVER?

  3. WHAT MAKES A FUNCTION?

  4. type Server = Request => Response? ?

  5. None
  6. WHAT REALLY MAKES A SERVER? Streaming Shared resources State Errors

    Routing Auth Serialization
  7. HTTP4S Purely functional, streaming HTTP server/client https://http4s.org

  8. HTTP4S Purely functional, streaming HTTP server/client Multiple backends https://http4s.org

  9. HTTP4S Purely functional, streaming HTTP server/client Multiple backends Built on

    cats-effect and fs2 https://http4s.org
  10. HTTP4S Purely functional, streaming HTTP server/client Multiple backends Built on

    cats-effect and fs2 Supports websockets (server) https://http4s.org
  11. Function1[_, _]

  12. Function1[_, _] + IO[_]

  13. Function1[_, _] + IO[_]

  14. HttpApp = Request => IO[Response] HttpRoutes = Request => IO[Option[Response]]

    HTTP4S SERVER: INTUITION
  15. Kleisli

  16. None
  17. KLEISLI case class Kleisli[F[_], A, B]( run: A => F[B]

    )
  18. KLEISLI case class Kleisli[F[_], A, B]( run: ) Function with

    effectful result A => F[B]
  19. KLEISLI val len: Kleisli[IO, Token, Int] = Kleisli { token

    => IO.pure(token.length) } val cloned: Kleisli[IO, Token, String] = Kleisli { token => IO.pure(token + token) }
  20. KLEISLI val f3 = for { a <- len b

    <- cloned } yield (a + b) val f4 = (len, cloned).tupled val len: Kleisli[IO, Token, Int] = Kleisli { token => IO.pure(token.length) } val cloned: Kleisli[IO, Token, String] = Kleisli { token => IO.pure(token + token) }
  21. OptionT

  22. OPTIONT case class OptionT[F[_], A]( value: F[Option[A]] )

  23. OPTIONT Option nested in an effect case class OptionT[F[_], A](

    value: ) F[Option[A]]
  24. OPTIONT val num: OptionT[IO, Int] = OptionT(IO.pure(42.some)) def text(num: Int):

    OptionT[IO, String] = num match { case n if n % 2 === 0 => OptionT(IO.pure("foo".some)) case _ => OptionT.none }
  25. OPTIONT val num: OptionT[IO, Int] = OptionT(IO.pure(42.some)) def text(num: Int):

    OptionT[IO, String] = num match { case n if n % 2 === 0 => OptionT(IO.pure("foo".some)) case _ => OptionT.none } val result = for { n <- num s <- text(n) s2 <- text(n + 1) s3 <- text(n + 2) } yield s + s2 + s3 val unwrapped: IO[Option[String]] = result.value
  26. HttpApp = Kleisli[IO, Request, Response] HttpRoutes = Kleisli[OptionT[IO, ?], Request,

    Response] HTTP4S SERVER: THE REAL THING (ALMOST)
  27. HTTP4S SERVER: THE REAL REAL THING HttpApp[F[_]] = Kleisli[F, Request[F],

    Response[F]] HttpRoutes[F[_]] = Kleisli[OptionT[F, ?], Request[F], Response[F]] Effect type of entity body stream
  28. HTTP4S SERVER: THE REAL REAL THING Http[F[_], G[_]] = Kleisli[F,

    Request[G], Response[G]] HttpApp[F[_]] = Http[F, F] HttpRoutes[F[_]] = Http[OptionT[F, ?], F]
  29. ALL THIS ABSTRACTION... WHY? (STAY TUNED) Spoiler: DRY

  30. ROUTING DSL

  31. object Main extends IOApp { val routes = HttpRoutes.of[IO] {

    case GET -> Root / "hello" => Ok("Hello, world!") } def run(args: List[String]): IO[ExitCode] = BlazeServerBuilder[IO] .withHttpApp(routes.orNotFound) .bindHttp(port = 8080) .resource .use(_ => IO.never) }
  32. object Main extends IOApp { val routes = HttpRoutes.of[IO] {

    case GET -> Root / "hello" => Ok("Hello, world!") } def run(args: List[String]): IO[ExitCode] = BlazeServerBuilder[IO] .withHttpApp( ) .bindHttp(port = 8080) .resource .use(_ => IO.never) } routes.orNotFound HttpRoutes => HttpApp
  33. object Main extends IOApp { val routes = HttpRoutes.of[IO] {

    case GET -> Root / "hello" => Ok("Hello, world!") } def run(args: List[String]): IO[ExitCode] = BlazeServerBuilder[IO] .withHttpApp(routes.orNotFound) .bindHttp(port = 8080) .resource .use(_ => IO.never) }
  34. HttpRoutes.of[IO] { case GET -> Root / "hello" => Ok("Hello,

    world!") }
  35. HttpRoutes.of[IO] { case -> Root / "hello" => Ok("Hello, world!")

    } GET
  36. HttpRoutes.of[IO] { case GET -> => Ok("Hello, world!") } Root

    / "hello"
  37. Ok("Hello, world!") HttpRoutes.of[IO] { case GET -> Root / "hello"

    => }
  38. HttpRoutes.of[IO] { case GET -> Root / "hello" => Ok("Hello,

    world!") }
  39. HttpRoutes.of[IO] { case request @ POST -> Root / "echo"

    => Ok(request.body) }
  40. HttpRoutes.of[IO] { case request @ POST -> Root / "echo"

    => Ok( ) } request.body
  41. HttpRoutes.of[IO] { case request @ POST -> Root / "echo"

    => Ok( ) } request.body Literally Stream[IO, Byte]
  42. CLIENT

  43. val request = Request[IO](uri = uri"https://http4s.org") val remoteCall: IO[String] =

    client.expect[String](request)
  44. val request = Request[IO](uri = uri"https://http4s.org") val remoteCall: IO[String] =

    client.expect[String](request) remoteCall BlazeClientBuilder[IO](executionContext).resource.use { client => }
  45. case GET -> Root / "remote" / "stream" => val

    response = client.stream(request).flatMap(_.body) Ok(response)
  46. trait Client[F[_]] { def run(req: Request[F]): Resource[F, Response[F]] //+ a

    bunch of other convenient methods }
  47. trait Client[F[_]] { def run(req: Request[F]): Resource[F, Response[F]] } def

    apply[F[_]](f: Request[F] => Resource[F, Response[F]] ): Client[F]
  48. trait Client[F[_]] { def run(req: Request[F]): Resource[F, Response[F]] } def

    apply[F[_]](f: Request[F] => Resource[F, Response[F]] ): Client[F]
  49. trait Client[F[_]] { def run(req: Request[F]): Resource[F, Response[F]] } def

    fromHttpApp[F[_]](app: HttpApp[F]): Client[F] def apply[F[_]](f: Request[F] => Resource[F, Response[F]] ): Client[F]
  50. T E S T I N G R E S

    O U R C E S A F E T Y E X T E N S I B I L I T Y
  51. T E S T I N G R E S

    O U R C E S A F E T Y E X T E N S I B I L I T Y
  52. TESTING

  53. TESTING Server routes

  54. TESTING Server routes Server HTTP interface

  55. TESTING Server routes Server HTTP interface Client calls

  56. TESTING ROUTES def routes(client: Client[IO]): HttpRoutes[IO] = { HttpRoutes.of[IO] {

    //... case GET -> Root / "remote" => val remoteCall: IO[String] = client.expect[String](request) for { result <- remoteCall response <- Ok(result) } yield response //... } }
  57. val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) TESTING ROUTES

  58. val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) val routes = Main.routes(remoteClient) TESTING ROUTES

  59. val body = Json.obj("foo" -> Json.fromString("bar")) val request = Request[IO](method

    = Method.POST, uri = uri"/echo").withEntity(body) val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) val routes = Main.routes(remoteClient) httsps://http4s.org TESTING ROUTES
  60. routes.run(request).value.flatMap(_.value.as[Json]).map(_ shouldBe body) val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) val routes =

    Main.routes(remoteClient) val body = Json.obj("foo" -> Json.fromString("bar")) val request = Request[IO](method = Method.POST, uri = uri"/echo").withEntity(body) TESTING ROUTES
  61. routes.run(request).value.flatMap(_.value.as[Json]).map(_ shouldBe body) val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) val routes =

    Main.routes(remoteClient) val body = Json.obj("foo" -> Json.fromString("bar")) val request = Request[IO](method = Method.POST, uri = uri"/echo").withEntity(body) TESTING ROUTES Client.fromHttpApp(routes.orNotFound).expect[Json](request).map(_ shouldBe body) or
  62. TESTING HTTP val blazeClient = BlazeClientBuilder[IO](ExecutionContext.global).resource

  63. TESTING HTTP val blazeClient = BlazeClientBuilder[IO](ExecutionContext.global).resource Main.server.use { server =>

    }
  64. TESTING HTTP val blazeClient = BlazeClientBuilder[IO](ExecutionContext.global).resource Main.server.use { server =>

    blazeClient.use { client => } }
  65. TESTING HTTP val blazeClient = BlazeClientBuilder[IO](ExecutionContext.global).resource Main.server.use { server =>

    blazeClient.use { client => } } client.expect[String](server.baseUri / "hello") .map(_ shouldBe "Hello world!")
  66. TESTING CLIENTS class TodoClient(client: Client[IO]) { def getTodo(id: Int): IO[Todo]

    = client.expect[Todo](Request[IO](uri = uri"/todos" / id.toString)) }
  67. TESTING CLIENTS class TodoClient(client: Client[IO]) { def getTodo(id: Int): IO[Todo]

    = client.expect[Todo](Request[IO](uri = uri"/todos" / id.toString)) } //in test val mockServer = HttpRoutes .of[IO] { case GET -> Root / "todos" / IntVar(id) => Ok(Todo(id, false)) } .orNotFound
  68. TESTING CLIENTS val mockServer = HttpRoutes .of[IO] { case GET

    -> Root / "todos" / IntVar(id) => Ok(Todo(id, false)) } .orNotFound
  69. TESTING CLIENTS val clientRaw = Client.fromHttpApp(mockServer) val todoClient = new

    TodoClient(clientRaw) val mockServer = HttpRoutes .of[IO] { case GET -> Root / "todos" / IntVar(id) => Ok(Todo(id, false)) } .orNotFound
  70. TESTING CLIENTS val mockServer = HttpRoutes .of[IO] { case GET

    -> Root / "todos" / IntVar(id) => Ok(Todo(id, false)) } .orNotFound val clientRaw = Client.fromHttpApp(mockServer) val todoClient = new TodoClient(clientRaw) todoClient.getTodo(1).map(_ shouldBe Todo(1, false))
  71. T E S T I N G R E S

    O U R C E S A F E T Y E X T E N S I B I L I T Y
  72. T E S T I N G R E S

    O U R C E S A F E T Y E X T E N S I B I L I T Y
  73. CATS.EFFECT.RESOURCE

  74. RESOURCE abstract class Resource[F[_], A] { def use[B](f: A =>

    F[B]): F[B] } object Resource { def make[F[_], A](acquire: F[A]) (release: A => F[Unit]): Resource[F, A] }
  75. RESOURCE Resource.make(IO(createPool()))(pool => IO(pool.close()))

  76. RESOURCE db <- Resource.make(IO(createPool()))(pool => IO(pool.close()))

  77. RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool =>

    IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds))
  78. RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool =>

    IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool
  79. RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool =>

    IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool 2. Start server
  80. RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool =>

    IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool 2. Start server 3. Sleep 10 seconds
  81. RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool =>

    IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool 2. Start server 3. Sleep 10 seconds 4. Stop server
  82. RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool =>

    IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool 2. Start server 3. Sleep 10 seconds 4. Stop server 5. Close pool
  83. FS2.STREAM[F[_], A] - 0 to ∞ values - Can have

    effects in F - Supports resource acquisition/cleanup
  84. FS2.STREAM[F[_], A] - 0 to ∞ values - Can have

    effects in F - Supports resource acquisition/cleanup Stream(1,2,3)
  85. FS2.STREAM[F[_], A] - 0 to ∞ values - Can have

    effects in F - Supports resource acquisition/cleanup Stream(1,2,3) Stream.eval(IO(util.Random.nextInt))
  86. FS2.STREAM[F[_], A] - 0 to ∞ values - Can have

    effects in F - Supports resource acquisition/cleanup Stream(1,2,3) Stream.eval(IO(util.Random.nextInt)) Stream.awakeDelay[IO](1.second)
  87. FS2.STREAM[F[_], A] - 0 to ∞ values - Can have

    effects in F - Supports resource acquisition/cleanup Stream(1,2,3) Stream.eval(IO(util.Random.nextInt)) Stream.awakeDelay[IO](1.second).compile.toList: IO[List[FiniteDuration]]
  88. FS2.STREAM[F[_], A] Stream(1,2,3) Stream.eval(IO(util.Random.nextInt)) Stream.awakeDelay[IO](1.second).compile.toList: IO[List[FiniteDuration]]

  89. FS2.STREAM[F[_], A] Stream(1,2,3) Stream.eval(IO(util.Random.nextInt)) Stream.awakeDelay[IO](1.second).compile.toList: IO[List[FiniteDuration]] + Stream.bracket + Stream.resource

  90. RESOURCE SAFETY IN HTTP4S

  91. RESOURCE SAFETY IN HTTP4S

  92. RESOURCE SAFETY IN HTTP4S BlazeServerBuilder[IO].resource: Resource[IO, Server[IO]]

  93. RESOURCE SAFETY IN HTTP4S BlazeServerBuilder[IO].resource: Resource[IO, Server[IO]] BlazeClientBuilder[IO](ec).resource: Resource[IO, Client[IO]]

  94. RESOURCE SAFETY IN HTTP4S BlazeServerBuilder[IO].resource: Resource[IO, Server[IO]] BlazeClientBuilder[IO](ec).resource: Resource[IO, Client[IO]]

    (request: Request[IO]).body: fs2.Stream[IO, Byte]
  95. RESOURCE SAFETY IN HTTP4S BlazeServerBuilder[IO].resource: Resource[IO, Server[IO]] BlazeClientBuilder[IO](ec).resource: Resource[IO, Client[IO]]

    (request: Request[IO]).body: fs2.Stream[IO, Byte] (response: Response[IO]).body: fs2.Stream[IO, Byte]
  96. RESOURCE SAFETY IN HTTP4S BlazeServerBuilder[IO].resource: Resource[IO, Server[IO]] BlazeClientBuilder[IO](ec).resource: Resource[IO, Client[IO]]

    (request: Request[IO]).body: fs2.Stream[IO, Byte] (response: Response[IO]).body: fs2.Stream[IO, Byte] ...and then there's Client
  97. trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def

    stream(req: Request[F]): Stream[F, Response[F]] }
  98. trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def

    stream(req: Request[F]): Stream[F, Response[F]] def fetch[A](req: Request[F])(f: Response[F] => F[A]) : F[A] def expect[A](req: Request[F])(implicit d: EntityDecoder[F, A]): F[A] } (these have like 10 overloads*) *some are about to be removed
  99. trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def

    stream(req: Request[F]): Stream[F, Response[F]] def fetch[A](req: Request[F])(f: Response[F] => F[A]) : F[A] def expect[A](req: Request[F])(implicit d: EntityDecoder[F, A]): F[A] def status(req: Request[F]). : F[Status] def successful(req: Request[F]): F[Boolean] }
  100. trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def

    stream(req: Request[F]): Stream[F, Response[F]] def fetch[A](req: Request[F])(f: Response[F] => F[A]) : F[A] def expect[A](req: Request[F])(implicit d: EntityDecoder[F, A]): F[A] def status(req: Request[F]). : F[Status] def successful(req: Request[F]): F[Boolean] def toKleisli[A](f: Response[F] => F[A]): Kleisli[F, Request[F], A] //... }
  101. trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def

    stream(req: Request[F]): Stream[F, Response[F]] def fetch[A](req: Request[F])(f: Response[F] => F[A]) : F[A] def expect[A](req: Request[F])(implicit d: EntityDecoder[F, A]): F[A] def status(req: Request[F]). : F[Status] def successful(req: Request[F]): F[Boolean] def toKleisli[A](f: Response[F] => F[A]): Kleisli[F, Request[F], A] //... //here be dragons - don't use unless you know what you're doing def toHttpApp: HttpApp[F] }
  102. T E S T I N G R E S

    O U R C E S A F E T Y E X T E N S I B I L I T Y
  103. T E S T I N G R E S

    O U R C E S A F E T Y E X T E N S I B I L I T Y
  104. EXTENSIBILITY

  105. EXTENSIBILITY If a server/client is still a function...

  106. EXTENSIBILITY If a server/client is still a function... what can

    we do with functions?
  107. case class Data(x: Int) val f: Data => Int =

    _.x
  108. case class Data(x: Int) val f: Data => Int =

    _.x def wrapF(f1: Data => Int): Data => Int = data => f1(data.copy(x = data.x + 1)) * 2
  109. case class Data(x: Int) val f: Data => Int =

    _.x def wrapF(f1: Data => Int): Data => Int = data => f1(data.copy(x = data.x + 1)) * 2 val f2: Data => Int = wrapF(f)
  110. case class Data(x: Int) val f: Data => Int =

    _.x val f2: Data => Int = wrapF(f) def wrapF(f1: Data => Int): Data => Int = data => f1( ) * 2 data.copy(x = data.x + 1)
  111. None
  112. None
  113. None
  114. HttpApp[F[_]] = Kleisli[F, Request[F], Response[F]]

  115. SERVER MIDDLEWARE http: HttpApp[F]

  116. SERVER MIDDLEWARE def apply[F[_]]( http: HttpApp[F]): HttpApp[F]

  117. SERVER MIDDLEWARE: RESPONSE TIMING object ResponseTiming { def apply[F[_]]( http:

    HttpApp[F], timeUnit: TimeUnit = MILLISECONDS, headerName: CaseInsensitiveString = CaseInsensitiveString("X-Response-Time"))( implicit F: Sync[F], clock: Clock[F]): HttpApp[F] = Kleisli { req => for { before <- clock.monotonic(timeUnit) resp <- http(req) after <- clock.monotonic(timeUnit) header = Header(headerName.value, s"${after - before}") } yield resp.putHeaders(header) } }
  118. SERVER MIDDLEWARE: RESPONSE TIMING object ResponseTiming { def apply[F[_]]( http:

    HttpApp[F], timeUnit: TimeUnit = MILLISECONDS, headerName: CaseInsensitiveString = CaseInsensitiveString("X-Response-Time"))( implicit F: Sync[F], clock: Clock[F]): HttpApp[F] = Kleisli { req => for { before <- clock.monotonic(timeUnit) after <- clock.monotonic(timeUnit) header = Header(headerName.value, s"${after - before}") } yield } } Kleisli { req => resp <- http(req) resp.putHeaders(header) }
  119. SERVER MIDDLEWARE: HEADER ECHO object HeaderEcho { def apply[F[_]: Functor,

    G[_]: Functor] (echoHeadersWhen: CaseInsensitiveString => Boolean) (http: Http[F, G]): Http[F, G] = Kleisli { req: Request[G] => val headersToEcho = req.headers.filter(h => echoHeadersWhen(h.name)) http(req).map(_.putHeaders(headersToEcho.toList: _*)) } }
  120. SERVER MIDDLEWARE: HEADER ECHO object HeaderEcho { def apply[F[_]: Functor,

    G[_]: Functor] (echoHeadersWhen: CaseInsensitiveString => Boolean) = Kleisli { req: Request[G] => val headersToEcho = req.headers.filter(h => echoHeadersWhen(h.name)) http(req).map(_.putHeaders(headersToEcho.toList: _*)) } } (http: Http[F, G]): Http[F, G]
  121. CLIENT MIDDLEWARE client: Client[F]

  122. CLIENT MIDDLEWARE def apply[F[_]]( client: Client[F] ): Client[F]

  123. CLIENT MIDDLEWARE: BASE URL object BaseUrl { def apply[F[_]](base: Uri)(

    client: Client[F] )(implicit F: Bracket[F, Throwable]): Client[F] = { Client.apply[F] { req => client.run(req.withUri(base.resolve(req.uri))) } } }
  124. SERVER CLIENT

  125. LEARN MORE http4s.org gitter.im/http4s/http4s gitter.im/typelevel/cats-effect

  126. ATTRIBUTION natural power by Icon Island from the Noun Project

    testing by tom from the Noun Project modules by mikicon from the Noun Project
  127. THANK YOU Slides: bit.ly/2WaFCOM Code: git.io/fjs4v Twitter: @kubukoz My blog:

    blog.kubukoz.com
  128. BONUS SLIDE: WEBSOCKETS case GET -> Root / "ws" /

    "echo" => Queue.bounded[IO, WebSocketFrame](100).flatMap { q => WebSocketBuilder[IO].build( send = q.dequeue, receive = q.enqueue ) }
  129. BONUS SLIDE: WEBSOCKETS case GET -> Root / "ws" /

    "echo" => Queue.bounded[IO, WebSocketFrame](100).flatMap { q => WebSocketBuilder[IO].build( ) } send = q.dequeue, receive = q.enqueue
  130. BONUS SLIDE: SWAGGER (BECAUSE SOMEONE ALWAYS ASKS)

  131. BONUS SLIDE: SWAGGER (BECAUSE SOMEONE ALWAYS ASKS)