A server is just a function: introduction to http4s

A server is just a function: introduction to http4s

If you're going to write a modern web service, you have a wide range of frameworks and libraries to choose from. One of them is http4s, a functional library for writing HTTP services and clients in Scala. I'm going to show you why you should consider it for your next application, for reasons including but not limited to:
- type-safe request/response models that are uniform between client and server
- resource-safe servers, response/request bodies, and streaming
- framework-agnostic testing.

At the end of this talk, you should have a clear understanding of how http4s can make it easier to build modern, runtime-safe services in Scala.

08f642741fba006656cb86fb61c160b3?s=128

Jakub Kozłowski

April 25, 2019
Tweet

Transcript

  1. 1.

    A SERVER IS JUST A FUNCTION A N I N

    T R O D U C T I O N TO H T T P 4 S J A K U B KO Z Ł O W S K I
  2. 5.
  3. 10.

    HTTP4S Purely functional, streaming HTTP server/client Multiple backends Built on

    cats-effect and fs2 Supports websockets (server) https://http4s.org
  4. 15.
  5. 16.
  6. 17.

  7. 20.

    KLEISLI val len: Kleisli[IO, Token, Int] = Kleisli { token

    => IO.pure(token.length) } val cloned: Kleisli[IO, Token, String] = Kleisli { token => IO.pure(token + token) }
  8. 21.

    KLEISLI val f3 = for { a <- len b

    <- cloned } yield (a + b) val f4 = (len, cloned).tupled val len: Kleisli[IO, Token, Int] = Kleisli { token => IO.pure(token.length) } val cloned: Kleisli[IO, Token, String] = Kleisli { token => IO.pure(token + token) }
  9. 22.
  10. 25.

    OPTIONT val num: OptionT[IO, Int] = OptionT(IO.pure(42.some)) def text(num: Int):

    OptionT[IO, String] = num match { case n if n % 2 === 0 => OptionT(IO.pure("foo".some)) case _ => OptionT.none }
  11. 26.

    OPTIONT val num: OptionT[IO, Int] = OptionT(IO.pure(42.some)) def text(num: Int):

    OptionT[IO, String] = num match { case n if n % 2 === 0 => OptionT(IO.pure("foo".some)) case _ => OptionT.none } val result = for { n <- num s <- text(n) s2 <- text(n + 1) s3 <- text(n + 2) } yield s + s2 + s3 val unwrapped: IO[Option[String]] = result.value
  12. 28.

    HTTP4S SERVER: THE REAL REAL THING HttpApp[F[_]] = Kleisli[F, Request[F],

    Response[F]] HttpRoutes[F[_]] = Kleisli[OptionT[F, ?], Request[F], Response[F]] Effect type of entity body stream
  13. 29.

    HTTP4S SERVER: THE REAL REAL THING Http[F[_], G[_]] = Kleisli[F,

    Request[G], Response[G]] HttpApp[F[_]] = Http[F, F] HttpRoutes[F[_]] = Http[OptionT[F, ?], F]
  14. 32.

    object Main extends IOApp { val routes = HttpRoutes.of[IO] {

    case GET -> Root / "hello" => Ok("Hello, world!") } def run(args: List[String]): IO[ExitCode] = BlazeServerBuilder[IO] .withHttpApp(routes.orNotFound) .bindHttp(port = 8080) .resource .use(_ => IO.never) }
  15. 33.

    object Main extends IOApp { val routes = HttpRoutes.of[IO] {

    case GET -> Root / "hello" => Ok("Hello, world!") } def run(args: List[String]): IO[ExitCode] = BlazeServerBuilder[IO] .withHttpApp( ) .bindHttp(port = 8080) .resource .use(_ => IO.never) } routes.orNotFound HttpRoutes => HttpApp
  16. 34.

    object Main extends IOApp { val routes = HttpRoutes.of[IO] {

    case GET -> Root / "hello" => Ok("Hello, world!") } def run(args: List[String]): IO[ExitCode] = BlazeServerBuilder[IO] .withHttpApp(routes.orNotFound) .bindHttp(port = 8080) .resource .use(_ => IO.never) }
  17. 42.

    HttpRoutes.of[IO] { case request @ POST -> Root / "echo"

    => Ok( ) } request.body Literally Stream[IO, Byte]
  18. 43.
  19. 45.

    val request = Request[IO](uri = uri"https://http4s.org") val remoteCall: IO[String] =

    client.expect[String](request) remoteCall BlazeClientBuilder[IO](executionContext).resource.use { client => }
  20. 46.

    case GET -> Root / "remote" / "stream" => val

    response = client.stream(request).flatMap(_.body) Ok(response)
  21. 48.

    trait Client[F[_]] { def run(req: Request[F]): Resource[F, Response[F]] } def

    apply[F[_]](f: Request[F] => Resource[F, Response[F]] ): Client[F]
  22. 49.

    trait Client[F[_]] { def run(req: Request[F]): Resource[F, Response[F]] } def

    apply[F[_]](f: Request[F] => Resource[F, Response[F]] ): Client[F]
  23. 50.

    trait Client[F[_]] { def run(req: Request[F]): Resource[F, Response[F]] } def

    fromHttpApp[F[_]](app: HttpApp[F]): Client[F] def apply[F[_]](f: Request[F] => Resource[F, Response[F]] ): Client[F]
  24. 51.

    T E S T I N G R E S

    O U R C E S A F E T Y E X T E N S I B I L I T Y
  25. 52.

    T E S T I N G R E S

    O U R C E S A F E T Y E X T E N S I B I L I T Y
  26. 53.
  27. 57.

    TESTING ROUTES def routes(client: Client[IO]): HttpRoutes[IO] = { HttpRoutes.of[IO] {

    //... case GET -> Root / "remote" => val remoteCall: IO[String] = client.expect[String](request) for { result <- remoteCall response <- Ok(result) } yield response //... } }
  28. 60.

    val body = Json.obj("foo" -> Json.fromString("bar")) val request = Request[IO](method

    = Method.POST, uri = uri"/echo").withEntity(body) val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) val routes = Main.routes(remoteClient) httsps://http4s.org TESTING ROUTES
  29. 61.

    routes.run(request).value.flatMap(_.value.as[Json]).map(_ shouldBe body) val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) val routes =

    Main.routes(remoteClient) val body = Json.obj("foo" -> Json.fromString("bar")) val request = Request[IO](method = Method.POST, uri = uri"/echo").withEntity(body) TESTING ROUTES
  30. 62.

    routes.run(request).value.flatMap(_.value.as[Json]).map(_ shouldBe body) val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) val routes =

    Main.routes(remoteClient) val body = Json.obj("foo" -> Json.fromString("bar")) val request = Request[IO](method = Method.POST, uri = uri"/echo").withEntity(body) TESTING ROUTES Client.fromHttpApp(routes.orNotFound).expect[Json](request).map(_ shouldBe body) or
  31. 66.

    TESTING HTTP val blazeClient = BlazeClientBuilder[IO](ExecutionContext.global).resource Main.server.use { server =>

    blazeClient.use { client => } } client.expect[String](server.baseUri / "hello") .map(_ shouldBe "Hello world!")
  32. 67.

    TESTING CLIENTS class TodoClient(client: Client[IO]) { def getTodo(id: Int): IO[Todo]

    = client.expect[Todo](Request[IO](uri = uri"/todos" / id.toString)) }
  33. 68.

    TESTING CLIENTS class TodoClient(client: Client[IO]) { def getTodo(id: Int): IO[Todo]

    = client.expect[Todo](Request[IO](uri = uri"/todos" / id.toString)) } //in test val mockServer = HttpRoutes .of[IO] { case GET -> Root / "todos" / IntVar(id) => Ok(Todo(id, false)) } .orNotFound
  34. 69.

    TESTING CLIENTS val mockServer = HttpRoutes .of[IO] { case GET

    -> Root / "todos" / IntVar(id) => Ok(Todo(id, false)) } .orNotFound
  35. 70.

    TESTING CLIENTS val clientRaw = Client.fromHttpApp(mockServer) val todoClient = new

    TodoClient(clientRaw) val mockServer = HttpRoutes .of[IO] { case GET -> Root / "todos" / IntVar(id) => Ok(Todo(id, false)) } .orNotFound
  36. 71.

    TESTING CLIENTS val mockServer = HttpRoutes .of[IO] { case GET

    -> Root / "todos" / IntVar(id) => Ok(Todo(id, false)) } .orNotFound val clientRaw = Client.fromHttpApp(mockServer) val todoClient = new TodoClient(clientRaw) todoClient.getTodo(1).map(_ shouldBe Todo(1, false))
  37. 72.

    T E S T I N G R E S

    O U R C E S A F E T Y E X T E N S I B I L I T Y
  38. 73.

    T E S T I N G R E S

    O U R C E S A F E T Y E X T E N S I B I L I T Y
  39. 75.

    RESOURCE abstract class Resource[F[_], A] { def use[B](f: A =>

    F[B]): F[B] } object Resource { def make[F[_], A](acquire: F[A]) (release: A => F[Unit]): Resource[F, A] }
  40. 78.

    RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool =>

    IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds))
  41. 79.

    RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool =>

    IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool
  42. 80.

    RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool =>

    IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool 2. Start server
  43. 81.

    RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool =>

    IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool 2. Start server 3. Sleep 10 seconds
  44. 82.

    RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool =>

    IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool 2. Start server 3. Sleep 10 seconds 4. Stop server
  45. 83.

    RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool =>

    IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool 2. Start server 3. Sleep 10 seconds 4. Stop server 5. Close pool
  46. 84.

    FS2.STREAM[F[_], A] - 0 to ∞ values - Can have

    effects in F - Supports resource acquisition/cleanup
  47. 85.

    FS2.STREAM[F[_], A] - 0 to ∞ values - Can have

    effects in F - Supports resource acquisition/cleanup Stream(1,2,3)
  48. 86.

    FS2.STREAM[F[_], A] - 0 to ∞ values - Can have

    effects in F - Supports resource acquisition/cleanup Stream(1,2,3) Stream.eval(IO(util.Random.nextInt))
  49. 87.

    FS2.STREAM[F[_], A] - 0 to ∞ values - Can have

    effects in F - Supports resource acquisition/cleanup Stream(1,2,3) Stream.eval(IO(util.Random.nextInt)) Stream.awakeDelay[IO](1.second)
  50. 88.

    FS2.STREAM[F[_], A] - 0 to ∞ values - Can have

    effects in F - Supports resource acquisition/cleanup Stream(1,2,3) Stream.eval(IO(util.Random.nextInt)) Stream.awakeDelay[IO](1.second).compile.toList: IO[List[FiniteDuration]]
  51. 96.

    RESOURCE SAFETY IN HTTP4S BlazeServerBuilder[IO].resource: Resource[IO, Server[IO]] BlazeClientBuilder[IO](ec).resource: Resource[IO, Client[IO]]

    (request: Request[IO]).body: fs2.Stream[IO, Byte] (response: Response[IO]).body: fs2.Stream[IO, Byte]
  52. 97.

    RESOURCE SAFETY IN HTTP4S BlazeServerBuilder[IO].resource: Resource[IO, Server[IO]] BlazeClientBuilder[IO](ec).resource: Resource[IO, Client[IO]]

    (request: Request[IO]).body: fs2.Stream[IO, Byte] (response: Response[IO]).body: fs2.Stream[IO, Byte] ...and then there's Client
  53. 98.

    trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def

    stream(req: Request[F]): Stream[F, Response[F]] }
  54. 99.

    trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def

    stream(req: Request[F]): Stream[F, Response[F]] def fetch[A](req: Request[F])(f: Response[F] => F[A]) : F[A] def expect[A](req: Request[F])(implicit d: EntityDecoder[F, A]): F[A] } (these have like 10 overloads*) *some are about to be removed
  55. 100.

    trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def

    stream(req: Request[F]): Stream[F, Response[F]] def fetch[A](req: Request[F])(f: Response[F] => F[A]) : F[A] def expect[A](req: Request[F])(implicit d: EntityDecoder[F, A]): F[A] def status(req: Request[F]). : F[Status] def successful(req: Request[F]): F[Boolean] }
  56. 101.

    trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def

    stream(req: Request[F]): Stream[F, Response[F]] def fetch[A](req: Request[F])(f: Response[F] => F[A]) : F[A] def expect[A](req: Request[F])(implicit d: EntityDecoder[F, A]): F[A] def status(req: Request[F]). : F[Status] def successful(req: Request[F]): F[Boolean] def toKleisli[A](f: Response[F] => F[A]): Kleisli[F, Request[F], A] //... }
  57. 102.

    trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def

    stream(req: Request[F]): Stream[F, Response[F]] def fetch[A](req: Request[F])(f: Response[F] => F[A]) : F[A] def expect[A](req: Request[F])(implicit d: EntityDecoder[F, A]): F[A] def status(req: Request[F]). : F[Status] def successful(req: Request[F]): F[Boolean] def toKleisli[A](f: Response[F] => F[A]): Kleisli[F, Request[F], A] //... //here be dragons - don't use unless you know what you're doing def toHttpApp: HttpApp[F] }
  58. 103.

    T E S T I N G R E S

    O U R C E S A F E T Y E X T E N S I B I L I T Y
  59. 104.

    T E S T I N G R E S

    O U R C E S A F E T Y E X T E N S I B I L I T Y
  60. 109.

    case class Data(x: Int) val f: Data => Int =

    _.x def wrapF(f1: Data => Int): Data => Int = data => f1(data.copy(x = data.x + 1)) * 2
  61. 110.

    case class Data(x: Int) val f: Data => Int =

    _.x def wrapF(f1: Data => Int): Data => Int = data => f1(data.copy(x = data.x + 1)) * 2 val f2: Data => Int = wrapF(f)
  62. 111.

    case class Data(x: Int) val f: Data => Int =

    _.x val f2: Data => Int = wrapF(f) def wrapF(f1: Data => Int): Data => Int = data => f1( ) * 2 data.copy(x = data.x + 1)
  63. 112.
  64. 113.
  65. 114.
  66. 118.

    SERVER MIDDLEWARE: RESPONSE TIMING object ResponseTiming { def apply[F[_]]( http:

    HttpApp[F], timeUnit: TimeUnit = MILLISECONDS, headerName: CaseInsensitiveString = CaseInsensitiveString("X-Response-Time"))( implicit F: Sync[F], clock: Clock[F]): HttpApp[F] = Kleisli { req => for { before <- clock.monotonic(timeUnit) resp <- http(req) after <- clock.monotonic(timeUnit) header = Header(headerName.value, s"${after - before}") } yield resp.putHeaders(header) } }
  67. 119.

    SERVER MIDDLEWARE: RESPONSE TIMING object ResponseTiming { def apply[F[_]]( http:

    HttpApp[F], timeUnit: TimeUnit = MILLISECONDS, headerName: CaseInsensitiveString = CaseInsensitiveString("X-Response-Time"))( implicit F: Sync[F], clock: Clock[F]): HttpApp[F] = Kleisli { req => for { before <- clock.monotonic(timeUnit) after <- clock.monotonic(timeUnit) header = Header(headerName.value, s"${after - before}") } yield } } Kleisli { req => resp <- http(req) resp.putHeaders(header) }
  68. 120.

    SERVER MIDDLEWARE: HEADER ECHO object HeaderEcho { def apply[F[_]: Functor,

    G[_]: Functor] (echoHeadersWhen: CaseInsensitiveString => Boolean) (http: Http[F, G]): Http[F, G] = Kleisli { req: Request[G] => val headersToEcho = req.headers.filter(h => echoHeadersWhen(h.name)) http(req).map(_.putHeaders(headersToEcho.toList: _*)) } }
  69. 121.

    SERVER MIDDLEWARE: HEADER ECHO object HeaderEcho { def apply[F[_]: Functor,

    G[_]: Functor] (echoHeadersWhen: CaseInsensitiveString => Boolean) = Kleisli { req: Request[G] => val headersToEcho = req.headers.filter(h => echoHeadersWhen(h.name)) http(req).map(_.putHeaders(headersToEcho.toList: _*)) } } (http: Http[F, G]): Http[F, G]
  70. 124.

    CLIENT MIDDLEWARE: BASE URL object BaseUrl { def apply[F[_]](base: Uri)(

    client: Client[F] )(implicit F: Bracket[F, Throwable]): Client[F] = { Client.apply[F] { req => client.run(req.withUri(base.resolve(req.uri))) } } }
  71. 127.

    ATTRIBUTION natural power by Icon Island from the Noun Project

    testing by tom from the Noun Project modules by mikicon from the Noun Project
  72. 129.

    BONUS SLIDE: WEBSOCKETS case GET -> Root / "ws" /

    "echo" => Queue.bounded[IO, WebSocketFrame](100).flatMap { q => WebSocketBuilder[IO].build( send = q.dequeue, receive = q.enqueue ) }
  73. 130.

    BONUS SLIDE: WEBSOCKETS case GET -> Root / "ws" /

    "echo" => Queue.bounded[IO, WebSocketFrame](100).flatMap { q => WebSocketBuilder[IO].build( ) } send = q.dequeue, receive = q.enqueue