A server is just a function: introduction to http4s

If you're going to write a modern web service, you have a wide range of frameworks and libraries to choose from. One of them is http4s, a functional library for writing HTTP services and clients in Scala. I'm going to show you why you should consider it for your next application, for reasons including but not limited to:
- type-safe request/response models that are uniform between client and server
- resource-safe servers, response/request bodies, and streaming
- framework-agnostic testing.

At the end of this talk, you should have a clear understanding of how http4s can make it easier to build modern, runtime-safe services in Scala.

Transcript

1. A SERVER IS JUST A FUNCTION A N I N T R O D U C T I O N TO H T T P 4 S J A K U B KO Z Ł O W S K I
2. WHAT MAKES A SERVER?
3. WHAT MAKES A FUNCTION?
4. type Server = Request => Response? ?
6. WHAT REALLY MAKES A SERVER? Streaming Shared resources State Errors Routing Auth Serialization
7. HTTP4S Purely functional, streaming HTTP server/client https://http4s.org
8. HTTP4S Purely functional, streaming HTTP server/client Multiple backends https://http4s.org
9. HTTP4S Purely functional, streaming HTTP server/client Multiple backends Built on cats-effect and fs2 https://http4s.org
10. HTTP4S Purely functional, streaming HTTP server/client Multiple backends Built on cats-effect and fs2 Supports websockets (server) https://http4s.org
11. Function1[_, _]
12. Function1[_, _] + IO[_]
13. Function1[_, _] + IO[_]
14. HttpApp = Request => IO[Response] HttpRoutes = Request => IO[Option[Response]] HTTP4S SERVER: INTUITION
15. Kleisli
17. ❌
18. KLEISLI case class Kleisli[F[_], A, B]( run: A => F[B] )
19. KLEISLI case class Kleisli[F[_], A, B]( run: ) Function with effectful result A => F[B]
20. KLEISLI val len: Kleisli[IO, Token, Int] = Kleisli { token => IO.pure(token.length) } val cloned: Kleisli[IO, Token, String] = Kleisli { token => IO.pure(token + token) }
21. KLEISLI val f3 = for { a <- len b <- cloned } yield (a + b) val f4 = (len, cloned).tupled val len: Kleisli[IO, Token, Int] = Kleisli { token => IO.pure(token.length) } val cloned: Kleisli[IO, Token, String] = Kleisli { token => IO.pure(token + token) }
22. OptionT
23. OPTIONT case class OptionT[F[_], A]( value: F[Option[A]] )
24. OPTIONT Option nested in an effect case class OptionT[F[_], A]( value: ) F[Option[A]]
25. OPTIONT val num: OptionT[IO, Int] = OptionT(IO.pure(42.some)) def text(num: Int): OptionT[IO, String] = num match { case n if n % 2 === 0 => OptionT(IO.pure("foo".some)) case _ => OptionT.none }
26. OPTIONT val num: OptionT[IO, Int] = OptionT(IO.pure(42.some)) def text(num: Int): OptionT[IO, String] = num match { case n if n % 2 === 0 => OptionT(IO.pure("foo".some)) case _ => OptionT.none } val result = for { n <- num s <- text(n) s2 <- text(n + 1) s3 <- text(n + 2) } yield s + s2 + s3 val unwrapped: IO[Option[String]] = result.value
27. HttpApp = Kleisli[IO, Request, Response] HttpRoutes = Kleisli[OptionT[IO, ?], Request, Response] HTTP4S SERVER: THE REAL THING (ALMOST)
28. HTTP4S SERVER: THE REAL REAL THING HttpApp[F[_]] = Kleisli[F, Request[F], Response[F]] HttpRoutes[F[_]] = Kleisli[OptionT[F, ?], Request[F], Response[F]] Effect type of entity body stream
29. HTTP4S SERVER: THE REAL REAL THING Http[F[_], G[_]] = Kleisli[F, Request[G], Response[G]] HttpApp[F[_]] = Http[F, F] HttpRoutes[F[_]] = Http[OptionT[F, ?], F]
30. ALL THIS ABSTRACTION... WHY? (STAY TUNED) Spoiler: DRY
31. ROUTING DSL
32. object Main extends IOApp { val routes = HttpRoutes.of[IO] { case GET -> Root / "hello" => Ok("Hello, world!") } def run(args: List[String]): IO[ExitCode] = BlazeServerBuilder[IO] .withHttpApp(routes.orNotFound) .bindHttp(port = 8080) .resource .use(_ => IO.never) }
33. object Main extends IOApp { val routes = HttpRoutes.of[IO] { case GET -> Root / "hello" => Ok("Hello, world!") } def run(args: List[String]): IO[ExitCode] = BlazeServerBuilder[IO] .withHttpApp( ) .bindHttp(port = 8080) .resource .use(_ => IO.never) } routes.orNotFound HttpRoutes => HttpApp
34. object Main extends IOApp { val routes = HttpRoutes.of[IO] { case GET -> Root / "hello" => Ok("Hello, world!") } def run(args: List[String]): IO[ExitCode] = BlazeServerBuilder[IO] .withHttpApp(routes.orNotFound) .bindHttp(port = 8080) .resource .use(_ => IO.never) }
35. HttpRoutes.of[IO] { case GET -> Root / "hello" => Ok("Hello, world!") }
36. HttpRoutes.of[IO] { case -> Root / "hello" => Ok("Hello, world!") } GET
37. HttpRoutes.of[IO] { case GET -> => Ok("Hello, world!") } Root / "hello"
38. Ok("Hello, world!") HttpRoutes.of[IO] { case GET -> Root / "hello" => }
39. HttpRoutes.of[IO] { case GET -> Root / "hello" => Ok("Hello, world!") }
40. HttpRoutes.of[IO] { case request @ POST -> Root / "echo" => Ok(request.body) }
41. HttpRoutes.of[IO] { case request @ POST -> Root / "echo" => Ok( ) } request.body
42. HttpRoutes.of[IO] { case request @ POST -> Root / "echo" => Ok( ) } request.body Literally Stream[IO, Byte]
43. CLIENT
44. val request = Request[IO](uri = uri"https://http4s.org") val remoteCall: IO[String] = client.expect[String](request)
45. val request = Request[IO](uri = uri"https://http4s.org") val remoteCall: IO[String] = client.expect[String](request) remoteCall BlazeClientBuilder[IO](executionContext).resource.use { client => }
46. case GET -> Root / "remote" / "stream" => val response = client.stream(request).flatMap(_.body) Ok(response)
47. trait Client[F[_]] { def run(req: Request[F]): Resource[F, Response[F]] //+ a bunch of other convenient methods }
48. trait Client[F[_]] { def run(req: Request[F]): Resource[F, Response[F]] } def apply[F[_]](f: Request[F] => Resource[F, Response[F]] ): Client[F]
49. trait Client[F[_]] { def run(req: Request[F]): Resource[F, Response[F]] } def apply[F[_]](f: Request[F] => Resource[F, Response[F]] ): Client[F]
50. trait Client[F[_]] { def run(req: Request[F]): Resource[F, Response[F]] } def fromHttpApp[F[_]](app: HttpApp[F]): Client[F] def apply[F[_]](f: Request[F] => Resource[F, Response[F]] ): Client[F]
51. T E S T I N G R E S O U R C E S A F E T Y E X T E N S I B I L I T Y
52. T E S T I N G R E S O U R C E S A F E T Y E X T E N S I B I L I T Y
53. TESTING
54. TESTING Server routes
55. TESTING Server routes Server HTTP interface
56. TESTING Server routes Server HTTP interface Client calls
57. TESTING ROUTES def routes(client: Client[IO]): HttpRoutes[IO] = { HttpRoutes.of[IO] { //... case GET -> Root / "remote" => val remoteCall: IO[String] = client.expect[String](request) for { result <- remoteCall response <- Ok(result) } yield response //... } }
58. val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) TESTING ROUTES
59. val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) val routes = Main.routes(remoteClient) TESTING ROUTES
60. val body = Json.obj("foo" -> Json.fromString("bar")) val request = Request[IO](method = Method.POST, uri = uri"/echo").withEntity(body) val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) val routes = Main.routes(remoteClient) httsps://http4s.org TESTING ROUTES
61. routes.run(request).value.flatMap(_.value.as[Json]).map(_ shouldBe body) val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) val routes = Main.routes(remoteClient) val body = Json.obj("foo" -> Json.fromString("bar")) val request = Request[IO](method = Method.POST, uri = uri"/echo").withEntity(body) TESTING ROUTES
62. routes.run(request).value.flatMap(_.value.as[Json]).map(_ shouldBe body) val remoteClient = Client.fromHttpApp(HttpApp.notFound[IO]) val routes = Main.routes(remoteClient) val body = Json.obj("foo" -> Json.fromString("bar")) val request = Request[IO](method = Method.POST, uri = uri"/echo").withEntity(body) TESTING ROUTES Client.fromHttpApp(routes.orNotFound).expect[Json](request).map(_ shouldBe body) or
63. TESTING HTTP val blazeClient = BlazeClientBuilder[IO](ExecutionContext.global).resource
64. TESTING HTTP val blazeClient = BlazeClientBuilder[IO](ExecutionContext.global).resource Main.server.use { server => }
65. TESTING HTTP val blazeClient = BlazeClientBuilder[IO](ExecutionContext.global).resource Main.server.use { server => blazeClient.use { client => } }
66. TESTING HTTP val blazeClient = BlazeClientBuilder[IO](ExecutionContext.global).resource Main.server.use { server => blazeClient.use { client => } } client.expect[String](server.baseUri / "hello") .map(_ shouldBe "Hello world!")
67. TESTING CLIENTS class TodoClient(client: Client[IO]) { def getTodo(id: Int): IO[Todo] = client.expect[Todo](Request[IO](uri = uri"/todos" / id.toString)) }
68. TESTING CLIENTS class TodoClient(client: Client[IO]) { def getTodo(id: Int): IO[Todo] = client.expect[Todo](Request[IO](uri = uri"/todos" / id.toString)) } //in test val mockServer = HttpRoutes .of[IO] { case GET -> Root / "todos" / IntVar(id) => Ok(Todo(id, false)) } .orNotFound
69. TESTING CLIENTS val mockServer = HttpRoutes .of[IO] { case GET -> Root / "todos" / IntVar(id) => Ok(Todo(id, false)) } .orNotFound
70. TESTING CLIENTS val clientRaw = Client.fromHttpApp(mockServer) val todoClient = new TodoClient(clientRaw) val mockServer = HttpRoutes .of[IO] { case GET -> Root / "todos" / IntVar(id) => Ok(Todo(id, false)) } .orNotFound
71. TESTING CLIENTS val mockServer = HttpRoutes .of[IO] { case GET -> Root / "todos" / IntVar(id) => Ok(Todo(id, false)) } .orNotFound val clientRaw = Client.fromHttpApp(mockServer) val todoClient = new TodoClient(clientRaw) todoClient.getTodo(1).map(_ shouldBe Todo(1, false))
72. T E S T I N G R E S O U R C E S A F E T Y E X T E N S I B I L I T Y
73. T E S T I N G R E S O U R C E S A F E T Y E X T E N S I B I L I T Y
74. CATS.EFFECT.RESOURCE
75. RESOURCE abstract class Resource[F[_], A] { def use[B](f: A => F[B]): F[B] } object Resource { def make[F[_], A](acquire: F[A]) (release: A => F[Unit]): Resource[F, A] }
76. RESOURCE Resource.make(IO(createPool()))(pool => IO(pool.close()))
77. RESOURCE db <- Resource.make(IO(createPool()))(pool => IO(pool.close()))
78. RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool => IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds))
79. RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool => IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool
80. RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool => IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool 2. Start server
81. RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool => IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool 2. Start server 3. Sleep 10 seconds
82. RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool => IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool 2. Start server 3. Sleep 10 seconds 4. Stop server
83. RESOURCE val server = for { db <- Resource.make(IO(createPool()))(pool => IO(pool.close())) server <- BlazeServerBuilder[IO].withHttpApp(routes(db)) } yield server server.use(_ => IO.sleep(10.seconds)) 1. Create pool 2. Start server 3. Sleep 10 seconds 4. Stop server 5. Close pool
84. FS2.STREAM[F[_], A] - 0 to ∞ values - Can have effects in F - Supports resource acquisition/cleanup
85. FS2.STREAM[F[_], A] - 0 to ∞ values - Can have effects in F - Supports resource acquisition/cleanup Stream(1,2,3)
86. FS2.STREAM[F[_], A] - 0 to ∞ values - Can have effects in F - Supports resource acquisition/cleanup Stream(1,2,3) Stream.eval(IO(util.Random.nextInt))
87. FS2.STREAM[F[_], A] - 0 to ∞ values - Can have effects in F - Supports resource acquisition/cleanup Stream(1,2,3) Stream.eval(IO(util.Random.nextInt)) Stream.awakeDelay[IO](1.second)
88. FS2.STREAM[F[_], A] - 0 to ∞ values - Can have effects in F - Supports resource acquisition/cleanup Stream(1,2,3) Stream.eval(IO(util.Random.nextInt)) Stream.awakeDelay[IO](1.second).compile.toList: IO[List[FiniteDuration]]
89. FS2.STREAM[F[_], A] Stream(1,2,3) Stream.eval(IO(util.Random.nextInt)) Stream.awakeDelay[IO](1.second).compile.toList: IO[List[FiniteDuration]]
90. FS2.STREAM[F[_], A] Stream(1,2,3) Stream.eval(IO(util.Random.nextInt)) Stream.awakeDelay[IO](1.second).compile.toList: IO[List[FiniteDuration]] + Stream.bracket + Stream.resource
91. RESOURCE SAFETY IN HTTP4S
92. RESOURCE SAFETY IN HTTP4S
93. RESOURCE SAFETY IN HTTP4S BlazeServerBuilder[IO].resource: Resource[IO, Server[IO]]
94. RESOURCE SAFETY IN HTTP4S BlazeServerBuilder[IO].resource: Resource[IO, Server[IO]] BlazeClientBuilder[IO](ec).resource: Resource[IO, Client[IO]]
95. RESOURCE SAFETY IN HTTP4S BlazeServerBuilder[IO].resource: Resource[IO, Server[IO]] BlazeClientBuilder[IO](ec).resource: Resource[IO, Client[IO]] (request: Request[IO]).body: fs2.Stream[IO, Byte]
96. RESOURCE SAFETY IN HTTP4S BlazeServerBuilder[IO].resource: Resource[IO, Server[IO]] BlazeClientBuilder[IO](ec).resource: Resource[IO, Client[IO]] (request: Request[IO]).body: fs2.Stream[IO, Byte] (response: Response[IO]).body: fs2.Stream[IO, Byte]
97. RESOURCE SAFETY IN HTTP4S BlazeServerBuilder[IO].resource: Resource[IO, Server[IO]] BlazeClientBuilder[IO](ec).resource: Resource[IO, Client[IO]] (request: Request[IO]).body: fs2.Stream[IO, Byte] (response: Response[IO]).body: fs2.Stream[IO, Byte] ...and then there's Client
98. trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def stream(req: Request[F]): Stream[F, Response[F]] }
99. trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def stream(req: Request[F]): Stream[F, Response[F]] def fetch[A](req: Request[F])(f: Response[F] => F[A]) : F[A] def expect[A](req: Request[F])(implicit d: EntityDecoder[F, A]): F[A] } (these have like 10 overloads*) *some are about to be removed
100. trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def stream(req: Request[F]): Stream[F, Response[F]] def fetch[A](req: Request[F])(f: Response[F] => F[A]) : F[A] def expect[A](req: Request[F])(implicit d: EntityDecoder[F, A]): F[A] def status(req: Request[F]). : F[Status] def successful(req: Request[F]): F[Boolean] }
101. trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def stream(req: Request[F]): Stream[F, Response[F]] def fetch[A](req: Request[F])(f: Response[F] => F[A]) : F[A] def expect[A](req: Request[F])(implicit d: EntityDecoder[F, A]): F[A] def status(req: Request[F]). : F[Status] def successful(req: Request[F]): F[Boolean] def toKleisli[A](f: Response[F] => F[A]): Kleisli[F, Request[F], A] //... }
102. trait Client[F[_]] { def run(req: Request[F]) : Resource[F, Response[F]] def stream(req: Request[F]): Stream[F, Response[F]] def fetch[A](req: Request[F])(f: Response[F] => F[A]) : F[A] def expect[A](req: Request[F])(implicit d: EntityDecoder[F, A]): F[A] def status(req: Request[F]). : F[Status] def successful(req: Request[F]): F[Boolean] def toKleisli[A](f: Response[F] => F[A]): Kleisli[F, Request[F], A] //... //here be dragons - don't use unless you know what you're doing def toHttpApp: HttpApp[F] }
103. T E S T I N G R E S O U R C E S A F E T Y E X T E N S I B I L I T Y
104. T E S T I N G R E S O U R C E S A F E T Y E X T E N S I B I L I T Y
105. EXTENSIBILITY
106. EXTENSIBILITY If a server/client is still a function...
107. EXTENSIBILITY If a server/client is still a function... what can we do with functions?
108. case class Data(x: Int) val f: Data => Int = _.x
109. case class Data(x: Int) val f: Data => Int = _.x def wrapF(f1: Data => Int): Data => Int = data => f1(data.copy(x = data.x + 1)) * 2
110. case class Data(x: Int) val f: Data => Int = _.x def wrapF(f1: Data => Int): Data => Int = data => f1(data.copy(x = data.x + 1)) * 2 val f2: Data => Int = wrapF(f)
111. case class Data(x: Int) val f: Data => Int = _.x val f2: Data => Int = wrapF(f) def wrapF(f1: Data => Int): Data => Int = data => f1( ) * 2 data.copy(x = data.x + 1)
115. HttpApp[F[_]] = Kleisli[F, Request[F], Response[F]]
116. SERVER MIDDLEWARE http: HttpApp[F]
117. SERVER MIDDLEWARE def apply[F[_]]( http: HttpApp[F]): HttpApp[F]
118. SERVER MIDDLEWARE: RESPONSE TIMING object ResponseTiming { def apply[F[_]]( http: HttpApp[F], timeUnit: TimeUnit = MILLISECONDS, headerName: CaseInsensitiveString = CaseInsensitiveString("X-Response-Time"))( implicit F: Sync[F], clock: Clock[F]): HttpApp[F] = Kleisli { req => for { before <- clock.monotonic(timeUnit) resp <- http(req) after <- clock.monotonic(timeUnit) header = Header(headerName.value, s"${after - before}") } yield resp.putHeaders(header) } }
119. SERVER MIDDLEWARE: RESPONSE TIMING object ResponseTiming { def apply[F[_]]( http: HttpApp[F], timeUnit: TimeUnit = MILLISECONDS, headerName: CaseInsensitiveString = CaseInsensitiveString("X-Response-Time"))( implicit F: Sync[F], clock: Clock[F]): HttpApp[F] = Kleisli { req => for { before <- clock.monotonic(timeUnit) after <- clock.monotonic(timeUnit) header = Header(headerName.value, s"${after - before}") } yield } } Kleisli { req => resp <- http(req) resp.putHeaders(header) }
120. SERVER MIDDLEWARE: HEADER ECHO object HeaderEcho { def apply[F[_]: Functor, G[_]: Functor] (echoHeadersWhen: CaseInsensitiveString => Boolean) (http: Http[F, G]): Http[F, G] = Kleisli { req: Request[G] => val headersToEcho = req.headers.filter(h => echoHeadersWhen(h.name)) http(req).map(_.putHeaders(headersToEcho.toList: _*)) } }
121. SERVER MIDDLEWARE: HEADER ECHO object HeaderEcho { def apply[F[_]: Functor, G[_]: Functor] (echoHeadersWhen: CaseInsensitiveString => Boolean) = Kleisli { req: Request[G] => val headersToEcho = req.headers.filter(h => echoHeadersWhen(h.name)) http(req).map(_.putHeaders(headersToEcho.toList: _*)) } } (http: Http[F, G]): Http[F, G]
122. CLIENT MIDDLEWARE client: Client[F]
123. CLIENT MIDDLEWARE def apply[F[_]]( client: Client[F] ): Client[F]
124. CLIENT MIDDLEWARE: BASE URL object BaseUrl { def apply[F[_]](base: Uri)( client: Client[F] )(implicit F: Bracket[F, Throwable]): Client[F] = { Client.apply[F] { req => client.run(req.withUri(base.resolve(req.uri))) } } }
125. SERVER CLIENT
126. LEARN MORE http4s.org gitter.im/http4s/http4s gitter.im/typelevel/cats-effect
127. ATTRIBUTION natural power by Icon Island from the Noun Project testing by tom from the Noun Project modules by mikicon from the Noun Project
128. THANK YOU Slides: bit.ly/2WaFCOM Code: git.io/fjs4v Twitter: @kubukoz My blog: blog.kubukoz.com
129. BONUS SLIDE: WEBSOCKETS case GET -> Root / "ws" / "echo" => Queue.bounded[IO, WebSocketFrame](100).flatMap { q => WebSocketBuilder[IO].build( send = q.dequeue, receive = q.enqueue ) }
130. BONUS SLIDE: WEBSOCKETS case GET -> Root / "ws" / "echo" => Queue.bounded[IO, WebSocketFrame](100).flatMap { q => WebSocketBuilder[IO].build( ) } send = q.dequeue, receive = q.enqueue
131. BONUS SLIDE: SWAGGER (BECAUSE SOMEONE ALWAYS ASKS)
132. BONUS SLIDE: SWAGGER (BECAUSE SOMEONE ALWAYS ASKS)