Building trait-based web services with Udash REST

Transcript

1. Udash REST by Roman Janusz, AVSystem, Kraków Scalar Conference, Warsaw, 06.04.2019
3. Udash Framework • Frontend and backend in the same language: Scala ◦ ScalaJS ◦ Scalatags & ScalaCSS by Li Haoyi ◦ UI: Routing, property binding, internationalization • Type safety continuous throughout the entire system ◦ Trait based client-server RPC ◦ Opaque-ish protocol • RPC based REST ◦ Well defined HTTP traffic ◦ Consumable by other systems in different languages
4. RPC vs REST
5. RPC vs REST
6. import io.udash.rest._ trait UserApi { def createUser(name: String): Future[User] } object UserApi extends DefaultRestApiCompanion[UserApi] REST API trait definition
7. import io.udash.rest._ trait UserApi { def createUser(name: String): Future[User] } object UserApi extends DefaultRestApiCompanion[UserApi] case class UserId(id: Long) extends AnyVal object UserId extends RestDataWrapperCompanion[Long, UserId] case class User(id: UserId, name: String) object User extends RestDataCompanion[User] Data types
8. Server class UserApiImpl extends UserApi { def createUser(name: String): Future[User] = ??? }
9. Server class UserApiImpl extends UserApi { def createUser(name: String): Future[User] = ??? } val serverHandle: HandleRequest = UserApi.asHandleRequest(new UserApiImpl)
10. Request handler function class UserApiImpl extends UserApi { def createUser(name: String): Future[User] = ??? } val serverHandle: HandleRequest = UserApi.asHandleRequest(new UserApiImpl) type HandleRequest = RestRequest => Async[RestResponse] type Callback[T] = Try[T] => Unit type Async[T] = Callback[T] => Unit
11. Server class UserApiImpl extends UserApi { def createUser(name: String): Future[User] = ??? } val serverHandle: HandleRequest = UserApi.asHandleRequest(new UserApiImpl) val servlet: javax.servlet.Servlet = new RestServlet(serverHandle)
12. Client implicit val sttpBackend: SttpBackend[Future, Nothing] = SttpRestClient.defaultBackend()
13. Client implicit val sttpBackend: SttpBackend[Future, Nothing] = SttpRestClient.defaultBackend() val clientHandle: HandleRequest = SttpRestClient.asHandleRequest("https://www.userapi.com")
14. Client implicit val sttpBackend: SttpBackend[Future, Nothing] = SttpRestClient.defaultBackend() val clientHandle: HandleRequest = SttpRestClient.asHandleRequest("https://www.userapi.com") val client: UserApi = UserApi.fromHandleRequest(clientHandle)
15. Client implicit val sttpBackend: SttpBackend[Future, Nothing] = SttpRestClient.defaultBackend() val clientHandle: HandleRequest = SttpRestClient.asHandleRequest("https://www.userapi.com") val client: UserApi = UserApi.fromHandleRequest(clientHandle) client.createUser("Fred").onComplete(...)
16. On the wire POST /createUser HTTP/1.1 Content-Type: application/json;charset=utf-8 content-length: 15 host: localhost:9090 {"name":"Fred"} def createUser(name: String): Future[User]
17. On the wire POST /createUser HTTP/1.1 Content-Type: application/json;charset=utf-8 content-length: 15 host: localhost:9090 {"name":"Fred"} def createUser(name: String): Future[User] HTTP/1.1 200 OK Content-Type: application/json;charset=utf-8 Content-Length: 22 {"id":0,"name":"Fred"}
18. import io.udash.rest._ trait UserApi { def createUser(name: String): Future[User] } object UserApi extends DefaultRestApiCompanion[UserApi] case class UserId(id: Long) extends AnyVal object UserId extends RestDataWrapperCompanion[Long, UserId] case class User(id: UserId, name: String) object User extends RestDataCompanion[User] How does it work?
19. import io.udash.rest._ trait UserApi { def createUser(name: String): Future[User] } object UserApi extends DefaultRestApiCompanion[UserApi](ǽƝǗǠƹ) case class UserId(id: Long) extends AnyVal object UserId extends RestDataWrapperCompanion[Long, UserId](ǽƝǗǠƹ) case class User(id: UserId, name: String) object User extends RestDataCompanion[User](ǽƝǗǠƹ) How does it work? Companions with macros!
20. Customizing mapping to HTTP with annotations • choosing HTTP method: @GET , @POST , @PUT , @PATCH , @DELETE
21. Customizing mapping to HTTP with annotations • choosing HTTP method: @GET , @POST , @PUT , @PATCH , @DELETE • customizing paths: @GET("api/user")
22. Customizing mapping to HTTP with annotations • choosing HTTP method: @GET , @POST , @PUT , @PATCH , @DELETE • customizing paths: @GET("api/user") • choosing parameter flavors and names: @Path , @Header("X-Header") , @Query , @Cookie , @Body
23. Customizing mapping to HTTP with annotations • choosing HTTP method: @GET , @POST , @PUT , @PATCH , @DELETE • customizing paths: @GET("api/user") • choosing parameter flavors and names: @Path , @Header("X-Header") , @Query , @Cookie , @Body • choosing body format: @JsonBody , @FormBody , @CustomBody
24. We’ve seen that already... Various Java frameworks, e.g. JAX-RS based
25. We’ve seen that already... Various Java frameworks, e.g. JAX-RS based Problems: • runtime reflection, runtime proxies, Java serialization • annotations form a severely limited, dynamic, unsafe, untyped pseudolanguage • hard to understand and debug • usually blocking, difficult to work with async
26. We write Scala, we can do better! • compile time reflection with macros
27. We write Scala, we can do better! • compile time reflection with macros • interface composition
28. We write Scala, we can do better! • compile time reflection with macros • interface composition • typeclass based serialization
29. We write Scala, we can do better! • compile time reflection with macros • interface composition • typeclass based serialization • effects for asynchronous & purely functional computations
30. We write Scala, we can do better! • compile time reflection with macros • interface composition • typeclass based serialization • effects for asynchronous & purely functional computations • platform independence
31. We write Scala, we can do better! • compile time reflection with macros • interface composition • typeclass based serialization • effects for asynchronous & purely functional computations • platform independence • readable, precise error messages
32. We write Scala, we can do better! • compile time reflection with macros • interface composition • typeclass based serialization • effects for asynchronous & purely functional computations • platform independence • readable, precise error messages • vastly better annotation processing
33. Interface composition: Prefix methods trait SystemApi { def users: UserApi def resources: ResourceApi } object SystemApi extends DefaultRestApiCompanion[SystemApi]
34. • you can organize, split and compose your APIs Interface composition: Prefix methods trait SystemApi { def users: UserApi def resources: ResourceApi } object SystemApi extends DefaultRestApiCompanion[SystemApi]
35. • you can organize, split and compose your APIs • you can capture common path fragment and parameters Interface composition: Prefix methods trait SystemApi { def users: UserApi def resources: ResourceApi } object SystemApi extends DefaultRestApiCompanion[SystemApi]
36. Prefix methods: authentication trait SeriousBusinessApi { … } object SeriousBusinessApi extends DefaultRestApiCompanion[SeriousBusinessApi]
37. Prefix methods: authentication trait SeriousBusinessApi { … } object SeriousBusinessApi extends DefaultRestApiCompanion[SeriousBusinessApi] trait AuthApi { @Prefix("") def auth( @Header("Authorization") authToken: String ): SeriousBusinessApi } object AuthApi extends DefaultRestApiCompanion[AuthApi]
38. OpenAPI • open standard for describing REST endpoints with JSON or YAML • interactive documentation for free (e.g. Swagger UI) • client & server stub code generation for various languages (e.g. Swagger Codegen)
39. OpenAPI • open standard for describing REST endpoints with JSON or YAML • interactive documentation for free (e.g. Swagger UI) • client & server stub code generation for various languages (e.g. Swagger Codegen) import io.udash.rest.openapi._ val userOpenApi: OpenApi = UserApi.openapiMetadata.openapi( Info("User API", "0.1", description = "User management API"), servers = List(Server("http://www.userapi.com")) )
40. OpenAPI • open standard for describing REST endpoints with JSON or YAML • interactive documentation for free (e.g. Swagger UI) • client & server stub code generation for various languages (e.g. Swagger Codegen) import io.udash.rest.openapi._ import com.avsystem.commons.serialization.json._ val userOpenApi: OpenApi = UserApi.openapiMetadata.openapi( Info("User API", "0.1", description = "User management API"), servers = List(Server("http://www.userapi.com")) ) println(JsonStringOutput.write(userOpenApi))
41. OpenAPI customization trait UserApi { @summary("Creates a user with given name") def createUser(name: String): Future[User] @summary("Fetches a user by ID") @GET def getUser(id: UserId): Future[User] } object UserApi extends DefaultRestApiCompanion[UserApi]
42. OpenAPI customization @description("Representation of system user") @example(User(UserId(0), "Fred")) case class User( id: UserId, @description("User name") name: String ) object User extends RestDataCompanion[User]
43. Swagger UI
44. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId, name: Option[String] = None ): Future[User] }
45. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId, name: Option[String] = None ): Future[User] } AsRaw[PlainValue, UserId] (client) AsReal[PlainValue, UserId] (server)
46. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId, name: Option[String] = None ): Future[User] } AsRaw[PlainValue, UserId] (client) AsReal[PlainValue, UserId] (server) AsRaw[JsonValue, Option[String]] (client) AsReal[JsonValue, Option[String]] (server)
47. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId, name: Option[String] = None ): Future[User] } AsRaw[PlainValue, UserId] (client) AsReal[PlainValue, UserId] (server) AsRaw[JsonValue, Option[String]] (client) AsReal[JsonValue, Option[String]] (server) AsReal[RestResponse, User] (client) AsRaw[RestResponse, User] (server)
48. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId, name: Option[String] = None ): Future[User] } AsRaw[PlainValue, UserId] (client) AsReal[PlainValue, UserId] (server) AsRaw[JsonValue, Option[String]] (client) AsReal[JsonValue, Option[String]] (server) AsReal[RestResponse, User] (client) AsRaw[RestResponse, User] (server) AsyncEffect[Future] (client & server)
49. • You must plug in a typeclass-based serialization library to provide AsReal and AsRaw instances Serialization
50. • You must plug in a typeclass-based serialization library to provide AsReal and AsRaw instances • e.g. AsRaw for JsonValue could be defined based on Circe Encoder • and AsReal for JsonValue could be defined based on Circe Decoder Serialization
51. • You must plug in a typeclass-based serialization library to provide AsReal and AsRaw instances • e.g. AsRaw for JsonValue could be defined based on Circe Encoder • and AsReal for JsonValue could be defined based on Circe Decoder • Bake it all into the base companion class! Serialization
52. • You must plug in a typeclass-based serialization library to provide AsReal and AsRaw instances • e.g. AsRaw for JsonValue could be defined based on Circe Encoder • and AsReal for JsonValue could be defined based on Circe Decoder • Bake it all into the base companion class! • DefaultRestApiCompanion injects GenCodec based serialization (AVSystem’s own serialization library) Serialization
53. • You must plug in a typeclass-based serialization library to provide AsReal and AsRaw instances • e.g. AsRaw for JsonValue could be defined based on Circe Encoder • and AsReal for JsonValue could be defined based on Circe Decoder • Bake it all into the base companion class! • DefaultRestApiCompanion injects GenCodec based serialization (AVSystem’s own serialization library) • It’s easy to make your own version of base companion class, tailored for your needs, e.g. CirceRestApiCompanion to use Circe for serialization Serialization
54. Asynchronicity and effects type Callback[T] = Try[T] => Unit type Async[T] = Callback[T] => Unit trait AsyncEffect[F[_]] { def toAsync[A](fa: F[A]): Async[A] def fromAsync[A](async: Async[A]): F[A] }
55. Asynchronicity and effects type Callback[T] = Try[T] => Unit type Async[T] = Callback[T] => Unit trait AsyncEffect[F[_]] { def toAsync[A](fa: F[A]): Async[A] def fromAsync[A](async: Async[A]): F[A] } • not only Future s but any effects which capture asynchronous, possibly lazy, repeatable computation - Monix Task , Cats IO , ZIO , etc.
56. Asynchronicity and effects type Callback[T] = Try[T] => Unit type Async[T] = Callback[T] => Unit trait AsyncEffect[F[_]] { def toAsync[A](fa: F[A]): Async[A] def fromAsync[A](async: Async[A]): F[A] } • not only Future s but any effects which capture asynchronous, possibly lazy, repeatable computation - Monix Task , Cats IO , ZIO , etc. • Future support is injected by DefaultRestApiCompanion , you can opt out
57. Effect-polymorphic APIs trait UserApi[F[_]] { def createUser(name: String): F[User] @GET def getUser(id: UserId): F[User] } object UserApi extends DefaultPolyRestApiCompanion[UserApi] Tagless final?
58. Compilation errors Serializer is missing because you forgot to declare companion for User ? @PUT @CustomBody def replaceUser(user: User): Future[Unit]
59. Compilation errors Serializer is missing because you forgot to declare companion for User ? [error] problem with parameter user of method replaceUser: [error] Cannot deserialize User from HttpBody, because: [error] Cannot deserialize User from JsonValue, because: [error] No GenCodec found for User [error] @PUT @CustomBody def replaceUser(user: User): Future[Unit] [error] ^ @PUT @CustomBody def replaceUser(user: User): Future[Unit]
60. Implicit trouble @implicitNotFound("${T} cannot be serialized") trait Codec[T] // [error] Int cannot be serialized implicitly[Codec[Int]]
61. Implicit trouble @implicitNotFound("LibCodec not found for ${T}") trait LibCodec[T] @implicitNotFound("${T} cannot be serialized") trait Codec[T] object Codec { implicit def fromLibCodec[T]( implicit libCodec: LibCodec[T] ): Codec[T] = ??? } // [error] Int cannot be serialized implicitly[Codec[Int]]
62. Extending @implicitNotFound @implicitNotFound("${T} cannot be serialized") trait Codec[T] object Codec { implicit def fromLibCodec[T]( implicit libCodec: LibCodec[T] ): Codec[T] = ??? }
63. Composable error messages @implicitNotFound("${T} cannot be serialized") trait Codec[T] object Codec { implicit def fromLibCodec[T]( implicit libCodec: LibCodec[T] ): Codec[T] = ??? @implicitNotFound("${T} cannot be serialized because: #{forLibCodec}") implicit def whenLibCodecNotFound[T]( implicit forLibCodec: ImplicitNotFound[LibCodec[T]] ): ImplicitNotFound[Codec[T]] = ImplicitNotFound() }
64. Composable error messages // [error] Int cannot be serialized because: LibCodec not found for Int Implicits.infer[Codec[Int]] • requires custom Implicits.infer macro • is used automatically by macro-originated implicit searches • extensible
65. Annotation processing in Scala • annotations are full blown classes
66. Annotation processing in Scala • annotations are full blown classes • can be accessed in macros
67. Annotation processing in Scala • annotations are full blown classes • can be accessed in macros as ASTs • we can implement our own annotation processing
68. Annotation processing in Scala • annotations are full blown classes • can be accessed in macros as ASTs • we can implement our own annotation processing • selective reification into runtime
69. Annotation processing in Scala • annotations are full blown classes • can be accessed in macros as ASTs • we can implement our own annotation processing • selective reification into runtime • aggregation - Don’t Repeat Yourself for annotations
70. Annotation extensibility trait UserApi { @summary("Creates a user with given name") def createUser(name: String): Future[User] }
71. Annotations are just classes trait UserApi { @summary("Creates a user with given name") def createUser(name: String): Future[User] } class summary(summary: String) extends OperationAdjuster { def adjustOperation(operation: Operation): Operation = operation.copy(summary = summary) }
72. Annotations are just classes trait UserApi { @summary("Creates a user with given name") def createUser(name: String): Future[User] } class summary(summary: String) extends OperationAdjuster { def adjustOperation(operation: Operation): Operation = operation.copy(summary = summary) } Make your own annotations!
73. DRY? trait MyApi { @PUT("users") @CustomBody @summary("Updates a user") def updateUser(user: User): Future[Unit] @PUT("groups") @CustomBody @summary("Updates a group") def updateGroup(group: Group): Future[Unit] @PUT("resources") @CustomBody @summary("Updates a resource") def updateResource(resource: Resource): Future[Unit] } We need a desiccant!
74. Yo dawg, I herd you like annotations class EntityPUT(what: String) extends AnnotationAggregate { @PUT(s"${what}s") @CustomBody @summary(s"Updates a $what") type Implied } trait MyApi { @EntityPUT("user") def updateUser(user: User): Future[Unit] @EntityPUT("group") def updateGroup(group: Group): Future[Unit] @EntityPUT("resource") def updateResource(resource: Resource): Future[Unit] }
75. Wrapping up Udash REST • use plain Scala traits and data types to define endpoints • type safety on many levels • pluggable native HTTP servers and clients • pluggable, extensible serialization • pluggable, extensible effects • superb error messages, also extensible • annotations on steroids • built using a powerful, generic metaprogramming engine
76. Bonus: the metaprogramming engine below • typeclass & schema derivation • arbitrarily designed schema - single macro
77. Bonus: the metaprogramming engine below • typeclass & schema derivation • arbitrarily designed schema - single macro case class RecordInfo[T]( @multi @adtParamMetadata paramInfos: List[ParamInfo[_]] ) extends TypedMetadata[T] object RecordInfo extends AdtMetadataCompanion[RecordInfo] case class ParamInfo[T]( @reifyName name: String, @optional @reifyAnnot nameAnnot: Option[name], @infer typeclass: Typeclass[T] ) extends TypedMetadata[T]