Save 37% off PRO during our Black Friday Sale! »

Building trait-based web services with Udash REST

Building trait-based web services with Udash REST

6c69347d855193796ad252188b22ff41?s=128

Roman Janusz

April 06, 2019
Tweet

Transcript

  1. Udash REST by Roman Janusz, AVSystem, Kraków Scalar Conference, Warsaw,

    06.04.2019
  2. None
  3. Udash Framework • Frontend and backend in the same language:

    Scala ◦ ScalaJS ◦ Scalatags & ScalaCSS by Li Haoyi ◦ UI: Routing, property binding, internationalization • Type safety continuous throughout the entire system ◦ Trait based client-server RPC ◦ Opaque-ish protocol • RPC based REST ◦ Well defined HTTP traffic ◦ Consumable by other systems in different languages
  4. RPC vs REST

  5. RPC vs REST

  6. import io.udash.rest._ trait UserApi { def createUser(name: String): Future[User] }

    object UserApi extends DefaultRestApiCompanion[UserApi] REST API trait definition
  7. import io.udash.rest._ trait UserApi { def createUser(name: String): Future[User] }

    object UserApi extends DefaultRestApiCompanion[UserApi] case class UserId(id: Long) extends AnyVal object UserId extends RestDataWrapperCompanion[Long, UserId] case class User(id: UserId, name: String) object User extends RestDataCompanion[User] Data types
  8. Server class UserApiImpl extends UserApi { def createUser(name: String): Future[User]

    = ??? }
  9. Server class UserApiImpl extends UserApi { def createUser(name: String): Future[User]

    = ??? } val serverHandle: HandleRequest = UserApi.asHandleRequest(new UserApiImpl)
  10. Request handler function class UserApiImpl extends UserApi { def createUser(name:

    String): Future[User] = ??? } val serverHandle: HandleRequest = UserApi.asHandleRequest(new UserApiImpl) type HandleRequest = RestRequest => Async[RestResponse] type Callback[T] = Try[T] => Unit type Async[T] = Callback[T] => Unit
  11. Server class UserApiImpl extends UserApi { def createUser(name: String): Future[User]

    = ??? } val serverHandle: HandleRequest = UserApi.asHandleRequest(new UserApiImpl) val servlet: javax.servlet.Servlet = new RestServlet(serverHandle)
  12. Client implicit val sttpBackend: SttpBackend[Future, Nothing] = SttpRestClient.defaultBackend()

  13. Client implicit val sttpBackend: SttpBackend[Future, Nothing] = SttpRestClient.defaultBackend() val clientHandle:

    HandleRequest = SttpRestClient.asHandleRequest("https://www.userapi.com")
  14. Client implicit val sttpBackend: SttpBackend[Future, Nothing] = SttpRestClient.defaultBackend() val clientHandle:

    HandleRequest = SttpRestClient.asHandleRequest("https://www.userapi.com") val client: UserApi = UserApi.fromHandleRequest(clientHandle)
  15. Client implicit val sttpBackend: SttpBackend[Future, Nothing] = SttpRestClient.defaultBackend() val clientHandle:

    HandleRequest = SttpRestClient.asHandleRequest("https://www.userapi.com") val client: UserApi = UserApi.fromHandleRequest(clientHandle) client.createUser("Fred").onComplete(...)
  16. On the wire POST /createUser HTTP/1.1 Content-Type: application/json;charset=utf-8 content-length: 15

    host: localhost:9090 {"name":"Fred"} def createUser(name: String): Future[User]
  17. On the wire POST /createUser HTTP/1.1 Content-Type: application/json;charset=utf-8 content-length: 15

    host: localhost:9090 {"name":"Fred"} def createUser(name: String): Future[User] HTTP/1.1 200 OK Content-Type: application/json;charset=utf-8 Content-Length: 22 {"id":0,"name":"Fred"}
  18. import io.udash.rest._ trait UserApi { def createUser(name: String): Future[User] }

    object UserApi extends DefaultRestApiCompanion[UserApi] case class UserId(id: Long) extends AnyVal object UserId extends RestDataWrapperCompanion[Long, UserId] case class User(id: UserId, name: String) object User extends RestDataCompanion[User] How does it work?
  19. import io.udash.rest._ trait UserApi { def createUser(name: String): Future[User] }

    object UserApi extends DefaultRestApiCompanion[UserApi](ǽƝǗǠƹ) case class UserId(id: Long) extends AnyVal object UserId extends RestDataWrapperCompanion[Long, UserId](ǽƝǗǠƹ) case class User(id: UserId, name: String) object User extends RestDataCompanion[User](ǽƝǗǠƹ) How does it work? Companions with macros!
  20. Customizing mapping to HTTP with annotations • choosing HTTP method:

    @GET , @POST , @PUT , @PATCH , @DELETE
  21. Customizing mapping to HTTP with annotations • choosing HTTP method:

    @GET , @POST , @PUT , @PATCH , @DELETE • customizing paths: @GET("api/user")
  22. Customizing mapping to HTTP with annotations • choosing HTTP method:

    @GET , @POST , @PUT , @PATCH , @DELETE • customizing paths: @GET("api/user") • choosing parameter flavors and names: @Path , @Header("X-Header") , @Query , @Cookie , @Body
  23. Customizing mapping to HTTP with annotations • choosing HTTP method:

    @GET , @POST , @PUT , @PATCH , @DELETE • customizing paths: @GET("api/user") • choosing parameter flavors and names: @Path , @Header("X-Header") , @Query , @Cookie , @Body • choosing body format: @JsonBody , @FormBody , @CustomBody
  24. We’ve seen that already... Various Java frameworks, e.g. JAX-RS based

  25. We’ve seen that already... Various Java frameworks, e.g. JAX-RS based

    Problems: • runtime reflection, runtime proxies, Java serialization • annotations form a severely limited, dynamic, unsafe, untyped pseudolanguage • hard to understand and debug • usually blocking, difficult to work with async
  26. We write Scala, we can do better! • compile time

    reflection with macros
  27. We write Scala, we can do better! • compile time

    reflection with macros • interface composition
  28. We write Scala, we can do better! • compile time

    reflection with macros • interface composition • typeclass based serialization
  29. We write Scala, we can do better! • compile time

    reflection with macros • interface composition • typeclass based serialization • effects for asynchronous & purely functional computations
  30. We write Scala, we can do better! • compile time

    reflection with macros • interface composition • typeclass based serialization • effects for asynchronous & purely functional computations • platform independence
  31. We write Scala, we can do better! • compile time

    reflection with macros • interface composition • typeclass based serialization • effects for asynchronous & purely functional computations • platform independence • readable, precise error messages
  32. We write Scala, we can do better! • compile time

    reflection with macros • interface composition • typeclass based serialization • effects for asynchronous & purely functional computations • platform independence • readable, precise error messages • vastly better annotation processing
  33. Interface composition: Prefix methods trait SystemApi { def users: UserApi

    def resources: ResourceApi } object SystemApi extends DefaultRestApiCompanion[SystemApi]
  34. • you can organize, split and compose your APIs Interface

    composition: Prefix methods trait SystemApi { def users: UserApi def resources: ResourceApi } object SystemApi extends DefaultRestApiCompanion[SystemApi]
  35. • you can organize, split and compose your APIs •

    you can capture common path fragment and parameters Interface composition: Prefix methods trait SystemApi { def users: UserApi def resources: ResourceApi } object SystemApi extends DefaultRestApiCompanion[SystemApi]
  36. Prefix methods: authentication trait SeriousBusinessApi { … } object SeriousBusinessApi

    extends DefaultRestApiCompanion[SeriousBusinessApi]
  37. Prefix methods: authentication trait SeriousBusinessApi { … } object SeriousBusinessApi

    extends DefaultRestApiCompanion[SeriousBusinessApi] trait AuthApi { @Prefix("") def auth( @Header("Authorization") authToken: String ): SeriousBusinessApi } object AuthApi extends DefaultRestApiCompanion[AuthApi]
  38. OpenAPI • open standard for describing REST endpoints with JSON

    or YAML • interactive documentation for free (e.g. Swagger UI) • client & server stub code generation for various languages (e.g. Swagger Codegen)
  39. OpenAPI • open standard for describing REST endpoints with JSON

    or YAML • interactive documentation for free (e.g. Swagger UI) • client & server stub code generation for various languages (e.g. Swagger Codegen) import io.udash.rest.openapi._ val userOpenApi: OpenApi = UserApi.openapiMetadata.openapi( Info("User API", "0.1", description = "User management API"), servers = List(Server("http://www.userapi.com")) )
  40. OpenAPI • open standard for describing REST endpoints with JSON

    or YAML • interactive documentation for free (e.g. Swagger UI) • client & server stub code generation for various languages (e.g. Swagger Codegen) import io.udash.rest.openapi._ import com.avsystem.commons.serialization.json._ val userOpenApi: OpenApi = UserApi.openapiMetadata.openapi( Info("User API", "0.1", description = "User management API"), servers = List(Server("http://www.userapi.com")) ) println(JsonStringOutput.write(userOpenApi))
  41. OpenAPI customization trait UserApi { @summary("Creates a user with given

    name") def createUser(name: String): Future[User] @summary("Fetches a user by ID") @GET def getUser(id: UserId): Future[User] } object UserApi extends DefaultRestApiCompanion[UserApi]
  42. OpenAPI customization @description("Representation of system user") @example(User(UserId(0), "Fred")) case class

    User( id: UserId, @description("User name") name: String ) object User extends RestDataCompanion[User]
  43. Swagger UI

  44. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId,

    name: Option[String] = None ): Future[User] }
  45. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId,

    name: Option[String] = None ): Future[User] } AsRaw[PlainValue, UserId] (client) AsReal[PlainValue, UserId] (server)
  46. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId,

    name: Option[String] = None ): Future[User] } AsRaw[PlainValue, UserId] (client) AsReal[PlainValue, UserId] (server) AsRaw[JsonValue, Option[String]] (client) AsReal[JsonValue, Option[String]] (server)
  47. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId,

    name: Option[String] = None ): Future[User] } AsRaw[PlainValue, UserId] (client) AsReal[PlainValue, UserId] (server) AsRaw[JsonValue, Option[String]] (client) AsReal[JsonValue, Option[String]] (server) AsReal[RestResponse, User] (client) AsRaw[RestResponse, User] (server)
  48. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId,

    name: Option[String] = None ): Future[User] } AsRaw[PlainValue, UserId] (client) AsReal[PlainValue, UserId] (server) AsRaw[JsonValue, Option[String]] (client) AsReal[JsonValue, Option[String]] (server) AsReal[RestResponse, User] (client) AsRaw[RestResponse, User] (server) AsyncEffect[Future] (client & server)
  49. • You must plug in a typeclass-based serialization library to

    provide AsReal and AsRaw instances Serialization
  50. • You must plug in a typeclass-based serialization library to

    provide AsReal and AsRaw instances • e.g. AsRaw for JsonValue could be defined based on Circe Encoder • and AsReal for JsonValue could be defined based on Circe Decoder Serialization
  51. • You must plug in a typeclass-based serialization library to

    provide AsReal and AsRaw instances • e.g. AsRaw for JsonValue could be defined based on Circe Encoder • and AsReal for JsonValue could be defined based on Circe Decoder • Bake it all into the base companion class! Serialization
  52. • You must plug in a typeclass-based serialization library to

    provide AsReal and AsRaw instances • e.g. AsRaw for JsonValue could be defined based on Circe Encoder • and AsReal for JsonValue could be defined based on Circe Decoder • Bake it all into the base companion class! • DefaultRestApiCompanion injects GenCodec based serialization (AVSystem’s own serialization library) Serialization
  53. • You must plug in a typeclass-based serialization library to

    provide AsReal and AsRaw instances • e.g. AsRaw for JsonValue could be defined based on Circe Encoder • and AsReal for JsonValue could be defined based on Circe Decoder • Bake it all into the base companion class! • DefaultRestApiCompanion injects GenCodec based serialization (AVSystem’s own serialization library) • It’s easy to make your own version of base companion class, tailored for your needs, e.g. CirceRestApiCompanion to use Circe for serialization Serialization
  54. Asynchronicity and effects type Callback[T] = Try[T] => Unit type

    Async[T] = Callback[T] => Unit trait AsyncEffect[F[_]] { def toAsync[A](fa: F[A]): Async[A] def fromAsync[A](async: Async[A]): F[A] }
  55. Asynchronicity and effects type Callback[T] = Try[T] => Unit type

    Async[T] = Callback[T] => Unit trait AsyncEffect[F[_]] { def toAsync[A](fa: F[A]): Async[A] def fromAsync[A](async: Async[A]): F[A] } • not only Future s but any effects which capture asynchronous, possibly lazy, repeatable computation - Monix Task , Cats IO , ZIO , etc.
  56. Asynchronicity and effects type Callback[T] = Try[T] => Unit type

    Async[T] = Callback[T] => Unit trait AsyncEffect[F[_]] { def toAsync[A](fa: F[A]): Async[A] def fromAsync[A](async: Async[A]): F[A] } • not only Future s but any effects which capture asynchronous, possibly lazy, repeatable computation - Monix Task , Cats IO , ZIO , etc. • Future support is injected by DefaultRestApiCompanion , you can opt out
  57. Effect-polymorphic APIs trait UserApi[F[_]] { def createUser(name: String): F[User] @GET

    def getUser(id: UserId): F[User] } object UserApi extends DefaultPolyRestApiCompanion[UserApi] Tagless final?
  58. Compilation errors Serializer is missing because you forgot to declare

    companion for User ? @PUT @CustomBody def replaceUser(user: User): Future[Unit]
  59. Compilation errors Serializer is missing because you forgot to declare

    companion for User ? [error] problem with parameter user of method replaceUser: [error] Cannot deserialize User from HttpBody, because: [error] Cannot deserialize User from JsonValue, because: [error] No GenCodec found for User [error] @PUT @CustomBody def replaceUser(user: User): Future[Unit] [error] ^ @PUT @CustomBody def replaceUser(user: User): Future[Unit]
  60. Implicit trouble @implicitNotFound("${T} cannot be serialized") trait Codec[T] // [error]

    Int cannot be serialized implicitly[Codec[Int]]
  61. Implicit trouble @implicitNotFound("LibCodec not found for ${T}") trait LibCodec[T] @implicitNotFound("${T}

    cannot be serialized") trait Codec[T] object Codec { implicit def fromLibCodec[T]( implicit libCodec: LibCodec[T] ): Codec[T] = ??? } // [error] Int cannot be serialized implicitly[Codec[Int]]
  62. Extending @implicitNotFound @implicitNotFound("${T} cannot be serialized") trait Codec[T] object Codec

    { implicit def fromLibCodec[T]( implicit libCodec: LibCodec[T] ): Codec[T] = ??? }
  63. Composable error messages @implicitNotFound("${T} cannot be serialized") trait Codec[T] object

    Codec { implicit def fromLibCodec[T]( implicit libCodec: LibCodec[T] ): Codec[T] = ??? @implicitNotFound("${T} cannot be serialized because: #{forLibCodec}") implicit def whenLibCodecNotFound[T]( implicit forLibCodec: ImplicitNotFound[LibCodec[T]] ): ImplicitNotFound[Codec[T]] = ImplicitNotFound() }
  64. Composable error messages // [error] Int cannot be serialized because:

    LibCodec not found for Int Implicits.infer[Codec[Int]] • requires custom Implicits.infer macro • is used automatically by macro-originated implicit searches • extensible
  65. Annotation processing in Scala • annotations are full blown classes

  66. Annotation processing in Scala • annotations are full blown classes

    • can be accessed in macros
  67. Annotation processing in Scala • annotations are full blown classes

    • can be accessed in macros as ASTs • we can implement our own annotation processing
  68. Annotation processing in Scala • annotations are full blown classes

    • can be accessed in macros as ASTs • we can implement our own annotation processing • selective reification into runtime
  69. Annotation processing in Scala • annotations are full blown classes

    • can be accessed in macros as ASTs • we can implement our own annotation processing • selective reification into runtime • aggregation - Don’t Repeat Yourself for annotations
  70. Annotation extensibility trait UserApi { @summary("Creates a user with given

    name") def createUser(name: String): Future[User] }
  71. Annotations are just classes trait UserApi { @summary("Creates a user

    with given name") def createUser(name: String): Future[User] } class summary(summary: String) extends OperationAdjuster { def adjustOperation(operation: Operation): Operation = operation.copy(summary = summary) }
  72. Annotations are just classes trait UserApi { @summary("Creates a user

    with given name") def createUser(name: String): Future[User] } class summary(summary: String) extends OperationAdjuster { def adjustOperation(operation: Operation): Operation = operation.copy(summary = summary) } Make your own annotations!
  73. DRY? trait MyApi { @PUT("users") @CustomBody @summary("Updates a user") def

    updateUser(user: User): Future[Unit] @PUT("groups") @CustomBody @summary("Updates a group") def updateGroup(group: Group): Future[Unit] @PUT("resources") @CustomBody @summary("Updates a resource") def updateResource(resource: Resource): Future[Unit] } We need a desiccant!
  74. Yo dawg, I herd you like annotations class EntityPUT(what: String)

    extends AnnotationAggregate { @PUT(s"${what}s") @CustomBody @summary(s"Updates a $what") type Implied } trait MyApi { @EntityPUT("user") def updateUser(user: User): Future[Unit] @EntityPUT("group") def updateGroup(group: Group): Future[Unit] @EntityPUT("resource") def updateResource(resource: Resource): Future[Unit] }
  75. Wrapping up Udash REST • use plain Scala traits and

    data types to define endpoints • type safety on many levels • pluggable native HTTP servers and clients • pluggable, extensible serialization • pluggable, extensible effects • superb error messages, also extensible • annotations on steroids • built using a powerful, generic metaprogramming engine
  76. Bonus: the metaprogramming engine below • typeclass & schema derivation

    • arbitrarily designed schema - single macro
  77. Bonus: the metaprogramming engine below • typeclass & schema derivation

    • arbitrarily designed schema - single macro case class RecordInfo[T]( @multi @adtParamMetadata paramInfos: List[ParamInfo[_]] ) extends TypedMetadata[T] object RecordInfo extends AdtMetadataCompanion[RecordInfo] case class ParamInfo[T]( @reifyName name: String, @optional @reifyAnnot nameAnnot: Option[name], @infer typeclass: Typeclass[T] ) extends TypedMetadata[T]