Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building trait-based web services with Udash REST

Building trait-based web services with Udash REST

Roman Janusz

April 06, 2019
Tweet

Other Decks in Programming

Transcript

  1. Udash REST by Roman Janusz, AVSystem, Kraków Scalar Conference, Warsaw,

    06.04.2019
  2. None
  3. Udash Framework • Frontend and backend in the same language:

    Scala ◦ ScalaJS ◦ Scalatags & ScalaCSS by Li Haoyi ◦ UI: Routing, property binding, internationalization • Type safety continuous throughout the entire system ◦ Trait based client-server RPC ◦ Opaque-ish protocol • RPC based REST ◦ Well defined HTTP traffic ◦ Consumable by other systems in different languages
  4. RPC vs REST

  5. RPC vs REST

  6. import io.udash.rest._ trait UserApi { def createUser(name: String): Future[User] }

    object UserApi extends DefaultRestApiCompanion[UserApi] REST API trait definition
  7. import io.udash.rest._ trait UserApi { def createUser(name: String): Future[User] }

    object UserApi extends DefaultRestApiCompanion[UserApi] case class UserId(id: Long) extends AnyVal object UserId extends RestDataWrapperCompanion[Long, UserId] case class User(id: UserId, name: String) object User extends RestDataCompanion[User] Data types
  8. Server class UserApiImpl extends UserApi { def createUser(name: String): Future[User]

    = ??? }
  9. Server class UserApiImpl extends UserApi { def createUser(name: String): Future[User]

    = ??? } val serverHandle: HandleRequest = UserApi.asHandleRequest(new UserApiImpl)
  10. Request handler function class UserApiImpl extends UserApi { def createUser(name:

    String): Future[User] = ??? } val serverHandle: HandleRequest = UserApi.asHandleRequest(new UserApiImpl) type HandleRequest = RestRequest => Async[RestResponse] type Callback[T] = Try[T] => Unit type Async[T] = Callback[T] => Unit
  11. Server class UserApiImpl extends UserApi { def createUser(name: String): Future[User]

    = ??? } val serverHandle: HandleRequest = UserApi.asHandleRequest(new UserApiImpl) val servlet: javax.servlet.Servlet = new RestServlet(serverHandle)
  12. Client implicit val sttpBackend: SttpBackend[Future, Nothing] = SttpRestClient.defaultBackend()

  13. Client implicit val sttpBackend: SttpBackend[Future, Nothing] = SttpRestClient.defaultBackend() val clientHandle:

    HandleRequest = SttpRestClient.asHandleRequest("https://www.userapi.com")
  14. Client implicit val sttpBackend: SttpBackend[Future, Nothing] = SttpRestClient.defaultBackend() val clientHandle:

    HandleRequest = SttpRestClient.asHandleRequest("https://www.userapi.com") val client: UserApi = UserApi.fromHandleRequest(clientHandle)
  15. Client implicit val sttpBackend: SttpBackend[Future, Nothing] = SttpRestClient.defaultBackend() val clientHandle:

    HandleRequest = SttpRestClient.asHandleRequest("https://www.userapi.com") val client: UserApi = UserApi.fromHandleRequest(clientHandle) client.createUser("Fred").onComplete(...)
  16. On the wire POST /createUser HTTP/1.1 Content-Type: application/json;charset=utf-8 content-length: 15

    host: localhost:9090 {"name":"Fred"} def createUser(name: String): Future[User]
  17. On the wire POST /createUser HTTP/1.1 Content-Type: application/json;charset=utf-8 content-length: 15

    host: localhost:9090 {"name":"Fred"} def createUser(name: String): Future[User] HTTP/1.1 200 OK Content-Type: application/json;charset=utf-8 Content-Length: 22 {"id":0,"name":"Fred"}
  18. import io.udash.rest._ trait UserApi { def createUser(name: String): Future[User] }

    object UserApi extends DefaultRestApiCompanion[UserApi] case class UserId(id: Long) extends AnyVal object UserId extends RestDataWrapperCompanion[Long, UserId] case class User(id: UserId, name: String) object User extends RestDataCompanion[User] How does it work?
  19. import io.udash.rest._ trait UserApi { def createUser(name: String): Future[User] }

    object UserApi extends DefaultRestApiCompanion[UserApi](ǽƝǗǠƹ) case class UserId(id: Long) extends AnyVal object UserId extends RestDataWrapperCompanion[Long, UserId](ǽƝǗǠƹ) case class User(id: UserId, name: String) object User extends RestDataCompanion[User](ǽƝǗǠƹ) How does it work? Companions with macros!
  20. Customizing mapping to HTTP with annotations • choosing HTTP method:

    @GET , @POST , @PUT , @PATCH , @DELETE
  21. Customizing mapping to HTTP with annotations • choosing HTTP method:

    @GET , @POST , @PUT , @PATCH , @DELETE • customizing paths: @GET("api/user")
  22. Customizing mapping to HTTP with annotations • choosing HTTP method:

    @GET , @POST , @PUT , @PATCH , @DELETE • customizing paths: @GET("api/user") • choosing parameter flavors and names: @Path , @Header("X-Header") , @Query , @Cookie , @Body
  23. Customizing mapping to HTTP with annotations • choosing HTTP method:

    @GET , @POST , @PUT , @PATCH , @DELETE • customizing paths: @GET("api/user") • choosing parameter flavors and names: @Path , @Header("X-Header") , @Query , @Cookie , @Body • choosing body format: @JsonBody , @FormBody , @CustomBody
  24. We’ve seen that already... Various Java frameworks, e.g. JAX-RS based

  25. We’ve seen that already... Various Java frameworks, e.g. JAX-RS based

    Problems: • runtime reflection, runtime proxies, Java serialization • annotations form a severely limited, dynamic, unsafe, untyped pseudolanguage • hard to understand and debug • usually blocking, difficult to work with async
  26. We write Scala, we can do better! • compile time

    reflection with macros
  27. We write Scala, we can do better! • compile time

    reflection with macros • interface composition
  28. We write Scala, we can do better! • compile time

    reflection with macros • interface composition • typeclass based serialization
  29. We write Scala, we can do better! • compile time

    reflection with macros • interface composition • typeclass based serialization • effects for asynchronous & purely functional computations
  30. We write Scala, we can do better! • compile time

    reflection with macros • interface composition • typeclass based serialization • effects for asynchronous & purely functional computations • platform independence
  31. We write Scala, we can do better! • compile time

    reflection with macros • interface composition • typeclass based serialization • effects for asynchronous & purely functional computations • platform independence • readable, precise error messages
  32. We write Scala, we can do better! • compile time

    reflection with macros • interface composition • typeclass based serialization • effects for asynchronous & purely functional computations • platform independence • readable, precise error messages • vastly better annotation processing
  33. Interface composition: Prefix methods trait SystemApi { def users: UserApi

    def resources: ResourceApi } object SystemApi extends DefaultRestApiCompanion[SystemApi]
  34. • you can organize, split and compose your APIs Interface

    composition: Prefix methods trait SystemApi { def users: UserApi def resources: ResourceApi } object SystemApi extends DefaultRestApiCompanion[SystemApi]
  35. • you can organize, split and compose your APIs •

    you can capture common path fragment and parameters Interface composition: Prefix methods trait SystemApi { def users: UserApi def resources: ResourceApi } object SystemApi extends DefaultRestApiCompanion[SystemApi]
  36. Prefix methods: authentication trait SeriousBusinessApi { … } object SeriousBusinessApi

    extends DefaultRestApiCompanion[SeriousBusinessApi]
  37. Prefix methods: authentication trait SeriousBusinessApi { … } object SeriousBusinessApi

    extends DefaultRestApiCompanion[SeriousBusinessApi] trait AuthApi { @Prefix("") def auth( @Header("Authorization") authToken: String ): SeriousBusinessApi } object AuthApi extends DefaultRestApiCompanion[AuthApi]
  38. OpenAPI • open standard for describing REST endpoints with JSON

    or YAML • interactive documentation for free (e.g. Swagger UI) • client & server stub code generation for various languages (e.g. Swagger Codegen)
  39. OpenAPI • open standard for describing REST endpoints with JSON

    or YAML • interactive documentation for free (e.g. Swagger UI) • client & server stub code generation for various languages (e.g. Swagger Codegen) import io.udash.rest.openapi._ val userOpenApi: OpenApi = UserApi.openapiMetadata.openapi( Info("User API", "0.1", description = "User management API"), servers = List(Server("http://www.userapi.com")) )
  40. OpenAPI • open standard for describing REST endpoints with JSON

    or YAML • interactive documentation for free (e.g. Swagger UI) • client & server stub code generation for various languages (e.g. Swagger Codegen) import io.udash.rest.openapi._ import com.avsystem.commons.serialization.json._ val userOpenApi: OpenApi = UserApi.openapiMetadata.openapi( Info("User API", "0.1", description = "User management API"), servers = List(Server("http://www.userapi.com")) ) println(JsonStringOutput.write(userOpenApi))
  41. OpenAPI customization trait UserApi { @summary("Creates a user with given

    name") def createUser(name: String): Future[User] @summary("Fetches a user by ID") @GET def getUser(id: UserId): Future[User] } object UserApi extends DefaultRestApiCompanion[UserApi]
  42. OpenAPI customization @description("Representation of system user") @example(User(UserId(0), "Fred")) case class

    User( id: UserId, @description("User name") name: String ) object User extends RestDataCompanion[User]
  43. Swagger UI

  44. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId,

    name: Option[String] = None ): Future[User] }
  45. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId,

    name: Option[String] = None ): Future[User] } AsRaw[PlainValue, UserId] (client) AsReal[PlainValue, UserId] (server)
  46. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId,

    name: Option[String] = None ): Future[User] } AsRaw[PlainValue, UserId] (client) AsReal[PlainValue, UserId] (server) AsRaw[JsonValue, Option[String]] (client) AsReal[JsonValue, Option[String]] (server)
  47. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId,

    name: Option[String] = None ): Future[User] } AsRaw[PlainValue, UserId] (client) AsReal[PlainValue, UserId] (server) AsRaw[JsonValue, Option[String]] (client) AsReal[JsonValue, Option[String]] (server) AsReal[RestResponse, User] (client) AsRaw[RestResponse, User] (server)
  48. Serialization trait UserApi { @PATCH def updateUser( @Path id: UserId,

    name: Option[String] = None ): Future[User] } AsRaw[PlainValue, UserId] (client) AsReal[PlainValue, UserId] (server) AsRaw[JsonValue, Option[String]] (client) AsReal[JsonValue, Option[String]] (server) AsReal[RestResponse, User] (client) AsRaw[RestResponse, User] (server) AsyncEffect[Future] (client & server)
  49. • You must plug in a typeclass-based serialization library to

    provide AsReal and AsRaw instances Serialization
  50. • You must plug in a typeclass-based serialization library to

    provide AsReal and AsRaw instances • e.g. AsRaw for JsonValue could be defined based on Circe Encoder • and AsReal for JsonValue could be defined based on Circe Decoder Serialization
  51. • You must plug in a typeclass-based serialization library to

    provide AsReal and AsRaw instances • e.g. AsRaw for JsonValue could be defined based on Circe Encoder • and AsReal for JsonValue could be defined based on Circe Decoder • Bake it all into the base companion class! Serialization
  52. • You must plug in a typeclass-based serialization library to

    provide AsReal and AsRaw instances • e.g. AsRaw for JsonValue could be defined based on Circe Encoder • and AsReal for JsonValue could be defined based on Circe Decoder • Bake it all into the base companion class! • DefaultRestApiCompanion injects GenCodec based serialization (AVSystem’s own serialization library) Serialization
  53. • You must plug in a typeclass-based serialization library to

    provide AsReal and AsRaw instances • e.g. AsRaw for JsonValue could be defined based on Circe Encoder • and AsReal for JsonValue could be defined based on Circe Decoder • Bake it all into the base companion class! • DefaultRestApiCompanion injects GenCodec based serialization (AVSystem’s own serialization library) • It’s easy to make your own version of base companion class, tailored for your needs, e.g. CirceRestApiCompanion to use Circe for serialization Serialization
  54. Asynchronicity and effects type Callback[T] = Try[T] => Unit type

    Async[T] = Callback[T] => Unit trait AsyncEffect[F[_]] { def toAsync[A](fa: F[A]): Async[A] def fromAsync[A](async: Async[A]): F[A] }
  55. Asynchronicity and effects type Callback[T] = Try[T] => Unit type

    Async[T] = Callback[T] => Unit trait AsyncEffect[F[_]] { def toAsync[A](fa: F[A]): Async[A] def fromAsync[A](async: Async[A]): F[A] } • not only Future s but any effects which capture asynchronous, possibly lazy, repeatable computation - Monix Task , Cats IO , ZIO , etc.
  56. Asynchronicity and effects type Callback[T] = Try[T] => Unit type

    Async[T] = Callback[T] => Unit trait AsyncEffect[F[_]] { def toAsync[A](fa: F[A]): Async[A] def fromAsync[A](async: Async[A]): F[A] } • not only Future s but any effects which capture asynchronous, possibly lazy, repeatable computation - Monix Task , Cats IO , ZIO , etc. • Future support is injected by DefaultRestApiCompanion , you can opt out
  57. Effect-polymorphic APIs trait UserApi[F[_]] { def createUser(name: String): F[User] @GET

    def getUser(id: UserId): F[User] } object UserApi extends DefaultPolyRestApiCompanion[UserApi] Tagless final?
  58. Compilation errors Serializer is missing because you forgot to declare

    companion for User ? @PUT @CustomBody def replaceUser(user: User): Future[Unit]
  59. Compilation errors Serializer is missing because you forgot to declare

    companion for User ? [error] problem with parameter user of method replaceUser: [error] Cannot deserialize User from HttpBody, because: [error] Cannot deserialize User from JsonValue, because: [error] No GenCodec found for User [error] @PUT @CustomBody def replaceUser(user: User): Future[Unit] [error] ^ @PUT @CustomBody def replaceUser(user: User): Future[Unit]
  60. Implicit trouble @implicitNotFound("${T} cannot be serialized") trait Codec[T] // [error]

    Int cannot be serialized implicitly[Codec[Int]]
  61. Implicit trouble @implicitNotFound("LibCodec not found for ${T}") trait LibCodec[T] @implicitNotFound("${T}

    cannot be serialized") trait Codec[T] object Codec { implicit def fromLibCodec[T]( implicit libCodec: LibCodec[T] ): Codec[T] = ??? } // [error] Int cannot be serialized implicitly[Codec[Int]]
  62. Extending @implicitNotFound @implicitNotFound("${T} cannot be serialized") trait Codec[T] object Codec

    { implicit def fromLibCodec[T]( implicit libCodec: LibCodec[T] ): Codec[T] = ??? }
  63. Composable error messages @implicitNotFound("${T} cannot be serialized") trait Codec[T] object

    Codec { implicit def fromLibCodec[T]( implicit libCodec: LibCodec[T] ): Codec[T] = ??? @implicitNotFound("${T} cannot be serialized because: #{forLibCodec}") implicit def whenLibCodecNotFound[T]( implicit forLibCodec: ImplicitNotFound[LibCodec[T]] ): ImplicitNotFound[Codec[T]] = ImplicitNotFound() }
  64. Composable error messages // [error] Int cannot be serialized because:

    LibCodec not found for Int Implicits.infer[Codec[Int]] • requires custom Implicits.infer macro • is used automatically by macro-originated implicit searches • extensible
  65. Annotation processing in Scala • annotations are full blown classes

  66. Annotation processing in Scala • annotations are full blown classes

    • can be accessed in macros
  67. Annotation processing in Scala • annotations are full blown classes

    • can be accessed in macros as ASTs • we can implement our own annotation processing
  68. Annotation processing in Scala • annotations are full blown classes

    • can be accessed in macros as ASTs • we can implement our own annotation processing • selective reification into runtime
  69. Annotation processing in Scala • annotations are full blown classes

    • can be accessed in macros as ASTs • we can implement our own annotation processing • selective reification into runtime • aggregation - Don’t Repeat Yourself for annotations
  70. Annotation extensibility trait UserApi { @summary("Creates a user with given

    name") def createUser(name: String): Future[User] }
  71. Annotations are just classes trait UserApi { @summary("Creates a user

    with given name") def createUser(name: String): Future[User] } class summary(summary: String) extends OperationAdjuster { def adjustOperation(operation: Operation): Operation = operation.copy(summary = summary) }
  72. Annotations are just classes trait UserApi { @summary("Creates a user

    with given name") def createUser(name: String): Future[User] } class summary(summary: String) extends OperationAdjuster { def adjustOperation(operation: Operation): Operation = operation.copy(summary = summary) } Make your own annotations!
  73. DRY? trait MyApi { @PUT("users") @CustomBody @summary("Updates a user") def

    updateUser(user: User): Future[Unit] @PUT("groups") @CustomBody @summary("Updates a group") def updateGroup(group: Group): Future[Unit] @PUT("resources") @CustomBody @summary("Updates a resource") def updateResource(resource: Resource): Future[Unit] } We need a desiccant!
  74. Yo dawg, I herd you like annotations class EntityPUT(what: String)

    extends AnnotationAggregate { @PUT(s"${what}s") @CustomBody @summary(s"Updates a $what") type Implied } trait MyApi { @EntityPUT("user") def updateUser(user: User): Future[Unit] @EntityPUT("group") def updateGroup(group: Group): Future[Unit] @EntityPUT("resource") def updateResource(resource: Resource): Future[Unit] }
  75. Wrapping up Udash REST • use plain Scala traits and

    data types to define endpoints • type safety on many levels • pluggable native HTTP servers and clients • pluggable, extensible serialization • pluggable, extensible effects • superb error messages, also extensible • annotations on steroids • built using a powerful, generic metaprogramming engine
  76. Bonus: the metaprogramming engine below • typeclass & schema derivation

    • arbitrarily designed schema - single macro
  77. Bonus: the metaprogramming engine below • typeclass & schema derivation

    • arbitrarily designed schema - single macro case class RecordInfo[T]( @multi @adtParamMetadata paramInfos: List[ParamInfo[_]] ) extends TypedMetadata[T] object RecordInfo extends AdtMetadataCompanion[RecordInfo] case class ParamInfo[T]( @reifyName name: String, @optional @reifyAnnot nameAnnot: Option[name], @infer typeclass: Typeclass[T] ) extends TypedMetadata[T]