Building trait-based web services with Udash REST

Transcript

1. Udash REST
   by Roman Janusz, AVSystem, Kraków
   Scalar Conference, Warsaw, 06.04.2019
   

   View Slide

2. View Slide

3. Udash Framework
   ●
   Frontend and backend in the same language: Scala
   ○
   ScalaJS
   ○
   Scalatags & ScalaCSS by Li Haoyi
   ○
   UI: Routing, property binding, internationalization
   ●
   Type safety continuous throughout the entire system
   ○
   Trait based client-server RPC
   ○
   Opaque-ish protocol
   ●
   RPC based REST
   ○
   Well defined HTTP traffic
   ○
   Consumable by other systems in different languages
   

   View Slide

4. RPC vs REST
   

   View Slide

5. RPC vs REST
   

   View Slide

6. import io.udash.rest._
   trait UserApi {
   def createUser(name: String): Future[User]
   }
   object UserApi extends DefaultRestApiCompanion[UserApi]
   REST API trait definition
   

   View Slide

7. import io.udash.rest._
   trait UserApi {
   def createUser(name: String): Future[User]
   }
   object UserApi extends DefaultRestApiCompanion[UserApi]
   case class UserId(id: Long) extends AnyVal
   object UserId extends RestDataWrapperCompanion[Long, UserId]
   case class User(id: UserId, name: String)
   object User extends RestDataCompanion[User]
   Data types
   

   View Slide

8. Server
   class UserApiImpl extends UserApi {
   def createUser(name: String): Future[User] = ???
   }
   

   View Slide

9. Server
   class UserApiImpl extends UserApi {
   def createUser(name: String): Future[User] = ???
   }
   val serverHandle: HandleRequest =
   UserApi.asHandleRequest(new UserApiImpl)
   

   View Slide

10. Request handler function
    class UserApiImpl extends UserApi {
    def createUser(name: String): Future[User] = ???
    }
    val serverHandle: HandleRequest =
    UserApi.asHandleRequest(new UserApiImpl)
    type HandleRequest =
    RestRequest => Async[RestResponse]
    type Callback[T] = Try[T] => Unit
    type Async[T] = Callback[T] => Unit
    

    View Slide

11. Server
    class UserApiImpl extends UserApi {
    def createUser(name: String): Future[User] = ???
    }
    val serverHandle: HandleRequest =
    UserApi.asHandleRequest(new UserApiImpl)
    val servlet: javax.servlet.Servlet =
    new RestServlet(serverHandle)
    

    View Slide

12. Client
    implicit val sttpBackend: SttpBackend[Future, Nothing] =
    SttpRestClient.defaultBackend()
    

    View Slide

13. Client
    implicit val sttpBackend: SttpBackend[Future, Nothing] =
    SttpRestClient.defaultBackend()
    val clientHandle: HandleRequest =
    SttpRestClient.asHandleRequest("https://www.userapi.com")
    

    View Slide

14. Client
    implicit val sttpBackend: SttpBackend[Future, Nothing] =
    SttpRestClient.defaultBackend()
    val clientHandle: HandleRequest =
    SttpRestClient.asHandleRequest("https://www.userapi.com")
    val client: UserApi =
    UserApi.fromHandleRequest(clientHandle)
    

    View Slide

15. Client
    implicit val sttpBackend: SttpBackend[Future, Nothing] =
    SttpRestClient.defaultBackend()
    val clientHandle: HandleRequest =
    SttpRestClient.asHandleRequest("https://www.userapi.com")
    val client: UserApi =
    UserApi.fromHandleRequest(clientHandle)
    client.createUser("Fred").onComplete(...)
    

    View Slide

16. On the wire
    POST /createUser HTTP/1.1
    Content-Type: application/json;charset=utf-8
    content-length: 15
    host: localhost:9090
    {"name":"Fred"}
    def createUser(name: String): Future[User]
    

    View Slide

17. On the wire
    POST /createUser HTTP/1.1
    Content-Type: application/json;charset=utf-8
    content-length: 15
    host: localhost:9090
    {"name":"Fred"}
    def createUser(name: String): Future[User]
    HTTP/1.1 200 OK
    Content-Type: application/json;charset=utf-8
    Content-Length: 22
    {"id":0,"name":"Fred"}
    

    View Slide

18. import io.udash.rest._
    trait UserApi {
    def createUser(name: String): Future[User]
    }
    object UserApi extends DefaultRestApiCompanion[UserApi]
    case class UserId(id: Long) extends AnyVal
    object UserId extends RestDataWrapperCompanion[Long, UserId]
    case class User(id: UserId, name: String)
    object User extends RestDataCompanion[User]
    How does it work?
    

    View Slide

19. import io.udash.rest._
    trait UserApi {
    def createUser(name: String): Future[User]
    }
    object UserApi extends DefaultRestApiCompanion[UserApi](ǽƝǗǠƹ)
    case class UserId(id: Long) extends AnyVal
    object UserId extends RestDataWrapperCompanion[Long, UserId](ǽƝǗǠƹ)
    case class User(id: UserId, name: String)
    object User extends RestDataCompanion[User](ǽƝǗǠƹ)
    How does it work? Companions with macros!
    

    View Slide

20. Customizing mapping to HTTP with annotations
    ●
    choosing HTTP method:
    @GET
    ,
    @POST
    ,
    @PUT
    ,
    @PATCH
    ,
    @DELETE
    

    View Slide

21. Customizing mapping to HTTP with annotations
    ●
    choosing HTTP method:
    @GET
    ,
    @POST
    ,
    @PUT
    ,
    @PATCH
    ,
    @DELETE
    ●
    customizing paths:
    @GET("api/user")
    

    View Slide

22. Customizing mapping to HTTP with annotations
    ●
    choosing HTTP method:
    @GET
    ,
    @POST
    ,
    @PUT
    ,
    @PATCH
    ,
    @DELETE
    ●
    customizing paths:
    @GET("api/user")
    ●
    choosing parameter flavors and names:
    @Path
    ,
    @Header("X-Header")
    ,
    @Query
    ,
    @Cookie
    ,
    @Body
    

    View Slide

23. Customizing mapping to HTTP with annotations
    ●
    choosing HTTP method:
    @GET
    ,
    @POST
    ,
    @PUT
    ,
    @PATCH
    ,
    @DELETE
    ●
    customizing paths:
    @GET("api/user")
    ●
    choosing parameter flavors and names:
    @Path
    ,
    @Header("X-Header")
    ,
    @Query
    ,
    @Cookie
    ,
    @Body
    ●
    choosing body format:
    @JsonBody
    ,
    @FormBody
    ,
    @CustomBody
    

    View Slide

24. We’ve seen that already...
    Various Java frameworks, e.g. JAX-RS based
    

    View Slide

25. We’ve seen that already...
    Various Java frameworks, e.g. JAX-RS based
    Problems:
    ●
    runtime reflection, runtime proxies, Java serialization
    ●
    annotations form a severely limited, dynamic, unsafe, untyped pseudolanguage
    ●
    hard to understand and debug
    ●
    usually blocking, difficult to work with async
    

    View Slide

26. We write Scala, we can do better!
    ●
    compile time reflection with macros
    

    View Slide

27. We write Scala, we can do better!
    ●
    compile time reflection with macros
    ●
    interface composition
    

    View Slide

28. We write Scala, we can do better!
    ●
    compile time reflection with macros
    ●
    interface composition
    ●
    typeclass based serialization
    

    View Slide

29. We write Scala, we can do better!
    ●
    compile time reflection with macros
    ●
    interface composition
    ●
    typeclass based serialization
    ●
    effects for asynchronous & purely functional computations
    

    View Slide

30. We write Scala, we can do better!
    ●
    compile time reflection with macros
    ●
    interface composition
    ●
    typeclass based serialization
    ●
    effects for asynchronous & purely functional computations
    ●
    platform independence
    

    View Slide

31. We write Scala, we can do better!
    ●
    compile time reflection with macros
    ●
    interface composition
    ●
    typeclass based serialization
    ●
    effects for asynchronous & purely functional computations
    ●
    platform independence
    ●
    readable, precise error messages
    

    View Slide

32. We write Scala, we can do better!
    ●
    compile time reflection with macros
    ●
    interface composition
    ●
    typeclass based serialization
    ●
    effects for asynchronous & purely functional computations
    ●
    platform independence
    ●
    readable, precise error messages
    ●
    vastly better annotation processing
    

    View Slide

33. Interface composition: Prefix methods
    trait SystemApi {
    def users: UserApi
    def resources: ResourceApi
    }
    object SystemApi extends DefaultRestApiCompanion[SystemApi]
    

    View Slide

34. ●
    you can organize, split and compose your APIs
    Interface composition: Prefix methods
    trait SystemApi {
    def users: UserApi
    def resources: ResourceApi
    }
    object SystemApi extends DefaultRestApiCompanion[SystemApi]
    

    View Slide

35. ●
    you can organize, split and compose your APIs
    ●
    you can capture common path fragment and parameters
    Interface composition: Prefix methods
    trait SystemApi {
    def users: UserApi
    def resources: ResourceApi
    }
    object SystemApi extends DefaultRestApiCompanion[SystemApi]
    

    View Slide

36. Prefix methods: authentication
    trait SeriousBusinessApi { … }
    object SeriousBusinessApi
    extends DefaultRestApiCompanion[SeriousBusinessApi]
    

    View Slide

37. Prefix methods: authentication
    trait SeriousBusinessApi { … }
    object SeriousBusinessApi
    extends DefaultRestApiCompanion[SeriousBusinessApi]
    trait AuthApi {
    @Prefix("") def auth(
    @Header("Authorization") authToken: String
    ): SeriousBusinessApi
    }
    object AuthApi extends DefaultRestApiCompanion[AuthApi]
    

    View Slide

38. OpenAPI
    ●
    open standard for describing REST endpoints with JSON or YAML
    ●
    interactive documentation for free (e.g. Swagger UI)
    ●
    client & server stub code generation for various languages (e.g. Swagger Codegen)
    

    View Slide

39. OpenAPI
    ●
    open standard for describing REST endpoints with JSON or YAML
    ●
    interactive documentation for free (e.g. Swagger UI)
    ●
    client & server stub code generation for various languages (e.g. Swagger Codegen)
    import io.udash.rest.openapi._
    val userOpenApi: OpenApi =
    UserApi.openapiMetadata.openapi(
    Info("User API", "0.1", description = "User management API"),
    servers = List(Server("http://www.userapi.com"))
    )
    

    View Slide

40. OpenAPI
    ●
    open standard for describing REST endpoints with JSON or YAML
    ●
    interactive documentation for free (e.g. Swagger UI)
    ●
    client & server stub code generation for various languages (e.g. Swagger Codegen)
    import io.udash.rest.openapi._
    import com.avsystem.commons.serialization.json._
    val userOpenApi: OpenApi =
    UserApi.openapiMetadata.openapi(
    Info("User API", "0.1", description = "User management API"),
    servers = List(Server("http://www.userapi.com"))
    )
    println(JsonStringOutput.write(userOpenApi))
    

    View Slide

41. OpenAPI customization
    trait UserApi {
    @summary("Creates a user with given name")
    def createUser(name: String): Future[User]
    @summary("Fetches a user by ID")
    @GET def getUser(id: UserId): Future[User]
    }
    object UserApi extends DefaultRestApiCompanion[UserApi]
    

    View Slide

42. OpenAPI customization
    @description("Representation of system user")
    @example(User(UserId(0), "Fred"))
    case class User(
    id: UserId,
    @description("User name") name: String
    )
    object User extends RestDataCompanion[User]
    

    View Slide

43. Swagger UI
    

    View Slide

44. Serialization
    trait UserApi {
    @PATCH def updateUser(
    @Path id: UserId,
    name: Option[String] = None
    ): Future[User]
    }
    

    View Slide

45. Serialization
    trait UserApi {
    @PATCH def updateUser(
    @Path id: UserId,
    name: Option[String] = None
    ): Future[User]
    }
    AsRaw[PlainValue, UserId]
    (client)
    AsReal[PlainValue, UserId]
    (server)
    

    View Slide

46. Serialization
    trait UserApi {
    @PATCH def updateUser(
    @Path id: UserId,
    name: Option[String] = None
    ): Future[User]
    }
    AsRaw[PlainValue, UserId]
    (client)
    AsReal[PlainValue, UserId]
    (server)
    AsRaw[JsonValue, Option[String]]
    (client)
    AsReal[JsonValue, Option[String]]
    (server)
    

    View Slide

47. Serialization
    trait UserApi {
    @PATCH def updateUser(
    @Path id: UserId,
    name: Option[String] = None
    ): Future[User]
    }
    AsRaw[PlainValue, UserId]
    (client)
    AsReal[PlainValue, UserId]
    (server)
    AsRaw[JsonValue, Option[String]]
    (client)
    AsReal[JsonValue, Option[String]]
    (server)
    AsReal[RestResponse, User]
    (client)
    AsRaw[RestResponse, User]
    (server)
    

    View Slide

48. Serialization
    trait UserApi {
    @PATCH def updateUser(
    @Path id: UserId,
    name: Option[String] = None
    ): Future[User]
    }
    AsRaw[PlainValue, UserId]
    (client)
    AsReal[PlainValue, UserId]
    (server)
    AsRaw[JsonValue, Option[String]]
    (client)
    AsReal[JsonValue, Option[String]]
    (server)
    AsReal[RestResponse, User]
    (client)
    AsRaw[RestResponse, User]
    (server)
    AsyncEffect[Future]
    (client & server)
    

    View Slide

49. ●
    You must plug in a typeclass-based serialization library to provide
    AsReal
    and
    AsRaw
    instances
    Serialization
    

    View Slide

50. ●
    You must plug in a typeclass-based serialization library to provide
    AsReal
    and
    AsRaw
    instances
    ●
    e.g.
    AsRaw
    for
    JsonValue
    could be defined based on Circe
    Encoder
    ●
    and
    AsReal
    for
    JsonValue
    could be defined based on Circe
    Decoder
    Serialization
    

    View Slide

51. ●
    You must plug in a typeclass-based serialization library to provide
    AsReal
    and
    AsRaw
    instances
    ●
    e.g.
    AsRaw
    for
    JsonValue
    could be defined based on Circe
    Encoder
    ●
    and
    AsReal
    for
    JsonValue
    could be defined based on Circe
    Decoder
    ●
    Bake it all into the base companion class!
    Serialization
    

    View Slide

52. ●
    You must plug in a typeclass-based serialization library to provide
    AsReal
    and
    AsRaw
    instances
    ●
    e.g.
    AsRaw
    for
    JsonValue
    could be defined based on Circe
    Encoder
    ●
    and
    AsReal
    for
    JsonValue
    could be defined based on Circe
    Decoder
    ●
    Bake it all into the base companion class!
    ● DefaultRestApiCompanion
    injects
    GenCodec
    based serialization
    (AVSystem’s own serialization library)
    Serialization
    

    View Slide

53. ●
    You must plug in a typeclass-based serialization library to provide
    AsReal
    and
    AsRaw
    instances
    ●
    e.g.
    AsRaw
    for
    JsonValue
    could be defined based on Circe
    Encoder
    ●
    and
    AsReal
    for
    JsonValue
    could be defined based on Circe
    Decoder
    ●
    Bake it all into the base companion class!
    ● DefaultRestApiCompanion
    injects
    GenCodec
    based serialization
    (AVSystem’s own serialization library)
    ●
    It’s easy to make your own version of base companion class, tailored for your
    needs, e.g.
    CirceRestApiCompanion
    to use Circe for serialization
    Serialization
    

    View Slide

54. Asynchronicity and effects
    type Callback[T] = Try[T] => Unit
    type Async[T] = Callback[T] => Unit
    trait AsyncEffect[F[_]] {
    def toAsync[A](fa: F[A]): Async[A]
    def fromAsync[A](async: Async[A]): F[A]
    }
    

    View Slide

55. Asynchronicity and effects
    type Callback[T] = Try[T] => Unit
    type Async[T] = Callback[T] => Unit
    trait AsyncEffect[F[_]] {
    def toAsync[A](fa: F[A]): Async[A]
    def fromAsync[A](async: Async[A]): F[A]
    }
    ●
    not only
    Future
    s but any effects which capture asynchronous, possibly lazy,
    repeatable computation - Monix
    Task
    , Cats
    IO
    ,
    ZIO
    , etc.
    

    View Slide

56. Asynchronicity and effects
    type Callback[T] = Try[T] => Unit
    type Async[T] = Callback[T] => Unit
    trait AsyncEffect[F[_]] {
    def toAsync[A](fa: F[A]): Async[A]
    def fromAsync[A](async: Async[A]): F[A]
    }
    ●
    not only
    Future
    s but any effects which capture asynchronous, possibly lazy,
    repeatable computation - Monix
    Task
    , Cats
    IO
    ,
    ZIO
    , etc.
    ● Future
    support is injected by
    DefaultRestApiCompanion
    , you can opt out
    

    View Slide

57. Effect-polymorphic APIs
    trait UserApi[F[_]] {
    def createUser(name: String): F[User]
    @GET def getUser(id: UserId): F[User]
    }
    object UserApi extends DefaultPolyRestApiCompanion[UserApi]
    Tagless final?
    

    View Slide

58. Compilation errors
    Serializer is missing because you forgot to declare companion for
    User
    ?
    @PUT @CustomBody def replaceUser(user: User): Future[Unit]
    

    View Slide

59. Compilation errors
    Serializer is missing because you forgot to declare companion for
    User
    ?
    [error] problem with parameter user of method replaceUser:
    [error] Cannot deserialize User from HttpBody, because:
    [error] Cannot deserialize User from JsonValue, because:
    [error] No GenCodec found for User
    [error] @PUT @CustomBody def replaceUser(user: User): Future[Unit]
    [error] ^
    @PUT @CustomBody def replaceUser(user: User): Future[Unit]
    

    View Slide

60. Implicit trouble
    @implicitNotFound("${T} cannot be serialized")
    trait Codec[T]
    // [error] Int cannot be serialized
    implicitly[Codec[Int]]
    

    View Slide

61. Implicit trouble
    @implicitNotFound("LibCodec not found for ${T}")
    trait LibCodec[T]
    @implicitNotFound("${T} cannot be serialized")
    trait Codec[T]
    object Codec {
    implicit def fromLibCodec[T](
    implicit libCodec: LibCodec[T]
    ): Codec[T] = ???
    }
    // [error] Int cannot be serialized
    implicitly[Codec[Int]]
    

    View Slide

62. Extending @implicitNotFound
    @implicitNotFound("${T} cannot be serialized")
    trait Codec[T]
    object Codec {
    implicit def fromLibCodec[T](
    implicit libCodec: LibCodec[T]
    ): Codec[T] = ???
    }
    

    View Slide

63. Composable error messages
    @implicitNotFound("${T} cannot be serialized")
    trait Codec[T]
    object Codec {
    implicit def fromLibCodec[T](
    implicit libCodec: LibCodec[T]
    ): Codec[T] = ???
    @implicitNotFound("${T} cannot be serialized because: #{forLibCodec}")
    implicit def whenLibCodecNotFound[T](
    implicit forLibCodec: ImplicitNotFound[LibCodec[T]]
    ): ImplicitNotFound[Codec[T]] = ImplicitNotFound()
    }
    

    View Slide

64. Composable error messages
    // [error] Int cannot be serialized because: LibCodec not found for Int
    Implicits.infer[Codec[Int]]
    ●
    requires custom
    Implicits.infer
    macro
    ●
    is used automatically by macro-originated implicit searches
    ●
    extensible
    

    View Slide

65. Annotation processing in Scala
    ●
    annotations are full blown classes
    

    View Slide

66. Annotation processing in Scala
    ●
    annotations are full blown classes
    ●
    can be accessed in macros
    

    View Slide

67. Annotation processing in Scala
    ●
    annotations are full blown classes
    ●
    can be accessed in macros as ASTs
    ●
    we can implement our own annotation processing
    

    View Slide

68. Annotation processing in Scala
    ●
    annotations are full blown classes
    ●
    can be accessed in macros as ASTs
    ●
    we can implement our own annotation processing
    ●
    selective reification into runtime
    

    View Slide

69. Annotation processing in Scala
    ●
    annotations are full blown classes
    ●
    can be accessed in macros as ASTs
    ●
    we can implement our own annotation processing
    ●
    selective reification into runtime
    ●
    aggregation - Don’t Repeat Yourself for annotations
    

    View Slide

70. Annotation extensibility
    trait UserApi {
    @summary("Creates a user with given name")
    def createUser(name: String): Future[User]
    }
    

    View Slide

71. Annotations are just classes
    trait UserApi {
    @summary("Creates a user with given name")
    def createUser(name: String): Future[User]
    }
    class summary(summary: String) extends OperationAdjuster {
    def adjustOperation(operation: Operation): Operation =
    operation.copy(summary = summary)
    }
    

    View Slide

72. Annotations are just classes
    trait UserApi {
    @summary("Creates a user with given name")
    def createUser(name: String): Future[User]
    }
    class summary(summary: String) extends OperationAdjuster {
    def adjustOperation(operation: Operation): Operation =
    operation.copy(summary = summary)
    }
    Make your own annotations!
    

    View Slide

73. DRY?
    trait MyApi {
    @PUT("users") @CustomBody @summary("Updates a user")
    def updateUser(user: User): Future[Unit]
    @PUT("groups") @CustomBody @summary("Updates a group")
    def updateGroup(group: Group): Future[Unit]
    @PUT("resources") @CustomBody @summary("Updates a resource")
    def updateResource(resource: Resource): Future[Unit]
    }
    We need a desiccant!
    

    View Slide

74. Yo dawg, I herd you like annotations
    class EntityPUT(what: String) extends AnnotationAggregate {
    @PUT(s"${what}s") @CustomBody @summary(s"Updates a $what") type Implied
    }
    trait MyApi {
    @EntityPUT("user") def updateUser(user: User): Future[Unit]
    @EntityPUT("group") def updateGroup(group: Group): Future[Unit]
    @EntityPUT("resource") def updateResource(resource: Resource): Future[Unit]
    }
    

    View Slide

75. Wrapping up Udash REST
    ●
    use plain Scala traits and data types to define endpoints
    ●
    type safety on many levels
    ●
    pluggable native HTTP servers and clients
    ●
    pluggable, extensible serialization
    ●
    pluggable, extensible effects
    ●
    superb error messages, also extensible
    ●
    annotations on steroids
    ●
    built using a powerful, generic metaprogramming engine
    

    View Slide

76. Bonus: the metaprogramming engine below
    ●
    typeclass & schema derivation
    ●
    arbitrarily designed schema - single macro
    

    View Slide

77. Bonus: the metaprogramming engine below
    ●
    typeclass & schema derivation
    ●
    arbitrarily designed schema - single macro
    case class RecordInfo[T](
    @multi @adtParamMetadata paramInfos: List[ParamInfo[_]]
    ) extends TypedMetadata[T]
    object RecordInfo extends AdtMetadataCompanion[RecordInfo]
    case class ParamInfo[T](
    @reifyName name: String,
    @optional @reifyAnnot nameAnnot: Option[name],
    @infer typeclass: Typeclass[T]
    ) extends TypedMetadata[T]
    

    View Slide