Mirrors for operations, not data

Can we derive endpoints from just a trait definition? One of Scala's strengths is domain modelling, and from a data definition, we can derive generic information in a Mirror, e.g. to generate type classes. For operations (such as endpoints) there is less support from metaprogramming, so we often create DSLs to describe them. I propose that we can extend Mirrors to operations, and use the most natural DSL of all - plain trait definitions.

(A talk from Scalar 2024, a conference for sharing ideas about the Scala programming language)

Transcript

1. Mirrors for operations, not data • Twitter: @bishabosha • Email: scala.center@epfl.ch Jamie Thompson
2. Agenda Goal statement Describing services as data is useful, can it be simpler? Intro to type class derivation Which tools does Scala give us to tackle the problem? The New: Operation Mirrors Introducing a simple model for describing services Comparison/Lookahead Is this better than alternatives?
3. Research Open Source 8000+ Github repos 1,200,000+ artifacts Industry
4. Not for profit, supported by donations Our priorities: • guide and support Scala community • coordinate and develop tools and libraries • provide high quality free education materials for Scala Come to our booth!
5. Come to our booth!
6. Fundraising Campaign Come to our booth!
7. Goal Statement To be able to derive type classes from operations
8. Goal Statement @fail[HttpError] trait HelloService derives HttpService { @get("/greet/{name}") def greet(@path name: String): String @post("/greet/{name}") def setGreeting( @path name: String, @body greeting: String ): Unit } Goal!
9. @error[ResponseError] trait LSP derives JsonRpcService { @method("$/progress") def progress(params: ProgressParams): Unit @method("textDocument/completion") def completion( params: CompletionParams ): Array[CompletionItem] } Goal Statement Goal!
10. trait HelloService derives HttpService trait LSP derives JsonRpcService Goal Statement Why is this useful? We can describe a service with data, from a single source of truth, and derive several implementations from that. e.g. server/client with the same API. effect polymorphism “for free”.
11. Type Class Derivation A standard mechanism to add new functionality to a type, based on the structure of the type.
12. Type Class Derivation trait Codec[T]: extension (x: T) def asJSON: JSON An instance of the type class Codec provides evidence that a value x of type T can serialize to JSON. Type class derivation is an opt-in mechanism to provide that evidence automatically.
13. Type Class Derivation case class Foo(x: Int) derives Codec
14. case class Foo(x: Int) derives Codec object Foo: given Codec[Foo] = Codec.derived derives Codec desugars to provide an instance of Codec[Foo] Type Class Derivation but how can you implement Codec.derived for all data types?
15. object Foo: given Codec[Foo] = Codec.derived object Codec: inline def derived [T](using Mirror.Of[T]): Codec[T] A typical declaration for derived uses the help of a Mirror, supplied automatically by the compiler, which provides information about the structure of type T. Type Class Derivation
16. What is a Mirror? A simplified model of the structure of data, it can be provided automatically.
17. enum IList: case ICons(x: Int, xs: IList) case INil A IList value conforms to only 1 of 2 possible shapes: • ICons which has a constructor with 2 parameters: ◦ x of type Int ◦ xs of type IList • INil which is a singleton value What is a Mirror?
18. What is a Mirror? val m: Mirror.Sum { type MirroredType = IList type MirroredElemTypes = (ICons, INil.type) type MirroredElemLabels = ("ICons", "INil") def ordinal(x: IList): Int } = ... Mirrors capture the structure of data in types, with zero runtime overhead. We can use metaprogramming to inspect types and generate code. val m = summon[scala.deriving.Mirror.Of[IList]] child types/labels sum type
19. val m = summon[scala.deriving.Mirror.Of[ICons]] What is a Mirror? val m: Mirror.Product { type MirroredType = ICons type MirroredElemTypes = (Int, IList) type MirroredElemLabels = ("x", "xs") def fromProduct(x: Product): ICons } = ... we can also summon a mirror for a specific case. field types/labels product type
20. A new kind of Mirror Why can’t I do all the things?
21. A new kind of Mirror trait HelloService derives HttpService: def greet: String @foo def setGreeting(greeting: String): Unit Operations are also a common target of reflection, but no mirrors yet! How could they look like?
22. A HelloService specifies 2 operations: • greet returning String, • setGreeting returning Unit, ◦ with param greeting of type String ◦ with annotation @foo A new kind of Mirror trait HelloService derives HttpService: def greet: String @foo def setGreeting(greeting: String): Unit
23. type HelloServiceMirror = OpsMirror { type MirroredType = HelloService type Metadata = EmptyTuple type MirroredOperationLabels = ("greet", "setGreeting") type MirroredOperations = (greetOp, setGreetingOp) } type greetOp = ... type setGreetingOp = ... A new kind of Mirror trait HelloService {...}
24. type HelloServiceMirror = ... type greetOp = Operation { type Metadata = EmptyTuple type InputTypes = EmptyTuple type InputLabels = EmptyTuple type OutputType = String } type setGreetingOp = ... A new kind of Mirror def greet: String
25. type HelloServiceMirror = ... type greetOp = ... type setGreetingOp = Operation { type Metadata = (Meta @foo,) type InputTypes = (String,) type InputLabels = ("greeting",) type OutputType = Unit } A new kind of Mirror @foo def setGreeting(greeting: String): Unit
26. A new kind of Mirror Meta @foo Annotations need a target type, Meta is a useful placeholder. foo and other metadata should derive from MetaAnnotation to appear in the Metadata tuple.
27. A new kind of Mirror enum method extends MetaAnnotation: case get(route: String) case post(route: String) case put(route: String) enum source extends MetaAnnotation: case path() case query() case body() by extending MetaAnnotation we can create DSLs for services.
28. Back to our Goal! @fail[HttpError] trait HelloService derives HttpService { @get("/greet/{name}") def greet(@path name: String): String @post("/greet/{name}") def setGreeting( @path name: String, @body greeting: String ): Unit } bishabosha / ops-mirror Demo time!
29. Alternatives Should we even do this?
30. Alternatives val booksListing: PublicEndpoint[(BooksQuery, Limit, AuthToken), String, List[Book], Any] = endpoint .get .in(("books" / path[String]("genre") / path[Int]("year")).mapTo[BooksQuery]) .in(query[Limit]("limit").description("Maximum number of books to retrieve")) .in(header[AuthToken]("X-Auth-Token")) .errorOut(stringBody) .out(jsonBody[List[Book]]) Tapir • define endpoints as data • derive both client and server • -- less readable • ++ more control
31. Alternatives service HelloService { operations: [Greet, SetGreeting] } @http(method: "GET", uri: "/greet/{name}") operation Greet { input := { @required @httpLabel name: String } output: String } ... Smithy • pure data • all code generation • -- new language • ++ extensible
32. Alternatives def reifyImpl[T: Type](using Quotes): Expr[Of[T]] = import quotes.reflect.* val tpe = TypeRepr.of[T] val cls = tpe.classSymbol.get val decls = cls.declaredMethods val labels = decls.map(m => ConstantType(StringConstant(m.name))) val ops = decls.map(m => ...) Scala 3 Macro API • ++ full control • -- not opinionated • -- less readable • -- verbose • -- repetitive
33. Conclusions Try it out please
34. bishabosha / ops-mirror Try it out on GitHub