Connect platforms with a Kotlin DSL

Connect platforms with a Kotlin DSL

Not everyone can jump right into a Kotlin Multiplatform world. Many of us have legacy codebases and need to support our businesses first. Yet we can already use Kotlin to connect our platforms and realize significant value. In this talk we’ll learn to build a DSL that defines a contract between an API and its clients. We’ll make our DSL into a common vocabulary that translates platform idiosyncrasies. Then, by connecting it to CI, we’ll enforce our contract to keep our platforms on speaking terms. The examples in this talk will be from Sprout, an internal library in development at Etsy. The Sprout Kotlin DSL is the seed that generates code for many platforms: Retrofit endpoints, Moshi models, Swift, PHP, and more. We’ll look at how to use Kotlin as a templating language, and how the DSL can even generate itself. Finally we’ll peek at the future potential of these types of DSLs. What does connecting a Figma file look like? What about Jetpack Compose? Let’s dream big and then write a DSL for it.

74af653995934bb432936a536761be33?s=128

Patrick Cousins

August 26, 2019
Tweet

Transcript

  1. 8.

    Domain Specific Language • Simplified mini language • Declarative •

    Can be data, function, configuration • Perfect for a builder pattern • Lots of tutorials online • Easier than you think
  2. 16.

    class Cat { var type: String = "" } fun

    cat(lambda: Cat.() -> Unit) : Cat { return Cat().apply(lambda) }
  3. 17.

    class Cat { var type: String = "" } fun

    cat(lambda: Cat.() -> Unit) : Cat { return Cat().apply(lambda) }
  4. 18.

    class Cat { var type: String = "" } fun

    cat(lambda: Cat.() -> Unit) : Cat { return Cat().apply(lambda) }
  5. 20.

    class Cat { var type: String = "" } class

    Pets { var petsList = mutableListOf<Cat>() }
  6. 22.
  7. 24.

    class Pets { var petsList = mutableListOf<Cat>() fun cat(lambda: Cat.()

    -> Unit) { petsList.add( Cat().apply(lambda) ) } }
  8. 25.
  9. 26.

    class Pets { var petsList = mutableListOf<Cat>() fun cat(lambda: Cat.()

    -> Unit) { petsList.add( Cat().apply(lambda) ) } }
  10. 28.
  11. 29.
  12. 31.

    What is Sprout? A code generation pipeline utilizing a custom

    Kotlin DSL to represent the nexus between platforms
  13. 35.

    Why Kotlin? Discoverable = Code completion and type safety help

    users discover what is possible in your DSL
  14. 36.
  15. 37.

    Why Kotlin? Easy to read = Syntax of a DSL

    is basically equivalent to the syntax of JSON
  16. 41.
  17. 44.

    class Listings_Fetch : Seedling { override fun seed(): Api {

    return api { apps { endpoint("Listings_Fetch") { route { get("/listings/:listing_id") } description = "Fetches a listing" resultType = Resource_ListingFetch() } } } } } } Seed
  18. 45.

    class Listings_Fetch : Seedling { override fun seed(): Api {

    return api { apps { endpoint("Listings_Fetch") { route { get("/listings/:listing_id") } description = "Fetches a listing" resultType = Resource_ListingFetch() } } } } } } Seed
  19. 46.

    class Resource_ListingCard(name: String = "") : Resource( name = name,

    info = { type = "Resource_ListingCard*" properties { id("listing_id") listingTitle("title") string("shop_average_rating").optional() } resources( Resource_ListingImage("img"), Resource_Money("price") ) } ) Resource
  20. 47.

    fun Api.printAndroidModels(): AndroidPrintableContent { val endpoint = this.service.endpoint val alreadyGenerated

    = mutableSetOf<Resource>() val toGenerate = generate(endpoint.resultType, mutableSetOf()) val files = mutableMapOf<String, String>() val typeAliases = mutableSetOf<String>() generate(toGenerate, alreadyGenerated, filesMap, typeAliases) return AndroidPrintableContent(files, typeAliases) } Printer
  21. 48.

    class PropertyFunctionTemplate(val name: String, val type: String):Template { override fun

    render(): String { return """ |@SproutDslMarker |fun Properties.$name(name: String): DataType { | val prop = $type() | _map[name] = prop | return prop |} """.trimIndent() } } Template
  22. 52.

    “The key abstraction of information in REST is a resource.

    Any information that can be named can be a resource: a document or image, a temporal service (e.g. “today’s weather in Los Angeles”), a collection of other resources, a non-virtual object (e.g. a person), and so on.” Roy Fielding
  23. 54.

    Resource • REST API request and response • machine readable

    • properties • String, Int, Long, Boolean • (sub) resources • cardinality • structure, type, nullability, optionality • what your JSON is going to look like
  24. 55.

    class ListingsCompare { public static function spec(ResourceSpec $def) { $def->property('user_id',

    new DataType_ID); $def->property('notes', new DataType_String); $def->property('comparison_id', new DataType_ID); $def->resource('listings', Resource_Listing::class.'*'); } } API Resource
  25. 56.

    $spec = [ 'user_id' => 'DataType_ID', 'notes' => 'DataType_String', 'comparison_id'

    => 'DataType_ID', 'listings' => [ ... ] ]; Resource spec
  26. 57.

    spec = { properties { int("user_id") string("notes") int("comparison_id") } resources(

    Resource_ListingCard("listings") ) } Sprout Resource DSL
  27. 58.

    spec = { properties { id("user_id") string("notes") id("comparison_id") } resources(

    Resource_ListingCard("listings") ) } Sprout $spec = [ 'user_id' => 'DataType_ID', 'notes' => 'DataType_String', 'comparison_id' => 'DataType_ID', 'listings' => [ ... ] ]; API
  28. 59.

    Sprout output $spec = [ 'user_id' => 'DataType_ID', 'notes' =>

    'DataType_String', 'comparison_id' => 'DataType_ID', 'listings' => [ ... ] ]; API $spec = [ 'user_id' => 'DataType_ID', 'notes' => 'DataType_String', 'comparison_id' => 'DataType_ID', 'listings' => [ ... ] ];
  29. 60.

    function array_diff_assoc_recursive($array1, $array2) { $difference = array(); foreach ($array1 as

    $key => $value) { if (is_array($value)) { if (!isset($array2[$key]) || !is_array($array2[$key])) { $difference[$key] = $value; } else { $new_diff = array_diff_assoc_recursive($value, $array2[$key]); if (!empty($new_diff)) $difference[$key] = $new_diff; } } else if (!array_key_exists($key, $array2) || $array2[$key] !== $value) { $difference[$key] = $value; } } return $difference; } diff in both directions
  30. 62.

    if (!empty($diff)) { echo "Mismatch between specs. The following were

    found in one spec but not the other. For a guide on how to fix this see http://go/sprout/mismatch"; print_r($diff); exit(1); } else { echo "specs match. here's a cat: ~/ᐠ.ꞈ.ᐟ\~"; exit(0); } Zero for success
  31. 64.
  32. 66.

    With the contract in place on the API, the apps

    need to adhere to it Why do we do this?
  33. 69.

    /** * ATTENTION! * If you are parsing strings in

    a BaseModel subclass (and I bet you are), * for your own safety use this wrapper, don't use JsonParser.valueAsString() * Assume you want all your strings stripped of HTML Escaped characters when delivered by the * API. If you show a string in the UI without cleaning up escaped characters, its going to * look ugly, Mother&#39'in ugly. * * (That's a joke about Mother's Day encoding looking like censored swearing, because it * happened to me, -redacted) */ public String parseString(JsonParser jp) throws IOException { return parseString(jp, false); } What could go wrong?
  34. 72.
  35. 77.

    While reading API resources into our DSL, we’ll encounter types

    we haven’t seen before Self-learning • Turns out our DSL is machine readable • We can generate it and... • Add new types not seen before into the DSL • Then we recompile ❤ Developer friendly Helps adoption
  36. 78.

    @SproutDslMarker fun Properties.html(name: String): DataType { val prop = DataType_Html()

    _map[name] = prop return prop } @SproutDslMarker fun Properties.id(name: String): DataType { val prop = DataType_ID() _map[name] = prop return prop } Property DSL function
  37. 79.

    spec = { properties { int("user_id") string("notes") int("comparison_id") } resources(

    Resource_ListingCard("listings") ) } Sprout Resource DSL
  38. 80.

    spec = { properties { id("user_id") html("notes") id("comparison_id") } resources(

    Resource_ListingCard("listings") ) } Sprout Resource DSL
  39. 81.

    @SproutDslMarker fun Properties.html(name: String): DataType { val prop = DataType_Html()

    _map[name] = prop return prop } @SproutDslMarker fun Properties.id(name: String): DataType { val prop = DataType_ID() _map[name] = prop return prop } Property DSL function
  40. 82.

    class PropertyFunctionTemplate(val name: String, val type: String):Template { override fun

    render(): String { return """ |@SproutDslMarker |fun Properties.$name(name: String): DataType { | val prop = $type() | _map[name] = prop | return prop |} """.trimIndent() } } Machine writeable
  41. 85.

    typealias PositiveInt = Int typealias ID = Long typealias PaymentMethod

    = String typealias SupportedLanguage = String typealias CurrencyCode = String typealias CurrencySymbol = String typealias Latitude = Float typealias Longitude = Float SproutTypeAliases.kt
  42. 86.

    @JsonClass(generateAdapter = true) data class ImageSize( @Json(name = "width") val

    width: PositiveInt, @Json(name = "height") val height: PositiveInt, @Json(name = "url") val url: String ) Type aliases used in models
  43. 88.

    Type aliases • Can be used on iOS and Android

    • doesn’t change the structure of the JSON • Assignment compatible • provides hint to the engineer working in native code
  44. 91.

    Styling makes great metadata • Doesn't change often • Machine

    readable/writeable • Too big to be experimented on?
  45. 92.
  46. 93.
  47. 94.
  48. 98.

    data class MaterialTypography( val h1: TextStyle = TextStyle( fontFamily =

    FontFamily("Roboto"), fontWeight = FontWeight.w100, fontSize = 96.sp), val h2: TextStyle = TextStyle( fontFamily = FontFamily("Roboto"), fontWeight = FontWeight.w100, fontSize = 60.sp), MaterialTheme.kt data class MaterialColors( val primary: Color = Color(0xFF6200EE.toInt()), val primaryVariant: Color = Color(0xFF3700B3.toInt()), val secondary: Color = Color(0xFF03DAC6.toInt()), val secondaryVariant: Color = Color(0xFF018786.toInt()), val background: Color = Color(0xFFFFFFFF.toInt()), Very machine writeable!
  49. 99.

    @Composable fun MaterialTheme() { Colors.Provider(value = colors) { Typography.Provider(value =

    typography) { CurrentTextStyleProvider(value = typography.body1) { MaterialRippleTheme { MaterialButtonShapeTheme(children = children) } } } } } MaterialTheme.kt
  50. 100.

    @Composable fun MaterialTheme() { Colors.Provider(value = colors) { Typography.Provider(value =

    typography) { CurrentTextStyleProvider(value = typography.body1) { if (myFeatureFlag) { SnazzyTheme { } } else { SwankyTheme { } } } } } } MaterialTheme.kt
  51. 104.

    fun main() { Kweb(port = 12345, plugins = listOf(fomanticUIPlugin, PrismPlugin()))

    { doc.body.new { div(fomantic.ui.container).new { div(fomantic.ui.grid).new { div(fomantic.three.wide.column).new { img(src = "sprout-logo.png", attributes = mapOf( "width" to "179", "height" to "120" )) } div(fomantic.eleven.wide.column.middle.aligned).new { var loadingSpinner:Element? = null val loadingDiv = div(fomantic.ui.hidden.middle.aligned).new { loadingSpinner = i(fomantic.notched.circle.hidden).gone() p().text("Loading...") Kweb
  52. 105.
  53. 107.

    Principles and aspirations • Un-opinionated • Does not impose complexity

    on existing systems • Easy to use, easy to learn • Designed to grow, learn • Lightweight (JVM, Vanila kotlin, no 3rd party libs or dependencies) • Run-books • go/link-all-the-things • Clear error messages • Easy to recover from errors • Do one thing at a time
  54. 108.

    dream big API Contract SDL iOS Android {JSON} test fixtures

    Figma React Swagger graph ql it should be easy to connect more platforms