Deep Dive into Kotlin DSL

Deep Dive into Kotlin DSL

A presentation at Kotlin Fest 2019 in Japan.

D4133d8efb46ae872d7a563b619bfc83?s=128

Matsuda Jumpei

August 24, 2019
Tweet

Transcript

  1. Deep Dive into Kotlin DSL Kotlin Fest 2019 @red_fat_daruma

  2. Self Introduction Jumpei Matsuda / Daruma / @red_fat_daruma ▸ Software

    Engineer @DeployGate ▸ GitHub: jmatsu / Twitter: red_fat_daruma ▸ JVM projects : Android apps/SDK, Scala server- side and Gradle plugins ▸ One of Kotlin users since M10 2
  3. Today's Goal Hopefully, Every Audience Will.... ▸ Know what is

    DSL and/or Kotlin DSL ▸ Understand what specifications/features of Kotlin support Kotlin DSL ▸ Have a motivation to develop the original Kotlin DSL and/or extend existing libraries through Kotlin DSL. 3
  4. Let me ask questions! 4

  5. Who thinks Kotlin DSL is for Gradle? ✋ 5

  6. I do not use a word *Kotlin DSL* to represent

    Kotlin DSL for Gradle Kotlin DSL Is Not Only Kotlin DSL for Gradle ▸ *Kotlin DSL* in this presentation does not mean Gradle's one ▸ Kotlin DSL for Gradle is just a DSL in Kotlin for Gradle ▸ Contracts, coroutines, etc. are also Kotlin DSL 6
  7. I do not use a word *Kotlin DSL* to represent

    Kotlin DSL for Gradle Kotlin DSL Is Not Only Kotlin DSL for Gradle ▸ *Kotlin DSL* in this presentation does not mean Gradle's one ▸ Kotlin DSL for Gradle is just a DSL in Kotlin for Gradle 7 Go to Next questions!
  8. Who have used and/or are using Kotlin DSL? ✋ 8

  9. Who know what techniques will help Kotlin DSL? ✋ 9

  10. Who can explain the definition of Kotlin DSL? ✋ 10

  11. Who can explain the definition of DSL? ✋ 11

  12. If you said YES to all questions, you are already

    Kotlin DSL deep diver! 12
  13. Today's topics Agenda ▸ The base knowledge of DSL ▸

    Specifications/techniques of Kotlin DSL based on levels ▸ Quick tips for DSL development 13
  14. The base knowledges of DSL Domain Specific Language ▸ A

    computer language specialized to a particular application domain ▸ In contrast: a general purpose language like Kotlin, Java 14 ref: https://en.wikipedia.org/wiki/Domain-specific_language
  15. Domain Specific Language ▸ A computer language specialized to a

    particular application domain ▸ In contrast: a general purpose language like Kotlin, Java ▸ What's a motivation to introduce/need DSL? 15 The base knowledges of DSL ref: https://en.wikipedia.org/wiki/Domain-specific_language
  16. Why People Want To Use DSL ▸ Simplify complex code

    ▸ More readable, understandable, fluent ▸ Use smart and/or minimum APIs for the domain ▸ Improve productivity and reduce human errors ▸ Get a power of type system etc. 16 The base knowledges of DSL
  17. Why People Want To Use DSL 17 The base knowledges

    of DSL NOTE: pseudo code DSL
  18. Aspects of Domain Specific Language ▸ External DSL and Internal

    DSL ▸ External: A language that's parsed independently of the host general purpose language ▸ Internal: A particular form of API in the host general purpose language for the domain 18 The base knowledges of DSL ref: https://martinfowler.com/books/dsl.html
  19. Examples in the Real World ▸ External DSL ▸ SQL,

    CSS, Regular expressions ▸ Internal DSL ▸ HTML, Mock Libraries, DI container 19 The base knowledges of DSL
  20. Examples in the Real World ▸ External DSL ▸ SQL,

    CSS, Regular expressions ▸ Internal DSL ▸ HTML, Mocking Libraries, DI container ▸ Kotlin DSL 20 The base knowledges of DSL
  21. Examples in the Real World ▸ External DSL ▸ SQL,

    CSS, Regular expressions ▸ Internal DSL ▸ HTML, Mocking Libraries, DI container ▸ Kotlin DSL ▸ NOT a special edition of Kotlin 21 The base knowledges of DSL
  22. Techniques for Internal DSL Depends on... ▸ The host general

    purpose language ▸ Type checking ▸ Syntax ▸ Syntactic extensions ▸ Compiler extensibility and interpolation 22 The base knowledges of DSL
  23. Techniques for Internal Kotlin DSL Depends on... ▸ A general

    purpose language Kotlin ▸ Type checking Static type checking ▸ Syntax Lambda expressions, etc. ▸ Syntactic extensions Method extension, operator overloading, etc. ▸ Compiler extensibility and/or interpolation Compiler plugins etc. 23 The base knowledges of DSL
  24. e.g. kotlinx.html is one of Kotlin DSL ▸ DSL for

    HTML domain ▸ Generate DOM tree from a Kotlin code 24 The base knowledges of DSL https://github.com/Kotlin/kotlinx.html
  25. e.g. kotlinx.html is one of Kotlin DSL ▸ DSL for

    HTML domain ▸ Generate DOM tree from a Kotlin code 25 Type safe builder pattern, lambda expressions, functions with receiver, higher-order functions, operator overloading, DSLMarker, etc. The base knowledges of DSL
  26. Today's topics Agenda ▸ The base knowledge of DSL ▸

    Specifications/techniques of Kotlin DSL based on levels ▸ Quick tips for DSL development 26 ▸ NOTE: DSL in following slides means Internal DSL unless specified
  27. Techniques of Kotlin DSL Techniques of Kotlin DSL ▸ Getting

    started ▸ Language specifications ▸ Intermediate ▸ Features for syntactic extensions ▸ Expertise ▸ Build environment stuff of Kotlin 27
  28. Techniques of Kotlin DSL ▸ Getting started ▸ Language specifications

    ▸ Intermediate ▸ Features for syntactic extensions ▸ Expertise ▸ Build environment stuff of Kotlin 28 Techniques of Kotlin DSL - Getting Started
  29. Language Specifications ▸ Type system ▸ Static type checking ▸

    Syntax related specifications ▸ Lambdas, Functions literals with receiver 29 Techniques of Kotlin DSL - Getting Started
  30. Static Type Checking ▸ Code analysis based type safety verification

    ▸ In contrast: Dynamic type checking ▸ Allow know type violations before executing 30 Techniques of Kotlin DSL - Getting Started - 1/3
  31. Static Type Checking ▸ Code analysis based type safety verification

    ▸ In contrast: Dynamic type checking ▸ Allow know type violations before executing 31 Groovy : no error but fail at runtime Techniques of Kotlin DSL - Getting Started - 1/3
  32. Static Type Checking ▸ Code analysis based type safety verification

    ▸ In contrast: Dynamic type checking ▸ Allow know type violations before executing 32 Groovy : no error but fail at runtime Kotlin : a compilation error Techniques of Kotlin DSL - Getting Started - 1/3
  33. Syntax-Related Specifications ▸ Lambda expressions ▸ Function literals with receiver

    33 Techniques of Kotlin DSL - Getting Started - 1.5/3
  34. Lambda Expressions ▸ Syntax sugar for function type variables and

    SAM ▸ Represent a function by braces 34 Techniques of Kotlin DSL - Getting Started - 2/3
  35. Lambda Expressions - Example 35 Techniques of Kotlin DSL -

    Getting Started - 2/3
  36. Lambda Expressions - Example 36 Techniques of Kotlin DSL -

    Getting Started - 2/3
  37. Lambda Expressions - Example 37 Techniques of Kotlin DSL -

    Getting Started - 2/3
  38. Lambda Expressions - Example 38 Techniques of Kotlin DSL -

    Getting Started - 2/3
  39. Lambda Expressions - Example 39 Techniques of Kotlin DSL -

    Getting Started - 2/3
  40. Function Literals With Receiver ▸ Kotlin allows specify what is

    this scope of a function ▸ Groovy also can do this but Java cannot. ▸ Function Type : Receiver.(Argument) -> Return ▸ Great help for block-code and fluent style coding 40 Techniques of Kotlin DSL - Getting Started - 3/3
  41. Function Literals With Receiver 41 Techniques of Kotlin DSL -

    Getting Started - 3/3
  42. A Primitive Kotlin DSL ▸ Techniques described before allow create

    a simple DSL 42 Techniques of Kotlin DSL - Getting Started - Closing
  43. A Primitive Kotlin DSL - Is It Enough for Your

    Better Development? ▸ Only techniques described before allow create a simple DSL 43 Techniques of Kotlin DSL - Getting Started - Closing
  44. Break ▸ We can build DSLs easily from scratch with

    knowledges I have explained ▸ It's enough if you want type-safe and/or lambda's elegant style. ▸ Let's check what techniques can improve usability of Kotlin DSL and how we can create Kotlin DSL for existing libraries/APIs. 44 Techniques of Kotlin DSL - Getting Started - Closing
  45. Break ▸ We can build DSLs easily from scratch with

    knowledges I have explained ▸ It's enough if you want type-safe and/or lambda's elegant style. ▸ Let's check what techniques can improve usability of Kotlin DSL and how we can create Kotlin DSL for existing libraries/APIs. 45 Techniques of Kotlin DSL - Getting Started - Closing
  46. Techniques of Kotlin DSL ▸ Getting started ▸ Language specifications

    ▸ Intermediate ▸ Features for syntactic extensions ▸ Expertise ▸ Build environment stuff of Kotlin 46 Techniques of Kotlin DSL - Intermediate
  47. Syntactic Extension ▸ Extension method ▸ Operator overloading ▸ Infix

    notation ▸ Method convention ▸ Delegation 47 Techniques of Kotlin DSL - Intermediate - 0/5
  48. Syntactic Extension ▸ Extension method ▸ Operator overloading ▸ Infix

    notation ▸ Method convention ▸ Delegation 48 Techniques of Kotlin DSL - Intermediate - 1/5
  49. Extension Method ▸ Define new member methods w/o inheritances ▸

    However, we cannot avoid member visibility restrictions ▸ The benefits of Extension Method for DSL ▸ Apply the techniques which I will explain later ▸ Can create DSLs without modifying 3rd party libraries 49 Techniques of Kotlin DSL - Intermediate - 1/5
  50. Syntactic Extension ▸ Extension method ▸ Operator overloading ▸ Infix

    notation ▸ Method convention ▸ Delegation 50 Techniques of Kotlin DSL - Intermediate - 2/5
  51. Example - kotlinx.html - DOM tree generator in Kotlin ▸

    +<String> would become a text node 51 Techniques of Kotlin DSL - Intermediate - 2/5 https://github.com/Kotlin/kotlinx.html
  52. Example - kotlinx.html - DOM tree generator in Kotlin ▸

    +"any text" means appending text to the node ▸ + is shorter than *text* 52 Techniques of Kotlin DSL - Intermediate - 2/5 https://github.com/Kotlin/kotlinx.html
  53. Operator Overloading ▸ Kotlin allow overload operators for developers ▸

    We can assign a logic to several operators like +, -... ▸ The benefits of Operator Overloading for DSL ▸ Basically, operators are shorter than method names ▸ A meaning of an operation is clear 53 Techniques of Kotlin DSL - Intermediate - 2/5
  54. Syntactic Extension ▸ Extension method ▸ Operator overloading ▸ Infix

    notation ▸ Method convention ▸ Delegation 54 Techniques of Kotlin DSL - Intermediate - 3/5
  55. Example - DSL for Hamcrest API ▸ Not fluent ▸

    It might be unclear for non-hamcrest users 55 Techniques of Kotlin DSL - Intermediate - 3/5
  56. Example - DSL for Hamcrest API ▸ Looks fluent ▸

    Hamcrest's knowledge is not required 56 Techniques of Kotlin DSL - Intermediate - 3/5
  57. Infix Notation ▸ Allow call a function like *Receiver infix-function

    argument* ▸ Can omit a dot and parentheses from a method call ▸ The benefits of Infix Notation for DSL ▸ Would be more fluent style ▸ Can reduce domain knowledge 57 Techniques of Kotlin DSL - Intermediate - 3/5
  58. Syntactic Extension ▸ Extension method ▸ Operator overloading ▸ Infix

    notation ▸ Method convention ▸ Delegation 58 Techniques of Kotlin DSL - Intermediate - 4/5
  59. Example - Kotlin DSL for Gradle ▸ debug is not

    an existing function which accepts closure ▸ It's dynamically generated and be configured by a passed closure 59 Techniques of Kotlin DSL - Intermediate - 4/5 Groovy code https://github.com/gradle/gradle
  60. Example - Kotlin DSL for Gradle ▸ NOTE: create(String) creates

    a configurable object for SigningConfig ▸ Prob.: configure is domain knowledge but hidden in Groovy DSL 60 Techniques of Kotlin DSL - Intermediate - 4/5 Groovy code Raw Kotlin code https://github.com/gradle/gradle
  61. Example - Kotlin DSL for Gradle ▸ Implement an invoke

    function which accepts a lambda ▸ Could hide configure from Kotlin DSL users 61 Techniques of Kotlin DSL - Intermediate - 4/5 https://github.com/gradle/gradle/blob/90974be832395cd5eabff97c18842791d7bee3a8/subprojects/kotlin-dsl/src/main/kotlin/org/gradle/kotlin/dsl/ NamedDomainObjectContainerExtensions.kt#L36 https://github.com/gradle/gradle
  62. Method Convention ▸ This is a kind of operator overloading

    ▸ Operators/syntax which Java does not implement manually will be available if a method satisfies a convention ▸ Index-access, invoke, range operator... ▸ The benefits of Method Convention for DSL ▸ Easy to create lambda expressions-based builder by overloading invoke ▸ Can hide domain knowledge bypass a common operator/syntax in Kotlin 62 Techniques of Kotlin DSL - Intermediate - 4/5
  63. Syntactic Extension ▸ Extension method ▸ Operator overloading ▸ Infix

    notation ▸ Method convention ▸ Delegation 63 Techniques of Kotlin DSL - Intermediate - 5/5
  64. Example - Kotpref - Android SharedPreferences DSL ▸ SharedPreferences API

    is low-level ▸ A simple command-query API sequence is required 64 Techniques of Kotlin DSL - Intermediate - 5/5 https://github.com/chibatching/Kotpref
  65. Example - Kotpref - Android SharedPreferences DSL ▸ Hide low-level

    API and domain knowledge ▸ Writing a value to a property will save it to the preferences but it's hidden 65 Techniques of Kotlin DSL - Intermediate - 5/5 https://github.com/chibatching/Kotpref https://github.com/chibatching/Kotpref
  66. Delegation ▸ A property delegates to another object when accessed

    ▸ The benefits of Delegations for DSL ▸ Can hide domain knowledge ▸ Effective for DRY when using read and write operations to properties 66 Techniques of Kotlin DSL - Intermediate - 5/5
  67. Techniques of Kotlin DSL ▸ Getting started ▸ Language specifications

    ▸ Intermediate ▸ Features for syntactic extensions ▸ Expertise ▸ Build environment stuff of Kotlin 67 Techniques of Kotlin DSL - Expertise
  68. Build Environment Stuff of Kotlin ▸ The plugin system for

    the compiler ▸ SAM with receiver compiler plugin ▸ Interpolators ▸ DslMarker 68 Techniques of Kotlin DSL - Expertise - 0/2
  69. Build Environment Stuff of Kotlin ▸ The plugin system for

    the compiler ▸ SAM with receiver compiler plugin ▸ Interpolators ▸ DslMarker 69 Techniques of Kotlin DSL - Expertise - 1/2
  70. Example - Kotlin DSL for Java Code ▸ Java cannot

    specify a receiver of SAM so Kotlin cannot use functions with receiver without extension methods 70 Java code Call Container#execute from Kotlin Techniques of Kotlin DSL - Expertise - 1/2
  71. ▸ Just annotate SAM interfaces and register the annotation to

    the plugin ▸ An argument of SAM will be a receiver in Kotlin! 71 Techniques of Kotlin DSL - Expertise - 1/2 Example - Kotlin DSL for Java Code
  72. SAM With Receiver ▸ Control a receiver in SAM conversions

    ▸ For Kotlin code, but need to modify Java code ▸ The benefits of SAM-with-receiver ▸ Useful when non-Kotlin libraries assume usability of Kotlin DSL ▸ Avoid a bit awkward API in Kotlin 72 Techniques of Kotlin DSL - Expertise - 1/2 https://github.com/JetBrains/kotlin/tree/master/plugins/sam-with-receiver
  73. SAM With Receiver - Example - Kotlin DSL for Gradle

    ▸ Action API is widely used so SAM-with-receiver is better than extensions 73 https://github.com/gradle/gradle/commit/18aebdba33cd9e1be966cc286a11d3ce86a28e4f Techniques of Kotlin DSL - Expertise - 1/2 https://github.com/gradle/gradle
  74. Build Environment Stuff of Kotlin ▸ The plugin system for

    the compiler ▸ SAM with receiver compiler plugin ▸ Interpolators ▸ DslMarker 74 Techniques of Kotlin DSL - Expertise - 1/2
  75. Example - Pseudo Dom Tree Builder ▸ head tag in

    head tag should be forbidden! 75 Techniques of Kotlin DSL - Expertise - 2/2
  76. ▸ Add an annotation of @DslMarker to receiver classes or

    superclasses (and functions if necessary) 76 Techniques of Kotlin DSL - Expertise - 2/2 Example - Pseudo Dom Tree Builder
  77. ▸ Show an error because of implicit receiver violation! 77

    Techniques of Kotlin DSL - Expertise - 2/2 Example - Pseudo Dom Tree Builder
  78. DslMarker ▸ Kotlin's only DSL specific feature as far as

    I know ▸ Add a constraint of a caller scope and can raise errors if invalid ▸ The benefits of DslMarker for DSL ▸ This is required to make your DSL safe ▸ Be developers' help because IntelliJ highlight marked functions 78 Techniques of Kotlin DSL - Expertise - 2/2
  79. DslMarker - Example - kotlinx.html ▸ Of course, kotlinx.html uses

    DslMarker to build DOM tree correctly ▸ html, head, body are marked functions so highlighted 79 Techniques of Kotlin DSL - Expertise - 2/2
  80. Build Environment Stuff of Kotlin - Limitations ▸ Compiler plugin

    stuff is undocumented ▸ Complex logic is hard to be implemented ▸ Maintainability... ▸ DslMarker needs be annotated to classes, so it's unavailable for existing libraries 80 Techniques of Kotlin DSL - Expertise - Closing
  81. Quick tips for DSL development 81

  82. Differences Between API and DSL ▸ Technically, nothing special but...

    ▸ The concepts of the interface design are different ▸ DSL is focusing on the domain ▸ DSL is well-designed API set for the domain but also limited ▸ We should not assume DSL can work as general APIs 82 Quick Tips 1/4
  83. Quick Tips 2/4 Adapt Kotlin DSL to Existing Libraries ▸

    Aggregate existing APIs by extension method ▸ Use Operator overloading, Infix operators, Method conventions ▸ Create DTOs if you would like to use DslMarker 83
  84. Quick Tips 3/4 How To Keep Both Usabilities of Kotlin

    DSL and Other JVM Languages ▸ Java -> Kotlin DSL ▸ Use SAM with receiver ▸ Kotlin DSL -> Other JVM languages ▸ Provide method chaining interface because of functions with receiver issue ▸ Provide alternative non-inline APIs because of inline features' availability ▸ Split API for other JVM languages and Kotlin DSL 84
  85. Quick Tips 3/4 How To Keep Both Usabilities of Kotlin

    DSL and Other JVM Languages ▸ Java -> Kotlin DSL ▸ Use SAM with receiver ▸ Kotlin DSL -> Other JVM languages ▸ Provide method chaining interface because of functions with receiver issue ▸ Provide alternative non-inline APIs because of inline features' availability ▸ Split API for other JVM languages and Kotlin DSL ▸ Other JVM languages -> Kotlin DSL : I don't know, sorry 85
  86. Quick Tips 4/4 For Advanced Steps ▸ Read Martin Fowler's

    DSL Patterns if you want to know DSL interfaces ▸ Good examples of Kotlin DSL ▸ See appendix! 86
  87. Closing 87

  88. Closing Summary 1/3 ▸ Kotlin DSL is ▸ A pure

    Kotlin code ▸ Not only for Kotlin libraries ▸ Built on several techniques 88
  89. Summary 2/3 ▸ Language specs ▸ Static type ▸ Lambda

    expressions w/ receiver control ▸ Language features ▸ Extension method, operator overloading, infix notation, method convention, delegation ▸ Build environment features ▸ SAM-with-receiver, DslMarker 89 Closing
  90. Summary 3/3 ▸ DslMarker is the only DSL specific feature

    but improves safety! ▸ Kotlin DSL can extend existing libraries with only a few constraints ▸ We can co-exist usability for the both of Kotlin and other JVM languages ▸ If you work hard 90 Closing
  91. Appendix 1/2 ▸ Books ▸ Domain Specific Languages by M.

    Fowler with R. Parsons ▸ DSLs in Action by D. Ghosh ▸ Dsl Engineering: Designing, Implementing and Using Domain-specific Languages by M. Voelter ▸ Kotlin in Action by D. Jemerrov and S. Isakova ▸ Web/Blog ▸ Catalog of DSL Patterns https://martinfowler.com/dslCatalog/ ▸ Kotlin DSL: From Theory to Practice https://dzone.com/articles/kotlin-dsl-from-theory-to-practice ▸ Writing a Kotlin DSL (series) https://proandroiddev.com/writing-dsls-in-kotlin-part-1-7f5d2193f277 ▸ Idiomatic Kotlin. Best Practices. https://phauer.com/2017/idiomatic-kotlin-best-practices/ ▸ and more... 91 Closing
  92. Appendix 2/2 ▸ Libraries ▸ Build system : gradle/gradle/tree/master/subprojects/kotlin-dsl ▸

    Android : android-ktx, Jetpack Compose, Kotlin/anko, JakeWharton/kotterknife, chibatching/Kotpref, infotech-group/ android-drawable-dsl, agoda-com/Kakao ▸ Web : Kotlin/kotlinx.html, perwendel/spark-kotlin, spring-projects/spring-fu, ktorio/ktor ▸ Testing libraries : MarkusAmshove/Kluent, winterbe/expekt, npryce/hamkrest, nhaarman/mockito-kotlin, kotlintest/ kotlintest, spekframework/spek ▸ DI containers : Kodein-Framework/Kodein-DI, Ekito/koin, authzee/kotlin-guice ▸ Otherwise: skrapeit/skrape.it, JetBrains/Exposed, x2bool/kuery, ruslanys/telegraff, ▸ Words ▸ Kotlin DSL, DSL, kotlish, type builder, kotlinize, idiomatic Kotlin 92 Closing
  93. Fin. Thank You for Listening! ▸ Jumpei Matsuda / daruma

    ▸ Software Engineer @DeployGate ▸ GitHub: jmatsu ▸ Twitter: red_fat_daruma ▸ Please feel free to ask questions ▸ ೔ຊޠŧŔŕŪũƄŝſ (Ӊ౎ٶҭͪͷ७೔ຊਓͰ͢) 93