Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Dive into Kotlin DSL

Deep Dive into Kotlin DSL

A presentation at Kotlin Fest 2019 in Japan.

Matsuda Jumpei

August 24, 2019
Tweet

More Decks by Matsuda Jumpei

Other Decks in Programming

Transcript

  1. Deep Dive into Kotlin DSL Kotlin Fest 2019 @red_fat_daruma

  2. Self Introduction Jumpei Matsuda / Daruma / @red_fat_daruma ▸ Software

    Engineer @DeployGate ▸ GitHub: jmatsu / Twitter: red_fat_daruma ▸ JVM projects : Android apps/SDK, Scala server- side and Gradle plugins ▸ One of Kotlin users since M10 2
  3. Today's Goal Hopefully, Every Audience Will.... ▸ Know what is

    DSL and/or Kotlin DSL ▸ Understand what specifications/features of Kotlin support Kotlin DSL ▸ Have a motivation to develop the original Kotlin DSL and/or extend existing libraries through Kotlin DSL. 3
  4. Let me ask questions! 4

  5. Who thinks Kotlin DSL is for Gradle? ✋ 5

  6. I do not use a word *Kotlin DSL* to represent

    Kotlin DSL for Gradle Kotlin DSL Is Not Only Kotlin DSL for Gradle ▸ *Kotlin DSL* in this presentation does not mean Gradle's one ▸ Kotlin DSL for Gradle is just a DSL in Kotlin for Gradle ▸ Contracts, coroutines, etc. are also Kotlin DSL 6
  7. I do not use a word *Kotlin DSL* to represent

    Kotlin DSL for Gradle Kotlin DSL Is Not Only Kotlin DSL for Gradle ▸ *Kotlin DSL* in this presentation does not mean Gradle's one ▸ Kotlin DSL for Gradle is just a DSL in Kotlin for Gradle 7 Go to Next questions!
  8. Who have used and/or are using Kotlin DSL? ✋ 8

  9. Who know what techniques will help Kotlin DSL? ✋ 9

  10. Who can explain the definition of Kotlin DSL? ✋ 10

  11. Who can explain the definition of DSL? ✋ 11

  12. If you said YES to all questions, you are already

    Kotlin DSL deep diver! 12
  13. Today's topics Agenda ▸ The base knowledge of DSL ▸

    Specifications/techniques of Kotlin DSL based on levels ▸ Quick tips for DSL development 13
  14. The base knowledges of DSL Domain Specific Language ▸ A

    computer language specialized to a particular application domain ▸ In contrast: a general purpose language like Kotlin, Java 14 ref: https://en.wikipedia.org/wiki/Domain-specific_language
  15. Domain Specific Language ▸ A computer language specialized to a

    particular application domain ▸ In contrast: a general purpose language like Kotlin, Java ▸ What's a motivation to introduce/need DSL? 15 The base knowledges of DSL ref: https://en.wikipedia.org/wiki/Domain-specific_language
  16. Why People Want To Use DSL ▸ Simplify complex code

    ▸ More readable, understandable, fluent ▸ Use smart and/or minimum APIs for the domain ▸ Improve productivity and reduce human errors ▸ Get a power of type system etc. 16 The base knowledges of DSL
  17. Why People Want To Use DSL 17 The base knowledges

    of DSL NOTE: pseudo code DSL
  18. Aspects of Domain Specific Language ▸ External DSL and Internal

    DSL ▸ External: A language that's parsed independently of the host general purpose language ▸ Internal: A particular form of API in the host general purpose language for the domain 18 The base knowledges of DSL ref: https://martinfowler.com/books/dsl.html
  19. Examples in the Real World ▸ External DSL ▸ SQL,

    CSS, Regular expressions ▸ Internal DSL ▸ HTML, Mock Libraries, DI container 19 The base knowledges of DSL
  20. Examples in the Real World ▸ External DSL ▸ SQL,

    CSS, Regular expressions ▸ Internal DSL ▸ HTML, Mocking Libraries, DI container ▸ Kotlin DSL 20 The base knowledges of DSL
  21. Examples in the Real World ▸ External DSL ▸ SQL,

    CSS, Regular expressions ▸ Internal DSL ▸ HTML, Mocking Libraries, DI container ▸ Kotlin DSL ▸ NOT a special edition of Kotlin 21 The base knowledges of DSL
  22. Techniques for Internal DSL Depends on... ▸ The host general

    purpose language ▸ Type checking ▸ Syntax ▸ Syntactic extensions ▸ Compiler extensibility and interpolation 22 The base knowledges of DSL
  23. Techniques for Internal Kotlin DSL Depends on... ▸ A general

    purpose language Kotlin ▸ Type checking Static type checking ▸ Syntax Lambda expressions, etc. ▸ Syntactic extensions Method extension, operator overloading, etc. ▸ Compiler extensibility and/or interpolation Compiler plugins etc. 23 The base knowledges of DSL
  24. e.g. kotlinx.html is one of Kotlin DSL ▸ DSL for

    HTML domain ▸ Generate DOM tree from a Kotlin code 24 The base knowledges of DSL https://github.com/Kotlin/kotlinx.html
  25. e.g. kotlinx.html is one of Kotlin DSL ▸ DSL for

    HTML domain ▸ Generate DOM tree from a Kotlin code 25 Type safe builder pattern, lambda expressions, functions with receiver, higher-order functions, operator overloading, DSLMarker, etc. The base knowledges of DSL
  26. Today's topics Agenda ▸ The base knowledge of DSL ▸

    Specifications/techniques of Kotlin DSL based on levels ▸ Quick tips for DSL development 26 ▸ NOTE: DSL in following slides means Internal DSL unless specified
  27. Techniques of Kotlin DSL Techniques of Kotlin DSL ▸ Getting

    started ▸ Language specifications ▸ Intermediate ▸ Features for syntactic extensions ▸ Expertise ▸ Build environment stuff of Kotlin 27
  28. Techniques of Kotlin DSL ▸ Getting started ▸ Language specifications

    ▸ Intermediate ▸ Features for syntactic extensions ▸ Expertise ▸ Build environment stuff of Kotlin 28 Techniques of Kotlin DSL - Getting Started
  29. Language Specifications ▸ Type system ▸ Static type checking ▸

    Syntax related specifications ▸ Lambdas, Functions literals with receiver 29 Techniques of Kotlin DSL - Getting Started
  30. Static Type Checking ▸ Code analysis based type safety verification

    ▸ In contrast: Dynamic type checking ▸ Allow know type violations before executing 30 Techniques of Kotlin DSL - Getting Started - 1/3
  31. Static Type Checking ▸ Code analysis based type safety verification

    ▸ In contrast: Dynamic type checking ▸ Allow know type violations before executing 31 Groovy : no error but fail at runtime Techniques of Kotlin DSL - Getting Started - 1/3
  32. Static Type Checking ▸ Code analysis based type safety verification

    ▸ In contrast: Dynamic type checking ▸ Allow know type violations before executing 32 Groovy : no error but fail at runtime Kotlin : a compilation error Techniques of Kotlin DSL - Getting Started - 1/3
  33. Syntax-Related Specifications ▸ Lambda expressions ▸ Function literals with receiver

    33 Techniques of Kotlin DSL - Getting Started - 1.5/3
  34. Lambda Expressions ▸ Syntax sugar for function type variables and

    SAM ▸ Represent a function by braces 34 Techniques of Kotlin DSL - Getting Started - 2/3
  35. Lambda Expressions - Example 35 Techniques of Kotlin DSL -

    Getting Started - 2/3
  36. Lambda Expressions - Example 36 Techniques of Kotlin DSL -

    Getting Started - 2/3
  37. Lambda Expressions - Example 37 Techniques of Kotlin DSL -

    Getting Started - 2/3
  38. Lambda Expressions - Example 38 Techniques of Kotlin DSL -

    Getting Started - 2/3
  39. Lambda Expressions - Example 39 Techniques of Kotlin DSL -

    Getting Started - 2/3
  40. Function Literals With Receiver ▸ Kotlin allows specify what is

    this scope of a function ▸ Groovy also can do this but Java cannot. ▸ Function Type : Receiver.(Argument) -> Return ▸ Great help for block-code and fluent style coding 40 Techniques of Kotlin DSL - Getting Started - 3/3
  41. Function Literals With Receiver 41 Techniques of Kotlin DSL -

    Getting Started - 3/3
  42. A Primitive Kotlin DSL ▸ Techniques described before allow create

    a simple DSL 42 Techniques of Kotlin DSL - Getting Started - Closing
  43. A Primitive Kotlin DSL - Is It Enough for Your

    Better Development? ▸ Only techniques described before allow create a simple DSL 43 Techniques of Kotlin DSL - Getting Started - Closing
  44. Break ▸ We can build DSLs easily from scratch with

    knowledges I have explained ▸ It's enough if you want type-safe and/or lambda's elegant style. ▸ Let's check what techniques can improve usability of Kotlin DSL and how we can create Kotlin DSL for existing libraries/APIs. 44 Techniques of Kotlin DSL - Getting Started - Closing
  45. Break ▸ We can build DSLs easily from scratch with

    knowledges I have explained ▸ It's enough if you want type-safe and/or lambda's elegant style. ▸ Let's check what techniques can improve usability of Kotlin DSL and how we can create Kotlin DSL for existing libraries/APIs. 45 Techniques of Kotlin DSL - Getting Started - Closing
  46. Techniques of Kotlin DSL ▸ Getting started ▸ Language specifications

    ▸ Intermediate ▸ Features for syntactic extensions ▸ Expertise ▸ Build environment stuff of Kotlin 46 Techniques of Kotlin DSL - Intermediate
  47. Syntactic Extension ▸ Extension method ▸ Operator overloading ▸ Infix

    notation ▸ Method convention ▸ Delegation 47 Techniques of Kotlin DSL - Intermediate - 0/5
  48. Syntactic Extension ▸ Extension method ▸ Operator overloading ▸ Infix

    notation ▸ Method convention ▸ Delegation 48 Techniques of Kotlin DSL - Intermediate - 1/5
  49. Extension Method ▸ Define new member methods w/o inheritances ▸

    However, we cannot avoid member visibility restrictions ▸ The benefits of Extension Method for DSL ▸ Apply the techniques which I will explain later ▸ Can create DSLs without modifying 3rd party libraries 49 Techniques of Kotlin DSL - Intermediate - 1/5
  50. Syntactic Extension ▸ Extension method ▸ Operator overloading ▸ Infix

    notation ▸ Method convention ▸ Delegation 50 Techniques of Kotlin DSL - Intermediate - 2/5
  51. Example - kotlinx.html - DOM tree generator in Kotlin ▸

    +<String> would become a text node 51 Techniques of Kotlin DSL - Intermediate - 2/5 https://github.com/Kotlin/kotlinx.html
  52. Example - kotlinx.html - DOM tree generator in Kotlin ▸

    +"any text" means appending text to the node ▸ + is shorter than *text* 52 Techniques of Kotlin DSL - Intermediate - 2/5 https://github.com/Kotlin/kotlinx.html
  53. Operator Overloading ▸ Kotlin allow overload operators for developers ▸

    We can assign a logic to several operators like +, -... ▸ The benefits of Operator Overloading for DSL ▸ Basically, operators are shorter than method names ▸ A meaning of an operation is clear 53 Techniques of Kotlin DSL - Intermediate - 2/5
  54. Syntactic Extension ▸ Extension method ▸ Operator overloading ▸ Infix

    notation ▸ Method convention ▸ Delegation 54 Techniques of Kotlin DSL - Intermediate - 3/5
  55. Example - DSL for Hamcrest API ▸ Not fluent ▸

    It might be unclear for non-hamcrest users 55 Techniques of Kotlin DSL - Intermediate - 3/5
  56. Example - DSL for Hamcrest API ▸ Looks fluent ▸

    Hamcrest's knowledge is not required 56 Techniques of Kotlin DSL - Intermediate - 3/5
  57. Infix Notation ▸ Allow call a function like *Receiver infix-function

    argument* ▸ Can omit a dot and parentheses from a method call ▸ The benefits of Infix Notation for DSL ▸ Would be more fluent style ▸ Can reduce domain knowledge 57 Techniques of Kotlin DSL - Intermediate - 3/5
  58. Syntactic Extension ▸ Extension method ▸ Operator overloading ▸ Infix

    notation ▸ Method convention ▸ Delegation 58 Techniques of Kotlin DSL - Intermediate - 4/5
  59. Example - Kotlin DSL for Gradle ▸ debug is not

    an existing function which accepts closure ▸ It's dynamically generated and be configured by a passed closure 59 Techniques of Kotlin DSL - Intermediate - 4/5 Groovy code https://github.com/gradle/gradle
  60. Example - Kotlin DSL for Gradle ▸ NOTE: create(String) creates

    a configurable object for SigningConfig ▸ Prob.: configure is domain knowledge but hidden in Groovy DSL 60 Techniques of Kotlin DSL - Intermediate - 4/5 Groovy code Raw Kotlin code https://github.com/gradle/gradle
  61. Example - Kotlin DSL for Gradle ▸ Implement an invoke

    function which accepts a lambda ▸ Could hide configure from Kotlin DSL users 61 Techniques of Kotlin DSL - Intermediate - 4/5 https://github.com/gradle/gradle/blob/90974be832395cd5eabff97c18842791d7bee3a8/subprojects/kotlin-dsl/src/main/kotlin/org/gradle/kotlin/dsl/ NamedDomainObjectContainerExtensions.kt#L36 https://github.com/gradle/gradle
  62. Method Convention ▸ This is a kind of operator overloading

    ▸ Operators/syntax which Java does not implement manually will be available if a method satisfies a convention ▸ Index-access, invoke, range operator... ▸ The benefits of Method Convention for DSL ▸ Easy to create lambda expressions-based builder by overloading invoke ▸ Can hide domain knowledge bypass a common operator/syntax in Kotlin 62 Techniques of Kotlin DSL - Intermediate - 4/5
  63. Syntactic Extension ▸ Extension method ▸ Operator overloading ▸ Infix

    notation ▸ Method convention ▸ Delegation 63 Techniques of Kotlin DSL - Intermediate - 5/5
  64. Example - Kotpref - Android SharedPreferences DSL ▸ SharedPreferences API

    is low-level ▸ A simple command-query API sequence is required 64 Techniques of Kotlin DSL - Intermediate - 5/5 https://github.com/chibatching/Kotpref
  65. Example - Kotpref - Android SharedPreferences DSL ▸ Hide low-level

    API and domain knowledge ▸ Writing a value to a property will save it to the preferences but it's hidden 65 Techniques of Kotlin DSL - Intermediate - 5/5 https://github.com/chibatching/Kotpref https://github.com/chibatching/Kotpref
  66. Delegation ▸ A property delegates to another object when accessed

    ▸ The benefits of Delegations for DSL ▸ Can hide domain knowledge ▸ Effective for DRY when using read and write operations to properties 66 Techniques of Kotlin DSL - Intermediate - 5/5
  67. Techniques of Kotlin DSL ▸ Getting started ▸ Language specifications

    ▸ Intermediate ▸ Features for syntactic extensions ▸ Expertise ▸ Build environment stuff of Kotlin 67 Techniques of Kotlin DSL - Expertise
  68. Build Environment Stuff of Kotlin ▸ The plugin system for

    the compiler ▸ SAM with receiver compiler plugin ▸ Interpolators ▸ DslMarker 68 Techniques of Kotlin DSL - Expertise - 0/2
  69. Build Environment Stuff of Kotlin ▸ The plugin system for

    the compiler ▸ SAM with receiver compiler plugin ▸ Interpolators ▸ DslMarker 69 Techniques of Kotlin DSL - Expertise - 1/2
  70. Example - Kotlin DSL for Java Code ▸ Java cannot

    specify a receiver of SAM so Kotlin cannot use functions with receiver without extension methods 70 Java code Call Container#execute from Kotlin Techniques of Kotlin DSL - Expertise - 1/2
  71. ▸ Just annotate SAM interfaces and register the annotation to

    the plugin ▸ An argument of SAM will be a receiver in Kotlin! 71 Techniques of Kotlin DSL - Expertise - 1/2 Example - Kotlin DSL for Java Code
  72. SAM With Receiver ▸ Control a receiver in SAM conversions

    ▸ For Kotlin code, but need to modify Java code ▸ The benefits of SAM-with-receiver ▸ Useful when non-Kotlin libraries assume usability of Kotlin DSL ▸ Avoid a bit awkward API in Kotlin 72 Techniques of Kotlin DSL - Expertise - 1/2 https://github.com/JetBrains/kotlin/tree/master/plugins/sam-with-receiver
  73. SAM With Receiver - Example - Kotlin DSL for Gradle

    ▸ Action API is widely used so SAM-with-receiver is better than extensions 73 https://github.com/gradle/gradle/commit/18aebdba33cd9e1be966cc286a11d3ce86a28e4f Techniques of Kotlin DSL - Expertise - 1/2 https://github.com/gradle/gradle
  74. Build Environment Stuff of Kotlin ▸ The plugin system for

    the compiler ▸ SAM with receiver compiler plugin ▸ Interpolators ▸ DslMarker 74 Techniques of Kotlin DSL - Expertise - 1/2
  75. Example - Pseudo Dom Tree Builder ▸ head tag in

    head tag should be forbidden! 75 Techniques of Kotlin DSL - Expertise - 2/2
  76. ▸ Add an annotation of @DslMarker to receiver classes or

    superclasses (and functions if necessary) 76 Techniques of Kotlin DSL - Expertise - 2/2 Example - Pseudo Dom Tree Builder
  77. ▸ Show an error because of implicit receiver violation! 77

    Techniques of Kotlin DSL - Expertise - 2/2 Example - Pseudo Dom Tree Builder
  78. DslMarker ▸ Kotlin's only DSL specific feature as far as

    I know ▸ Add a constraint of a caller scope and can raise errors if invalid ▸ The benefits of DslMarker for DSL ▸ This is required to make your DSL safe ▸ Be developers' help because IntelliJ highlight marked functions 78 Techniques of Kotlin DSL - Expertise - 2/2
  79. DslMarker - Example - kotlinx.html ▸ Of course, kotlinx.html uses

    DslMarker to build DOM tree correctly ▸ html, head, body are marked functions so highlighted 79 Techniques of Kotlin DSL - Expertise - 2/2
  80. Build Environment Stuff of Kotlin - Limitations ▸ Compiler plugin

    stuff is undocumented ▸ Complex logic is hard to be implemented ▸ Maintainability... ▸ DslMarker needs be annotated to classes, so it's unavailable for existing libraries 80 Techniques of Kotlin DSL - Expertise - Closing
  81. Quick tips for DSL development 81

  82. Differences Between API and DSL ▸ Technically, nothing special but...

    ▸ The concepts of the interface design are different ▸ DSL is focusing on the domain ▸ DSL is well-designed API set for the domain but also limited ▸ We should not assume DSL can work as general APIs 82 Quick Tips 1/4
  83. Quick Tips 2/4 Adapt Kotlin DSL to Existing Libraries ▸

    Aggregate existing APIs by extension method ▸ Use Operator overloading, Infix operators, Method conventions ▸ Create DTOs if you would like to use DslMarker 83
  84. Quick Tips 3/4 How To Keep Both Usabilities of Kotlin

    DSL and Other JVM Languages ▸ Java -> Kotlin DSL ▸ Use SAM with receiver ▸ Kotlin DSL -> Other JVM languages ▸ Provide method chaining interface because of functions with receiver issue ▸ Provide alternative non-inline APIs because of inline features' availability ▸ Split API for other JVM languages and Kotlin DSL 84
  85. Quick Tips 3/4 How To Keep Both Usabilities of Kotlin

    DSL and Other JVM Languages ▸ Java -> Kotlin DSL ▸ Use SAM with receiver ▸ Kotlin DSL -> Other JVM languages ▸ Provide method chaining interface because of functions with receiver issue ▸ Provide alternative non-inline APIs because of inline features' availability ▸ Split API for other JVM languages and Kotlin DSL ▸ Other JVM languages -> Kotlin DSL : I don't know, sorry 85
  86. Quick Tips 4/4 For Advanced Steps ▸ Read Martin Fowler's

    DSL Patterns if you want to know DSL interfaces ▸ Good examples of Kotlin DSL ▸ See appendix! 86
  87. Closing 87

  88. Closing Summary 1/3 ▸ Kotlin DSL is ▸ A pure

    Kotlin code ▸ Not only for Kotlin libraries ▸ Built on several techniques 88
  89. Summary 2/3 ▸ Language specs ▸ Static type ▸ Lambda

    expressions w/ receiver control ▸ Language features ▸ Extension method, operator overloading, infix notation, method convention, delegation ▸ Build environment features ▸ SAM-with-receiver, DslMarker 89 Closing
  90. Summary 3/3 ▸ DslMarker is the only DSL specific feature

    but improves safety! ▸ Kotlin DSL can extend existing libraries with only a few constraints ▸ We can co-exist usability for the both of Kotlin and other JVM languages ▸ If you work hard 90 Closing
  91. Appendix 1/2 ▸ Books ▸ Domain Specific Languages by M.

    Fowler with R. Parsons ▸ DSLs in Action by D. Ghosh ▸ Dsl Engineering: Designing, Implementing and Using Domain-specific Languages by M. Voelter ▸ Kotlin in Action by D. Jemerrov and S. Isakova ▸ Web/Blog ▸ Catalog of DSL Patterns https://martinfowler.com/dslCatalog/ ▸ Kotlin DSL: From Theory to Practice https://dzone.com/articles/kotlin-dsl-from-theory-to-practice ▸ Writing a Kotlin DSL (series) https://proandroiddev.com/writing-dsls-in-kotlin-part-1-7f5d2193f277 ▸ Idiomatic Kotlin. Best Practices. https://phauer.com/2017/idiomatic-kotlin-best-practices/ ▸ and more... 91 Closing
  92. Appendix 2/2 ▸ Libraries ▸ Build system : gradle/gradle/tree/master/subprojects/kotlin-dsl ▸

    Android : android-ktx, Jetpack Compose, Kotlin/anko, JakeWharton/kotterknife, chibatching/Kotpref, infotech-group/ android-drawable-dsl, agoda-com/Kakao ▸ Web : Kotlin/kotlinx.html, perwendel/spark-kotlin, spring-projects/spring-fu, ktorio/ktor ▸ Testing libraries : MarkusAmshove/Kluent, winterbe/expekt, npryce/hamkrest, nhaarman/mockito-kotlin, kotlintest/ kotlintest, spekframework/spek ▸ DI containers : Kodein-Framework/Kodein-DI, Ekito/koin, authzee/kotlin-guice ▸ Otherwise: skrapeit/skrape.it, JetBrains/Exposed, x2bool/kuery, ruslanys/telegraff, ▸ Words ▸ Kotlin DSL, DSL, kotlish, type builder, kotlinize, idiomatic Kotlin 92 Closing
  93. Fin. Thank You for Listening! ▸ Jumpei Matsuda / daruma

    ▸ Software Engineer @DeployGate ▸ GitHub: jmatsu ▸ Twitter: red_fat_daruma ▸ Please feel free to ask questions ▸ ೔ຊޠŧŔŕŪũƄŝſ (Ӊ౎ٶҭͪͷ७೔ຊਓͰ͢) 93