Code Generation in Kotlin with Annotation and KotlinPoet

©2019 Wantedly, Inc.
Code Generation in Kotlin
with Annotation and KotlinPoet
Kotlin Fest 2019
Aug 24, 2019 - Malvin Sutanto
Photo by Dave Michuda on Unsplash

View Slide

Introduction
Malvin Sutanto
Software Engineer @Wantedly
Android, Kotlin.
Twitter/ Medium: @malvinsutanto

View Slide

Metaprogramming

View Slide

Treat other programs as their data
To add or change behavior and functionalities
• Runtime: Reflection and parser
• Compile time: Generator
Metaprogramming https://jakewharton.com/mechanisms-of-metaprogramming/

View Slide

Read files and properties during runtime
Read from Java or Kotlin classes
• Read variables
• Invoke functions
Interpret files, like XML or JSON
Metaprogramming Reflection and parser

View Slide

Considered a bad practice
and difficult to maintain
Page Title Page Subtitle

View Slide

Generate files based on input
Annotation
Can output source code, byte code, or any type of files
Can be compile safe
Better performance during runtime
Dagger 2 for dependency injection
Metaprogramming Generator

View Slide

Dagger 2 Example
Metaprogramming
@Inject
lateinit var repository: Repository
!!...
DaggerComponent.inject(this)

View Slide

Annotation Processor

View Slide

API added in Java 6
Reads representation of your code
• Not the code itself
• Code has not been compiled yet
• Special APIs to read code
Annotation Processor Annotation

View Slide

Annotation Processor Convert data class to a map
@AutoMap
data class Person(
val id: Long,
val name: String?,
val address: Address
)
/**
* Converts [Person] to [Map].
!*/
fun Person.toMap(): Map {
val map: Map = mapOf(
"id" to id,
"name" to name,
"address" to address.toMap()
)
return map
}
@AutoMap
data class Address(
val streetName: String,
val city: String
)
/**
* Converts [Address] to [Map].
!*/
fun Address.toMap(): Map {
"streetName" to streetName,
"city" to city
)
return map
}

View Slide

Annotation Processor Annotation module
!// :annotation/build.gradle
apply plugin: 'kotlin'
dependencies {
implementation “org.jetbrains.kotlin:kotlin-stdlib:1.3.41”
}
Java/Kotlin module

View Slide

@Target(AnnotationTarget.CLASS)
@Retention(AnnotationRetention.SOURCE)
annotation class AutoMap
Annotation Processor Annotation class

View Slide

Annotation Processor Processor module
!// :processor/build.gradle
apply plugin: ‘kotlin-kapt'
dependencies {
implementation project(':annotation')
!// Kotlin metadata
implementation “me.eugeniomarletti.kotlin.metadata:kotlin-metadata:1.4.0”
!// Configuration generator for service providers
implementation “com.google.auto.service:auto-service:1.0-rc6“
kapt "com.google.auto.service:auto-service:1.0-rc6"
!// Code generation library for kotlin
implementation “com.squareup:kotlinpoet:1.2.0”
}

View Slide

Annotation Processor Processor module
dependencies {
!// Kotlin metadata
}
Java/Kotlin module

View Slide

Annotation Processor Dependency graph
BQQMJDBUJPO
BOOPUBUJPO
QSPDFTTPS
Implementation
kapt
implementation

View Slide

AutoMapProcessor

View Slide

dependencies {
!// Kotlin metadata
}
AutoMapProcessor Kotlin metadata

View Slide

dependencies {
!// Kotlin metadata
}

View Slide

@Metadata(
mv = {1, 1, 15},
bv = {1, 0, 3},
k = 1,
d1 =
{"\u00002\n\u0002\u0018\u0002\n\u0002\u0010\u0000\n\u0000\n\u0002\u0010\t\n\u0000\n\u0002\u0010\u00
0e\n\u0000\n\u0002\u0010
\n\u0002\u0018\u0002\n\u0002\b\f\n\u0002\u0010\u000b\n\u0002\b\u0002\n\u0002\u0010\b\n\u0002\b\u000
2\b\u0086\b\u0018\u00002\u00020\u0001B%
\u0012\u0006\u0010\u0002\u001a\u00020\u0003\u0012\b\u0010\u0004\u001a\u0004\u0018\u00010\u0005\u001
2\f\u0010\u0006\u001a\b\u0012\u0004\u0012\u00020\b0\u0007¢
\u0006\u0002\u0010\tJ\t\u0010\u0010\u001a\u0002!!...”},
d2 = {"Lcom/malvinstn/myannotationprocessing/Person;", "", "id", "", "name", "", "address", "",
"Lcom/malvinstn/myannotationprocessing/Address;", "(JLjava/lang/String;Ljava/util/List;)V",
"getAddress", "()Ljava/util/List;", "getId", "()J", "getName", "()Ljava/lang/String;",
"component1", "component2", "component3", "copy", "equals", "", "other", "hashCode", "",
"toString", "app_debug"}
)
public final class Person {

View Slide

Kotlinx metadata
• From JetBrains
• https://github.com/JetBrains/kotlin/tree/master/libraries/kotlinx-metadata/jvm
AutoMapProcessor Kotlinx metadata

View Slide

AutoMapProcessor Processor class
class AutoMapProcessor : KotlinAbstractProcessor(), KotlinMetadataUtils {
override fun getSupportedAnnotationTypes(): Set =
setOf(AutoMap!::class.java.canonicalName)
override fun process(annotations: Set, roundEnv: RoundEnvironment): Boolean {
TODO("Process kotlin code here")
}
}

View Slide

dependencies {
!// Kotlin metadata
}
AutoMapProcessor Auto service

View Slide

AutoMapProcessor Auto service
dependencies {
!// Kotlin metadata
}

View Slide

AutoMapProcessor Processor class
}
}

View Slide

AutoMapProcessor Processor class annotated with AutoService
@AutoService(Processor!::class)
}
}

View Slide

AutoMapProcessor Kotlin class syntax
class KotlinClass {
private var count: Int = 0
fun incrementCount() {
count += 1
}
fun incrementCountBy(value: Int) {
count += value
}
}
fun KotlinClass.printCount(): String = "Count is: $count"

View Slide

AutoMapProcessor Type
class KotlinClass {
count += 1
}
count += value
}
}

View Slide

AutoMapProcessor Members
class KotlinClass {
count += 1
}
count += value
}
}

View Slide

AutoMapProcessor Parameter
class KotlinClass {
count += 1
}
count += value
}
}

View Slide

AutoMapProcessor Top level member
class KotlinClass {
count += 1
}
count += value
}
}

View Slide

AutoMapProcessor Return type
class KotlinClass {
count += 1
}
count += value
}
}

View Slide

AutoMapProcessor Visibility modifier
class KotlinClass {
count += 1
}
count += value
}
}

View Slide

AutoMapProcessor Kotlin class syntax
class KotlinClass {
count += 1
}
count += value
}
}

View Slide

Processing Kotlin code

View Slide

Processing Kotlin code Get annotated element
val elements = roundEnv.getElementsAnnotatedWith(AutoMap!::class.java)
!// Loop through all annotated element
for (element in elements) {
!// Check if it is a public data class.
!// Read the declared variables of each element.
!// Check if a declared variable’s Type is also annotated with @AutoMap.
!// Generate code with KotlinPoet.
}
return true
}

View Slide

Processing Kotlin code Get annotated element
val elements = roundEnv.getElementsAnnotatedWith(AutoMap!::class.java)
!// Loop through all annotated element
!// Check if it is a public data class.
!// Generate code with KotlinPoet.
}
return true
}
for (element in elements)

View Slide

Processing Kotlin code Validating element
!// Check if it is a valid element.
val metadata: KotlinClassMetadata? = element.kotlinMetadata as? KotlinClassMetadata
if (element !is TypeElement !|| metadata !== null) {
messager.printMessage(Diagnostic.Kind.ERROR, "Not a class", element)
continue
}
val classProto: ProtoBuf.Class = metadata.data.classProto
!!...
}
for (element in elements)

View Slide

continue
}
!!...
}

View Slide

Processing Kotlin code ProtoBuf.Class
classProto.constructorList
classProto.supertypeList
classProto.typeParameterList
classProto.functionList
classProto.modality
!!...

View Slide

continue
}
!!...
}

View Slide

!!...
if (!classProto.isDataClass !|| !// Is not a data class.
classProto.visibility !!= ProtoBuf.Visibility.PUBLIC !|| !// Is not public.
classProto.isInnerClass !// Is an inner class.
) {
messager.printMessage(Diagnostic.Kind.ERROR, "Not a public data class", element)
continue
}
}

View Slide

!!...
) {
continue
}
}

View Slide

Processing Kotlin code Reading class’ variables
!!...
val variables: List> = element.enclosedElements
.filterIsInstance()
.map { variable !->
val varType: TypeElement? = typeUtils.asElement(variable.asType()) as? TypeElement
val hasAnnotation: Boolean = varType!?.getAnnotation(AutoMap!::class.java) !!= null
variable.simpleName.toString() to varType!?.takeIf { hasAnnotation }
}
generateCode(element, variables)
}

View Slide

Generate Kotlin Code
with KotlinPoet

View Slide

dependencies {
!// Kotlin metadata
}
Generate Kotlin Code KotlinPoet

View Slide

Generate Kotlin Code KotlinPoet
dependencies {
!// Kotlin metadata
}

View Slide

@AutoMap
data class Person(
val id: Long,
val name: String?,
)
/**
!*/
"id" to id,
"name" to name,
)
return map
}
Sample

View Slide

/**
!*/
"id" to id,
"name" to name,
)
return map
}
Sample

View Slide

Generate Kotlin Code Code block
val variables: List> = !!...
val codeBlockBuilder: CodeBlock.Builder = CodeBlock.builder()

View Slide

.addStatement("val map: %T = %M(", returnType, mapOfFunction)

View Slide

val returnType: ParameterizedTypeName = !!...

View Slide

val returnType: ParameterizedTypeName = ClassName("kotlin.collections", “Map") !// Map
.plusParameter(String!::class.asTypeName())
.plusParameter(Any!::class.asTypeName().copy(nullable = true))

View Slide

val returnType: ParameterizedTypeName = ClassName("kotlin.collections", "Map") !// Map

View Slide

val mapOfFunction: MemberName = MemberName("kotlin.collections", “mapOf")

View Slide

.indent()
codeBlockBuilder.unindent()
.addStatement(")")
.addStatement("return map")

View Slide

.indent()
!// Loop through all variables
variables.forEachIndexed { !!... }
codeBlockBuilder.unindent()
.addStatement(")")
.addStatement("return map")
variables.forEachIndexed

View Slide

Generate Kotlin Code Code block, cont’d
val codeBlockBuilder: CodeBlock.Builder = !!...
variables.forEachIndexed { index: Int, (name: String, varType: TypeElement?) !->
val comma = if (index < variables.size - 1) "," else ""
if (varType !!= null) {
!// Another class that is annotated with @AutoMap, use toMap instead.
val toMap = MemberName(varType.asClassName().packageName, "toMap")
codeBlockBuilder.addStatement("%S to %L.%M()%L", name, name, toMap, comma)
} else {
codeBlockBuilder.addStatement("%S to %L%L", name, name, comma)
}
}
variables.forEachIndexed

View Slide

} else {
codeBlockBuilder.addStatement("%S to %L%L", name, name, comma)
}
}

View Slide

} else {
codeBlockBuilder.addStatement("%S to %L%L", name, name, comma) !// !-> “id” to id,
}
}

View Slide

val funSpec = FunSpec.builder("toMap")
.addCode(codeBlockBuilder.build())
.build()

View Slide

/**
!*/
"id" to id,
"name" to name,
)
return map
}
Sample

View Slide

Generate Kotlin Code Receiver type
val className: ClassName = element.asClassName()
.receiver(className)
.build()

View Slide

Generate Kotlin Code Receiver type
val className: ClassName = element.asClassName()
.build()
fun Person.toMap(): Map { !!... }

View Slide

Generate Kotlin Code Parameterized return type
val className: ClassName = !!...
.returns(returnType)
.build()

View Slide

.build()
fun Person.toMap(): Map { !!... }

View Slide

Generate Kotlin Code FunSpec, other options
FunSpec.builder(!!...)
.addKDoc(!!...)
.addAnnotation(MyAnnotation!::class)
.addParameter(ParameterSpec)
.addModifiers(!!...)
.beginControlFlow()
.endControlFlow()
.nextControlFlow(!!...)
!!...

View Slide

.build()

View Slide

Generate Kotlin Code FileSpec
!!...
FileSpec.builder(className.packageName, className.simpleName)
.addFunction(funSpec)
!!...

View Slide

Generate Kotlin Code FileSpec
!!...
!!...

View Slide

Generate Kotlin Code FileSpec, other options
FileSpec.builder(!!...)
.addType(TypeSpec)
.addAnnotation(MyAnnotation!::class)
.addProperty(PropertySpec)
.addComment("This is a generated file. Do not edit”)
!!...

View Slide

Generate Kotlin Code Documentation
val mapClass = ClassName("kotlin.collections", "Map")
.addKdoc("Converts [%T] to [%T].", className, mapClass)
.build()
.addComment("This is a generated file. Do not edit")
.build()
.writeTo(outputDir)

View Slide

Generate Kotlin Code End result
package com.malvinstn.myannotationprocessing
import com.malvinstn.annotation.AutoMap
@AutoMap
data class Person(
val id: Long,
val name: String?,
)
!// This is a generated file. Do not edit
package com.malvinstn.myannotationprocessing
import kotlin.Any
import kotlin.String
import kotlin.collections.Map
import kotlin.collections.mapOf
/**
!*/
"id" to id,
"name" to name,
)
return map
}

View Slide

Testing Generated Code

View Slide

BQQMJDBUJPO
BOOPUBUJPO
QSPDFTTPS
Implementation
kapt
implementation
Testing Generated Code Test module
UFTU
implementation
kapt

View Slide

Testing Generated Code Test module
class PersonTest {
@Test
fun `test when name is null map"name" is null`() {
val person = Person(
id = 1,
name = null,
address = Address(
streetName = "streetName",
city = "city"
)
)
val map = person.toMap()
assertNull(map["name"])
}
}

View Slide

Good for testing happy paths
But unable to test and validate error cases

View Slide

Kotlin Compile Testing
• https://github.com/tschuchortdev/kotlin-compile-testing
Kompile Testing
• https://github.com/permissions-dispatcher/kompile-testing
Testing Generated Code Testing libraries

View Slide

1. Add dependency as testImplementation
• To :processor module
2. Write source code in String format
• Or load from a File
3. Use compile API from the library
4. Validate compiler’s result
• And validate generated classes
Testing Generated Code Testing libraries

View Slide

Testing Generated Code Kotlin compile testing
@Test
fun `test AutoMap, when applied to data class, then generate toMap function`() {
val result = KotlinCompilation().apply {
sources = listOf(
SourceFile.new(
“Person.kt", """
@AutoMap
data class Person(val id: Long, val name: String?)
"""
)
)
annotationProcessors = listOf(AutoMapProcessor())
!!...
}.compile()
!!...
}

View Slide

@Test
fun `test AutoMap, when applied to data class, then generate toMap function`() {
val result = !!...{
assertEquals(KotlinCompilation.ExitCode.OK, result.exitCode)
!// Assert generated classes
val kClazz = result.classLoader.loadClass("PersonKt")
val paramClazz = result.classLoader.loadClass("Person")
kClazz.getDeclaredMethod("toMap", paramClazz)
}

View Slide

public final class PersonKt {
@NotNull
public static final Map toMap(@NotNull Person $this$toMap) {
Intrinsics.checkParameterIsNotNull($this$toMap, "$this$toMap");
Map map = MapsKt.mapOf(new Pair[]{TuplesKt.to("id", $this$toMap.getId()), TuplesKt.to("name",
$this$toMap.getName()), TuplesKt.to("address", AddressKt.toMap($this$toMap.getAddress()))});
return map;
}
}

View Slide

Testing Generated Code Kotlin compile testing - testing error case
@Test
fun `test AutoMap, when applied to non data class, then exit with compilation error`() {
val result = KotlinCompilation().apply {
sources = listOf(
SourceFile.new(
"Person.kt", """
@AutoMap
class Person(val id: Long, val name: String?)
"""
)
)
!!...
}.compile()
assertEquals(KotlinCompilation.ExitCode.COMPILATION_ERROR, result.exitCode)
}

View Slide

Things to Consider
Limitations and drawbacks

View Slide

Increase the overall build time
• All steps need to be performed during compile time
• Implement incremental kapt for better performance
• Supported since Kotlin 1.3.30
• https://docs.gradle.org/5.0/userguide/
java_plugin.html#sec:incremental_annotation_processing
• https://blog.jetbrains.com/kotlin/2019/04/kotlin-1-3-30-released/
Things to Consider Build time overhead

View Slide

Increased size of your app
• Unable to modify existing files or classes
• Only create additional classes and functions
Things to Consider Code footprint

View Slide

Generated code are not accessible until kapt
is run
• “Unresolved references” error.
• Might shadow the root cause of a compilation error
Things to Consider Error handling and code visibility

View Slide

Should We Use Annotation Processor?

View Slide

1. Generate code based on members of an
element
2. Focus on API over readability
3. Deal with repeated codes
• Can be automated
• Reduce boilerplate codes
4. Time investment
Should We Use Annotation Processor?

View Slide

Interface and Abstract Class
Sometimes they’re easier to implement

View Slide

Sample Annotation Processor
at Wantedly

View Slide

Variable List for Java and Kotlin Objects
For Api Consumption
@ApiEntity
data class User(
val id: Long,
val name: String,
val wanted_score: Int,
val profile: Profile
)
@ApiEntity
data class Profile(
val introduction: String
)
val fields: List = listOf(
"id",
"name",
“wanted_score”,
“profile.introduction”,
)

View Slide

Sealed Class to Bundle
For Analytics Log
@AnalyticsEvent
sealed class MyEvent {
data class ShareImage(val imageName:
String, val fullString: String) : MyEvent()
object ButtonTapped : MyEvent()
}
/**
* Converts [MyEvent] to event name and
params and logs it using
[EventTracker.logEvent].
*
* This is a generated function. Do not edit.
!*/
fun EventTracker.logEvent(event: MyEvent) {
val name: String
val params: Bundle
when (event) {
is MyEvent.ShareImage !-> {
name = "share_image"
params = bundleOf(
"image_name" to
event.imageName,
"full_string" to
event.fullString
)
}
is MyEvent.ButtonTapped !-> {
name = "button_tapped"
params = Bundle()
}
}
logEvent(name, params)
}
medium.com/@malvinsutanto

View Slide

1. Kotlin Metadata: https://github.com/Takhion/kotlin-metadata
2. KotlinPoet: https://square.github.io/kotlinpoet/
3. Auto Service: https://github.com/google/auto/tree/master/service
4. Kotlin compile testing: https://github.com/tschuchortdev/kotlin-compile-testing
5. Kompile Testing: https://github.com/permissions-dispatcher/kompile-testing
6. Kotlin Metadata documentation: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-metadata/index.html
7. Kotlinx Metadata: https://github.com/JetBrains/kotlin/tree/master/libraries/kotlinx-metadata/jvm
8. Better Analytics in Android with Annotation Processing and KotlinPoet: https://link.medium.com/Bh8wRWOY6Y
9. Mechanisms of Metaprogramming: https://jakewharton.com/mechanisms-of-metaprogramming/
10.KotlinFest 2018 - Annotation Processing in a Kotlin World by Zac Sweers: https://youtu.be/a2RoLFzrFG0
11.Moshi: https://github.com/square/moshi
Thank You! References and links
Twitter/Medium: @malvinsutanto

View Slide

Code Generation in Kotlin with Annotation and KotlinPoet

Code Generation in Kotlin with Annotation and KotlinPoet

Malvin Sutanto

More Decks by Malvin Sutanto

Other Decks in Programming

Featured

Transcript