Develop your CI tools

Nowadays, setting up a basic unit and syntax testing pipeline for your mobile app has never been easier! That’s a great starting point, but what comes next? We can leverage the CI pipeline to do so much more! Things like automatic code generation and advanced static analysis—all with an aim to ensuring the reliability and performance of your applications.

In this talk, we’ll take a look at several tools and techniques that you can use to improve your quality of life as a developer, largely by letting the CI do the hard work for you.

Transcript

1. Develop your
   CI Tools
   

   View Slide

2. Xavier F. Gouchet
   Senior Software Engineer at Datadog
   @xgouchet / @datadoghq
   

   View Slide

3. View Slide

4. 0. Introduction
   

   View Slide

5. Small team
   ▫ 3 engineers
   ▫ Lots of features
   ▫ Lots of bug fixes
   ▫ Lots of improvements
   ▫ Small bandwidth
   Unsplash photo by @marvelous
   

   View Slide

6. CI setup
   ▫ Gitlab
   ▫ Bitrise *
   photo by @xgouchet
   

   View Slide

7. Delegating to the CI
   ▫ Runs automatically
   ▫ Strict and thorough
   ▫ Great way to offload the PR Review
   

   View Slide

8. Code Review Pyramid
   Credit: Gunnar Morling (CC-BY-SA 4.0)
   CODE STYLE
   TESTS
   IMPLEMENTATION
   API SEMANTICS
   DOCUMENTATION
   

   View Slide

9. 1. Detekt
   Your code, your rules…
   

   View Slide

10. What is Detekt?
    ▫ Kotlin Static Analysis
    ▫ Code smells
    ▫ Performance Issues
    ▫ Code complexity
    ▫ Documentation Missing
    ▫ …
    

    View Slide

11. Detekt Report
    

    View Slide

12. How does Detekt work
    ▫ Visitor pattern over the Syntax Tree (PSI)
    ▫ With Type Resolution
    ▫ Report issues on specific patterns
    

    View Slide

13. Custom Rule
    class MyCustomRule : Rule() {
    override val issue: Issue = Issue(
    javaClass.simpleName,
    Severity.CodeSmell,
    "A description of the issue…",
    Debt.TWENTY_MINS
    )
    }
    

    View Slide

14. Custom Rule
    class MyCustomRule : Rule() {
    override val issue: Issue = Issue(
    javaClass.simpleName,
    Severity.CodeSmell,
    "A description of the issue…",
    Debt.TWENTY_MINS
    )
    }
    

    View Slide

15. Custom Rule
    class MyCustomRule : Rule() {
    override val issue: Issue = Issue(
    javaClass.simpleName,
    Severity.CodeSmell,
    "A description of the issue…",
    Debt.TWENTY_MINS
    )
    }
    

    View Slide

16. Custom Rule
    override fun visitExpression(expression: KtExpression) {
    if (invalidExpression(expression))
    report(
    CodeSmell(
    issue, Entity.from(expression), "A message"
    )
    )
    }
    

    View Slide

17. Custom Rule
    override fun visitExpression(expression: KtExpression) {
    if (invalidExpression(expression))
    report(
    CodeSmell(
    issue, Entity.from(expression), "A message"
    )
    )
    }
    

    View Slide

18. Custom Rule
    override fun visitExpression(expression: KtExpression) {
    if (invalidExpression(expression))
    report(
    CodeSmell(
    issue, Entity.from(expression), "A message"
    )
    )
    }
    

    View Slide

19. A few examples of our custom rules
    ▫ Forbid throwing exceptions
    ▫ Forbid using check or require
    ▫ Forbid usage of !!
    ▫ Flag unsafe third party methods
    ▫ Flag TODO comments without a Jira ticket
    

    View Slide

20. Going Further
    ▫ Make the rule configurable from detekt.yml
    

    View Slide

21. Custom Rule
    class MyCustomRule(
    config: Config
    ) : Rule(config) {
    private val enableFoo: Boolean
    by config(defaultValue = true)
    //…
    }
    

    View Slide

22. Custom Rule
    class MyCustomRule(
    config: Config
    ) : Rule(config) {
    private val enableFoo: Boolean
    by config(defaultValue = true)
    //…
    }
    

    View Slide

23. Custom Rule
    class MyCustomRule(
    config: Config
    ) : Rule(config) {
    private val enableFoo: Boolean
    by config(defaultValue = true)
    //…
    }
    

    View Slide

24. More info…
    ▫ https:/
    /detekt.dev/
    ▫ https:/
    /github.com/detekt/detekt
    ▫ https:/
    /thebakery.dev/30/
    

    View Slide

25. 2. Kotlin Symbol Processor
    

    View Slide

26. What is KSP?
    ▫ Lightweight Compiler Plugin
    ▫ Similar to (K)APT but faster
    ▫ Source code is read-only
    ▫ Code generation
    

    View Slide

27. How does KSP work
    ▫ Called at compile time
    ▫ Processor can query files and/or declarations
    ▫ Can generate new code (but not modify existing one)
    

    View Slide

28. Custom Processor
    class MyCustomSymbolProcessor(
    val codeGenerator: CodeGenerator,
    val logger: KSPLogger
    ) : SymbolProcessor {
    override fun process(resolver: Resolver): List {
    return emptyList()
    }
    }
    

    View Slide

29. Custom Processor
    class MyCustomSymbolProcessor(
    val codeGenerator: CodeGenerator,
    val logger: KSPLogger
    ) : SymbolProcessor {
    override fun process(resolver: Resolver): List {
    return emptyList()
    }
    }
    

    View Slide

30. Custom Processor
    class MyCustomSymbolProcessor(
    val codeGenerator: CodeGenerator,
    val logger: KSPLogger
    ) : SymbolProcessor {
    override fun process(resolver: Resolver): List {
    return emptyList()
    }
    }
    

    View Slide

31. Custom Processor
    override fun process(resolver: Resolver): List {
    val deferred = mutableListOf()
    resolver.getSymbolsWithAnnotation("com.example.MyAnnotation")
    .forEach {
    if (!generateSomething(it))
    deferred.add(it)
    }
    return deferred
    }
    

    View Slide

32. Custom Processor
    override fun process(resolver: Resolver): List {
    val deferred = mutableListOf()
    resolver.getSymbolsWithAnnotation("com.example.MyAnnotation")
    .forEach {
    if (!generateSomething(it))
    deferred.add(it)
    }
    return deferred
    }
    

    View Slide

33. Custom Processor
    override fun process(resolver: Resolver): List {
    val deferred = mutableListOf()
    resolver.getSymbolsWithAnnotation("com.example.MyAnnotation")
    .forEach {
    if (!generateSomething(it))
    deferred.add(it)
    }
    return deferred
    }
    

    View Slide

34. Custom Processor
    override fun process(resolver: Resolver): List {
    val deferred = mutableListOf()
    resolver.getSymbolsWithAnnotation("com.example.MyAnnotation")
    .forEach {
    if (!generateSomething(it))
    deferred.add(it)
    }
    return deferred
    }
    

    View Slide

35. Custom Processor
    override fun process(resolver: Resolver): List {
    val deferred = mutableListOf()
    resolver.getSymbolsWithAnnotation("com.example.MyAnnotation")
    .forEach {
    if (!generateSomething(it))
    deferred.add(it)
    }
    return deferred
    }
    

    View Slide

36. An example of custom processor
    ▫ Generate no-op implementation of annotated
    interfaces
    

    View Slide

37. DataWriter.kt
    @NoOpImplementation
    internal interface DataWriter {
    fun write(element: T)
    fun write(data: List)
    }
    

    View Slide

38. NoOpDataWriter.kt
    internal class NoOpDataWriter : DataWriter {
    public override fun write(element: T) {
    }
    public override fun write(data: List) {
    }
    }
    

    View Slide

39. DataReader.kt
    @NoOpImplementation
    internal interface DataReader {
    fun readNext(): T?
    }
    

    View Slide

40. NoOpDataReader.kt
    internal class NoOpDataReader : DataReader {
    public override fun readNext(): T? {
    return null
    }
    }
    

    View Slide

41. Going Further
    ▫ KotlinPoet library
    

    View Slide

42. KotlinPoet
    val file = FileSpec.builder("", "Greeter")
    .addType(
    TypeSpec.classBuilder("Greeter")
    .addFunction(
    FunSpec.builder("greet")
    .addParameter("name", String::class)
    .addStatement("println(%P)", "Hello, \$name").build()
    ).build()
    ).build()
    

    View Slide

43. KotlinPoet
    val file = FileSpec.builder("", "Greeter")
    .addType(
    TypeSpec.classBuilder("Greeter")
    .addFunction(
    FunSpec.builder("greet")
    .addParameter("name", String::class)
    .addStatement("println(%P)", "Hello, \$name").build()
    ).build()
    ).build()
    

    View Slide

44. KotlinPoet
    val file = FileSpec.builder("", "Greeter")
    .addType(
    TypeSpec.classBuilder("Greeter")
    .addFunction(
    FunSpec.builder("greet")
    .addParameter("name", String::class)
    .addStatement("println(%P)", "Hello, \$name").build()
    ).build()
    ).build()
    

    View Slide

45. KotlinPoet
    val file = FileSpec.builder("", "Greeter")
    .addType(
    TypeSpec.classBuilder("Greeter")
    .addFunction(
    FunSpec.builder("greet")
    .addParameter("name", String::class)
    .addStatement("println(%P)", "Hello, \$name").build()
    ).build()
    ).build()
    

    View Slide

46. KotlinPoet
    class Greeter() {
    fun greet(name: String) {
    println("""Hello, $name""")
    }
    }
    

    View Slide

47. More info…
    ▫ https:/
    /kotlinlang.org/docs/ksp-overview.html
    ▫ https:/
    /github.com/google/ksp
    ▫ https:/
    /square.github.io/kotlinpoet/
    

    View Slide

48. 3. Gradle Plugin
    

    View Slide

49. What is Gradle?
    ▫ Custom tasks
    ▫ Compile time checks
    ▫ Helper classes
    

    View Slide

50. How does Gradle work
    ▫ Project object created by Gradle
    ▫ Plugin adds extension/tasks to it
    

    View Slide

51. Custom Task
    open class MyCustomTask : DefaultTask() {
    init {
    group = "MyCompany"
    description = "Description of the task"
    }
    @TaskAction
    fun applyTask() { }
    }
    

    View Slide

52. Custom Task
    open class MyCustomTask : DefaultTask() {
    init {
    group = "MyCompany"
    description = "Description of the task"
    }
    @TaskAction
    fun applyTask() { }
    }
    

    View Slide

53. Custom Task
    open class MyCustomTask : DefaultTask() {
    init {
    group = "MyCompany"
    description = "Description of the task"
    }
    @TaskAction
    fun applyTask() { }
    }
    

    View Slide

54. Custom Extension
    open class MyCustomExtension(
    var outputFile: File = File(DEFAULT_OUTPUT_FILENAME),
    var enableSomething: Boolean = false
    )
    

    View Slide

55. Custom Plugin
    class MyCustomPlugin : Plugin {
    override fun apply(target: Project) {
    val myExtension = target.extensions
    .create("myExtension", MyCustomExtension::class.java)
    val myTask = target.tasks
    .create("myTask", MyCustomTask::class.java)
    myTask.extension = myExtension
    }
    }
    

    View Slide

56. Custom Plugin
    class MyCustomPlugin : Plugin {
    override fun apply(target: Project) {
    val myExtension = target.extensions
    .create("myExtension", MyCustomExtension::class.java)
    val myTask = target.tasks
    .create("myTask", MyCustomTask::class.java)
    myTask.extension = myExtension
    }
    }
    

    View Slide

57. Custom Plugin
    class MyCustomPlugin : Plugin {
    override fun apply(target: Project) {
    val myExtension = target.extensions
    .create("myExtension", MyCustomExtension::class.java)
    val myTask = target.tasks
    .create("myTask", MyCustomTask::class.java)
    myTask.extension = myExtension
    }
    }
    

    View Slide

58. Custom Plugin
    class MyCustomPlugin : Plugin {
    override fun apply(target: Project) {
    val myExtension = target.extensions
    .create("myExtension", MyCustomExtension::class.java)
    val myTask = target.tasks
    .create("myTask", MyCustomTask::class.java)
    myTask.extension = myExtension
    }
    }
    

    View Slide

59. Custom Plugin
    class MyCustomPlugin : Plugin {
    override fun apply(target: Project) {
    val myExtension = target.extensions
    .create("myExtension", MyCustomExtension::class.java)
    val myTask = target.tasks
    .create("myTask", MyCustomTask::class.java)
    myTask.extension = myExtension
    }
    }
    

    View Slide

60. An example of custom Gradle Tasks
    ▫ Compute dependencies size and OSS Licenses
    ▫ Keep track of API Surface
    ▫ Generate Data Class from JSON Schema
    ▫ From a remote git repository
    

    View Slide

61. Going Further
    ▫ Link tasks together
    

    View Slide

62. Custom Plugin
    override fun apply(target: Project) {
    // ...
    target.afterEvaluate { p ->
    p.tasks.withType(KotlinCompile::class.java) {
    dependsOn(myTask)
    }
    }
    }
    

    View Slide

63. More info…
    ▫ https:/
    /docs.gradle.org/current/userguide/custom_plugins.html
    ▫ https:/
    /www.youtube.com/watch?v=ww0D8WMWPXc
    ▫ https:/
    /speakerdeck.com/xgouchet/rock-the-gradle-mobileera
    

    View Slide

64. 4. Kotlin Compiler Plugin
    Let’s start with the first set of slides
    

    View Slide

65. What is it?
    ▫ Plugin getting access to Kotlin source code / IR
    ▫ Can manipulate the code before compilation into
    JVM/JS/…
    ▫ Similar to Aspect Oriented Programming
    

    View Slide

66. Cross-cutting concerns
    

    View Slide

67. How does it work
    Parsing Resolution
    Analysis Code
    Generation
    Source
    Code
    PSI
    (AST)
    IR
    JVM
    JS
    Native
    

    View Slide

68. Compiler Plugin
    ▫ Kotlin Compiler Plugin
    ▫ Gradle Plugin / Maven Plugin
    ▫ IDE Plugin (icons / syntax)
    

    View Slide

69. Examples of Kotlin Compiler Plugins
    ▫ Generate Compose boilerplate code
    ▫ All-open makes all class open by default
    

    View Slide

70. More info…
    ▫ https:/
    /www.youtube.com/watch?v=w-GMlaziIyo
    ▫ https:/
    /arrow-kt.io/docs/meta/
    ▫ https:/
    /medium.com/@heyitsmohit/writing-kotlin-compiler-pl
    ugin-with-arrow-meta-cf7b3689aa3e
    

    View Slide

71. *. A little bit more
    Let’s start with the first set of slides
    

    View Slide

72. PSI Viewer
    ▫ IDE Plugin for Intellij Idea and Android Studio
    ▫ Displays the actual PSI for the open tab
    

    View Slide

73. View Slide

74. *. Conclusion
    Let’s start with the first set of slides
    

    View Slide

75. Detekt
    Gradle
    KSP
    KCP
    

    View Slide

76. “
    Your Detekt rules, KSP, Gradle scripts are
    still code. Keep them as clean,
    maintainable and trustworthy
    as your production code
    

    View Slide

77. Useful links
    ▫ https:/
    /github.com/datadog/dd-sdk-android
    

    View Slide

78. Thank You!
    Any questions?
    @xgouchet
    @datadoghq
    Unsplash photo by @marcosjluiz
    

    View Slide

79. Credits
    Special thanks to all the people who made and released
    these awesome resources for free:
    ▫ Presentation template by SlidesCarnival
    ▫ Photographs by Unsplash
    79
    

    View Slide