Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Develop your CI tools

Develop your CI tools

Nowadays, setting up a basic unit and syntax testing pipeline for your mobile app has never been easier! That’s a great starting point, but what comes next? We can leverage the CI pipeline to do so much more! Things like automatic code generation and advanced static analysis—all with an aim to ensuring the reliability and performance of your applications.

In this talk, we’ll take a look at several tools and techniques that you can use to improve your quality of life as a developer, largely by letting the CI do the hard work for you.

Xavier Gouchet

April 25, 2022
Tweet

More Decks by Xavier Gouchet

Other Decks in Programming

Transcript

  1. Develop your
    CI Tools

    View full-size slide

  2. Xavier F. Gouchet
    Senior Software Engineer at Datadog
    @xgouchet / @datadoghq

    View full-size slide

  3. 0. Introduction

    View full-size slide

  4. Small team
    ▫ 3 engineers
    ▫ Lots of features
    ▫ Lots of bug fixes
    ▫ Lots of improvements
    ▫ Small bandwidth
    Unsplash photo by @marvelous

    View full-size slide

  5. CI setup
    ▫ Gitlab
    ▫ Bitrise *
    photo by @xgouchet

    View full-size slide

  6. Delegating to the CI
    ▫ Runs automatically
    ▫ Strict and thorough
    ▫ Great way to offload the PR Review

    View full-size slide

  7. Code Review Pyramid
    Credit: Gunnar Morling (CC-BY-SA 4.0)
    CODE STYLE
    TESTS
    IMPLEMENTATION
    API SEMANTICS
    DOCUMENTATION

    View full-size slide

  8. 1. Detekt
    Your code, your rules…

    View full-size slide

  9. What is Detekt?
    ▫ Kotlin Static Analysis
    ▫ Code smells
    ▫ Performance Issues
    ▫ Code complexity
    ▫ Documentation Missing
    ▫ …

    View full-size slide

  10. Detekt Report

    View full-size slide

  11. How does Detekt work
    ▫ Visitor pattern over the Syntax Tree (PSI)
    ▫ With Type Resolution
    ▫ Report issues on specific patterns

    View full-size slide

  12. Custom Rule
    class MyCustomRule : Rule() {
    override val issue: Issue = Issue(
    javaClass.simpleName,
    Severity.CodeSmell,
    "A description of the issue…",
    Debt.TWENTY_MINS
    )
    }

    View full-size slide

  13. Custom Rule
    class MyCustomRule : Rule() {
    override val issue: Issue = Issue(
    javaClass.simpleName,
    Severity.CodeSmell,
    "A description of the issue…",
    Debt.TWENTY_MINS
    )
    }

    View full-size slide

  14. Custom Rule
    class MyCustomRule : Rule() {
    override val issue: Issue = Issue(
    javaClass.simpleName,
    Severity.CodeSmell,
    "A description of the issue…",
    Debt.TWENTY_MINS
    )
    }

    View full-size slide

  15. Custom Rule
    override fun visitExpression(expression: KtExpression) {
    if (invalidExpression(expression))
    report(
    CodeSmell(
    issue, Entity.from(expression), "A message"
    )
    )
    }

    View full-size slide

  16. Custom Rule
    override fun visitExpression(expression: KtExpression) {
    if (invalidExpression(expression))
    report(
    CodeSmell(
    issue, Entity.from(expression), "A message"
    )
    )
    }

    View full-size slide

  17. Custom Rule
    override fun visitExpression(expression: KtExpression) {
    if (invalidExpression(expression))
    report(
    CodeSmell(
    issue, Entity.from(expression), "A message"
    )
    )
    }

    View full-size slide

  18. A few examples of our custom rules
    ▫ Forbid throwing exceptions
    ▫ Forbid using check or require
    ▫ Forbid usage of !!
    ▫ Flag unsafe third party methods
    ▫ Flag TODO comments without a Jira ticket

    View full-size slide

  19. Going Further
    ▫ Make the rule configurable from detekt.yml

    View full-size slide

  20. Custom Rule
    class MyCustomRule(
    config: Config
    ) : Rule(config) {
    private val enableFoo: Boolean
    by config(defaultValue = true)
    //…
    }

    View full-size slide

  21. Custom Rule
    class MyCustomRule(
    config: Config
    ) : Rule(config) {
    private val enableFoo: Boolean
    by config(defaultValue = true)
    //…
    }

    View full-size slide

  22. Custom Rule
    class MyCustomRule(
    config: Config
    ) : Rule(config) {
    private val enableFoo: Boolean
    by config(defaultValue = true)
    //…
    }

    View full-size slide

  23. More info…
    ▫ https:/
    /detekt.dev/
    ▫ https:/
    /github.com/detekt/detekt
    ▫ https:/
    /thebakery.dev/30/

    View full-size slide

  24. 2. Kotlin Symbol Processor

    View full-size slide

  25. What is KSP?
    ▫ Lightweight Compiler Plugin
    ▫ Similar to (K)APT but faster
    ▫ Source code is read-only
    ▫ Code generation

    View full-size slide

  26. How does KSP work
    ▫ Called at compile time
    ▫ Processor can query files and/or declarations
    ▫ Can generate new code (but not modify existing one)

    View full-size slide

  27. Custom Processor
    class MyCustomSymbolProcessor(
    val codeGenerator: CodeGenerator,
    val logger: KSPLogger
    ) : SymbolProcessor {
    override fun process(resolver: Resolver): List {
    return emptyList()
    }
    }

    View full-size slide

  28. Custom Processor
    class MyCustomSymbolProcessor(
    val codeGenerator: CodeGenerator,
    val logger: KSPLogger
    ) : SymbolProcessor {
    override fun process(resolver: Resolver): List {
    return emptyList()
    }
    }

    View full-size slide

  29. Custom Processor
    class MyCustomSymbolProcessor(
    val codeGenerator: CodeGenerator,
    val logger: KSPLogger
    ) : SymbolProcessor {
    override fun process(resolver: Resolver): List {
    return emptyList()
    }
    }

    View full-size slide

  30. Custom Processor
    override fun process(resolver: Resolver): List {
    val deferred = mutableListOf()
    resolver.getSymbolsWithAnnotation("com.example.MyAnnotation")
    .forEach {
    if (!generateSomething(it))
    deferred.add(it)
    }
    return deferred
    }

    View full-size slide

  31. Custom Processor
    override fun process(resolver: Resolver): List {
    val deferred = mutableListOf()
    resolver.getSymbolsWithAnnotation("com.example.MyAnnotation")
    .forEach {
    if (!generateSomething(it))
    deferred.add(it)
    }
    return deferred
    }

    View full-size slide

  32. Custom Processor
    override fun process(resolver: Resolver): List {
    val deferred = mutableListOf()
    resolver.getSymbolsWithAnnotation("com.example.MyAnnotation")
    .forEach {
    if (!generateSomething(it))
    deferred.add(it)
    }
    return deferred
    }

    View full-size slide

  33. Custom Processor
    override fun process(resolver: Resolver): List {
    val deferred = mutableListOf()
    resolver.getSymbolsWithAnnotation("com.example.MyAnnotation")
    .forEach {
    if (!generateSomething(it))
    deferred.add(it)
    }
    return deferred
    }

    View full-size slide

  34. Custom Processor
    override fun process(resolver: Resolver): List {
    val deferred = mutableListOf()
    resolver.getSymbolsWithAnnotation("com.example.MyAnnotation")
    .forEach {
    if (!generateSomething(it))
    deferred.add(it)
    }
    return deferred
    }

    View full-size slide

  35. An example of custom processor
    ▫ Generate no-op implementation of annotated
    interfaces

    View full-size slide

  36. DataWriter.kt
    @NoOpImplementation
    internal interface DataWriter {
    fun write(element: T)
    fun write(data: List)
    }

    View full-size slide

  37. NoOpDataWriter.kt
    internal class NoOpDataWriter : DataWriter {
    public override fun write(element: T) {
    }
    public override fun write(data: List) {
    }
    }

    View full-size slide

  38. DataReader.kt
    @NoOpImplementation
    internal interface DataReader {
    fun readNext(): T?
    }

    View full-size slide

  39. NoOpDataReader.kt
    internal class NoOpDataReader : DataReader {
    public override fun readNext(): T? {
    return null
    }
    }

    View full-size slide

  40. Going Further
    ▫ KotlinPoet library

    View full-size slide

  41. KotlinPoet
    val file = FileSpec.builder("", "Greeter")
    .addType(
    TypeSpec.classBuilder("Greeter")
    .addFunction(
    FunSpec.builder("greet")
    .addParameter("name", String::class)
    .addStatement("println(%P)", "Hello, \$name").build()
    ).build()
    ).build()

    View full-size slide

  42. KotlinPoet
    val file = FileSpec.builder("", "Greeter")
    .addType(
    TypeSpec.classBuilder("Greeter")
    .addFunction(
    FunSpec.builder("greet")
    .addParameter("name", String::class)
    .addStatement("println(%P)", "Hello, \$name").build()
    ).build()
    ).build()

    View full-size slide

  43. KotlinPoet
    val file = FileSpec.builder("", "Greeter")
    .addType(
    TypeSpec.classBuilder("Greeter")
    .addFunction(
    FunSpec.builder("greet")
    .addParameter("name", String::class)
    .addStatement("println(%P)", "Hello, \$name").build()
    ).build()
    ).build()

    View full-size slide

  44. KotlinPoet
    val file = FileSpec.builder("", "Greeter")
    .addType(
    TypeSpec.classBuilder("Greeter")
    .addFunction(
    FunSpec.builder("greet")
    .addParameter("name", String::class)
    .addStatement("println(%P)", "Hello, \$name").build()
    ).build()
    ).build()

    View full-size slide

  45. KotlinPoet
    class Greeter() {
    fun greet(name: String) {
    println("""Hello, $name""")
    }
    }

    View full-size slide

  46. More info…
    ▫ https:/
    /kotlinlang.org/docs/ksp-overview.html
    ▫ https:/
    /github.com/google/ksp
    ▫ https:/
    /square.github.io/kotlinpoet/

    View full-size slide

  47. 3. Gradle Plugin

    View full-size slide

  48. What is Gradle?
    ▫ Custom tasks
    ▫ Compile time checks
    ▫ Helper classes

    View full-size slide

  49. How does Gradle work
    ▫ Project object created by Gradle
    ▫ Plugin adds extension/tasks to it

    View full-size slide

  50. Custom Task
    open class MyCustomTask : DefaultTask() {
    init {
    group = "MyCompany"
    description = "Description of the task"
    }
    @TaskAction
    fun applyTask() { }
    }

    View full-size slide

  51. Custom Task
    open class MyCustomTask : DefaultTask() {
    init {
    group = "MyCompany"
    description = "Description of the task"
    }
    @TaskAction
    fun applyTask() { }
    }

    View full-size slide

  52. Custom Task
    open class MyCustomTask : DefaultTask() {
    init {
    group = "MyCompany"
    description = "Description of the task"
    }
    @TaskAction
    fun applyTask() { }
    }

    View full-size slide

  53. Custom Extension
    open class MyCustomExtension(
    var outputFile: File = File(DEFAULT_OUTPUT_FILENAME),
    var enableSomething: Boolean = false
    )

    View full-size slide

  54. Custom Plugin
    class MyCustomPlugin : Plugin {
    override fun apply(target: Project) {
    val myExtension = target.extensions
    .create("myExtension", MyCustomExtension::class.java)
    val myTask = target.tasks
    .create("myTask", MyCustomTask::class.java)
    myTask.extension = myExtension
    }
    }

    View full-size slide

  55. Custom Plugin
    class MyCustomPlugin : Plugin {
    override fun apply(target: Project) {
    val myExtension = target.extensions
    .create("myExtension", MyCustomExtension::class.java)
    val myTask = target.tasks
    .create("myTask", MyCustomTask::class.java)
    myTask.extension = myExtension
    }
    }

    View full-size slide

  56. Custom Plugin
    class MyCustomPlugin : Plugin {
    override fun apply(target: Project) {
    val myExtension = target.extensions
    .create("myExtension", MyCustomExtension::class.java)
    val myTask = target.tasks
    .create("myTask", MyCustomTask::class.java)
    myTask.extension = myExtension
    }
    }

    View full-size slide

  57. Custom Plugin
    class MyCustomPlugin : Plugin {
    override fun apply(target: Project) {
    val myExtension = target.extensions
    .create("myExtension", MyCustomExtension::class.java)
    val myTask = target.tasks
    .create("myTask", MyCustomTask::class.java)
    myTask.extension = myExtension
    }
    }

    View full-size slide

  58. Custom Plugin
    class MyCustomPlugin : Plugin {
    override fun apply(target: Project) {
    val myExtension = target.extensions
    .create("myExtension", MyCustomExtension::class.java)
    val myTask = target.tasks
    .create("myTask", MyCustomTask::class.java)
    myTask.extension = myExtension
    }
    }

    View full-size slide

  59. An example of custom Gradle Tasks
    ▫ Compute dependencies size and OSS Licenses
    ▫ Keep track of API Surface
    ▫ Generate Data Class from JSON Schema
    ▫ From a remote git repository

    View full-size slide

  60. Going Further
    ▫ Link tasks together

    View full-size slide

  61. Custom Plugin
    override fun apply(target: Project) {
    // ...
    target.afterEvaluate { p ->
    p.tasks.withType(KotlinCompile::class.java) {
    dependsOn(myTask)
    }
    }
    }

    View full-size slide

  62. More info…
    ▫ https:/
    /docs.gradle.org/current/userguide/custom_plugins.html
    ▫ https:/
    /www.youtube.com/watch?v=ww0D8WMWPXc
    ▫ https:/
    /speakerdeck.com/xgouchet/rock-the-gradle-mobileera

    View full-size slide

  63. 4. Kotlin Compiler Plugin
    Let’s start with the first set of slides

    View full-size slide

  64. What is it?
    ▫ Plugin getting access to Kotlin source code / IR
    ▫ Can manipulate the code before compilation into
    JVM/JS/…
    ▫ Similar to Aspect Oriented Programming

    View full-size slide

  65. Cross-cutting concerns

    View full-size slide

  66. How does it work
    Parsing Resolution
    Analysis Code
    Generation
    Source
    Code
    PSI
    (AST)
    IR
    JVM
    JS
    Native

    View full-size slide

  67. Compiler Plugin
    ▫ Kotlin Compiler Plugin
    ▫ Gradle Plugin / Maven Plugin
    ▫ IDE Plugin (icons / syntax)

    View full-size slide

  68. Examples of Kotlin Compiler Plugins
    ▫ Generate Compose boilerplate code
    ▫ All-open makes all class open by default

    View full-size slide

  69. More info…
    ▫ https:/
    /www.youtube.com/watch?v=w-GMlaziIyo
    ▫ https:/
    /arrow-kt.io/docs/meta/
    ▫ https:/
    /medium.com/@heyitsmohit/writing-kotlin-compiler-pl
    ugin-with-arrow-meta-cf7b3689aa3e

    View full-size slide

  70. *. A little bit more
    Let’s start with the first set of slides

    View full-size slide

  71. PSI Viewer
    ▫ IDE Plugin for Intellij Idea and Android Studio
    ▫ Displays the actual PSI for the open tab

    View full-size slide

  72. *. Conclusion
    Let’s start with the first set of slides

    View full-size slide

  73. Detekt
    Gradle
    KSP
    KCP

    View full-size slide


  74. Your Detekt rules, KSP, Gradle scripts are
    still code. Keep them as clean,
    maintainable and trustworthy
    as your production code

    View full-size slide

  75. Useful links
    ▫ https:/
    /github.com/datadog/dd-sdk-android

    View full-size slide

  76. Thank You!
    Any questions?
    @xgouchet
    @datadoghq
    Unsplash photo by @marcosjluiz

    View full-size slide

  77. Credits
    Special thanks to all the people who made and released
    these awesome resources for free:
    ▫ Presentation template by SlidesCarnival
    ▫ Photographs by Unsplash
    79

    View full-size slide