Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Future of Jira Software Powered by Kotlin

Future of Jira Software Powered by Kotlin

Atlassian is on a decomposition journey. We split our monoliths in microservices and we build our microservices in Kotlin. Our services are reactive and resilient and working on them is kool.

This is a story of Jira Software services that we built in Kotlin with Spring Boot and Project Reactor, our success in interoperability with Java and how we solved issues when crossing the boundary between the reactive and non-reactive world.

Martin T. Varga

August 24, 2019
Tweet

More Decks by Martin T. Varga

Other Decks in Programming

Transcript

  1. MARTIN VARGA | SOFTWARE DEVELOPER | @MARTINTEEVARGA
    Future of Jira Software
    Powered by Kotlin

    View full-size slide

  2. kotlinlang.org

    View full-size slide

  3. Google
    Pixel 2
    Just Black
    Simply drop
    your screenshot
    on to this
    media
    placeholder
    TRELLO MOBILE

    View full-size slide

  4. Jira
    For cloud and server
    Confluence
    For cloud and server
    OpsGenie
    Our recent acquisition
    Not just Trello

    View full-size slide

  5. Not just mobile!

    View full-size slide

  6. FUTURE OF JIRA SOFTWARE
    POWERED BY KOTLIN

    View full-size slide

  7. JIRA SOFTWARE?

    View full-size slide

  8. Plan, track and
    release software.
    JIRA SOFTWARE

    View full-size slide

  9. The Story of Jira
    From the developer’s perspective

    View full-size slide

  10. Server Cloud
    1001001
    0 0 1 0 0 1
    0 0 1 0 0 1
    1101010
    1010101
    0 0 1 0 0 1
    1110001001
    0011110010
    1100101011
    01111110001
    Single Code Base

    View full-size slide

  11. Server Cloud
    1001001
    0 0 1 0 0 1
    0 0 1 0 0 1
    1101010
    1010101
    0 0 1 0 0 1

    View full-size slide

  12. 1101010
    1010101
    0 0 1 0 0 1
    1101010
    1010101
    0 0 1 0 0 1

    View full-size slide

  13. 1101010
    1010101
    0 0 1 0 0 1

    View full-size slide

  14. 11010101010101
    0010010101001
    00101010111010
    1010101010010
    0101010010010
    10101110101010

    View full-size slide

  15. 110101010101010010010101
    001001010101110101010101
    010010010101001001010101
    110101010101010010010101
    001001010101110101010101
    010010010101001001010101
    110101010101010010010101
    001001010101110101010101
    010010010101001001010101
    110101010101010010010101
    001001010101110101010101

    View full-size slide

  16. 110101010101010010010101
    001001010101110101010101
    010010010101001001010101
    110101010101010010010101
    001001010101110101010101
    010010010101001001010101
    110101010101010010010101
    001001010101110101010101
    010010010101001001010101
    110101010101010010010101
    001001010101110101010101

    View full-size slide

  17. 110101010101010010010101
    001001010101110101010101
    010010010101001001010101
    110101010101010010010101
    001001010101110101010101
    010010010101001001010101
    110101010101010010010101
    001001010101110101010101
    010010010101001001010101
    110101010101010010010101
    001001010101110101010101

    View full-size slide

  18. 110101010101010010010101
    001001010101110101010101
    010010010101001001010101
    110101010101010010010101
    001001010101110101010101
    010010010101001001010101
    110101010101010010010101
    001001010101110101010101
    010010010101001001010101
    110101010101010010010101
    001001010101110101010101

    View full-size slide

  19. 1101010101010100
    1001010100100101
    0101110101010101
    010010010101001
    0010101011101010
    1010101001001010
    1001001010101110
    1 0 1 0 1 0
    1010111
    001001

    View full-size slide

  20. 1101010101
    010100100
    101010010
    0101010111
    0 1 0 1 0 1
    1011100
    1 0 0 1 0 1
    1 0 1 0 1 0
    1010111
    001001

    View full-size slide

  21. 1101010
    1010101
    001001
    0 1 0 1 0 1
    1011100
    1 0 0 1 0 1
    1 0 1 0 1 0
    1010111
    001001
    0 1 0 1 0 1
    1011100
    1 0 0 1 0 1

    View full-size slide

  22. 1101010
    1010101
    001001
    0 1 0 1 0 1
    1011100
    1 0 0 1 0 1
    1 0 1 0 1 0
    1010111
    001001
    0 1 0 1 0 1
    1011100
    1 0 0 1 0 1

    View full-size slide

  23. 1101010
    1010101
    001001
    0 1 0 1 0 1
    1011100
    1 0 0 1 0 1

    View full-size slide

  24. How Atlassian builds services
    Making developers productive with platform as a service

    View full-size slide

  25. PaaS that allows
    devs to create and
    deploy a service in
    matter of minutes.
    MICROS

    View full-size slide

  26. name: "animalfacts"
    compose:
    animalfacts:
    image: internal.docker.repo/animalfacts
    tag: 'PROJECT_VERSION'
    ports:
    - 8080:8080
    MICROS DESCRIPTOR

    View full-size slide

  27. links:
    healthcheck:
    uri: heartbeat
    port: 8080
    source:
    url: "[email protected]:atlassianlabs/animal-facts-
    graphql-gateway-demo.git"
    notifications:
    email: "[email protected]"
    resources:
    - type: dynamo-db
    name: facts
    attributes:
    ReadWriteCapacityMode: ON_DEMAND

    View full-size slide

  28. Jira Software Tech Stack
    DynamoDB or PostgreSQL
    Depends on the application.
    Project Reactor
    Mature reactive library.
    Kotlin
    For all new services.
    Spring Boot
    Custom wrapper that makes
    monitoring etc. easier.

    View full-size slide

  29. Powered by Kotlin
    Making developers happy

    View full-size slide

  30. Experience
    Have we learned Kotlin enough? Was it a
    success?
    Compatibility
    How well is it going to play with our current tech
    stacks?
    Staffing
    Are we going to have enough Kotlin developers
    available?
    Adopting
    Kotlin

    View full-size slide

  31. Velocity
    Does it make our developers more productive?
    Industry
    Is it already adopted by the developers
    community?
    Adopting
    Kotlin

    View full-size slide

  32. Experienced with Kotlin.
    Android Developer Ad

    View full-size slide

  33. Experience or interest in Kotlin
    development.
    Software Engineer, Developer Productivity, Jira Cloud Ad

    View full-size slide

  34. We happen to have a variety of
    languages within our stack
    including Kotlin, Python, and
    Ruby.
    Senior Backend Developer Ad

    View full-size slide

  35. A case for a reactive app
    A real world example of an app that we built recently

    View full-size slide

  36. LATENCY
    250ms
    250ms
    250ms
    750ms

    View full-size slide

  37. LATENCY
    10ms
    250ms
    10ms
    10ms
    280ms

    View full-size slide

  38. Operators
    Useful for modifying and combining sources.
    Allows composition of libraries that add
    resilience, tracing etc.
    Async
    Calls a service, then does mostly nothing and
    waits. Perfect fit.
    Project
    reactor
    Clean code
    No callback hell.

    View full-size slide

  39. val currentUser = "martinId".toMono()
    MONO

    View full-size slide

  40. val recentUsers = listOf("alexId", "bobbieId",
    "charlieId", "drewId").toFlux()
    FLUX

    View full-size slide

  41. val currentUser = “martinId”.toMono()
    val recentUsers = listOf("alexId", "bobbieId",
    "charlieId", "drewId").toFlux()
    currentUser.concatWith(recentUsers)
    .map {
    userService.getFirstName(it)
    }
    .take(3)
    .subscribe {
    println(it)
    }
    COMPOSITION & CLEAN CODE

    View full-size slide

  42. Martin
    Alex
    Bobbie
    COMPOSITION & CLEAN CODE

    View full-size slide

  43. currentUser.concatWith(recentUsers)
    .map {
    userService.getFirstName(it)
    }
    .timeout(Duration.ofMillis(2500))
    .onErrorReturn("Recent user")
    .take(3)
    .subscribe {
    println(it)
    }
    COMPOSITION & CLEAN CODE

    View full-size slide

  44. Martin
    Recent User
    Bobbie
    COMPOSITION & CLEAN CODE

    View full-size slide

  45. ThreadLocal value = new ThreadLocal<>();
    value.set(42);

    Integer integer = value.get();
    REACTOR GOTCHAS

    View full-size slide

  46. currentUser.concatWith(recentUsers)
    .subscriberContext { sc ->
    sc.put("key", value)
    }
    .flatMap { userId ->
    Mono.subscriberContext().map { sc ->
    userService.getFullName(userId, sc.get("key"))
    }
    }
    .subscribe {
    println(it)
    }
    SUBSCRIBER CONTEXT

    View full-size slide

  47. Get exactly what you want
    Query multiple sources at once, specify what you
    want.
    Strongly typed schema
    Schema is a living documentation.
    Mature web clients
    Clients like Apollo let us remove dependency on
    Redux in ReactJs web apps.
    GraphQL

    View full-size slide

  48. type Query {
    board(id: ID): Board
    }
    type Board {
    id: ID
    name: String
    issues: [Issue]
    }
    type Issue {
    id: ID
    key: String
    }
    SCHEMA

    View full-size slide

  49. POST /jsw/graphql
    query {
    board(id: 42) {
    name
    issues {
    key
    }
    }
    }
    QUERY

    View full-size slide

  50. {
    "data": {
    "board": {
    "name": "Software board”
    "issues": [
    {
    "key": "KEY-1"
    }
    ]
    }
    }
    }
    RESPONSE

    View full-size slide

  51. Project Reactor + GraphQL Java
    OUR SOLUTION

    View full-size slide

  52. DataFetching
    Environment
    Subscriber
    Context
    PROJECT REACTOR GRAPHQL-JAVA

    View full-size slide

  53. Let’s write some code
    GraphQL gateway, Jira Software style

    View full-size slide

  54. GET https://catfact.ninja/fact
    {
    "fact": "The leopard is the most widespread of all big
    cats.",
    "length": 51
    }
    CAT FACTS API

    View full-size slide

  55. GET https://some-random-api.ml/facts/dog
    {
    "fact": "A dogs' first sense to develop is touch."
    }
    DOG FACTS API

    View full-size slide

  56. Cat Facts
    Dog Facts

    View full-size slide

  57. WHY NOT BOTH?

    View full-size slide

  58. POST https://animalfactsgateway.com/grapqhl
    query {
    dog {
    fact
    length
    }
    cat {
    fact
    length
    }
    }
    ANIMAL FACTS GRAPHQL API

    View full-size slide

  59. {
    "data": {
    "dog": {
    "fact": "Dog fact",
    "length": 8
    },
    "cat": {
    "fact": "Cat fact",
    "length": 8
    }
    }
    }

    View full-size slide

  60. import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
    plugins {
    id("org.springframework.boot") version "2.1.6.RELEASE"
    id("io.spring.dependency-management") version
    "1.0.7.RELEASE"
    kotlin("jvm") version "1.2.71"
    kotlin("plugin.spring") version "1.2.71"
    }
    PROJECT DESCRIPTOR

    View full-size slide

  61. dependencies {
    implementation("org.springframework.boot:spring-boot-
    starter-webflux")
    implementation("com.fasterxml.jackson.module:jackson-
    module-kotlin")
    implementation("org.jetbrains.kotlin:kotlin-reflect")
    implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
    implementation("com.graphql-java:graphql-java:11.0")
    }
    tasks.withType {
    kotlinOptions {
    freeCompilerArgs = listOf("-Xjsr305=strict")
    jvmTarget = "1.8"
    }
    }

    View full-size slide

  62. package com.atlassian.jsw.factsdemo
    import o.s.b.autoconfigure.SpringBootApplication
    import o.s.b.runApplication
    @SpringBootApplication
    class DemoApplication
    fun main(args: Array) {
    runApplication(*args)
    }
    APPLICATION

    View full-size slide

  63. type Query {
    cat: Fact
    dog: Fact
    }
    type Fact {
    fact: String
    length: Int
    }
    GRAPHQL SCHEMA

    View full-size slide

  64. interface FactFetcher: DataFetcher>
    FETCHERS
    public interface DataFetcher {
    T get(DataFetchingEnvironment env) throws Exception;
    }

    View full-size slide

  65. data class Fact(val fact: String, val length: Int)
    FACT CLASS

    View full-size slide

  66. @Component
    class CatFactFetcher(val webClient: WebClient) :
    FactFetcher {
    override fun get(environment:
    DataFetchingEnvironment): CompletionStage =
    webClient.get().uri("https://catfact.ninja/fact")
    .retrieve()
    .bodyToMono(CatFact::class.java)
    .map { Fact(it.fact, it.length) }
    .toFuture()
    }
    data class CatFact(val fact: String, val length: Int)
    CAT FACTS FETCHER

    View full-size slide

  67. @Component
    class DogFactFetcher(val webClient: WebClient) :
    FactFetcher {
    override fun get(environment:
    DataFetchingEnvironment): CompletionStage =
    webClient.get().uri("https://.../facts/dog")
    .retrieve()
    .bodyToMono(DogFact::class.java)
    .map { Fact(it.fact, it.fact.length) }
    .toFuture()
    }
    data class DogFact(val fact: String)
    DOG FACTS FETCHER

    View full-size slide

  68. @Configuration
    class GraphqlConfig {
    @Bean
    fun schema(wiring: RuntimeWiring): GraphQLSchema =
    SchemaGenerator().makeExecutableSchema(
    parseSchema("/schema.graphqls"),
    wiring
    )
    }
    GRAPHQL CONFIGURATION

    View full-size slide

  69. @Configuration
    class GraphqlConfig {
    @Bean
    fun factsWiring(
    dogFactFetcher: DogFactFetcher,
    catFactFetcher: CatFactFetcher
    ): RuntimeWiring =
    RuntimeWiring.newRuntimeWiring()
    .type(
    newTypeWiring("Query")
    .dataFetcher("dog", dogFactFetcher)
    .dataFetcher("cat", catFactFetcher)
    )
    .build()
    }

    View full-size slide

  70. override fun processQuery(
    query: String
    ): Mono = Mono.fromFuture(
    GraphQL
    .newGraphQL(graphQLSchema)
    .build()
    .executeAsync(
    ExecutionInput.newExecutionInput()
    .query(query)
    .build()
    )
    )
    SERVICE METHOD

    View full-size slide

  71. @PostMapping(
    path = ["/graphql"],
    consumes = [MediaType.APPLICATION_JSON_VALUE],
    produces = [MediaType.APPLICATION_JSON_VALUE]
    )
    fun graphql(
    @RequestBody body: Map
    ): Mono> {
    val query: String = body["query"] as String? ?: "{}"
    return graphQLService.processQuery(query)
    .map { it.toSpecification() }
    }
    REACTIVE API

    View full-size slide

  72. {
    "data": {
    "dog": {
    "fact": "“Him” and “Her” were the names of President
    Lyndon Johnson's two beagles.",
    "length": 73
    },
    "cat": {
    "fact": "Jaguars are the only big cats that don't
    roar.",
    "length": 46
    }
    }
    }
    RESULT

    View full-size slide

  73. Resilience
    Protect the downstream services from the upstream.

    View full-size slide


  74. 250ms 380ms
    380ms

    View full-size slide

  75. CAT FACTS BROKEN



    250ms 30s
    30s

    View full-size slide

  76. {
    "data": {
    "dog": {…},
    "cat": null
    },
    "errors": [
    {
    "message": "Exception while fetching data (/cat) :
    Operation timed out",
    "path": [
    "cat"
    ]
    }
    ]
    }

    View full-size slide

  77. TIMEOUT



    250ms 5000ms
    5000ms

    View full-size slide

  78. class CatFactFetcher(
    val webClient: WebClient
    ) : FactFetcher {
    override fun get(environment:
    DataFetchingEnvironment): CompletionStage =
    webClient.get().uri("https://catfact.ninja/fact")
    .retrieve()
    .bodyToMono(CatFact::class.java)
    .map { … }
    .timeout(Duration.ofMillis(5000))
    .toFuture()
    }
    CAT FACTS WITH TIMEOUT

    View full-size slide

  79. {
    "data": {
    "dog": {…},
    "cat": null
    },
    "errors": [
    {
    "message": "Exception while fetching data (/cat) :
    Did not observe any item or terminal signal within 5000ms
    in 'map' (and no fallback has been configured)",
    "path": [
    “cat"
    ]
    }
    ]
    }

    View full-size slide

  80. CIRCUIT BREAKER



    250ms 0ms
    250ms

    View full-size slide

  81. CIRCUIT BREAKER
    Closed
    Open
    Half-open


    View full-size slide

  82. class CatFactFetcher(
    val webClient: WebClient,
    val catBreaker: CircuitBreaker
    ) : FactFetcher {
    override fun get(environment:
    DataFetchingEnvironment): CompletionStage =
    webClient.get()


    .timeout(Duration.ofMillis(2500))
    .transform(
    CircuitBreakerOperator.of(catBreaker)
    )
    .toFuture()
    }
    CAT FACTS FETCHER WITH CIRCUIT BREAKER

    View full-size slide

  83. @Bean
    fun catCircuitBreaker(): CircuitBreaker {
    val circuitBreakerConfig =
    CircuitBreakerConfig.custom()
    .ringBufferSizeInClosedState(3)
    .ringBufferSizeInHalfOpenState(3)
    .waitDurationInOpenState(Duration.ofMillis(5000))
    .failureRateThreshold(0.5f)
    .build()
    return circuitBreakerRegistry.circuitBreaker(
    "CatCCB",
    circuitBreakerConfig
    )
    }

    View full-size slide

  84. {
    "data": {
    "dog": {…},
    "cat": null
    },
    "errors": [
    {
    "message": "Exception while fetching data (/cat) :
    CircuitBreaker 'CatCCB' is OPEN and does not permit
    further calls",
    "path": [
    "cat"
    ]
    }
    ]
    }

    View full-size slide

  85. CIRCUIT BREAKER



    250ms 0ms
    250ms
    But we might miss some cat facts.

    View full-size slide

  86. Takeaways
    Even if you forget everything else I said, please try…

    View full-size slide

  87. GraphQL
    For APIs consumed from the front-end.
    Resilience
    To prevent propagating errors downstream.
    Reactive
    For I/O heavy apps.
    Takeaways

    View full-size slide

  88. Beyond Future
    Because you were going to ask about coroutines…

    View full-size slide

  89. GraphQL Java
    There are already pure Kotlin libraries. Spring
    boot has a set of starters too.
    Coroutines
    Support in Spring. New utilities for Project
    reactor.
    Alternatives

    View full-size slide

  90. fun processQuery(query: String): Map
    @PostMapping(
    path = ["/graphql"],
    consumes = [MediaType.APPLICATION_JSON_VALUE],
    produces = [MediaType.APPLICATION_JSON_VALUE]
    )
    fun graphqlCoroutines(
    @RequestBody body: Map
    ): Mono> = mono(Unconfined) {
    val query: String = body["query"] as String? ?: "{}"
    graphQLService.processQuery(query)
    }
    REACTOR WITH COROUTINES

    View full-size slide

  91. Resources
    Write your own reactive and resilient GraphQL gateway in Kotlin

    View full-size slide

  92. 1
    2
    3
    Animal Facts GraphQL Gateway
    https://bitbucket.org/atlassianlabs/animal-facts-graphql-gateway-demo/src/
    master/
    graphql-java
    https://www.graphql-java.com/
    Resilience4j
    https://github.com/resilience4j/resilience4j
    Resources / Code

    View full-size slide

  93. 4 GraphiQL
    https://github.com/graphql/graphiql
    Resources / Code
    5 Coroutines-reactor
    https://github.com/Kotlin/kotlinx.coroutines/tree/master/reactive/kotlinx-coroutines-
    reactor

    View full-size slide

  94. 1
    2
    3
    Kotlin at Trello
    Trello tech blog:

    https://tech.trello.com/kotlin-at-trello/
    Engineering stateless, high-availability cloud services at Atlassian
    Atlassian Engineering Blog:
    https://www.atlassian.com/blog/technology/aws-scaling-multi-region-low-latency-
    service
    More about Micros
    Slightly outdated, but still relevant.

    https://www.youtube.com/watch?v=dc2nqzgqH24
    Resources / Further information

    View full-size slide

  95. 1 Why not both?
    https://knowyourmeme.com/memes/why-not-both-why-dont-we-have-both
    Resources / Memes

    View full-size slide

  96. MARTIN VARGA | SOFTWARE DEVELOPER | @MARTINTEEVARGA
    Future of Jira Software
    Powered by Kotlin

    View full-size slide