Future of Jira Software Powered by Kotlin

Future of Jira Software Powered by Kotlin

Atlassian is on a decomposition journey. We split our monoliths in microservices and we build our microservices in Kotlin. Our services are reactive and resilient and working on them is kool.

This is a story of Jira Software services that we built in Kotlin with Spring Boot and Project Reactor, our success in interoperability with Java and how we solved issues when crossing the boundary between the reactive and non-reactive world.

11ebf5b478b3da4d1b9d8cc719abba8a?s=128

Martin T. Varga

August 24, 2019
Tweet

Transcript

  1. None
  2. MARTIN VARGA | SOFTWARE DEVELOPER | @MARTINTEEVARGA Future of Jira

    Software Powered by Kotlin
  3. None
  4. kotlinlang.org

  5. None
  6. Google Pixel 2 Just Black Simply drop your screenshot on

    to this media placeholder TRELLO MOBILE
  7. Jira For cloud and server Confluence For cloud and server

    OpsGenie Our recent acquisition Not just Trello
  8. Not just mobile!

  9. FUTURE OF JIRA SOFTWARE POWERED BY KOTLIN

  10. JIRA SOFTWARE?

  11. Plan, track and release software. JIRA SOFTWARE

  12. None
  13. The Story of Jira From the developer’s perspective

  14. Server Cloud 1001001 0 0 1 0 0 1 0

    0 1 0 0 1 1101010 1010101 0 0 1 0 0 1 1110001001 0011110010 1100101011 01111110001 Single Code Base
  15. Server Cloud 1001001 0 0 1 0 0 1 0

    0 1 0 0 1 1101010 1010101 0 0 1 0 0 1
  16. 1101010 1010101 0 0 1 0 0 1 1101010 1010101

    0 0 1 0 0 1
  17. None
  18. 1101010 1010101 0 0 1 0 0 1

  19. 11010101010101 0010010101001 00101010111010 1010101010010 0101010010010 10101110101010

  20. 110101010101010010010101 001001010101110101010101 010010010101001001010101 110101010101010010010101 001001010101110101010101 010010010101001001010101 110101010101010010010101 001001010101110101010101 010010010101001001010101 110101010101010010010101

    001001010101110101010101
  21. None
  22. 110101010101010010010101 001001010101110101010101 010010010101001001010101 110101010101010010010101 001001010101110101010101 010010010101001001010101 110101010101010010010101 001001010101110101010101 010010010101001001010101 110101010101010010010101

    001001010101110101010101
  23. 110101010101010010010101 001001010101110101010101 010010010101001001010101 110101010101010010010101 001001010101110101010101 010010010101001001010101 110101010101010010010101 001001010101110101010101 010010010101001001010101 110101010101010010010101

    001001010101110101010101
  24. 110101010101010010010101 001001010101110101010101 010010010101001001010101 110101010101010010010101 001001010101110101010101 010010010101001001010101 110101010101010010010101 001001010101110101010101 010010010101001001010101 110101010101010010010101

    001001010101110101010101
  25. 1101010101010100 1001010100100101 0101110101010101 010010010101001 0010101011101010 1010101001001010 1001001010101110 1 0 1

    0 1 0 1010111 001001
  26. 1101010101 010100100 101010010 0101010111 0 1 0 1 0 1

    1011100 1 0 0 1 0 1 1 0 1 0 1 0 1010111 001001
  27. 1101010 1010101 001001 0 1 0 1 0 1 1011100

    1 0 0 1 0 1 1 0 1 0 1 0 1010111 001001 0 1 0 1 0 1 1011100 1 0 0 1 0 1
  28. 1101010 1010101 001001 0 1 0 1 0 1 1011100

    1 0 0 1 0 1 1 0 1 0 1 0 1010111 001001 0 1 0 1 0 1 1011100 1 0 0 1 0 1
  29. 1101010 1010101 001001 0 1 0 1 0 1 1011100

    1 0 0 1 0 1
  30. None
  31. How Atlassian builds services Making developers productive with platform as

    a service
  32. PaaS that allows devs to create and deploy a service

    in matter of minutes. MICROS
  33. MICROS

  34. name: "animalfacts" compose: animalfacts: image: internal.docker.repo/animalfacts tag: 'PROJECT_VERSION' ports: -

    8080:8080 MICROS DESCRIPTOR
  35. links: healthcheck: uri: heartbeat port: 8080 source: url: "git@bitbucket.org:atlassianlabs/animal-facts- graphql-gateway-demo.git"

    notifications: email: "animalfactsowner@atlassian.com" resources: - type: dynamo-db name: facts attributes: ReadWriteCapacityMode: ON_DEMAND
  36. Jira Software Tech Stack DynamoDB or PostgreSQL Depends on the

    application. Project Reactor Mature reactive library. Kotlin For all new services. Spring Boot Custom wrapper that makes monitoring etc. easier.
  37. Powered by Kotlin Making developers happy

  38. Experience Have we learned Kotlin enough? Was it a success?

    Compatibility How well is it going to play with our current tech stacks? Staffing Are we going to have enough Kotlin developers available? Adopting Kotlin
  39. Velocity Does it make our developers more productive? Industry Is

    it already adopted by the developers community? Adopting Kotlin
  40. Experienced with Kotlin. Android Developer Ad

  41. Experience or interest in Kotlin development. Software Engineer, Developer Productivity,

    Jira Cloud Ad
  42. We happen to have a variety of languages within our

    stack including Kotlin, Python, and Ruby. Senior Backend Developer Ad
  43. A case for a reactive app A real world example

    of an app that we built recently
  44. LATENCY 250ms 250ms 250ms 750ms

  45. LATENCY 10ms 250ms 10ms 10ms 280ms

  46. Operators Useful for modifying and combining sources. Allows composition of

    libraries that add resilience, tracing etc. Async Calls a service, then does mostly nothing and waits. Perfect fit. Project reactor Clean code No callback hell.
  47. val currentUser = "martinId".toMono() MONO

  48. val recentUsers = listOf("alexId", "bobbieId", "charlieId", "drewId").toFlux() FLUX

  49. val currentUser = “martinId”.toMono() val recentUsers = listOf("alexId", "bobbieId", "charlieId",

    "drewId").toFlux() currentUser.concatWith(recentUsers) .map { userService.getFirstName(it) } .take(3) .subscribe { println(it) } COMPOSITION & CLEAN CODE
  50. Martin Alex Bobbie COMPOSITION & CLEAN CODE

  51. currentUser.concatWith(recentUsers) .map { userService.getFirstName(it) } .timeout(Duration.ofMillis(2500)) .onErrorReturn("Recent user") .take(3) .subscribe

    { println(it) } COMPOSITION & CLEAN CODE
  52. Martin Recent User Bobbie COMPOSITION & CLEAN CODE

  53. ThreadLocal<Integer> value = new ThreadLocal<>(); value.set(42); … Integer integer =

    value.get(); REACTOR GOTCHAS
  54. currentUser.concatWith(recentUsers) .subscriberContext { sc -> sc.put("key", value) } .flatMap {

    userId -> Mono.subscriberContext().map { sc -> userService.getFullName(userId, sc.get("key")) } } .subscribe { println(it) } SUBSCRIBER CONTEXT
  55. Get exactly what you want Query multiple sources at once,

    specify what you want. Strongly typed schema Schema is a living documentation. Mature web clients Clients like Apollo let us remove dependency on Redux in ReactJs web apps. GraphQL
  56. type Query { board(id: ID): Board } type Board {

    id: ID name: String issues: [Issue] } type Issue { id: ID key: String } SCHEMA
  57. POST /jsw/graphql query { board(id: 42) { name issues {

    key } } } QUERY
  58. { "data": { "board": { "name": "Software board” "issues": [

    { "key": "KEY-1" } ] } } } RESPONSE
  59. Project Reactor + GraphQL Java OUR SOLUTION

  60. DataFetching Environment Subscriber Context PROJECT REACTOR GRAPHQL-JAVA

  61. Let’s write some code GraphQL gateway, Jira Software style

  62. GET https://catfact.ninja/fact { "fact": "The leopard is the most widespread

    of all big cats.", "length": 51 } CAT FACTS API
  63. GET https://some-random-api.ml/facts/dog { "fact": "A dogs' first sense to develop

    is touch." } DOG FACTS API
  64. Cat Facts Dog Facts

  65. WHY NOT BOTH?

  66. POST https://animalfactsgateway.com/grapqhl query { dog { fact length } cat

    { fact length } } ANIMAL FACTS GRAPHQL API
  67. { "data": { "dog": { "fact": "Dog fact", "length": 8

    }, "cat": { "fact": "Cat fact", "length": 8 } } }
  68. import org.jetbrains.kotlin.gradle.tasks.KotlinCompile plugins { id("org.springframework.boot") version "2.1.6.RELEASE" id("io.spring.dependency-management") version "1.0.7.RELEASE"

    kotlin("jvm") version "1.2.71" kotlin("plugin.spring") version "1.2.71" } PROJECT DESCRIPTOR
  69. dependencies { implementation("org.springframework.boot:spring-boot- starter-webflux") implementation("com.fasterxml.jackson.module:jackson- module-kotlin") implementation("org.jetbrains.kotlin:kotlin-reflect") implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8") implementation("com.graphql-java:graphql-java:11.0") }

    tasks.withType<KotlinCompile> { kotlinOptions { freeCompilerArgs = listOf("-Xjsr305=strict") jvmTarget = "1.8" } }
  70. package com.atlassian.jsw.factsdemo import o.s.b.autoconfigure.SpringBootApplication import o.s.b.runApplication @SpringBootApplication class DemoApplication fun

    main(args: Array<String>) { runApplication<DemoApplication>(*args) } APPLICATION
  71. type Query { cat: Fact dog: Fact } type Fact

    { fact: String length: Int } GRAPHQL SCHEMA
  72. interface FactFetcher: DataFetcher<CompletionStage<Fact?>> FETCHERS public interface DataFetcher<T> { T get(DataFetchingEnvironment

    env) throws Exception; }
  73. data class Fact(val fact: String, val length: Int) FACT CLASS

  74. @Component class CatFactFetcher(val webClient: WebClient) : FactFetcher { override fun

    get(environment: DataFetchingEnvironment): CompletionStage<Fact?> = webClient.get().uri("https://catfact.ninja/fact") .retrieve() .bodyToMono(CatFact::class.java) .map { Fact(it.fact, it.length) } .toFuture() } data class CatFact(val fact: String, val length: Int) CAT FACTS FETCHER
  75. @Component class DogFactFetcher(val webClient: WebClient) : FactFetcher { override fun

    get(environment: DataFetchingEnvironment): CompletionStage<Fact?> = webClient.get().uri("https://.../facts/dog") .retrieve() .bodyToMono(DogFact::class.java) .map { Fact(it.fact, it.fact.length) } .toFuture() } data class DogFact(val fact: String) DOG FACTS FETCHER
  76. @Configuration class GraphqlConfig { @Bean fun schema(wiring: RuntimeWiring): GraphQLSchema =

    SchemaGenerator().makeExecutableSchema( parseSchema("/schema.graphqls"), wiring ) } GRAPHQL CONFIGURATION
  77. @Configuration class GraphqlConfig { @Bean fun factsWiring( dogFactFetcher: DogFactFetcher, catFactFetcher:

    CatFactFetcher ): RuntimeWiring = RuntimeWiring.newRuntimeWiring() .type( newTypeWiring("Query") .dataFetcher("dog", dogFactFetcher) .dataFetcher("cat", catFactFetcher) ) .build() }
  78. override fun processQuery( query: String ): Mono<ExecutionResult> = Mono.fromFuture( GraphQL

    .newGraphQL(graphQLSchema) .build() .executeAsync( ExecutionInput.newExecutionInput() .query(query) .build() ) ) SERVICE METHOD
  79. @PostMapping( path = ["/graphql"], consumes = [MediaType.APPLICATION_JSON_VALUE], produces = [MediaType.APPLICATION_JSON_VALUE]

    ) fun graphql( @RequestBody body: Map<String, Any> ): Mono<Map<String, Any>> { val query: String = body["query"] as String? ?: "{}" return graphQLService.processQuery(query) .map { it.toSpecification() } } REACTIVE API
  80. { "data": { "dog": { "fact": "“Him” and “Her” were

    the names of President Lyndon Johnson's two beagles.", "length": 73 }, "cat": { "fact": "Jaguars are the only big cats that don't roar.", "length": 46 } } } RESULT
  81. Resilience Protect the downstream services from the upstream.

  82. 250ms 380ms 380ms

  83. CAT FACTS BROKEN 250ms 30s 30s

  84. { "data": { "dog": {…}, "cat": null }, "errors": [

    { "message": "Exception while fetching data (/cat) : Operation timed out", "path": [ "cat" ] } ] }
  85. TIMEOUT 250ms 5000ms 5000ms

  86. class CatFactFetcher( val webClient: WebClient ) : FactFetcher { override

    fun get(environment: DataFetchingEnvironment): CompletionStage<Fact?> = webClient.get().uri("https://catfact.ninja/fact") .retrieve() .bodyToMono(CatFact::class.java) .map { … } .timeout(Duration.ofMillis(5000)) .toFuture() } CAT FACTS WITH TIMEOUT
  87. { "data": { "dog": {…}, "cat": null }, "errors": [

    { "message": "Exception while fetching data (/cat) : Did not observe any item or terminal signal within 5000ms in 'map' (and no fallback has been configured)", "path": [ “cat" ] } ] }
  88. CIRCUIT BREAKER 250ms 0ms 250ms

  89. CIRCUIT BREAKER Closed Open Half-open

  90. class CatFactFetcher( val webClient: WebClient, val catBreaker: CircuitBreaker ) :

    FactFetcher { override fun get(environment: DataFetchingEnvironment): CompletionStage<Fact?> = webClient.get()
 … .timeout(Duration.ofMillis(2500)) .transform( CircuitBreakerOperator.of(catBreaker) ) .toFuture() } CAT FACTS FETCHER WITH CIRCUIT BREAKER
  91. @Bean fun catCircuitBreaker(): CircuitBreaker { val circuitBreakerConfig = CircuitBreakerConfig.custom() .ringBufferSizeInClosedState(3)

    .ringBufferSizeInHalfOpenState(3) .waitDurationInOpenState(Duration.ofMillis(5000)) .failureRateThreshold(0.5f) .build() return circuitBreakerRegistry.circuitBreaker( "CatCCB", circuitBreakerConfig ) }
  92. { "data": { "dog": {…}, "cat": null }, "errors": [

    { "message": "Exception while fetching data (/cat) : CircuitBreaker 'CatCCB' is OPEN and does not permit further calls", "path": [ "cat" ] } ] }
  93. CIRCUIT BREAKER 250ms 0ms 250ms But we might miss some

    cat facts.
  94. Takeaways Even if you forget everything else I said, please

    try…
  95. GraphQL For APIs consumed from the front-end. Resilience To prevent

    propagating errors downstream. Reactive For I/O heavy apps. Takeaways
  96. Beyond Future Because you were going to ask about coroutines…

  97. GraphQL Java There are already pure Kotlin libraries. Spring boot

    has a set of starters too. Coroutines Support in Spring. New utilities for Project reactor. Alternatives
  98. fun processQuery(query: String): Map<String, Any> @PostMapping( path = ["/graphql"], consumes

    = [MediaType.APPLICATION_JSON_VALUE], produces = [MediaType.APPLICATION_JSON_VALUE] ) fun graphqlCoroutines( @RequestBody body: Map<String, Any> ): Mono<Map<String, Any>> = mono(Unconfined) { val query: String = body["query"] as String? ?: "{}" graphQLService.processQuery(query) } REACTOR WITH COROUTINES
  99. Resources Write your own reactive and resilient GraphQL gateway in

    Kotlin
  100. 1 2 3 Animal Facts GraphQL Gateway https://bitbucket.org/atlassianlabs/animal-facts-graphql-gateway-demo/src/ master/ graphql-java

    https://www.graphql-java.com/ Resilience4j https://github.com/resilience4j/resilience4j Resources / Code
  101. 4 GraphiQL https://github.com/graphql/graphiql Resources / Code 5 Coroutines-reactor https://github.com/Kotlin/kotlinx.coroutines/tree/master/reactive/kotlinx-coroutines- reactor

  102. 1 2 3 Kotlin at Trello Trello tech blog:
 https://tech.trello.com/kotlin-at-trello/

    Engineering stateless, high-availability cloud services at Atlassian Atlassian Engineering Blog: https://www.atlassian.com/blog/technology/aws-scaling-multi-region-low-latency- service More about Micros Slightly outdated, but still relevant.
 https://www.youtube.com/watch?v=dc2nqzgqH24 Resources / Further information
  103. 1 Why not both? https://knowyourmeme.com/memes/why-not-both-why-dont-we-have-both Resources / Memes

  104. MARTIN VARGA | SOFTWARE DEVELOPER | @MARTINTEEVARGA Future of Jira

    Software Powered by Kotlin
  105. None