Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Spring-Flavored Kotlin Coroutines

Spring-Flavored Kotlin Coroutines

Kotlin has built-in support for asynchronous programming with Kotlin Coroutines that are designed to allow for simple and easy-to-understand code. We'll witness that reduction in complexity that coroutines bring and dissect what kind of magic is working behind the scenes to make it possible, how it all integrates with Spring and how can you integrate coroutines with any kind of asynchronous code you write.

F9c354e780ce562daea0e21b99bfdc0d?s=128

Roman Elizarov

October 08, 2019
Tweet

More Decks by Roman Elizarov

Other Decks in Programming

Transcript

  1. Spring-Flavored Kotlin Coroutines Roman Elizarov October 7–10, 2019 Austin Convention

    Center
  2. Speaker: Roman Elizarov Professional developer since 2000 Previously developed high-perf

    trading software @ Devexperts Teach concurrent & distributed programming @ St. Petersburg ITMO University Chief judge @ Northern Eurasia Contest / ICPC Now team lead in Kotlin Libraries @ JetBrains elizarov @ relizarov
  3. Kotlin – Programming Language for

  4. Kotlin – Programming Language for

  5. Backend evolution Starting with “good old days”

  6. ET 1 ET 2 ET N … DB Old-school client-server

    monolith Clients Executor Threads
  7. ET 1 ET 2 ET N … DB Incoming request

    Clients Executor Threads
  8. ET 1 ET 2 ET N … DB Blocks tread

    Clients Executor Threads
  9. ET 1 ET 2 ET N … DB Sizing threads

    – easy Clients N = number of DB connections Executor Threads
  10. Services

  11. ET 1 ET 2 ET N … DB Old-school client-server

    monolith Clients Executor Threads
  12. ET 1 ET 2 ET N … DB Now with

    Services Service Clients Executor Threads
  13. ET 1 ET 2 ET N … Services everywhere …

    Service K Service 1 Service 2 Clients Executor Threads
  14. Complex business logic fun placeOrder(order: Order): Response { … }

  15. Complex business logic fun placeOrder(order: Order): Response { val account

    = accountService.loadAccout(order.accountId) … }
  16. Complex business logic fun placeOrder(order: Order): Response { val account

    = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) … }
  17. Complex business logic fun placeOrder(order: Order): Response { val account

    = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } … }
  18. Complex business logic fun placeOrder(order: Order): Response { val account

    = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } … }
  19. Complex business logic fun placeOrder(order: Order): Response { val account

    = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
  20. Complex business logic fun placeOrder(order: Order): Response { val account

    = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
  21. What if a service is slow? fun placeOrder(order: Order): Response

    { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
  22. Slow service @Service class MarginService { private val rest =

    RestTemplate() fun loadMargin(account: Account): Margin = rest.getForObject( "http://localhost:9090/margin/${account.id}", Margin::class.java)!! }
  23. Slow service @Service class MarginService { private val rest =

    RestTemplate() fun loadMargin(account: Account): Margin = rest.getForObject( "http://localhost:9090/margin/${account.id}", Margin::class.java)!! }
  24. Demo Slow service, blocking

  25. ET 2 ET N … Clients Blocks threads … Service

    K Service 1 Service 2 ET 1 Executor Threads
  26. ET 2 ET N … Clients Blocks threads … Service

    K Service 1 Service 2 ET 1 Executor Threads
  27. ET 2 ET N … Clients Blocks threads … Service

    K Service 1 Service 2 ET 1 Executor Threads
  28. Code that waits

  29. Asynchronous programming Writing code that waits

  30. ET 2 ET N … Clients Instead of blocking… …

    Service K Service 1 Service 2 ET 1 Executor Threads
  31. ET 2 ET N … Release the thread … Service

    K Service 1 Service 2 ET 1 Clients Executor Threads
  32. Clients ET 2 ET N … Resume operation later …

    Service K Service 1 Service 2 ET 1 Executor Threads
  33. But how? fun loadMargin(account: Account): Margin

  34. But how? Callbacks fun loadMargin(account: Account, callback: (Margin) -> Unit)

  35. But how? Callbacks Futures/Promises fun loadMargin(account: Account): CompletableFuture<Margin>

  36. But how? Callbacks Futures/Promises/Reactive fun loadMargin(account: Account): Mono<Margin>

  37. Reactive

  38. The logic fun placeOrder(order: Order): Response { val account =

    accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
  39. Going reactive fun placeOrder(order: Order): Mono<Response>

  40. Going reactive fun placeOrder(order: Order): Mono<Response> =

  41. Going reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId)

  42. Going reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap {

    account -> … }
  43. Going reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap {

    account -> if (account.isOptionsAccount) { marginService.loadMargin(account) } else { Mono.just(defaultMargin) } }
  44. Going reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap {

    account -> if (account.isOptionsAccount) { marginService.loadMargin(account) } else { Mono.just(defaultMargin) } }
  45. Going reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap {

    account -> if (account.isOptionsAccount) { marginService.loadMargin(account) } else { Mono.just(defaultMargin) } } .map { margin -> … }
  46. Going reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap {

    account -> if (account.isOptionsAccount) { marginService.loadMargin(account) } else { Mono.just(defaultMargin) } } .map { margin -> validateOrder(order, margin) }
  47. Reactive Service @Service class MarginService { private val client =

    WebClient.create("http://localhost:9090/") fun loadMargin(account: Account): Mono<Margin> = client .get() .uri("margin/${account.id}") .retrieve() .bodyToMono(Margin::class.java) }
  48. Reactive Service @Service class MarginService { private val client =

    WebClient.create("http://localhost:9090/") fun loadMargin(account: Account): Mono<Margin> = client .get() .uri("margin/${account.id}") .retrieve() .bodyToMono(Margin::class.java) }
  49. Demo: Reactive Was it worth it?

  50. Choice 1: Direct code, blocking fun placeOrder(order: Order): Response {

    val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
  51. Choice 2: Complex code, non-blocking fun placeOrder(order: Order): Mono<Response> =

    accountService.loadAccount(order.accountId) .flatMap { account -> if (account.isOptionsAccount) { marginService.loadMargin(account) } else { Mono.just(defaultMargin) } } .map { margin -> validateOrder(order, margin) }
  52. Why not both?

  53. But how? Callbacks Futures/Promises/Reactive fun loadMargin(account: Account): Mono<Margin>

  54. But how? Callbacks Futures/Promises/Reactive Kotlin Coroutines suspend fun loadMargin(account: Account):

    Margin
  55. Direct code: Blocking fun placeOrder(order: Order): Response { val account

    = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
  56. Direct code: Coroutines suspend fun placeOrder(order: Order): Response { val

    account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) } Write regular code! Call suspending funs
  57. Coroutines Service @Service class MarginService { private val client =

    WebClient.create("http://localhost:9090/") suspend fun loadMargin(account: Account): Margin = client .get() .uri("margin/${account.id}") .retrieve() .awaitBody() }
  58. Coroutines Service @Service class MarginService { private val client =

    WebClient.create("http://localhost:9090/") suspend fun loadMargin(account: Account): Margin = client .get() .uri("margin/${account.id}") .retrieve() .awaitBody() }
  59. Demo: Coroutines

  60. None
  61. Suspend behind the scenes suspend fun loadMargin(account: Account): Margin

  62. Suspend behind the scenes suspend fun loadMargin(account: Account): Margin fun

    loadMargin(account: Account, cont: Continuation<Margin>) But why callback and not future/mono?
  63. Performance! Callback is a simple, low-level primitive Integration with async

    IO libraries is easy
  64. Service integration suspend fun loadMargin(account: Account): Margin

  65. Service integration suspend fun loadMargin(account: Account): Margin = suspendCoroutine {

    cont -> // install callback & use cont to resume }
  66. Server integrated with coroutines @PostMapping("/order") suspend fun placeOrder(order: Order): Response

    { … }
  67. Server not integrated with coroutines @PostMapping("/order") fun placeOrder(order: Order): Mono<Response>

    = mono { … } Coroutine builder
  68. Blocking integration @Component class Initializer { @EventListener(ApplicationStartedEvent::class) fun initData() {

    … } }
  69. Blocking integration @Component class Initializer { @EventListener(ApplicationStartedEvent::class) fun initData() =

    runBlocking<Unit> { … } } Coroutine builder
  70. Suspend is efficient

  71. Suspend is efficient suspend fun placeOrder(order: Order): Response { val

    account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) } One object allocated
  72. Reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap { account

    -> if (account.isOptionsAccount) { marginService.loadMargin(account) } else { Mono.just(defaultMargin) } } .map { margin -> validateOrder(order, margin) } Lambda allocated* Mono allocated Lambda allocated Mono allocated
  73. Let’s go deeper fun placeOrder(params: Params): Mono<Response> { // check

    pre-conditions return actuallyPlaceOrder(order) } fun actuallyPlaceOrder(order: Order): Mono<Response>
  74. Let’s go deeper (with coroutines) suspend fun placeOrder(params: Params): Response

    { // check pre-conditions return actuallyPlaceOrder(order) } suspend fun actuallyPlaceOrder(params: Params): Response Tail call optimization Tail call
  75. Call stack with coroutines Coroutine Builder placeOrder actuallyPlaceOrder moreLogic marginService.loadMargin

    suspendCoroutine
  76. Call stack with coroutines Coroutine Builder placeOrder actuallyPlaceOrder moreLogic marginService.loadMargin

    suspendCoroutine unwind Continuation in heap
  77. Learn more KotlinConf San Francisco 2017 GOTO Copenhagen 2018

  78. Scaling with coroutines With thread pools

  79. ET 2 ET N … Clients Thread pools ET 1

    Executor Threads
  80. ET 2 ET N … Clients Thread pools ET 1

    ST 2 ST M1 … S1 1 N = number of CPU cores M1 = depends Service 1 Threads Executor Threads
  81. IO-bound (blocking) fun loadAccount(order: Order): Account { // some blocking

    code here.... }
  82. IO-bound suspend fun loadAccount(order: Order): Account { // some blocking

    code here.... }
  83. IO-bound withContext suspend fun loadAccount(order: Order): Account = withContext(dispatcher) {

    // some blocking code here.... }
  84. IO-bound withContext suspend fun loadAccount(order: Order): Account = withContext(dispatcher) {

    // some blocking code here.... } val dispatcher = Executors.newFixedThreadPool(M2).asCoroutineDispatcher()
  85. CPU-bound code fun validateOrder(order: Order, margin: Margin): Response { //

    perform CPU-consuming computation }
  86. CPU-bound code suspend fun validateOrder(order: Order, margin: Margin): Response =

    withContext(compute) { // perform CPU-consuming computation } val compute = Executors.newFixedThreadPool(M3).asCoroutineDispatcher()
  87. ET 2 ET N … Clients Fine-grained control and encapsulation

    ET 1 S1 1 ST M1 Service 2 Threads S1 1 ST M2 Service 3 Threads S1 1 ST M3 Async IO-bound CPU- bound Never blocked Service 1 Threads Executor Threads
  88. But there’s more!

  89. Cancellation

  90. withTimeout suspend fun placeOrder(order: Order): Response = withTimeout(1000) { //

    code before loadMargin(account) // code after }
  91. withTimeout propagation suspend fun placeOrder(order: Order): Response = withTimeout(1000) {

    // code before loadMargin(account) // code after } suspend fun loadMargin(account: Account): Margin = suspendCoroutine { cont -> // install callback & use cont to resume }
  92. withTimeout propagation suspend fun placeOrder(order: Order): Response = withTimeout(1000) {

    // code before loadMargin(account) // code after } suspend fun loadMargin(account: Account): Margin = suspendCancellableCoroutine { cont -> // install callback & use cont to resume }
  93. withTimeout propagation suspend fun placeOrder(order: Order): Response = withTimeout(1000) {

    // code before loadMargin(account) // code after } suspend fun loadMargin(account: Account): Margin = suspendCancellableCoroutine { cont -> // install callback & use cont to resume cont.invokeOnCancellation { … } }
  94. Concurrency Multiple things at the same time

  95. Example fun placeOrder(order: Order): Response { val account = accountService.loadAccount(order)

    val margin = marginService.loadMargin(order) return validateOrder(order, account, margin) }
  96. Example fun placeOrder(order: Order): Response { val account = accountService.loadAccount(order)

    val margin = marginService.loadMargin(order) return validateOrder(order, account, margin) } No data dependencies
  97. Concurrency with async (futures) fun placeOrder(order: Order): Response { val

    account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) }
  98. Concurrency with async (futures) fun placeOrder(order: Order): Response { val

    account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) }
  99. Concurrency with async (futures) fun placeOrder(order: Order): Response { val

    account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) } Fails?
  100. Concurrency with async (futures) fun placeOrder(order: Order): Response { val

    account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) } Fails? Leaks !
  101. Structured concurrency

  102. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) }
  103. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) }
  104. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) }
  105. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Fails?
  106. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Fails? Cancels
  107. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Fails? Cancels Cancels
  108. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Waits for completion of all children
  109. Enforcing structure

  110. Without coroutine scope? suspend fun placeOrder(order: Order): Response { val

    account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } return validateOrder(order, account.await(), margin.await()) }
  111. Without coroutine scope? suspend fun placeOrder(order: Order): Response { val

    account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } return validateOrder(order, account.await(), margin.await()) } ERROR: Unresolved reference.
  112. Extensions of CoroutineScope fun <T> CoroutineScope.async( context: CoroutineContext = EmptyCoroutineContext,

    start: CoroutineStart = CoroutineStart.DEFAULT, block: suspend CoroutineScope.() -> T ): Deferred<T>
  113. Convention fun CoroutineScope.bg(params: Params) = launch { // … }

    Launches new coroutine
  114. Types as documentation fun foo(params: Params): Response suspend fun foo(params:

    Params): Response fun CoroutineScope.foo(params: Params): Response Fast, local Remote, or slow Side effect - bg
  115. Types are enforced fun foo(params: Params): Response suspend fun foo(params:

    Params): Response fun CoroutineScope.foo(params: Params): Response Not allowed But must provide scope explicitly Using coroutineScope { … } Fast, local Remote, or slow Side effect - bg
  116. Green threads / fibers Alternative way to async

  117. Green threads / Fibers ET 2 ET N … ET

    1 F 2 F M … Fibers F 1 ~ Coroutines Hidden from developer Executor Threads
  118. But marking with suspend pays off at scale Fibers –

    everything suspending
  119. Thread switching And how to avoid it

  120. ET 2 ET N … Clients Threads ET 1 S1

    1 ST M1 S1 1 ST M2 S1 1 ST M3 Service 2 Threads Service 3 Threads Service 1 Threads Executor Threads
  121. ET 2 ET N … Clients Solution – shared thread

    pool ET 1 Executor Threads
  122. ET 2 ET N … Clients Solution – shared thread

    pool ET 1 Executor Threads
  123. ET 2 ET N … Clients Solution – shared thread

    pool ET 1 ET N+1 Executor Threads
  124. ET 2 ET N … Clients Solution – shared thread

    pool ET 1 ET N+1 ET N+2 Executor Threads
  125. ET 2 ET N … Clients Solution – shared thread

    pool ET 1 ET N+1 … ET M ET N+2 ET N+M Executor Threads
  126. withContext for IO suspend fun loadAccount(order: Order): Account = withContext(dispatcher)

    { // some blocking code here.... } val dispatcher = Executors.newFixedThreadPool(M2).asCoroutineDispatcher()
  127. withContext for Dispatсhers.IO suspend fun loadAccount(order: Order): Account = withContext(Dispatchers.IO)

    { // some blocking code here.... } No thread switch from Dispatchers.Default pool
  128. ET 2 ET N … Clients Solution – shared thread

    pool ET 1 Dispatchers.Default Executor Threads
  129. ET 2 ET N … Clients Solution – shared thread

    pool ET 1 ET N+1 … ET M ET N+2 ET N+M Dispatchers.Default Dispatchers.IO Executor Threads
  130. Coroutines and data streams

  131. Returning many responses suspend fun foo(params: Params): Response One response

    suspend fun foo(params: Params): List<Response> Many responses
  132. Returning many responses suspend fun foo(params: Params): Response One response

    suspend fun foo(params: Params): List<Response> Many responses fun foo(params: Params): Flow<Response> Many responses async
  133. Kotlin Flows Reactive streams for coroutines

  134. fun foo(): Flow<Int> ~ Reactive Publisher / Flux

  135. fun foo(): Flow<Int> = flow { … }

  136. fun foo(): Flow<Int> = flow { for (i in 1..10)

    { emit(i) delay(100) } }
  137. fun foo(): Flow<Int> = flow { for (i in 1..10)

    { emit(i) delay(100) } } suspend fun main() { foo().collect { x -> println(x) } }
  138. fun foo(): Flow<Int> = flow { for (i in 1..10)

    { emit(i) delay(100) } } suspend fun main() { foo() } Flow is cold: describes the data, does not run it until collected
  139. Flow operators fun foo(): Flow<Int> = flow { for (i

    in 1..10) { emit(i) delay(100) } } suspend fun main() { foo() .map { it * it } .toList() }
  140. Why flow?

  141. A A’ mapper fun map(mapper: (T) -> R): Flux<R> fun

    flatMap(mapper: (T) -> Publisher<R>): Flux<R> Synchronous Asynchronous Flux<T>
  142. A A’ mapper fun map(mapper: (T) -> R): Flux<R> fun

    flatMap(mapper: (T) -> Publisher<R>): Flux<R> fun filter(predicate: (T) -> Boolean): Flux<T> A A predicate Synchronous Asynchronous Synchronous Asynchronous Flux<T> fun filterWhen(predicate: (T) -> Publisher<Boolean>): Flux<T>
  143. A A’ Flow<T> fun map(transform: suspend (T) -> R): Flow<R>

    transform
  144. A A’ transform Flow<T> fun map(transform: suspend (T) -> R):

    Flow<R>
  145. A A’ transform fun map(transform: suspend (T) -> R): Flow<R>

    Flow<T> fun filter(predicate: suspend (T) -> Boolean): Flow<T> A predicate A
  146. Operator avoidance startWith(value) onStart { emit(value) } delaySubscription(time) onStart {

    delay(time) } startWith(flow) onStart { emitAll(flow) } delayElements(time) onEach { delay(time) } onErrorReturn(value) catch { emit(value) } onErrorResume(flow) catch { emitAll(flow) } generate(…) flow { … } Composable
  147. Reactive + = ❤

  148. Spring ❤ Flow @GetMapping("/sse/{n}", produces = [MediaType.TEXT_EVENT_STREAM_VALUE]) fun greetings(@PathVariable n:

    String): Flow<Greeting>
  149. Spring ❤ Flow @GetMapping("/sse/{n}", produces = [MediaType.TEXT_EVENT_STREAM_VALUE]) fun greetings(@PathVariable n:

    String): Flow<Greeting> = flow { while(true) { emit(Greeting("Hello from coroutines $n @ ${Instant.now()}")) delay(1000) } }
  150. Demo: Spring Flow Support

  151. Thank you Questions? elizarov @ relizarov

  152. Kotlin Coroutines For The Win! #springone @s1p