Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Server-Side Kotlin Coroutines

Server-Side Kotlin Coroutines

Kotlin was designed as a general-purpose programming language and with coroutines writing server-side code is a easy as never before on JVM. We'll dissect scalability and reliability problems of a traditional thread-based stacks and discuss the process of mitigating those issues by introducing asynchrony with Kotlin coroutines. We'll see how coroutine-based design naturally avoids common pitfalls of traditional asynchronous programming such as resource management, error handling and request cancellation, producing safe and reliable code using a concept of structured concurrency.

F9c354e780ce562daea0e21b99bfdc0d?s=128

Roman Elizarov

April 30, 2019
Tweet

More Decks by Roman Elizarov

Other Decks in Programming

Transcript

  1. Server-side Kotlin with Coroutines Roman Elizarov relizarov

  2. Speaker: Roman Elizarov • Professional developer since 2000 • Previously

    developed high-perf trading software @ Devexperts • Teach concurrent & distributed programming @ St. Petersburg ITMO University • Chief judge @ Northern Eurasia Contest / ICPC • Now team lead in Kotlin Libraries @ JetBrains elizarov @ relizarov
  3. Kotlin – Programming Language for

  4. Kotlin – Programming Language for This talk Server-side

  5. Backend evolution Starting with “good old days”

  6. ET 1 ET 2 ET N … Executor Threads DB

    Old-school client-server monolith Clients
  7. ET 1 ET 2 ET N … Executor Threads DB

    Incoming request Clients
  8. ET 1 ET 2 ET N … Executor Threads DB

    Blocks tread ! Clients
  9. ET 1 ET 2 ET N … Executor Threads DB

    Sizing threads – easy Clients N = number of DB connections
  10. Services

  11. ET 1 ET 2 ET N … Executor Threads DB

    Old-school client-server monolith Clients
  12. ET 1 ET 2 ET N … Executor Threads DB

    Now with Services Service Clients
  13. ET 1 ET 2 ET N … Executor Threads Services

    everywhere … Service K Service 1 Service 2 Clients
  14. ET 1 ET 2 ET N … Executor Threads Sizing

    threads – not easy … N = ????? Service K Service 1 Service 2 Clients
  15. Complex business logic fun placeOrder(order: Order): Response { … }

  16. Complex business logic fun placeOrder(order: Order): Response { val account

    = accountService.loadAccout(order.accountId) … }
  17. Complex business logic fun placeOrder(order: Order): Response { val account

    = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) … }
  18. Complex business logic fun placeOrder(order: Order): Response { val account

    = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } … }
  19. Complex business logic fun placeOrder(order: Order): Response { val account

    = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
  20. Complex business logic fun placeOrder(order: Order): Response { val account

    = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
  21. What if a service is slow? fun placeOrder(order: Order): Response

    { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) } !
  22. ET 2 ET N … Executor Threads Clients Blocks threads

    … Service K Service 1 Service 2 ET 1! "
  23. ET 2 ET N … Executor Threads Clients Blocks threads

    … Service K Service 1 Service 2 ET 1! " !
  24. ET 2 ET N … Executor Threads Clients Blocks threads

    … Service K Service 1 Service 2 ET 1! " ! !
  25. Code that waits

  26. Asynchronous programming Writing code that waits

  27. ET 2 ET N … Executor Threads Clients Instead of

    blocking… … Service K Service 1 Service 2 ET 1! "
  28. ET 2 ET N … Executor Threads Release the thread

    … Service K Service 1 Service 2 ET 1 ! Clients
  29. Clients ET 2 ET N … Executor Threads Resume operation

    later … Service K Service 1 Service 2 ET 1 !
  30. But how? fun loadMargin(account: Account): Margin

  31. But how? •Callbacks fun loadMargin(account: Account, callback: (Margin) -> Unit)

  32. But how? •Callbacks •Futures/Promises fun loadMargin(account: Account): Future<Margin>

  33. But how? •Callbacks •Futures/Promises/Reactive fun loadMargin(account: Account): Mono<Margin>

  34. But how? •Callbacks •Futures/Promises/Reactive •async/await async fun loadMargin(account: Account): Task<Margin>

  35. But how? •Callbacks •Futures/Promises/Reactive •async/await •Kotlin Coroutines suspend fun loadMargin(account:

    Account): Margin
  36. Learn more KotlinConf (San Francisco) 2017 GOTO Copenhagen 2018

  37. Suspend behind the scenes suspend fun loadMargin(account: Account): Margin

  38. Suspend behind the scenes suspend fun loadMargin(account: Account): Margin fun

    loadMargin(account: Account, cont: Continuation<Margin>) But why callback and not future?
  39. Performance! •Future is a synchronization primitive •Callback is a lower-level

    primitive •Integration with async IO libraries is easy
  40. Integration suspend fun loadMargin(account: Account): Margin

  41. Integration suspend fun loadMargin(account: Account): Margin = suspendCoroutine { cont

    -> // install callback & use cont to resume }
  42. Integration at scale Going beyond slide-ware

  43. ET 2 ET N … Executor Threads Clients Release thread?

    … Service K Service 1 Service 2 ET 1! "
  44. Blocking server fun placeOrder(order: Order): Response { // must return

    response }
  45. Asynchronous server fun placeOrder(order: Order): Mono<Response> { // may return

    without response }
  46. Convenient? fun placeOrder(order: Order): Mono<Response> { // response from placed

    order cache return Mono.just(response) }
  47. Server integrated with coroutines suspend fun placeOrder(order: Order): Response {

    // response from placed order cache return response }
  48. Server not integrated with coroutines fun placeOrder(order: Order) = GlobalScope.mono

    { // response from placed order cache return@mono response } Coroutine builder
  49. The server shall support asynchrony is some way

  50. Suspend suspend fun placeOrder(order: Order): Response { val account =

    accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
  51. Suspend suspend fun placeOrder(order: Order): Response { val account =

    accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) } Invoke suspending funs
  52. Suspend is convenient suspend fun placeOrder(order: Order): Response { val

    account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) } Invoke suspending funs Write regular code!
  53. Suspend is efficient suspend fun placeOrder(order: Order): Response { val

    account = accountService.loadAccount(order.accountId) val margin = marginService.loadMargin(account) return validateOrder(order, margin) } One object allocated
  54. Futures/Promises/Reactive – less efficient fun placeOrder(order: Order): Mono<Response> = accountService.loadAccountAsync(order.accountId)

    .flatMap { account -> marginService.loadMargin(account) } .map { margin -> validateOrder(order, margin) } Lambda allocated* Future allocated Lambda allocated Future allocated
  55. Let’s go deeper fun placeOrder(params: Params): Mono<Response> { // check

    pre-conditions return actuallyPlaceOrder(order) } fun actuallyPlaceOrder(order: Order): Mono<Response>
  56. Let’s go deeper (with coroutines) suspend fun placeOrder(params: Params): Response

    { // check pre-conditions return actuallyPlaceOrder(order) } suspend fun actuallyPlaceOrder(params: Params): Response Tail call optimization Tail call
  57. Call stack with coroutines Coroutine Builder placeOrder actuallyPlaceOrder moreLogic marginService.loadMargin

    suspendCoroutine
  58. Call stack with coroutines Coroutine Builder placeOrder actuallyPlaceOrder moreLogic marginService.loadMargin

    suspendCoroutine unwind Continuation in heap
  59. Scaling with coroutines With thread pools

  60. ET 2 ET N … Executor Threads Clients Thread pools

    ET 1
  61. ET 2 ET N … Executor Threads Clients Thread pools

    ET 1 Service 1 Threads ST 2 ST M1 … S1 1 N = number of CPU cores M 1 = depends
  62. IO-bound (blocking) fun loadAccount(order: Order): Account { // some blocking

    code here.... }
  63. IO-bound suspend fun loadAccount(order: Order): Account { // some blocking

    code here.... }
  64. IO-bound withContext suspend fun loadAccount(order: Order): Account = withContext(dispatcher) {

    // some blocking code here.... }
  65. IO-bound withContext suspend fun loadAccount(order: Order): Account = withContext(dispatcher) {

    // some blocking code here.... } val dispatcher = Executors.newFixedThreadPool(M2).asCoroutineDispatcher()
  66. CPU-bound code fun validateOrder(order: Order, margin: Margin): Response { //

    perform CPU-consuming computation }
  67. CPU-bound code suspend fun validateOrder(order: Order, margin: Margin): Response =

    withContext(compute) { // perform CPU-consuming computation } val compute = Executors.newFixedThreadPool(M3).asCoroutineDispatcher()
  68. ET 2 ET N … Executor Threads Clients Fine-grained control

    and encapsulation ET 1 Service 1 Threads S1 1 ST M1 Service 2 Threads S1 1 ST M 2 Service 3 Threads S1 1 ST M 3 Async IO-bound CPU-bound Never blocked
  69. But there’s more!

  70. Cancellation

  71. withTimeout suspend fun placeOrder(order: Order): Response = withTimeout(1000) { //

    code before loadMargin(account) // code after }
  72. withTimeout propagation suspend fun placeOrder(order: Order): Response = withTimeout(1000) {

    // code before loadMargin(account) // code after } suspend fun loadMargin(account: Account): Margin = suspendCoroutine { cont -> // install callback & use cont to resume }
  73. withTimeout propagation suspend fun placeOrder(order: Order): Response = withTimeout(1000) {

    // code before loadMargin(account) // code after } suspend fun loadMargin(account: Account): Margin = suspendCancellableCoroutine { cont -> // install callback & use cont to resume }
  74. withTimeout propagation suspend fun placeOrder(order: Order): Response = withTimeout(1000) {

    // code before loadMargin(account) // code after } suspend fun loadMargin(account: Account): Margin = suspendCancellableCoroutine { cont -> // install callback & use cont to resume cont.invokeOnCancellation { … } }
  75. Concurrency Multiple things at the same time

  76. Example fun placeOrder(order: Order): Response { val account = accountService.loadAccount(order)

    val margin = marginService.loadMargin(order) return validateOrder(order, account, margin) }
  77. Example fun placeOrder(order: Order): Response { val account = accountService.loadAccount(order)

    val margin = marginService.loadMargin(order) return validateOrder(order, account, margin) } No data dependencies
  78. Concurrency with async (futures) fun placeOrder(order: Order): Response { val

    account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) }
  79. Concurrency with async (futures) fun placeOrder(order: Order): Response { val

    account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) }
  80. Concurrency with async (futures) fun placeOrder(order: Order): Response { val

    account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) } Fails?
  81. Concurrency with async (futures) fun placeOrder(order: Order): Response { val

    account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) } Fails? Leaks!
  82. Structured concurrency

  83. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) }
  84. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) }
  85. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) }
  86. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Fails?
  87. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Fails? Cancels
  88. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Fails? Cancels Cancels
  89. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope

    { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Waits for completion of all children
  90. Enforcing structure

  91. Without coroutine scope? suspend fun placeOrder(order: Order): Response { val

    account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } return validateOrder(order, account.await(), margin.await()) }
  92. Without coroutine scope? suspend fun placeOrder(order: Order): Response { val

    account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } return validateOrder(order, account.await(), margin.await()) } ERROR: Unresolved reference.
  93. Extensions of CoroutineScope fun <T> CoroutineScope.async( context: CoroutineContext = EmptyCoroutineContext,

    start: CoroutineStart = CoroutineStart.DEFAULT, block: suspend CoroutineScope.() -> T ): Deferred<T>
  94. Convention fun CoroutineScope.bg(params: Params) = launch { // … }

    Launches new coroutine
  95. Types as documentation fun foo(params: Params): Response suspend fun foo(params:

    Params): Response fun CoroutineScope.foo(params: Params): Response Fast, local Remote, or slow Side effect - bg
  96. Types are enforced fun foo(params: Params): Response suspend fun foo(params:

    Params): Response fun CoroutineScope.foo(params: Params): Response Not allowed But must provide scope explicitly Using coroutineScope { … } Fast, local Remote, or slow Side effect - bg
  97. Green threads / fibers Alternative way to async

  98. Green threads / Fibers ET 2 ET N … Executor

    Threads ET 1 F 2 F M … Fibers F 1 ~ Coroutines Hidden from developer
  99. Fibers promise •Develop just like with threads •Everything is effectively

    suspendable
  100. Marking with suspend pays off at scale

  101. Thread switching And how to avoid it

  102. ET 2 ET N … Executor Threads Clients Threads ET

    1 Service 1 Threads S1 1 ST M1 Service 2 Threads S1 1 ST M 2 Service 3 Threads S1 1 ST M 3
  103. ET 2 ET N … Executor Threads Clients Solution –

    shared thread pool ET 1
  104. ET 2 ET N … Executor Threads Clients Solution –

    shared thread pool ET 1!
  105. ET 2 ET N … Executor Threads Clients Solution –

    shared thread pool ET 1! ET N+1
  106. ET 2 ET N … Executor Threads Clients Solution –

    shared thread pool ET 1! ET N+1 ! ET N+2
  107. ET 2 ET N … Executor Threads Clients Solution –

    shared thread pool ET 1! ET N+1 ! … ET M! ET N+2 ET N+M
  108. withContext for IO suspend fun loadAccount(order: Order): Account = withContext(dispatcher)

    { // some blocking code here.... } val dispatcher = Executors.newFixedThreadPool(M2).asCoroutineDispatcher()
  109. withContext for Dispatсhers.IO suspend fun loadAccount(order: Order): Account = withContext(Dispatchers.IO)

    { // some blocking code here.... } No thread switch from Dispatchers.Default pool
  110. ET 2 ET N … Executor Threads Clients Solution –

    shared thread pool ET 1 Dispatchers.Default
  111. ET 2 ET N … Executor Threads Clients Solution –

    shared thread pool ET 1! ET N+1 ! … ET M! ET N+2 ET N+M Dispatchers.Default Dispatchers.IO
  112. Coroutines and data streams

  113. Returning many responses suspend fun foo(params: Params): Response One response

    suspend fun foo(params: Params): List<Response> Many responses suspend fun foo(params: Params): ????<Response> Many responses async?
  114. Channel receive() send()

  115. Producer Builder fun CoroutineScope.foo(): ReceiveChannel<Int> = produce { for (i

    in 1..10) { send(i) delay(100) } } Channel type Can be async
  116. Consumer fun CoroutineScope.foo(): ReceiveChannel<Int> = produce { for (i in

    1..10) { send(i) delay(100) } } fun main() = runBlocking<Unit> { for (x in foo()) { println(x) } }
  117. Where’s the catch? fun CoroutineScope.foo(): ReceiveChannel<Int> = produce { for

    (i in 1..10) { send(i) delay(100) } } fun main() = runBlocking<Unit> { for (x in foo()) { println(x) } }
  118. Where’s the catch? fun CoroutineScope.foo(): ReceiveChannel<Int> = produce { for

    (i in 1..10) { send(i) delay(100) } } fun main() = runBlocking<Unit> { for (x in foo()) { println(x) } } Creates coroutine
  119. Try this! fun CoroutineScope.foo(): ReceiveChannel<Int> = produce { for (i

    in 1..10) { send(i) delay(100) } } fun main() = runBlocking<Unit> { foo() } Waits for completion of children !
  120. Kotlin Flows Disclaimer: available in preview only, not stable yet

  121. Flow example fun bar(): Flow<Int> = flow { for (i

    in 1..10) { emit(i) delay(100) } } ~ Asynchronous sequence
  122. Flow example fun bar(): Flow<Int> = flow { for (i

    in 1..10) { emit(i) delay(100) } } fun main() = runBlocking<Unit> { bar().collect { x -> println(x) } }
  123. Try this! fun bar(): Flow<Int> = flow { for (i

    in 1..10) { emit(i) delay(100) } } fun main() = runBlocking<Unit> { bar() } Flow is cold: describes the data, does not run it until collected !
  124. Flow example fun bar(): Flow<Int> = flow { for (i

    in 1..10) { emit(i) delay(100) } } fun main() = runBlocking<Unit> { bar() .map { it * it } .toList() } Write regular code! Similar to collections / sequences
  125. Thank you Want to learn more? Questions? elizarov @ Roman

    Elizarov relizarov
  126. None