$30 off During Our Annual Pro Sale. View Details »

Server-Side Kotlin Coroutines

Server-Side Kotlin Coroutines

Kotlin was designed as a general-purpose programming language and with coroutines writing server-side code is a easy as never before on JVM. We'll dissect scalability and reliability problems of a traditional thread-based stacks and discuss the process of mitigating those issues by introducing asynchrony with Kotlin coroutines. We'll see how coroutine-based design naturally avoids common pitfalls of traditional asynchronous programming such as resource management, error handling and request cancellation, producing safe and reliable code using a concept of structured concurrency.

Roman Elizarov

April 30, 2019
Tweet

More Decks by Roman Elizarov

Other Decks in Programming

Transcript

  1. Server-side Kotlin
    with Coroutines
    Roman Elizarov
    relizarov

    View Slide

  2. Speaker: Roman Elizarov
    • Professional developer since 2000
    • Previously developed high-perf trading software
    @ Devexperts
    • Teach concurrent & distributed programming
    @ St. Petersburg ITMO University
    • Chief judge
    @ Northern Eurasia Contest / ICPC
    • Now team lead in Kotlin Libraries
    @ JetBrains
    elizarov @
    relizarov

    View Slide

  3. Kotlin – Programming Language for

    View Slide

  4. Kotlin – Programming Language for
    This talk
    Server-side

    View Slide

  5. Backend evolution
    Starting with “good old days”

    View Slide

  6. ET 1
    ET 2
    ET N

    Executor Threads
    DB
    Old-school client-server monolith
    Clients

    View Slide

  7. ET 1
    ET 2
    ET N

    Executor Threads
    DB
    Incoming request
    Clients

    View Slide

  8. ET 1
    ET 2
    ET N

    Executor Threads
    DB
    Blocks tread
    !
    Clients

    View Slide

  9. ET 1
    ET 2
    ET N

    Executor Threads
    DB
    Sizing threads – easy
    Clients
    N = number of DB
    connections

    View Slide

  10. Services

    View Slide

  11. ET 1
    ET 2
    ET N

    Executor Threads
    DB
    Old-school client-server monolith
    Clients

    View Slide

  12. ET 1
    ET 2
    ET N

    Executor Threads
    DB
    Now with Services
    Service
    Clients

    View Slide

  13. ET 1
    ET 2
    ET N

    Executor Threads
    Services everywhere

    Service K
    Service 1
    Service 2
    Clients

    View Slide

  14. ET 1
    ET 2
    ET N

    Executor Threads
    Sizing threads – not easy

    N = ?????
    Service K
    Service 1
    Service 2
    Clients

    View Slide

  15. Complex business logic
    fun placeOrder(order: Order): Response {

    }

    View Slide

  16. Complex business logic
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)

    }

    View Slide

  17. Complex business logic
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount)

    }

    View Slide

  18. Complex business logic
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }

    }

    View Slide

  19. Complex business logic
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    return validateOrder(order, margin)
    }

    View Slide

  20. Complex business logic
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    return validateOrder(order, margin)
    }

    View Slide

  21. What if a service is slow?
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    return validateOrder(order, margin)
    }
    !

    View Slide

  22. ET 2
    ET N

    Executor Threads
    Clients
    Blocks threads

    Service K
    Service 1
    Service 2
    ET 1!
    "

    View Slide

  23. ET 2
    ET N

    Executor Threads
    Clients
    Blocks threads

    Service K
    Service 1
    Service 2
    ET 1!
    "
    !

    View Slide

  24. ET 2
    ET N

    Executor Threads
    Clients
    Blocks threads

    Service K
    Service 1
    Service 2
    ET 1!
    "
    !
    !

    View Slide

  25. Code that waits

    View Slide

  26. Asynchronous programming
    Writing code that waits

    View Slide

  27. ET 2
    ET N

    Executor Threads
    Clients
    Instead of blocking…

    Service K
    Service 1
    Service 2
    ET 1!
    "

    View Slide

  28. ET 2
    ET N

    Executor Threads
    Release the thread

    Service K
    Service 1
    Service 2
    ET 1
    !
    Clients

    View Slide

  29. Clients
    ET 2
    ET N

    Executor Threads
    Resume operation later

    Service K
    Service 1
    Service 2
    ET 1
    !

    View Slide

  30. But how?
    fun loadMargin(account: Account): Margin

    View Slide

  31. But how?
    •Callbacks
    fun loadMargin(account: Account, callback: (Margin) -> Unit)

    View Slide

  32. But how?
    •Callbacks
    •Futures/Promises
    fun loadMargin(account: Account): Future

    View Slide

  33. But how?
    •Callbacks
    •Futures/Promises/Reactive
    fun loadMargin(account: Account): Mono

    View Slide

  34. But how?
    •Callbacks
    •Futures/Promises/Reactive
    •async/await
    async fun loadMargin(account: Account): Task

    View Slide

  35. But how?
    •Callbacks
    •Futures/Promises/Reactive
    •async/await
    •Kotlin Coroutines
    suspend fun loadMargin(account: Account): Margin

    View Slide

  36. Learn more
    KotlinConf (San Francisco) 2017 GOTO Copenhagen 2018

    View Slide

  37. Suspend behind the scenes
    suspend fun loadMargin(account: Account): Margin

    View Slide

  38. Suspend behind the scenes
    suspend fun loadMargin(account: Account): Margin
    fun loadMargin(account: Account, cont: Continuation)
    But why callback and not future?

    View Slide

  39. Performance!
    •Future is a synchronization primitive
    •Callback is a lower-level primitive
    •Integration with async IO libraries is easy

    View Slide

  40. Integration
    suspend fun loadMargin(account: Account): Margin

    View Slide

  41. Integration
    suspend fun loadMargin(account: Account): Margin =
    suspendCoroutine { cont ->
    // install callback & use cont to resume
    }

    View Slide

  42. Integration at scale
    Going beyond slide-ware

    View Slide

  43. ET 2
    ET N

    Executor Threads
    Clients
    Release thread?

    Service K
    Service 1
    Service 2
    ET 1!
    "

    View Slide

  44. Blocking server
    fun placeOrder(order: Order): Response {
    // must return response
    }

    View Slide

  45. Asynchronous server
    fun placeOrder(order: Order): Mono {
    // may return without response
    }

    View Slide

  46. Convenient?
    fun placeOrder(order: Order): Mono {
    // response from placed order cache
    return Mono.just(response)
    }

    View Slide

  47. Server integrated with coroutines
    suspend fun placeOrder(order: Order): Response {
    // response from placed order cache
    return response
    }

    View Slide

  48. Server not integrated with coroutines
    fun placeOrder(order: Order) = GlobalScope.mono {
    // response from placed order cache
    return@mono response
    }
    Coroutine builder

    View Slide

  49. The server shall support
    asynchrony is some way

    View Slide

  50. Suspend
    suspend fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    return validateOrder(order, margin)
    }

    View Slide

  51. Suspend
    suspend fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    return validateOrder(order, margin)
    }
    Invoke suspending funs

    View Slide

  52. Suspend is convenient
    suspend fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    return validateOrder(order, margin)
    }
    Invoke suspending funs
    Write regular code!

    View Slide

  53. Suspend is efficient
    suspend fun placeOrder(order: Order): Response {
    val account = accountService.loadAccount(order.accountId)
    val margin = marginService.loadMargin(account)
    return validateOrder(order, margin)
    }
    One object allocated

    View Slide

  54. Futures/Promises/Reactive – less efficient
    fun placeOrder(order: Order): Mono =
    accountService.loadAccountAsync(order.accountId)
    .flatMap { account -> marginService.loadMargin(account) }
    .map { margin -> validateOrder(order, margin) }
    Lambda allocated*
    Future allocated
    Lambda allocated
    Future allocated

    View Slide

  55. Let’s go deeper
    fun placeOrder(params: Params): Mono {
    // check pre-conditions
    return actuallyPlaceOrder(order)
    }
    fun actuallyPlaceOrder(order: Order): Mono

    View Slide

  56. Let’s go deeper (with coroutines)
    suspend fun placeOrder(params: Params): Response {
    // check pre-conditions
    return actuallyPlaceOrder(order)
    }
    suspend fun actuallyPlaceOrder(params: Params): Response
    Tail call optimization
    Tail call

    View Slide

  57. Call stack with coroutines
    Coroutine Builder
    placeOrder
    actuallyPlaceOrder
    moreLogic
    marginService.loadMargin
    suspendCoroutine

    View Slide

  58. Call stack with coroutines
    Coroutine Builder
    placeOrder
    actuallyPlaceOrder
    moreLogic
    marginService.loadMargin
    suspendCoroutine
    unwind
    Continuation in heap

    View Slide

  59. Scaling with coroutines
    With thread pools

    View Slide

  60. ET 2
    ET N

    Executor Threads
    Clients
    Thread pools
    ET 1

    View Slide

  61. ET 2
    ET N

    Executor Threads
    Clients
    Thread pools
    ET 1
    Service 1 Threads
    ST 2
    ST M1

    S1 1
    N = number of CPU cores M
    1
    = depends

    View Slide

  62. IO-bound (blocking)
    fun loadAccount(order: Order): Account {
    // some blocking code here....
    }

    View Slide

  63. IO-bound
    suspend fun loadAccount(order: Order): Account {
    // some blocking code here....
    }

    View Slide

  64. IO-bound withContext
    suspend fun loadAccount(order: Order): Account =
    withContext(dispatcher) {
    // some blocking code here....
    }

    View Slide

  65. IO-bound withContext
    suspend fun loadAccount(order: Order): Account =
    withContext(dispatcher) {
    // some blocking code here....
    }
    val dispatcher =
    Executors.newFixedThreadPool(M2).asCoroutineDispatcher()

    View Slide

  66. CPU-bound code
    fun validateOrder(order: Order, margin: Margin): Response {
    // perform CPU-consuming computation
    }

    View Slide

  67. CPU-bound code
    suspend fun validateOrder(order: Order, margin: Margin): Response =
    withContext(compute) {
    // perform CPU-consuming computation
    }
    val compute =
    Executors.newFixedThreadPool(M3).asCoroutineDispatcher()

    View Slide

  68. ET 2
    ET N

    Executor Threads
    Clients
    Fine-grained control and encapsulation
    ET 1
    Service 1 Threads
    S1 1
    ST M1
    Service 2 Threads
    S1 1
    ST M
    2
    Service 3 Threads
    S1 1
    ST M
    3
    Async
    IO-bound
    CPU-bound
    Never blocked

    View Slide

  69. But there’s more!

    View Slide

  70. Cancellation

    View Slide

  71. withTimeout
    suspend fun placeOrder(order: Order): Response =
    withTimeout(1000) {
    // code before
    loadMargin(account)
    // code after
    }

    View Slide

  72. withTimeout propagation
    suspend fun placeOrder(order: Order): Response =
    withTimeout(1000) {
    // code before
    loadMargin(account)
    // code after
    }
    suspend fun loadMargin(account: Account): Margin =
    suspendCoroutine { cont ->
    // install callback & use cont to resume
    }

    View Slide

  73. withTimeout propagation
    suspend fun placeOrder(order: Order): Response =
    withTimeout(1000) {
    // code before
    loadMargin(account)
    // code after
    }
    suspend fun loadMargin(account: Account): Margin =
    suspendCancellableCoroutine { cont ->
    // install callback & use cont to resume
    }

    View Slide

  74. withTimeout propagation
    suspend fun placeOrder(order: Order): Response =
    withTimeout(1000) {
    // code before
    loadMargin(account)
    // code after
    }
    suspend fun loadMargin(account: Account): Margin =
    suspendCancellableCoroutine { cont ->
    // install callback & use cont to resume
    cont.invokeOnCancellation { … }
    }

    View Slide

  75. Concurrency
    Multiple things at the same time

    View Slide

  76. Example
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccount(order)
    val margin = marginService.loadMargin(order)
    return validateOrder(order, account, margin)
    }

    View Slide

  77. Example
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccount(order)
    val margin = marginService.loadMargin(order)
    return validateOrder(order, account, margin)
    }
    No data dependencies

    View Slide

  78. Concurrency with async (futures)
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccountAsync(order)
    val margin = marginService.loadMarginAsync(order)
    return validateOrder(order, account.await(), margin.await())
    }

    View Slide

  79. Concurrency with async (futures)
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccountAsync(order)
    val margin = marginService.loadMarginAsync(order)
    return validateOrder(order, account.await(), margin.await())
    }

    View Slide

  80. Concurrency with async (futures)
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccountAsync(order)
    val margin = marginService.loadMarginAsync(order)
    return validateOrder(order, account.await(), margin.await())
    }
    Fails?

    View Slide

  81. Concurrency with async (futures)
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccountAsync(order)
    val margin = marginService.loadMarginAsync(order)
    return validateOrder(order, account.await(), margin.await())
    }
    Fails?
    Leaks!

    View Slide

  82. Structured concurrency

    View Slide

  83. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }

    View Slide

  84. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }

    View Slide

  85. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }

    View Slide

  86. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }
    Fails?

    View Slide

  87. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }
    Fails?
    Cancels

    View Slide

  88. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }
    Fails?
    Cancels
    Cancels

    View Slide

  89. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }
    Waits for completion of all children

    View Slide

  90. Enforcing structure

    View Slide

  91. Without coroutine scope?
    suspend fun placeOrder(order: Order): Response {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    return validateOrder(order, account.await(), margin.await())
    }

    View Slide

  92. Without coroutine scope?
    suspend fun placeOrder(order: Order): Response {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    return validateOrder(order, account.await(), margin.await())
    }
    ERROR: Unresolved reference.

    View Slide

  93. Extensions of CoroutineScope
    fun CoroutineScope.async(
    context: CoroutineContext = EmptyCoroutineContext,
    start: CoroutineStart = CoroutineStart.DEFAULT,
    block: suspend CoroutineScope.() -> T
    ): Deferred

    View Slide

  94. Convention
    fun CoroutineScope.bg(params: Params) = launch {
    // …
    }
    Launches new coroutine

    View Slide

  95. Types as documentation
    fun foo(params: Params): Response
    suspend fun foo(params: Params): Response
    fun CoroutineScope.foo(params: Params): Response
    Fast, local
    Remote, or slow
    Side effect - bg

    View Slide

  96. Types are enforced
    fun foo(params: Params): Response
    suspend fun foo(params: Params): Response
    fun CoroutineScope.foo(params: Params): Response
    Not allowed
    But must provide
    scope explicitly
    Using coroutineScope { … }
    Fast, local
    Remote, or slow
    Side effect - bg

    View Slide

  97. Green threads / fibers
    Alternative way to async

    View Slide

  98. Green threads / Fibers
    ET 2
    ET N

    Executor Threads
    ET 1
    F 2
    F M

    Fibers
    F 1
    ~ Coroutines Hidden from developer

    View Slide

  99. Fibers promise
    •Develop just like with threads
    •Everything is effectively suspendable

    View Slide

  100. Marking with suspend
    pays off at scale

    View Slide

  101. Thread switching
    And how to avoid it

    View Slide

  102. ET 2
    ET N

    Executor Threads
    Clients
    Threads
    ET 1
    Service 1 Threads
    S1 1
    ST M1
    Service 2 Threads
    S1 1
    ST M
    2
    Service 3 Threads
    S1 1
    ST M
    3

    View Slide

  103. ET 2
    ET N

    Executor Threads
    Clients
    Solution – shared thread pool
    ET 1

    View Slide

  104. ET 2
    ET N

    Executor Threads
    Clients
    Solution – shared thread pool
    ET 1!

    View Slide

  105. ET 2
    ET N

    Executor Threads
    Clients
    Solution – shared thread pool
    ET 1! ET N+1

    View Slide

  106. ET 2
    ET N

    Executor Threads
    Clients
    Solution – shared thread pool
    ET 1! ET N+1
    ! ET N+2

    View Slide

  107. ET 2
    ET N

    Executor Threads
    Clients
    Solution – shared thread pool
    ET 1! ET N+1
    !

    ET M!
    ET N+2
    ET N+M

    View Slide

  108. withContext for IO
    suspend fun loadAccount(order: Order): Account =
    withContext(dispatcher) {
    // some blocking code here....
    }
    val dispatcher =
    Executors.newFixedThreadPool(M2).asCoroutineDispatcher()

    View Slide

  109. withContext for Dispatсhers.IO
    suspend fun loadAccount(order: Order): Account =
    withContext(Dispatchers.IO) {
    // some blocking code here....
    }
    No thread switch from
    Dispatchers.Default pool

    View Slide

  110. ET 2
    ET N

    Executor Threads
    Clients
    Solution – shared thread pool
    ET 1
    Dispatchers.Default

    View Slide

  111. ET 2
    ET N

    Executor Threads
    Clients
    Solution – shared thread pool
    ET 1! ET N+1
    !

    ET M!
    ET N+2
    ET N+M
    Dispatchers.Default
    Dispatchers.IO

    View Slide

  112. Coroutines and data streams

    View Slide

  113. Returning many responses
    suspend fun foo(params: Params): Response One response
    suspend fun foo(params: Params): List Many responses
    suspend fun foo(params: Params): ???? Many responses async?

    View Slide

  114. Channel
    receive()
    send()

    View Slide

  115. Producer
    Builder
    fun CoroutineScope.foo(): ReceiveChannel = produce {
    for (i in 1..10) {
    send(i)
    delay(100)
    }
    }
    Channel type
    Can be async

    View Slide

  116. Consumer
    fun CoroutineScope.foo(): ReceiveChannel = produce {
    for (i in 1..10) {
    send(i)
    delay(100)
    }
    }
    fun main() = runBlocking {
    for (x in foo()) {
    println(x)
    }
    }

    View Slide

  117. Where’s the catch?
    fun CoroutineScope.foo(): ReceiveChannel = produce {
    for (i in 1..10) {
    send(i)
    delay(100)
    }
    }
    fun main() = runBlocking {
    for (x in foo()) {
    println(x)
    }
    }

    View Slide

  118. Where’s the catch?
    fun CoroutineScope.foo(): ReceiveChannel = produce {
    for (i in 1..10) {
    send(i)
    delay(100)
    }
    }
    fun main() = runBlocking {
    for (x in foo()) {
    println(x)
    }
    }
    Creates coroutine

    View Slide

  119. Try this!
    fun CoroutineScope.foo(): ReceiveChannel = produce {
    for (i in 1..10) {
    send(i)
    delay(100)
    }
    }
    fun main() = runBlocking {
    foo()
    }
    Waits for completion of children
    !

    View Slide

  120. Kotlin Flows
    Disclaimer: available in preview only, not stable yet

    View Slide

  121. Flow example
    fun bar(): Flow = flow {
    for (i in 1..10) {
    emit(i)
    delay(100)
    }
    }
    ~ Asynchronous sequence

    View Slide

  122. Flow example
    fun bar(): Flow = flow {
    for (i in 1..10) {
    emit(i)
    delay(100)
    }
    }
    fun main() = runBlocking {
    bar().collect { x ->
    println(x)
    }
    }

    View Slide

  123. Try this!
    fun bar(): Flow = flow {
    for (i in 1..10) {
    emit(i)
    delay(100)
    }
    }
    fun main() = runBlocking {
    bar()
    }
    Flow is cold: describes the data,
    does not run it until collected
    !

    View Slide

  124. Flow example
    fun bar(): Flow = flow {
    for (i in 1..10) {
    emit(i)
    delay(100)
    }
    }
    fun main() = runBlocking {
    bar()
    .map { it * it }
    .toList()
    }
    Write regular code!
    Similar to collections / sequences

    View Slide

  125. Thank you
    Want to learn more?
    Questions?
    elizarov @
    Roman Elizarov
    relizarov

    View Slide

  126. View Slide