Server-Side Kotlin Coroutines

Kotlin was designed as a general-purpose programming language and with coroutines writing server-side code is a easy as never before on JVM. We'll dissect scalability and reliability problems of a traditional thread-based stacks and discuss the process of mitigating those issues by introducing asynchrony with Kotlin coroutines. We'll see how coroutine-based design naturally avoids common pitfalls of traditional asynchronous programming such as resource management, error handling and request cancellation, producing safe and reliable code using a concept of structured concurrency.

Transcript

1. Server-side Kotlin
   with Coroutines
   Roman Elizarov
   relizarov
   

   View Slide

2. Speaker: Roman Elizarov
   • Professional developer since 2000
   • Previously developed high-perf trading software
   @ Devexperts
   • Teach concurrent & distributed programming
   @ St. Petersburg ITMO University
   • Chief judge
   @ Northern Eurasia Contest / ICPC
   • Now team lead in Kotlin Libraries
   @ JetBrains
   elizarov @
   relizarov
   

   View Slide

3. Kotlin – Programming Language for
   

   View Slide

4. Kotlin – Programming Language for
   This talk
   Server-side
   

   View Slide

5. Backend evolution
   Starting with “good old days”
   

   View Slide

6. ET 1
   ET 2
   ET N
   …
   Executor Threads
   DB
   Old-school client-server monolith
   Clients
   

   View Slide

7. ET 1
   ET 2
   ET N
   …
   Executor Threads
   DB
   Incoming request
   Clients
   

   View Slide

8. ET 1
   ET 2
   ET N
   …
   Executor Threads
   DB
   Blocks tread
   !
   Clients
   

   View Slide

9. ET 1
   ET 2
   ET N
   …
   Executor Threads
   DB
   Sizing threads – easy
   Clients
   N = number of DB
   connections
   

   View Slide

10. Services
    

    View Slide

11. ET 1
    ET 2
    ET N
    …
    Executor Threads
    DB
    Old-school client-server monolith
    Clients
    

    View Slide

12. ET 1
    ET 2
    ET N
    …
    Executor Threads
    DB
    Now with Services
    Service
    Clients
    

    View Slide

13. ET 1
    ET 2
    ET N
    …
    Executor Threads
    Services everywhere
    …
    Service K
    Service 1
    Service 2
    Clients
    

    View Slide

14. ET 1
    ET 2
    ET N
    …
    Executor Threads
    Sizing threads – not easy
    …
    N = ?????
    Service K
    Service 1
    Service 2
    Clients
    

    View Slide

15. Complex business logic
    fun placeOrder(order: Order): Response {
    …
    }
    

    View Slide

16. Complex business logic
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    …
    }
    

    View Slide

17. Complex business logic
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount)
    …
    }
    

    View Slide

18. Complex business logic
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    …
    }
    

    View Slide

19. Complex business logic
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    return validateOrder(order, margin)
    }
    

    View Slide

20. Complex business logic
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    return validateOrder(order, margin)
    }
    

    View Slide

21. What if a service is slow?
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    return validateOrder(order, margin)
    }
    !
    

    View Slide

22. ET 2
    ET N
    …
    Executor Threads
    Clients
    Blocks threads
    …
    Service K
    Service 1
    Service 2
    ET 1!
    "
    

    View Slide

23. ET 2
    ET N
    …
    Executor Threads
    Clients
    Blocks threads
    …
    Service K
    Service 1
    Service 2
    ET 1!
    "
    !
    

    View Slide

24. ET 2
    ET N
    …
    Executor Threads
    Clients
    Blocks threads
    …
    Service K
    Service 1
    Service 2
    ET 1!
    "
    !
    !
    

    View Slide

25. Code that waits
    

    View Slide

26. Asynchronous programming
    Writing code that waits
    

    View Slide

27. ET 2
    ET N
    …
    Executor Threads
    Clients
    Instead of blocking…
    …
    Service K
    Service 1
    Service 2
    ET 1!
    "
    

    View Slide

28. ET 2
    ET N
    …
    Executor Threads
    Release the thread
    …
    Service K
    Service 1
    Service 2
    ET 1
    !
    Clients
    

    View Slide

29. Clients
    ET 2
    ET N
    …
    Executor Threads
    Resume operation later
    …
    Service K
    Service 1
    Service 2
    ET 1
    !
    

    View Slide

30. But how?
    fun loadMargin(account: Account): Margin
    

    View Slide

31. But how?
    •Callbacks
    fun loadMargin(account: Account, callback: (Margin) -> Unit)
    

    View Slide

32. But how?
    •Callbacks
    •Futures/Promises
    fun loadMargin(account: Account): Future
    

    View Slide

33. But how?
    •Callbacks
    •Futures/Promises/Reactive
    fun loadMargin(account: Account): Mono
    

    View Slide

34. But how?
    •Callbacks
    •Futures/Promises/Reactive
    •async/await
    async fun loadMargin(account: Account): Task
    

    View Slide

35. But how?
    •Callbacks
    •Futures/Promises/Reactive
    •async/await
    •Kotlin Coroutines
    suspend fun loadMargin(account: Account): Margin
    

    View Slide

36. Learn more
    KotlinConf (San Francisco) 2017 GOTO Copenhagen 2018
    

    View Slide

37. Suspend behind the scenes
    suspend fun loadMargin(account: Account): Margin
    

    View Slide

38. Suspend behind the scenes
    suspend fun loadMargin(account: Account): Margin
    fun loadMargin(account: Account, cont: Continuation)
    But why callback and not future?
    

    View Slide

39. Performance!
    •Future is a synchronization primitive
    •Callback is a lower-level primitive
    •Integration with async IO libraries is easy
    

    View Slide

40. Integration
    suspend fun loadMargin(account: Account): Margin
    

    View Slide

41. Integration
    suspend fun loadMargin(account: Account): Margin =
    suspendCoroutine { cont ->
    // install callback & use cont to resume
    }
    

    View Slide

42. Integration at scale
    Going beyond slide-ware
    

    View Slide

43. ET 2
    ET N
    …
    Executor Threads
    Clients
    Release thread?
    …
    Service K
    Service 1
    Service 2
    ET 1!
    "
    

    View Slide

44. Blocking server
    fun placeOrder(order: Order): Response {
    // must return response
    }
    

    View Slide

45. Asynchronous server
    fun placeOrder(order: Order): Mono {
    // may return without response
    }
    

    View Slide

46. Convenient?
    fun placeOrder(order: Order): Mono {
    // response from placed order cache
    return Mono.just(response)
    }
    

    View Slide

47. Server integrated with coroutines
    suspend fun placeOrder(order: Order): Response {
    // response from placed order cache
    return response
    }
    

    View Slide

48. Server not integrated with coroutines
    fun placeOrder(order: Order) = GlobalScope.mono {
    // response from placed order cache
    return@mono response
    }
    Coroutine builder
    

    View Slide

49. The server shall support
    asynchrony is some way
    

    View Slide

50. Suspend
    suspend fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    return validateOrder(order, margin)
    }
    

    View Slide

51. Suspend
    suspend fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    return validateOrder(order, margin)
    }
    Invoke suspending funs
    

    View Slide

52. Suspend is convenient
    suspend fun placeOrder(order: Order): Response {
    val account = accountService.loadAccout(order.accountId)
    val margin = if (account.isOptionsAccount) {
    marginService.loadMargin(account)
    } else {
    defaultMargin
    }
    return validateOrder(order, margin)
    }
    Invoke suspending funs
    Write regular code!
    

    View Slide

53. Suspend is efficient
    suspend fun placeOrder(order: Order): Response {
    val account = accountService.loadAccount(order.accountId)
    val margin = marginService.loadMargin(account)
    return validateOrder(order, margin)
    }
    One object allocated
    

    View Slide

54. Futures/Promises/Reactive – less efficient
    fun placeOrder(order: Order): Mono =
    accountService.loadAccountAsync(order.accountId)
    .flatMap { account -> marginService.loadMargin(account) }
    .map { margin -> validateOrder(order, margin) }
    Lambda allocated*
    Future allocated
    Lambda allocated
    Future allocated
    

    View Slide

55. Let’s go deeper
    fun placeOrder(params: Params): Mono {
    // check pre-conditions
    return actuallyPlaceOrder(order)
    }
    fun actuallyPlaceOrder(order: Order): Mono
    

    View Slide

56. Let’s go deeper (with coroutines)
    suspend fun placeOrder(params: Params): Response {
    // check pre-conditions
    return actuallyPlaceOrder(order)
    }
    suspend fun actuallyPlaceOrder(params: Params): Response
    Tail call optimization
    Tail call
    

    View Slide

57. Call stack with coroutines
    Coroutine Builder
    placeOrder
    actuallyPlaceOrder
    moreLogic
    marginService.loadMargin
    suspendCoroutine
    

    View Slide

58. Call stack with coroutines
    Coroutine Builder
    placeOrder
    actuallyPlaceOrder
    moreLogic
    marginService.loadMargin
    suspendCoroutine
    unwind
    Continuation in heap
    

    View Slide

59. Scaling with coroutines
    With thread pools
    

    View Slide

60. ET 2
    ET N
    …
    Executor Threads
    Clients
    Thread pools
    ET 1
    

    View Slide

61. ET 2
    ET N
    …
    Executor Threads
    Clients
    Thread pools
    ET 1
    Service 1 Threads
    ST 2
    ST M1
    …
    S1 1
    N = number of CPU cores M
    1
    = depends
    

    View Slide

62. IO-bound (blocking)
    fun loadAccount(order: Order): Account {
    // some blocking code here....
    }
    

    View Slide

63. IO-bound
    suspend fun loadAccount(order: Order): Account {
    // some blocking code here....
    }
    

    View Slide

64. IO-bound withContext
    suspend fun loadAccount(order: Order): Account =
    withContext(dispatcher) {
    // some blocking code here....
    }
    

    View Slide

65. IO-bound withContext
    suspend fun loadAccount(order: Order): Account =
    withContext(dispatcher) {
    // some blocking code here....
    }
    val dispatcher =
    Executors.newFixedThreadPool(M2).asCoroutineDispatcher()
    

    View Slide

66. CPU-bound code
    fun validateOrder(order: Order, margin: Margin): Response {
    // perform CPU-consuming computation
    }
    

    View Slide

67. CPU-bound code
    suspend fun validateOrder(order: Order, margin: Margin): Response =
    withContext(compute) {
    // perform CPU-consuming computation
    }
    val compute =
    Executors.newFixedThreadPool(M3).asCoroutineDispatcher()
    

    View Slide

68. ET 2
    ET N
    …
    Executor Threads
    Clients
    Fine-grained control and encapsulation
    ET 1
    Service 1 Threads
    S1 1
    ST M1
    Service 2 Threads
    S1 1
    ST M
    2
    Service 3 Threads
    S1 1
    ST M
    3
    Async
    IO-bound
    CPU-bound
    Never blocked
    

    View Slide

69. But there’s more!
    

    View Slide

70. Cancellation
    

    View Slide

71. withTimeout
    suspend fun placeOrder(order: Order): Response =
    withTimeout(1000) {
    // code before
    loadMargin(account)
    // code after
    }
    

    View Slide

72. withTimeout propagation
    suspend fun placeOrder(order: Order): Response =
    withTimeout(1000) {
    // code before
    loadMargin(account)
    // code after
    }
    suspend fun loadMargin(account: Account): Margin =
    suspendCoroutine { cont ->
    // install callback & use cont to resume
    }
    

    View Slide

73. withTimeout propagation
    suspend fun placeOrder(order: Order): Response =
    withTimeout(1000) {
    // code before
    loadMargin(account)
    // code after
    }
    suspend fun loadMargin(account: Account): Margin =
    suspendCancellableCoroutine { cont ->
    // install callback & use cont to resume
    }
    

    View Slide

74. withTimeout propagation
    suspend fun placeOrder(order: Order): Response =
    withTimeout(1000) {
    // code before
    loadMargin(account)
    // code after
    }
    suspend fun loadMargin(account: Account): Margin =
    suspendCancellableCoroutine { cont ->
    // install callback & use cont to resume
    cont.invokeOnCancellation { … }
    }
    

    View Slide

75. Concurrency
    Multiple things at the same time
    

    View Slide

76. Example
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccount(order)
    val margin = marginService.loadMargin(order)
    return validateOrder(order, account, margin)
    }
    

    View Slide

77. Example
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccount(order)
    val margin = marginService.loadMargin(order)
    return validateOrder(order, account, margin)
    }
    No data dependencies
    

    View Slide

78. Concurrency with async (futures)
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccountAsync(order)
    val margin = marginService.loadMarginAsync(order)
    return validateOrder(order, account.await(), margin.await())
    }
    

    View Slide

79. Concurrency with async (futures)
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccountAsync(order)
    val margin = marginService.loadMarginAsync(order)
    return validateOrder(order, account.await(), margin.await())
    }
    

    View Slide

80. Concurrency with async (futures)
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccountAsync(order)
    val margin = marginService.loadMarginAsync(order)
    return validateOrder(order, account.await(), margin.await())
    }
    Fails?
    

    View Slide

81. Concurrency with async (futures)
    fun placeOrder(order: Order): Response {
    val account = accountService.loadAccountAsync(order)
    val margin = marginService.loadMarginAsync(order)
    return validateOrder(order, account.await(), margin.await())
    }
    Fails?
    Leaks!
    

    View Slide

82. Structured concurrency
    

    View Slide

83. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }
    

    View Slide

84. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }
    

    View Slide

85. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }
    

    View Slide

86. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }
    Fails?
    

    View Slide

87. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }
    Fails?
    Cancels
    

    View Slide

88. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }
    Fails?
    Cancels
    Cancels
    

    View Slide

89. Concurrency with coroutines
    suspend fun placeOrder(order: Order): Response =
    coroutineScope {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    validateOrder(order, account.await(), margin.await())
    }
    Waits for completion of all children
    

    View Slide

90. Enforcing structure
    

    View Slide

91. Without coroutine scope?
    suspend fun placeOrder(order: Order): Response {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    return validateOrder(order, account.await(), margin.await())
    }
    

    View Slide

92. Without coroutine scope?
    suspend fun placeOrder(order: Order): Response {
    val account = async { accountService.loadAccount(order) }
    val margin = async { marginService.loadMargin(order) }
    return validateOrder(order, account.await(), margin.await())
    }
    ERROR: Unresolved reference.
    

    View Slide

93. Extensions of CoroutineScope
    fun CoroutineScope.async(
    context: CoroutineContext = EmptyCoroutineContext,
    start: CoroutineStart = CoroutineStart.DEFAULT,
    block: suspend CoroutineScope.() -> T
    ): Deferred
    

    View Slide

94. Convention
    fun CoroutineScope.bg(params: Params) = launch {
    // …
    }
    Launches new coroutine
    

    View Slide

95. Types as documentation
    fun foo(params: Params): Response
    suspend fun foo(params: Params): Response
    fun CoroutineScope.foo(params: Params): Response
    Fast, local
    Remote, or slow
    Side effect - bg
    

    View Slide

96. Types are enforced
    fun foo(params: Params): Response
    suspend fun foo(params: Params): Response
    fun CoroutineScope.foo(params: Params): Response
    Not allowed
    But must provide
    scope explicitly
    Using coroutineScope { … }
    Fast, local
    Remote, or slow
    Side effect - bg
    

    View Slide

97. Green threads / fibers
    Alternative way to async
    

    View Slide

98. Green threads / Fibers
    ET 2
    ET N
    …
    Executor Threads
    ET 1
    F 2
    F M
    …
    Fibers
    F 1
    ~ Coroutines Hidden from developer
    

    View Slide

99. Fibers promise
    •Develop just like with threads
    •Everything is effectively suspendable
    

    View Slide

100. Marking with suspend
     pays off at scale
     

     View Slide

101. Thread switching
     And how to avoid it
     

     View Slide

102. ET 2
     ET N
     …
     Executor Threads
     Clients
     Threads
     ET 1
     Service 1 Threads
     S1 1
     ST M1
     Service 2 Threads
     S1 1
     ST M
     2
     Service 3 Threads
     S1 1
     ST M
     3
     

     View Slide

103. ET 2
     ET N
     …
     Executor Threads
     Clients
     Solution – shared thread pool
     ET 1
     

     View Slide

104. ET 2
     ET N
     …
     Executor Threads
     Clients
     Solution – shared thread pool
     ET 1!
     

     View Slide

105. ET 2
     ET N
     …
     Executor Threads
     Clients
     Solution – shared thread pool
     ET 1! ET N+1
     

     View Slide

106. ET 2
     ET N
     …
     Executor Threads
     Clients
     Solution – shared thread pool
     ET 1! ET N+1
     ! ET N+2
     

     View Slide

107. ET 2
     ET N
     …
     Executor Threads
     Clients
     Solution – shared thread pool
     ET 1! ET N+1
     !
     …
     ET M!
     ET N+2
     ET N+M
     

     View Slide

108. withContext for IO
     suspend fun loadAccount(order: Order): Account =
     withContext(dispatcher) {
     // some blocking code here....
     }
     val dispatcher =
     Executors.newFixedThreadPool(M2).asCoroutineDispatcher()
     

     View Slide

109. withContext for Dispatсhers.IO
     suspend fun loadAccount(order: Order): Account =
     withContext(Dispatchers.IO) {
     // some blocking code here....
     }
     No thread switch from
     Dispatchers.Default pool
     

     View Slide

110. ET 2
     ET N
     …
     Executor Threads
     Clients
     Solution – shared thread pool
     ET 1
     Dispatchers.Default
     

     View Slide

111. ET 2
     ET N
     …
     Executor Threads
     Clients
     Solution – shared thread pool
     ET 1! ET N+1
     !
     …
     ET M!
     ET N+2
     ET N+M
     Dispatchers.Default
     Dispatchers.IO
     

     View Slide

112. Coroutines and data streams
     

     View Slide

113. Returning many responses
     suspend fun foo(params: Params): Response One response
     suspend fun foo(params: Params): List Many responses
     suspend fun foo(params: Params): ???? Many responses async?
     

     View Slide

114. Channel
     receive()
     send()
     

     View Slide

115. Producer
     Builder
     fun CoroutineScope.foo(): ReceiveChannel = produce {
     for (i in 1..10) {
     send(i)
     delay(100)
     }
     }
     Channel type
     Can be async
     

     View Slide

116. Consumer
     fun CoroutineScope.foo(): ReceiveChannel = produce {
     for (i in 1..10) {
     send(i)
     delay(100)
     }
     }
     fun main() = runBlocking {
     for (x in foo()) {
     println(x)
     }
     }
     

     View Slide

117. Where’s the catch?
     fun CoroutineScope.foo(): ReceiveChannel = produce {
     for (i in 1..10) {
     send(i)
     delay(100)
     }
     }
     fun main() = runBlocking {
     for (x in foo()) {
     println(x)
     }
     }
     

     View Slide

118. Where’s the catch?
     fun CoroutineScope.foo(): ReceiveChannel = produce {
     for (i in 1..10) {
     send(i)
     delay(100)
     }
     }
     fun main() = runBlocking {
     for (x in foo()) {
     println(x)
     }
     }
     Creates coroutine
     

     View Slide

119. Try this!
     fun CoroutineScope.foo(): ReceiveChannel = produce {
     for (i in 1..10) {
     send(i)
     delay(100)
     }
     }
     fun main() = runBlocking {
     foo()
     }
     Waits for completion of children
     !
     

     View Slide

120. Kotlin Flows
     Disclaimer: available in preview only, not stable yet
     

     View Slide

121. Flow example
     fun bar(): Flow = flow {
     for (i in 1..10) {
     emit(i)
     delay(100)
     }
     }
     ~ Asynchronous sequence
     

     View Slide

122. Flow example
     fun bar(): Flow = flow {
     for (i in 1..10) {
     emit(i)
     delay(100)
     }
     }
     fun main() = runBlocking {
     bar().collect { x ->
     println(x)
     }
     }
     

     View Slide

123. Try this!
     fun bar(): Flow = flow {
     for (i in 1..10) {
     emit(i)
     delay(100)
     }
     }
     fun main() = runBlocking {
     bar()
     }
     Flow is cold: describes the data,
     does not run it until collected
     !
     

     View Slide

124. Flow example
     fun bar(): Flow = flow {
     for (i in 1..10) {
     emit(i)
     delay(100)
     }
     }
     fun main() = runBlocking {
     bar()
     .map { it * it }
     .toList()
     }
     Write regular code!
     Similar to collections / sequences
     

     View Slide

125. Thank you
     Want to learn more?
     Questions?
     elizarov @
     Roman Elizarov
     relizarov
     

     View Slide

126. View Slide