Server-Side Kotlin Coroutines

Kotlin was designed as a general-purpose programming language and with coroutines writing server-side code is a easy as never before on JVM. We'll dissect scalability and reliability problems of a traditional thread-based stacks and discuss the process of mitigating those issues by introducing asynchrony with Kotlin coroutines. We'll see how coroutine-based design naturally avoids common pitfalls of traditional asynchronous programming such as resource management, error handling and request cancellation, producing safe and reliable code using a concept of structured concurrency.

Transcript

1. Server-side Kotlin with Coroutines Roman Elizarov relizarov
2. Speaker: Roman Elizarov • Professional developer since 2000 • Previously developed high-perf trading software @ Devexperts • Teach concurrent & distributed programming @ St. Petersburg ITMO University • Chief judge @ Northern Eurasia Contest / ICPC • Now team lead in Kotlin Libraries @ JetBrains elizarov @ relizarov
3. Kotlin – Programming Language for
4. Kotlin – Programming Language for This talk Server-side
5. Backend evolution Starting with “good old days”
6. ET 1 ET 2 ET N … Executor Threads DB Old-school client-server monolith Clients
7. ET 1 ET 2 ET N … Executor Threads DB Incoming request Clients
8. ET 1 ET 2 ET N … Executor Threads DB Blocks tread ! Clients
9. ET 1 ET 2 ET N … Executor Threads DB Sizing threads – easy Clients N = number of DB connections
10. Services
11. ET 1 ET 2 ET N … Executor Threads DB Old-school client-server monolith Clients
12. ET 1 ET 2 ET N … Executor Threads DB Now with Services Service Clients
13. ET 1 ET 2 ET N … Executor Threads Services everywhere … Service K Service 1 Service 2 Clients
14. ET 1 ET 2 ET N … Executor Threads Sizing threads – not easy … N = ????? Service K Service 1 Service 2 Clients
15. Complex business logic fun placeOrder(order: Order): Response { … }
16. Complex business logic fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) … }
17. Complex business logic fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) … }
18. Complex business logic fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } … }
19. Complex business logic fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
20. Complex business logic fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
21. What if a service is slow? fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) } !
22. ET 2 ET N … Executor Threads Clients Blocks threads … Service K Service 1 Service 2 ET 1! "
23. ET 2 ET N … Executor Threads Clients Blocks threads … Service K Service 1 Service 2 ET 1! " !
24. ET 2 ET N … Executor Threads Clients Blocks threads … Service K Service 1 Service 2 ET 1! " ! !
25. Code that waits
26. Asynchronous programming Writing code that waits
27. ET 2 ET N … Executor Threads Clients Instead of blocking… … Service K Service 1 Service 2 ET 1! "
28. ET 2 ET N … Executor Threads Release the thread … Service K Service 1 Service 2 ET 1 ! Clients
29. Clients ET 2 ET N … Executor Threads Resume operation later … Service K Service 1 Service 2 ET 1 !
30. But how? fun loadMargin(account: Account): Margin
31. But how? •Callbacks fun loadMargin(account: Account, callback: (Margin) -> Unit)
32. But how? •Callbacks •Futures/Promises fun loadMargin(account: Account): Future<Margin>
33. But how? •Callbacks •Futures/Promises/Reactive fun loadMargin(account: Account): Mono<Margin>
34. But how? •Callbacks •Futures/Promises/Reactive •async/await async fun loadMargin(account: Account): Task<Margin>
35. But how? •Callbacks •Futures/Promises/Reactive •async/await •Kotlin Coroutines suspend fun loadMargin(account: Account): Margin
36. Learn more KotlinConf (San Francisco) 2017 GOTO Copenhagen 2018
37. Suspend behind the scenes suspend fun loadMargin(account: Account): Margin
38. Suspend behind the scenes suspend fun loadMargin(account: Account): Margin fun loadMargin(account: Account, cont: Continuation<Margin>) But why callback and not future?
39. Performance! •Future is a synchronization primitive •Callback is a lower-level primitive •Integration with async IO libraries is easy
40. Integration suspend fun loadMargin(account: Account): Margin
41. Integration suspend fun loadMargin(account: Account): Margin = suspendCoroutine { cont -> // install callback & use cont to resume }
42. Integration at scale Going beyond slide-ware
43. ET 2 ET N … Executor Threads Clients Release thread? … Service K Service 1 Service 2 ET 1! "
44. Blocking server fun placeOrder(order: Order): Response { // must return response }
45. Asynchronous server fun placeOrder(order: Order): Mono<Response> { // may return without response }
46. Convenient? fun placeOrder(order: Order): Mono<Response> { // response from placed order cache return Mono.just(response) }
47. Server integrated with coroutines suspend fun placeOrder(order: Order): Response { // response from placed order cache return response }
48. Server not integrated with coroutines fun placeOrder(order: Order) = GlobalScope.mono { // response from placed order cache return@mono response } Coroutine builder
49. The server shall support asynchrony is some way
50. Suspend suspend fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
51. Suspend suspend fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) } Invoke suspending funs
52. Suspend is convenient suspend fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) } Invoke suspending funs Write regular code!
53. Suspend is efficient suspend fun placeOrder(order: Order): Response { val account = accountService.loadAccount(order.accountId) val margin = marginService.loadMargin(account) return validateOrder(order, margin) } One object allocated
54. Futures/Promises/Reactive – less efficient fun placeOrder(order: Order): Mono<Response> = accountService.loadAccountAsync(order.accountId) .flatMap { account -> marginService.loadMargin(account) } .map { margin -> validateOrder(order, margin) } Lambda allocated* Future allocated Lambda allocated Future allocated
55. Let’s go deeper fun placeOrder(params: Params): Mono<Response> { // check pre-conditions return actuallyPlaceOrder(order) } fun actuallyPlaceOrder(order: Order): Mono<Response>
56. Let’s go deeper (with coroutines) suspend fun placeOrder(params: Params): Response { // check pre-conditions return actuallyPlaceOrder(order) } suspend fun actuallyPlaceOrder(params: Params): Response Tail call optimization Tail call
57. Call stack with coroutines Coroutine Builder placeOrder actuallyPlaceOrder moreLogic marginService.loadMargin suspendCoroutine
58. Call stack with coroutines Coroutine Builder placeOrder actuallyPlaceOrder moreLogic marginService.loadMargin suspendCoroutine unwind Continuation in heap
59. Scaling with coroutines With thread pools
60. ET 2 ET N … Executor Threads Clients Thread pools ET 1
61. ET 2 ET N … Executor Threads Clients Thread pools ET 1 Service 1 Threads ST 2 ST M1 … S1 1 N = number of CPU cores M 1 = depends
62. IO-bound (blocking) fun loadAccount(order: Order): Account { // some blocking code here.... }
63. IO-bound suspend fun loadAccount(order: Order): Account { // some blocking code here.... }
64. IO-bound withContext suspend fun loadAccount(order: Order): Account = withContext(dispatcher) { // some blocking code here.... }
65. IO-bound withContext suspend fun loadAccount(order: Order): Account = withContext(dispatcher) { // some blocking code here.... } val dispatcher = Executors.newFixedThreadPool(M2).asCoroutineDispatcher()
66. CPU-bound code fun validateOrder(order: Order, margin: Margin): Response { // perform CPU-consuming computation }
67. CPU-bound code suspend fun validateOrder(order: Order, margin: Margin): Response = withContext(compute) { // perform CPU-consuming computation } val compute = Executors.newFixedThreadPool(M3).asCoroutineDispatcher()
68. ET 2 ET N … Executor Threads Clients Fine-grained control and encapsulation ET 1 Service 1 Threads S1 1 ST M1 Service 2 Threads S1 1 ST M 2 Service 3 Threads S1 1 ST M 3 Async IO-bound CPU-bound Never blocked
69. But there’s more!
70. Cancellation
71. withTimeout suspend fun placeOrder(order: Order): Response = withTimeout(1000) { // code before loadMargin(account) // code after }
72. withTimeout propagation suspend fun placeOrder(order: Order): Response = withTimeout(1000) { // code before loadMargin(account) // code after } suspend fun loadMargin(account: Account): Margin = suspendCoroutine { cont -> // install callback & use cont to resume }
73. withTimeout propagation suspend fun placeOrder(order: Order): Response = withTimeout(1000) { // code before loadMargin(account) // code after } suspend fun loadMargin(account: Account): Margin = suspendCancellableCoroutine { cont -> // install callback & use cont to resume }
74. withTimeout propagation suspend fun placeOrder(order: Order): Response = withTimeout(1000) { // code before loadMargin(account) // code after } suspend fun loadMargin(account: Account): Margin = suspendCancellableCoroutine { cont -> // install callback & use cont to resume cont.invokeOnCancellation { … } }
75. Concurrency Multiple things at the same time
76. Example fun placeOrder(order: Order): Response { val account = accountService.loadAccount(order) val margin = marginService.loadMargin(order) return validateOrder(order, account, margin) }
77. Example fun placeOrder(order: Order): Response { val account = accountService.loadAccount(order) val margin = marginService.loadMargin(order) return validateOrder(order, account, margin) } No data dependencies
78. Concurrency with async (futures) fun placeOrder(order: Order): Response { val account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) }
79. Concurrency with async (futures) fun placeOrder(order: Order): Response { val account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) }
80. Concurrency with async (futures) fun placeOrder(order: Order): Response { val account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) } Fails?
81. Concurrency with async (futures) fun placeOrder(order: Order): Response { val account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) } Fails? Leaks!
82. Structured concurrency
83. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) }
84. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) }
85. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) }
86. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Fails?
87. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Fails? Cancels
88. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Fails? Cancels Cancels
89. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Waits for completion of all children
90. Enforcing structure
91. Without coroutine scope? suspend fun placeOrder(order: Order): Response { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } return validateOrder(order, account.await(), margin.await()) }
92. Without coroutine scope? suspend fun placeOrder(order: Order): Response { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } return validateOrder(order, account.await(), margin.await()) } ERROR: Unresolved reference.
93. Extensions of CoroutineScope fun <T> CoroutineScope.async( context: CoroutineContext = EmptyCoroutineContext, start: CoroutineStart = CoroutineStart.DEFAULT, block: suspend CoroutineScope.() -> T ): Deferred<T>
94. Convention fun CoroutineScope.bg(params: Params) = launch { // … } Launches new coroutine
95. Types as documentation fun foo(params: Params): Response suspend fun foo(params: Params): Response fun CoroutineScope.foo(params: Params): Response Fast, local Remote, or slow Side effect - bg
96. Types are enforced fun foo(params: Params): Response suspend fun foo(params: Params): Response fun CoroutineScope.foo(params: Params): Response Not allowed But must provide scope explicitly Using coroutineScope { … } Fast, local Remote, or slow Side effect - bg
97. Green threads / fibers Alternative way to async
98. Green threads / Fibers ET 2 ET N … Executor Threads ET 1 F 2 F M … Fibers F 1 ~ Coroutines Hidden from developer
99. Fibers promise •Develop just like with threads •Everything is effectively suspendable
100. Marking with suspend pays off at scale
101. Thread switching And how to avoid it
102. ET 2 ET N … Executor Threads Clients Threads ET 1 Service 1 Threads S1 1 ST M1 Service 2 Threads S1 1 ST M 2 Service 3 Threads S1 1 ST M 3
103. ET 2 ET N … Executor Threads Clients Solution – shared thread pool ET 1
104. ET 2 ET N … Executor Threads Clients Solution – shared thread pool ET 1!
105. ET 2 ET N … Executor Threads Clients Solution – shared thread pool ET 1! ET N+1
106. ET 2 ET N … Executor Threads Clients Solution – shared thread pool ET 1! ET N+1 ! ET N+2
107. ET 2 ET N … Executor Threads Clients Solution – shared thread pool ET 1! ET N+1 ! … ET M! ET N+2 ET N+M
108. withContext for IO suspend fun loadAccount(order: Order): Account = withContext(dispatcher) { // some blocking code here.... } val dispatcher = Executors.newFixedThreadPool(M2).asCoroutineDispatcher()
109. withContext for Dispatсhers.IO suspend fun loadAccount(order: Order): Account = withContext(Dispatchers.IO) { // some blocking code here.... } No thread switch from Dispatchers.Default pool
110. ET 2 ET N … Executor Threads Clients Solution – shared thread pool ET 1 Dispatchers.Default
111. ET 2 ET N … Executor Threads Clients Solution – shared thread pool ET 1! ET N+1 ! … ET M! ET N+2 ET N+M Dispatchers.Default Dispatchers.IO
112. Coroutines and data streams
113. Returning many responses suspend fun foo(params: Params): Response One response suspend fun foo(params: Params): List<Response> Many responses suspend fun foo(params: Params): ????<Response> Many responses async?
114. Channel receive() send()
115. Producer Builder fun CoroutineScope.foo(): ReceiveChannel<Int> = produce { for (i in 1..10) { send(i) delay(100) } } Channel type Can be async
116. Consumer fun CoroutineScope.foo(): ReceiveChannel<Int> = produce { for (i in 1..10) { send(i) delay(100) } } fun main() = runBlocking<Unit> { for (x in foo()) { println(x) } }
117. Where’s the catch? fun CoroutineScope.foo(): ReceiveChannel<Int> = produce { for (i in 1..10) { send(i) delay(100) } } fun main() = runBlocking<Unit> { for (x in foo()) { println(x) } }
118. Where’s the catch? fun CoroutineScope.foo(): ReceiveChannel<Int> = produce { for (i in 1..10) { send(i) delay(100) } } fun main() = runBlocking<Unit> { for (x in foo()) { println(x) } } Creates coroutine
119. Try this! fun CoroutineScope.foo(): ReceiveChannel<Int> = produce { for (i in 1..10) { send(i) delay(100) } } fun main() = runBlocking<Unit> { foo() } Waits for completion of children !
120. Kotlin Flows Disclaimer: available in preview only, not stable yet
121. Flow example fun bar(): Flow<Int> = flow { for (i in 1..10) { emit(i) delay(100) } } ~ Asynchronous sequence
122. Flow example fun bar(): Flow<Int> = flow { for (i in 1..10) { emit(i) delay(100) } } fun main() = runBlocking<Unit> { bar().collect { x -> println(x) } }
123. Try this! fun bar(): Flow<Int> = flow { for (i in 1..10) { emit(i) delay(100) } } fun main() = runBlocking<Unit> { bar() } Flow is cold: describes the data, does not run it until collected !
124. Flow example fun bar(): Flow<Int> = flow { for (i in 1..10) { emit(i) delay(100) } } fun main() = runBlocking<Unit> { bar() .map { it * it } .toList() } Write regular code! Similar to collections / sequences
125. Thank you Want to learn more? Questions? elizarov @ Roman Elizarov relizarov