Spring-Flavored Kotlin Coroutines

Kotlin has built-in support for asynchronous programming with Kotlin Coroutines that are designed to allow for simple and easy-to-understand code. We'll witness that reduction in complexity that coroutines bring and dissect what kind of magic is working behind the scenes to make it possible, how it all integrates with Spring and how can you integrate coroutines with any kind of asynchronous code you write.

Transcript

1. Spring-Flavored Kotlin Coroutines Roman Elizarov October 7–10, 2019 Austin Convention Center
2. Speaker: Roman Elizarov Professional developer since 2000 Previously developed high-perf trading software @ Devexperts Teach concurrent & distributed programming @ St. Petersburg ITMO University Chief judge @ Northern Eurasia Contest / ICPC Now team lead in Kotlin Libraries @ JetBrains elizarov @ relizarov
3. Kotlin – Programming Language for
4. Kotlin – Programming Language for
5. Backend evolution Starting with “good old days”
6. ET 1 ET 2 ET N … DB Old-school client-server monolith Clients Executor Threads
7. ET 1 ET 2 ET N … DB Incoming request Clients Executor Threads
8. ET 1 ET 2 ET N … DB Blocks tread Clients Executor Threads
9. ET 1 ET 2 ET N … DB Sizing threads – easy Clients N = number of DB connections Executor Threads
10. Services
11. ET 1 ET 2 ET N … DB Old-school client-server monolith Clients Executor Threads
12. ET 1 ET 2 ET N … DB Now with Services Service Clients Executor Threads
13. ET 1 ET 2 ET N … Services everywhere … Service K Service 1 Service 2 Clients Executor Threads
14. Complex business logic fun placeOrder(order: Order): Response { … }
15. Complex business logic fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) … }
16. Complex business logic fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) … }
17. Complex business logic fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } … }
18. Complex business logic fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } … }
19. Complex business logic fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
20. Complex business logic fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
21. What if a service is slow? fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
22. Slow service @Service class MarginService { private val rest = RestTemplate() fun loadMargin(account: Account): Margin = rest.getForObject( "http://localhost:9090/margin/${account.id}", Margin::class.java)!! }
23. Slow service @Service class MarginService { private val rest = RestTemplate() fun loadMargin(account: Account): Margin = rest.getForObject( "http://localhost:9090/margin/${account.id}", Margin::class.java)!! }
24. Demo Slow service, blocking
25. ET 2 ET N … Clients Blocks threads … Service K Service 1 Service 2 ET 1 Executor Threads
26. ET 2 ET N … Clients Blocks threads … Service K Service 1 Service 2 ET 1 Executor Threads
27. ET 2 ET N … Clients Blocks threads … Service K Service 1 Service 2 ET 1 Executor Threads
28. Code that waits
29. Asynchronous programming Writing code that waits
30. ET 2 ET N … Clients Instead of blocking… … Service K Service 1 Service 2 ET 1 Executor Threads
31. ET 2 ET N … Release the thread … Service K Service 1 Service 2 ET 1 Clients Executor Threads
32. Clients ET 2 ET N … Resume operation later … Service K Service 1 Service 2 ET 1 Executor Threads
33. But how? fun loadMargin(account: Account): Margin
34. But how? Callbacks fun loadMargin(account: Account, callback: (Margin) -> Unit)
35. But how? Callbacks Futures/Promises fun loadMargin(account: Account): CompletableFuture<Margin>
36. But how? Callbacks Futures/Promises/Reactive fun loadMargin(account: Account): Mono<Margin>
37. Reactive
38. The logic fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
39. Going reactive fun placeOrder(order: Order): Mono<Response>
40. Going reactive fun placeOrder(order: Order): Mono<Response> =
41. Going reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId)
42. Going reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap { account -> … }
43. Going reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap { account -> if (account.isOptionsAccount) { marginService.loadMargin(account) } else { Mono.just(defaultMargin) } }
44. Going reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap { account -> if (account.isOptionsAccount) { marginService.loadMargin(account) } else { Mono.just(defaultMargin) } }
45. Going reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap { account -> if (account.isOptionsAccount) { marginService.loadMargin(account) } else { Mono.just(defaultMargin) } } .map { margin -> … }
46. Going reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap { account -> if (account.isOptionsAccount) { marginService.loadMargin(account) } else { Mono.just(defaultMargin) } } .map { margin -> validateOrder(order, margin) }
47. Reactive Service @Service class MarginService { private val client = WebClient.create("http://localhost:9090/") fun loadMargin(account: Account): Mono<Margin> = client .get() .uri("margin/${account.id}") .retrieve() .bodyToMono(Margin::class.java) }
48. Reactive Service @Service class MarginService { private val client = WebClient.create("http://localhost:9090/") fun loadMargin(account: Account): Mono<Margin> = client .get() .uri("margin/${account.id}") .retrieve() .bodyToMono(Margin::class.java) }
49. Demo: Reactive Was it worth it?
50. Choice 1: Direct code, blocking fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
51. Choice 2: Complex code, non-blocking fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap { account -> if (account.isOptionsAccount) { marginService.loadMargin(account) } else { Mono.just(defaultMargin) } } .map { margin -> validateOrder(order, margin) }
52. Why not both?
53. But how? Callbacks Futures/Promises/Reactive fun loadMargin(account: Account): Mono<Margin>
54. But how? Callbacks Futures/Promises/Reactive Kotlin Coroutines suspend fun loadMargin(account: Account): Margin
55. Direct code: Blocking fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) }
56. Direct code: Coroutines suspend fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) } Write regular code! Call suspending funs
57. Coroutines Service @Service class MarginService { private val client = WebClient.create("http://localhost:9090/") suspend fun loadMargin(account: Account): Margin = client .get() .uri("margin/${account.id}") .retrieve() .awaitBody() }
58. Coroutines Service @Service class MarginService { private val client = WebClient.create("http://localhost:9090/") suspend fun loadMargin(account: Account): Margin = client .get() .uri("margin/${account.id}") .retrieve() .awaitBody() }
59. Demo: Coroutines
61. Suspend behind the scenes suspend fun loadMargin(account: Account): Margin
62. Suspend behind the scenes suspend fun loadMargin(account: Account): Margin fun loadMargin(account: Account, cont: Continuation<Margin>) But why callback and not future/mono?
63. Performance! Callback is a simple, low-level primitive Integration with async IO libraries is easy
64. Service integration suspend fun loadMargin(account: Account): Margin
65. Service integration suspend fun loadMargin(account: Account): Margin = suspendCoroutine { cont -> // install callback & use cont to resume }
66. Server integrated with coroutines @PostMapping("/order") suspend fun placeOrder(order: Order): Response { … }
67. Server not integrated with coroutines @PostMapping("/order") fun placeOrder(order: Order): Mono<Response> = mono { … } Coroutine builder
68. Blocking integration @Component class Initializer { @EventListener(ApplicationStartedEvent::class) fun initData() { … } }
69. Blocking integration @Component class Initializer { @EventListener(ApplicationStartedEvent::class) fun initData() = runBlocking<Unit> { … } } Coroutine builder
70. Suspend is efficient
71. Suspend is efficient suspend fun placeOrder(order: Order): Response { val account = accountService.loadAccout(order.accountId) val margin = if (account.isOptionsAccount) { marginService.loadMargin(account) } else { defaultMargin } return validateOrder(order, margin) } One object allocated
72. Reactive fun placeOrder(order: Order): Mono<Response> = accountService.loadAccount(order.accountId) .flatMap { account -> if (account.isOptionsAccount) { marginService.loadMargin(account) } else { Mono.just(defaultMargin) } } .map { margin -> validateOrder(order, margin) } Lambda allocated* Mono allocated Lambda allocated Mono allocated
73. Let’s go deeper fun placeOrder(params: Params): Mono<Response> { // check pre-conditions return actuallyPlaceOrder(order) } fun actuallyPlaceOrder(order: Order): Mono<Response>
74. Let’s go deeper (with coroutines) suspend fun placeOrder(params: Params): Response { // check pre-conditions return actuallyPlaceOrder(order) } suspend fun actuallyPlaceOrder(params: Params): Response Tail call optimization Tail call
75. Call stack with coroutines Coroutine Builder placeOrder actuallyPlaceOrder moreLogic marginService.loadMargin suspendCoroutine
76. Call stack with coroutines Coroutine Builder placeOrder actuallyPlaceOrder moreLogic marginService.loadMargin suspendCoroutine unwind Continuation in heap
77. Learn more KotlinConf San Francisco 2017 GOTO Copenhagen 2018
78. Scaling with coroutines With thread pools
79. ET 2 ET N … Clients Thread pools ET 1 Executor Threads
80. ET 2 ET N … Clients Thread pools ET 1 ST 2 ST M1 … S1 1 N = number of CPU cores M1 = depends Service 1 Threads Executor Threads
81. IO-bound (blocking) fun loadAccount(order: Order): Account { // some blocking code here.... }
82. IO-bound suspend fun loadAccount(order: Order): Account { // some blocking code here.... }
83. IO-bound withContext suspend fun loadAccount(order: Order): Account = withContext(dispatcher) { // some blocking code here.... }
84. IO-bound withContext suspend fun loadAccount(order: Order): Account = withContext(dispatcher) { // some blocking code here.... } val dispatcher = Executors.newFixedThreadPool(M2).asCoroutineDispatcher()
85. CPU-bound code fun validateOrder(order: Order, margin: Margin): Response { // perform CPU-consuming computation }
86. CPU-bound code suspend fun validateOrder(order: Order, margin: Margin): Response = withContext(compute) { // perform CPU-consuming computation } val compute = Executors.newFixedThreadPool(M3).asCoroutineDispatcher()
87. ET 2 ET N … Clients Fine-grained control and encapsulation ET 1 S1 1 ST M1 Service 2 Threads S1 1 ST M2 Service 3 Threads S1 1 ST M3 Async IO-bound CPU- bound Never blocked Service 1 Threads Executor Threads
88. But there’s more!
89. Cancellation
90. withTimeout suspend fun placeOrder(order: Order): Response = withTimeout(1000) { // code before loadMargin(account) // code after }
91. withTimeout propagation suspend fun placeOrder(order: Order): Response = withTimeout(1000) { // code before loadMargin(account) // code after } suspend fun loadMargin(account: Account): Margin = suspendCoroutine { cont -> // install callback & use cont to resume }
92. withTimeout propagation suspend fun placeOrder(order: Order): Response = withTimeout(1000) { // code before loadMargin(account) // code after } suspend fun loadMargin(account: Account): Margin = suspendCancellableCoroutine { cont -> // install callback & use cont to resume }
93. withTimeout propagation suspend fun placeOrder(order: Order): Response = withTimeout(1000) { // code before loadMargin(account) // code after } suspend fun loadMargin(account: Account): Margin = suspendCancellableCoroutine { cont -> // install callback & use cont to resume cont.invokeOnCancellation { … } }
94. Concurrency Multiple things at the same time
95. Example fun placeOrder(order: Order): Response { val account = accountService.loadAccount(order) val margin = marginService.loadMargin(order) return validateOrder(order, account, margin) }
96. Example fun placeOrder(order: Order): Response { val account = accountService.loadAccount(order) val margin = marginService.loadMargin(order) return validateOrder(order, account, margin) } No data dependencies
97. Concurrency with async (futures) fun placeOrder(order: Order): Response { val account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) }
98. Concurrency with async (futures) fun placeOrder(order: Order): Response { val account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) }
99. Concurrency with async (futures) fun placeOrder(order: Order): Response { val account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) } Fails?
100. Concurrency with async (futures) fun placeOrder(order: Order): Response { val account = accountService.loadAccountAsync(order) val margin = marginService.loadMarginAsync(order) return validateOrder(order, account.await(), margin.await()) } Fails? Leaks !
101. Structured concurrency
102. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) }
103. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) }
104. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) }
105. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Fails?
106. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Fails? Cancels
107. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Fails? Cancels Cancels
108. Concurrency with coroutines suspend fun placeOrder(order: Order): Response = coroutineScope { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } validateOrder(order, account.await(), margin.await()) } Waits for completion of all children
109. Enforcing structure
110. Without coroutine scope? suspend fun placeOrder(order: Order): Response { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } return validateOrder(order, account.await(), margin.await()) }
111. Without coroutine scope? suspend fun placeOrder(order: Order): Response { val account = async { accountService.loadAccount(order) } val margin = async { marginService.loadMargin(order) } return validateOrder(order, account.await(), margin.await()) } ERROR: Unresolved reference.
112. Extensions of CoroutineScope fun <T> CoroutineScope.async( context: CoroutineContext = EmptyCoroutineContext, start: CoroutineStart = CoroutineStart.DEFAULT, block: suspend CoroutineScope.() -> T ): Deferred<T>
113. Convention fun CoroutineScope.bg(params: Params) = launch { // … } Launches new coroutine
114. Types as documentation fun foo(params: Params): Response suspend fun foo(params: Params): Response fun CoroutineScope.foo(params: Params): Response Fast, local Remote, or slow Side effect - bg
115. Types are enforced fun foo(params: Params): Response suspend fun foo(params: Params): Response fun CoroutineScope.foo(params: Params): Response Not allowed But must provide scope explicitly Using coroutineScope { … } Fast, local Remote, or slow Side effect - bg
116. Green threads / fibers Alternative way to async
117. Green threads / Fibers ET 2 ET N … ET 1 F 2 F M … Fibers F 1 ~ Coroutines Hidden from developer Executor Threads
118. But marking with suspend pays off at scale Fibers – everything suspending
119. Thread switching And how to avoid it
120. ET 2 ET N … Clients Threads ET 1 S1 1 ST M1 S1 1 ST M2 S1 1 ST M3 Service 2 Threads Service 3 Threads Service 1 Threads Executor Threads
121. ET 2 ET N … Clients Solution – shared thread pool ET 1 Executor Threads
122. ET 2 ET N … Clients Solution – shared thread pool ET 1 Executor Threads
123. ET 2 ET N … Clients Solution – shared thread pool ET 1 ET N+1 Executor Threads
124. ET 2 ET N … Clients Solution – shared thread pool ET 1 ET N+1 ET N+2 Executor Threads
125. ET 2 ET N … Clients Solution – shared thread pool ET 1 ET N+1 … ET M ET N+2 ET N+M Executor Threads
126. withContext for IO suspend fun loadAccount(order: Order): Account = withContext(dispatcher) { // some blocking code here.... } val dispatcher = Executors.newFixedThreadPool(M2).asCoroutineDispatcher()
127. withContext for Dispatсhers.IO suspend fun loadAccount(order: Order): Account = withContext(Dispatchers.IO) { // some blocking code here.... } No thread switch from Dispatchers.Default pool
128. ET 2 ET N … Clients Solution – shared thread pool ET 1 Dispatchers.Default Executor Threads
129. ET 2 ET N … Clients Solution – shared thread pool ET 1 ET N+1 … ET M ET N+2 ET N+M Dispatchers.Default Dispatchers.IO Executor Threads
130. Coroutines and data streams
131. Returning many responses suspend fun foo(params: Params): Response One response suspend fun foo(params: Params): List<Response> Many responses
132. Returning many responses suspend fun foo(params: Params): Response One response suspend fun foo(params: Params): List<Response> Many responses fun foo(params: Params): Flow<Response> Many responses async
133. Kotlin Flows Reactive streams for coroutines
134. fun foo(): Flow<Int> ~ Reactive Publisher / Flux
135. fun foo(): Flow<Int> = flow { … }
136. fun foo(): Flow<Int> = flow { for (i in 1..10) { emit(i) delay(100) } }
137. fun foo(): Flow<Int> = flow { for (i in 1..10) { emit(i) delay(100) } } suspend fun main() { foo().collect { x -> println(x) } }
138. fun foo(): Flow<Int> = flow { for (i in 1..10) { emit(i) delay(100) } } suspend fun main() { foo() } Flow is cold: describes the data, does not run it until collected
139. Flow operators fun foo(): Flow<Int> = flow { for (i in 1..10) { emit(i) delay(100) } } suspend fun main() { foo() .map { it * it } .toList() }
140. Why flow?
141. A A’ mapper fun map(mapper: (T) -> R): Flux<R> fun flatMap(mapper: (T) -> Publisher<R>): Flux<R> Synchronous Asynchronous Flux<T>
142. A A’ mapper fun map(mapper: (T) -> R): Flux<R> fun flatMap(mapper: (T) -> Publisher<R>): Flux<R> fun filter(predicate: (T) -> Boolean): Flux<T> A A predicate Synchronous Asynchronous Synchronous Asynchronous Flux<T> fun filterWhen(predicate: (T) -> Publisher<Boolean>): Flux<T>
143. A A’ Flow<T> fun map(transform: suspend (T) -> R): Flow<R> transform
144. A A’ transform Flow<T> fun map(transform: suspend (T) -> R): Flow<R>
145. A A’ transform fun map(transform: suspend (T) -> R): Flow<R> Flow<T> fun filter(predicate: suspend (T) -> Boolean): Flow<T> A predicate A
146. Operator avoidance startWith(value) onStart { emit(value) } delaySubscription(time) onStart { delay(time) } startWith(flow) onStart { emitAll(flow) } delayElements(time) onEach { delay(time) } onErrorReturn(value) catch { emit(value) } onErrorResume(flow) catch { emitAll(flow) } generate(…) flow { … } Composable
147. Reactive + = ❤
148. Spring ❤ Flow @GetMapping("/sse/{n}", produces = [MediaType.TEXT_EVENT_STREAM_VALUE]) fun greetings(@PathVariable n: String): Flow<Greeting>
149. Spring ❤ Flow @GetMapping("/sse/{n}", produces = [MediaType.TEXT_EVENT_STREAM_VALUE]) fun greetings(@PathVariable n: String): Flow<Greeting> = flow { while(true) { emit(Greeting("Hello from coroutines $n @ ${Instant.now()}")) delay(1000) } }
150. Demo: Spring Flow Support
151. Thank you Questions? elizarov @ relizarov
152. Kotlin Coroutines For The Win! #springone @s1p