Kotlin Coroutines for asynchronous programming on the JVM

Introductory session on using Kotlin coroutines for asynchronous programming on the JVM

Transcript

1. Kotlin Coroutines for asynchronous programming on the JVM Pedro Felix Sky Technology Center – Portugal June 2018
2. Agenda • Asynchronous programming: motivation and models • Kotlin’s coroutine model • Building blocks • High-level functions • Coroutine internals • Coroutines and threads • Comparison with the async-await model
3. Kotlin • Statically typed programming language • Created by JetBrains circa 2011 - Apache 2 licence • Initially targeted the JVM with emphasis on Java interoperability • Currently also targets javascript and native • Excellent integration with the JVM tooling ecosystem • IntelliJ IDE • Gradle and Maven build systems • E.g. Kotlin and Java source code side-by-side in the same module • Official language for Android development, since 2017 • Supported by Spring Boot 2.0, since 2017
4. Programming in connected systems ... ... is “mostly” waiting for external things to happen However blocking waits do consume threads
5. My precious... ...UI Thread ...HTTP Server Threads
6. Why asynchronous, i.e., non-blocking? • Avoid blocking the thread • Responsivess in UI frameworks with thread affinity (or a single thread) • Avoid blocking any thread • Scalability in “intermediate” servers • E.g. API gateways/frontends • A thread uses significant resources (e.g. 1Mb on the JVM) • Made possible by asynchronous/non-blocking IO • and asynchronous synchronization • A wait should not require a thread to be blocked • Mandatory in some environments • E.g. Android’s activities, javascript • Optional on others • E.g. Spring
7. 500 RPS, 2s latency   1K simultaneously blocked threads   1GB just for the thread stack memory (assuming 1MB per thread stack)
8. Why asynchronous, i.e., non-blocking? • Not the same as parallel programming • We may have an asynchronous flow without any parallelism • Will not make things necessarely faster • Asynchronous I/O will be slower than synchronous I/O • Better • Resource (thread) usage • Responsiveness • Better scalability
9. Asynchronous programming models • Callbacks • Asynchronous methods receive one or more callbacks • E.g. Java’s NIO, NodeJS • May lead to the so called “callback hell” • Futures and Reactive Streams • Objects representing asynchronous operations • With composition methods (e.g. then, map/flatMap, ...) • E.g. CompletableFuture and Flow.Publisher • Fit nicely into a functional programming style • Functions created by the composition of other functions
10. However... • Futures do not fit so nicely in an imperative programming style • Functions created by sequences of statements • With complex control logic (ifs, whiles, try-catchs)
11. Futures and Observables/Reactive Streams ...flatMap everywhere flatMap...
12. async-await • async-await adapts futures into the imperative programming style • Functions created by sequences of statements • On async functions, an await converts a Future<V> into a V, without blocking • Increasing popularity • Available in C#, Javascript, Python, Dart, ... • Instead of async-await, Kotlin uses a different (yet related) concept Coroutines
13. Language vs. Library suspendCoroutine function suspend modifier startCoroutine function await async future launch CommonPool Unconfined UI Language & intrinsics Libraries async modifier await operator Kotlin C#/JS
14. Library oriented model • In C#/JS futures are well-defined • C# - Task<T> type • JS – Promises+ specification • In the JVM world the scenario is quite the opposite • Guava – ListenableFuture • Spring – DeferredResult • JDK 8 – CompletableFuture • Apache HTTP Client – callbacks • A library based approach work best in this heterogenous scenario • E.g. Multiple await extension functions, one for each future type.
15. So, let’s start... fun simpleFunctionWithABlockingWait(a: Int, b: Int): Int { log.info("step 1") Thread.sleep(1000) log.info("step 2") return a + b }
16. val executor = Executors.newScheduledThreadPool(1) suspend fun suspendFunctionNonBlockingWait(a: Int, b: Int): Int { log.info("step 1") suspendCoroutine<Unit> { continuation -> executor.schedule({ continuation.resume(Unit) }, 1000, TimeUnit.MILLISECONDS) } log.info("step 2") return a + b }
17. val executor = Executors.newScheduledThreadPool(1) suspend fun suspendFunctionNonBlockingWait(a: Int, b: Int): Int { log.info("step 1") suspendCoroutine<Unit> { continuation -> executor.schedule({ continuation.resume(Unit) }, 1000, TimeUnit.MILLISECONDS) } log.info("step 2") return a + b }
18. val executor = Executors.newScheduledThreadPool(1) suspend fun suspendFunctionNonBlockingWait(a: Int, b: Int): Int { log.info("step 1") suspendCoroutine<Unit> { continuation -> executor.schedule({ continuation.resume(Unit) }, 1000, TimeUnit.MILLISECONDS) } log.info("step 2") return a + b }
19. val executor = Executors.newScheduledThreadPool(1) suspend fun suspendFunctionNonBlockingWait(a: Int, b: Int): Int { log.info("step 1") suspendCoroutine<Unit> { continuation -> executor.schedule({ continuation.resume(Unit) }, 1000, TimeUnit.MILLISECONDS) } log.info("step 2") return a + b }
20. val executor = Executors.newScheduledThreadPool(1) suspend fun suspendFunctionNonBlockingWait(a: Int, b: Int): Int { log.info("step 1") suspendCoroutine<Unit> { continuation -> executor.schedule({ continuation.resume(Unit) }, 1000, TimeUnit.MILLISECONDS) } log.info("step 2") return a + b }
21. val executor = Executors.newScheduledThreadPool(1) suspend fun suspendFunctionNonBlockingWait(a: Int, b: Int): Int { log.info("step 1") suspendCoroutine<Unit> { continuation -> executor.schedule({ continuation.resume(Unit) }, 1000, TimeUnit.MILLISECONDS) } log.info("step 2") return a + b }
22. val executor = Executors.newScheduledThreadPool(1) suspend fun suspendFunctionNonBlockingWait(a: Int, b: Int): Int { log.info("step 1") suspendCoroutine<Unit> { continuation -> executor.schedule({ continuation.resume(Unit) }, 1000, TimeUnit.MILLISECONDS) } log.info("step 2") return a + b } ... 14:21:58.491 [main] INFO intro - step 1 ... 14:21:59.497 [pool-1-thread-1] INFO intro - step 2 ...
23. val executor = Executors.newScheduledThreadPool(1) suspend fun suspendFunctionNonBlockingWait(a: Int, b: Int): Int { log.info("step 1") suspendCoroutine<Unit> { continuation -> executor.schedule({ continuation.resume(Unit) }, 1000, TimeUnit.MILLISECONDS) } log.info("step 2") return a + b }
24. ... suspendCoroutine {cont -> ... } everywhere suspendCoroutine { cont -> ... } ...
25. suspend fun delay(ms: Long) { suspendCoroutine<Unit> { continuation -> executor.schedule({ continuation.resume(Unit) }, ms, TimeUnit.MILLISECONDS) } } suspend fun suspendFunctionNonBlockingWait2(a: Int, b: Int): Int { log.info("step 1") delay(1000) log.info("step 2") return a + b }
26. suspend fun delay(ms: Long) { suspendCoroutine<Unit> { continuation -> executor.schedule({ continuation.resume(Unit) }, ms, TimeUnit.MILLISECONDS) } } suspend fun suspendFunctionNonBlockingWait2(a: Int, b: Int): Int { log.info("step 1") delay(1000) log.info("step 2") return a + b } same scope
27. suspend fun delay(ms: Long) { suspendCoroutine<Unit> { continuation -> executor.schedule({ continuation.resume(Unit) }, ms, TimeUnit.MILLISECONDS) } } suspend fun suspendFunctionNonBlockingWait2(a: Int, b: Int): Int { log.info("step 1") delay(1000) log.info("step 2") return a + b } same scope Run on different threads
28. Complex control logic suspend fun suspendFunctionWithLoopWithConditionalLogic(a: Int, b: Int): Int { for(i in 0..3) { log.info("step 1 of iteration $i") if(i % 2 == 0) { delay(1000) } log.info("step 2 of iteration $i") } return a + b } 14:51:25.344 [main] INFO intro - step 1 of iteration 0 14:51:26.349 [pool-1-thread-1] INFO intro - step 2 of iteration 0 14:51:26.349 [pool-1-thread-1] INFO intro - step 1 of iteration 1 14:51:26.349 [pool-1-thread-1] INFO intro - step 2 of iteration 1 14:51:26.349 [pool-1-thread-1] INFO intro - step 1 of iteration 2 14:51:27.349 [pool-1-thread-3] INFO intro - step 2 of iteration 2 14:51:27.350 [pool-1-thread-3] INFO intro - step 1 of iteration 3 14:51:27.350 [pool-1-thread-3] INFO intro - step 2 of iteration 3 14:51:27.350 [pool-1-thread-3] INFO intro - result is 42
29. Something more interesting... • Using the Apache Async HTTP client with suspend functions suspend fun HttpAsyncClient.execute(request: HttpUriRequest): HttpResponse = suspendCoroutine {continuation -> this.execute(request, object : FutureCallback<HttpResponse> { override fun cancelled() { continuation.resumeWithException(TimeoutException()) } override fun completed(result: HttpResponse?) { continuation.resume(result!!) } override fun failed(ex: Exception?) { continuation.resumeWithException(ex!!) } }) } suspend fun HttpAsyncClient.get(url: String, token: String) = this.execute(HttpGet(url).apply { addHeader("Authorization", "Bearer $token") })
30. suspend fun showTagsForReposInOrgs(orgUrl: String) { HttpAsyncClients.createDefault().use { client -> client.start() log.info("Getting repo list from $orgUrl") val resp = client.get(orgUrl, token) val repoList = mapper.readValue<List<Repo>>(resp.entity.content) repoList.forEach { log.info("Getting tags from ${it.tags_url}") val resp = client.get(it.tags_url, token) mapper.readValue<List<Tag>>(resp.entity.content).forEach { log.info("tag ${it.name} found") } } }} Something more interesting...
31. And now with CompletableFuture suspend fun <T> CompletableFuture<T>.await(): T = suspendCoroutine<T> { continuation -> this.whenComplete({value, error -> if (error != null) continuation.resumeWithException(error) else continuation.resume(value) })} await is an extension function on CompletableFuture Converts a CompletableFuture<T> into a T • without blocking • Similar to the await operator from C#/JS, however defined as a function
32. And now with AsyncHttpClient suspend fun showTagsForReposInOrgsUsingAsyncHttpClient(orgUrl: String) { DefaultAsyncHttpClient().use { client -> log.info("Getting repo list from $orgUrl") val resp = client.prepareGet(orgUrl).withToken() .execute().toCompletableFuture().await() val repoList = mapper.readValue<List<Repo>>(resp.responseBody) repoList.map { log.info("Getting tags from ${it.tags_url}") val resp = client.prepareGet(it.tags_url).withToken() .execute().toCompletableFuture().await() mapper.readValue<List<Tag>>(resp.responseBody).forEach { log.info("tag ${it.name} found") } } } }
33. Callback based method suspendCoroutine
34. Callback based method suspendCoroutine suspend
35. Callback based method suspendCoroutine suspend suspend suspend
36. Callback based method suspendCoroutine suspend suspend suspend suspend suspend suspend suspend
37. Callback based method suspendCoroutine suspend suspend is “viral” suspend suspend suspend suspend suspend suspend regular function • suspend functions can only be called from other suspend functions • E.g. main cannot be a suspend function • E.g. a suspend method cannot override a non-suspend method • E.g. a suspend method cannot be a Spring MVC controller handler
38. ...suspend everywhere suspend...
39. How to get out of suspend? • fun <T> (suspend () -> T).startCoroutine(completion: Continuation<T>) • Is an extension method • For suspend functions that return a T • Receives a Continuation<T>, i.e., a callback based interface • It is not a suspend method so can be called from any function • suspendCoroutine converts callback/futures into suspend functions • startCoroutine converts back from suspend functions into callbacks/futures Callback/future suspendCoroutine suspend suspend startCoroutine suspend regular function using callbacks
40. Example: Spring Controller handler @GetMapping("/tags") fun getTags(): DeferredResult<List<RepoAndTags>> = deferred { getTagsForReposInOrgs("https://api.github.com/users/ktorio/repos") } • Plain old Spring controller handler (DeferredResult is a Spring type) • getTagsForReposInOrgs is a suspend function • deferred converts from suspend into DeferredResult using startCoroutine fun <T> deferred(block: suspend () -> T): DeferredResult<T> { val result = DeferredResult<T>() block.startCoroutine(object : Continuation<T> { override fun resume(value: T) { result.setResult(value) } override fun resumeWithException(exception: Throwable) { result.setErrorResult(exception) } (...) } return result }
41. Library support • suspendCoroutine and startCoroutine are low-level building blocks • Rarely used directly in application code • Library provided high level functions • To convert into suspend (uses suspendCoroutine internally) • await: CompletableFuture<T> → suspend T • await: ListenableFuture<T> → suspend T • await: Deferred<T> → suspend T • To convert from suspend – (uses startCoroutine internally) • future: suspend T → CompletableFuture<T> • future: suspend T → ListenableFuture<T> • launch: suspend → Job • async: suspend T → Deferred<T>
42. Internals • Compiler transforms each suspend function into a state machine • With a different signature – continuation passing style • A suspend fun someFunction(a: Int, b: Int): Int • Is converted into Object someFunction(a: Int, b: Int, Continuation<Int> continuation) • An internal call to a suspend fuction automatically uses the next state as the continuation • The next state is made available via the suspendCoroutine lambda • That’s why a normal function cannot call a suspend function directly • The continuation needs to be passed in • Which is what startCoroutine does • A coroutine is an instance of the state machine defined by a suspend function
43. States and threads dispatch State 1 State 2 State 3 State machine defined by a suspend function
44. States and threads dispatch State 1 State 2 State 3 suspendCoroutine continuation
45. States and threads dispatch State 1 State 2 State 3 callback continuation suspendCoroutine
46. States and threads dispatch State 1 State 2 State 3 continuation callback
47. Coroutines and threading • On which threads do the coroutines run? • It depends on the active CoroutineDispatcher • Unconfined • First segment runs on thread that called startCoroutine... • ...and then on the threads where the continuation methods where called • I.e. No thread switching is done by the Kotlin runtime • CommonPool • Coroutine runs on threads from the Java ForkJoinPool • UI • Coroutine runs on Android’s UI thread • Others • Dispatcher is defined when calling the coroutine builder – • E.g. launch(UI) {...}
48. States and threads dispatch State 1 State 2 State 3 suspendCoroutine continuation D
49. States and threads dispatch State 1 State 2 State 3 callback continuation suspendCoroutine D
50. States and threads dispatch State 1 State 2 State 3 continuation callback D
51. Sequential vs Parallel var img1 = await loadImage(...) var img2 = await loadImage(...) doSomething(img1, img2) val img1 = loadImage(...) val img2 = loadImage(...) doSomething(img1, img2) Kotlin C#/JS Sequential (yet asynchronous) Based on example present in https://qconsf.com/sf2017/system/files/presentation-slides/2017_qcon_sf_-_fresh_async_with_kotlin.pdf by Roman Elizarov
52. Sequential vs Parallel var img1 = await loadImage(...) var img2 = await loadImage(...) doSomething(img1, img2) val img1 = loadImage(...) val img2 = loadImage(...) doSomething(img1, img2) var f1 = loadImage(...) var f2 = loadImage(...) doSomething(await f1, await f2) val f1 = async { loadImage(...) } val f2 = async { loadImage(...) } doSomething(f1.await(), f2.await()) Kotlin C#/JS Sequential (yet asynchronous) Parallel Based on example present in https://qconsf.com/sf2017/system/files/presentation-slides/2017_qcon_sf_-_fresh_async_with_kotlin.pdf by Roman Elizarov
53. async-await vs coroutines async-await Kotlin coroutines Changes return type from T to Task<T> Return type is still T async is a private implementation detail suspend is visible on the signature async function can be called by any function suspend function can only be called by another suspend function (or startCoroutine) Consumer can use the Task<T> directly or obtain the T via an await Consumer only sees the T (no await required) Parallel by default Sequential by default Sequential needs explicit await Parallel needs explicit async await is a language operator - C#: operand must have an GetAwaiter method - JS: operand must be Promise/thenable await are library provided functions - Can be added, as an extension function, to any future-like type Task<T> is C#. In JS it would be a Promise/thenable
54. Final remarks • Important topics not covered in this presentation • Coroutine context • Cancellation and timeouts • Resources • The two KotlinConf 2017 sessions on coroutines by Roman Elizarov • https://github.com/Kotlin/kotlinx.coroutines/blob/master/coroutines-guide.md • Coroutines in action • https://ktor.io – framework for building asynchronous servers and clients
55. Thanks! Q & A