CodeFest 2019. Всеволод Толстопятов (JetBrains) — Корутины изнутри

Поддержка асинхронного программирования сейчас представлена примерно везде, и Kotlin — не исключение. В Kotlin 1.3 стабилизировались корутины, удобный и прозрачный механизм для асинхронного программирования.

В этом докладе мы посмотрим на них изнутри: как в принципе представить метод в виде приостанавливаемого вычисления и какие грабли разложены на этом пути. Потом пройдемся по оптимизациям, которые пришлось поддержать в компиляторе, узнаем, где и зачем можно экономить на объектах, и как обхитрить Hotspot. Кроме того, увидим, как отвязать потокобезопасную реализацию корутин от спецификации языка, причём здесь задача о консенсусе и что нужно сделать, чтобы написать отладчик для корутин без перекомпилирования кода.

Transcript

1. Coroutines under the hood — Толстопятов Всеволод @qwwdfsad 1
2. Agenda This talk is about coroutines implementation and its implications ,2
3. Part I. The importance of being asynchronous • Remote calls • Microservices • Interactions ,3
4. Part I. The importance of being asynchronous • Remote calls • Microservices • Interactions ,4
5. Threading • Thread per connection • Footprint (~1M stack, 1M VM metadata) • Scalability ,5
6. Callbacks http://thecodebarbarian.com/2015/03/20/callback-hell-is-a-myth ,6
7. Reactive programming https://medium.com/netflix-techblog/reactive-programming-in-the-netflix-api-with- rxjava-7811c3a1496a ,7
8. Reactive programming ,8
9. Lesser of two evils Project Loom: Fibers and Continuations for Java by Alan Bateman ,9
10. Part II. Continuations A continuation is a representation of the computation ,10
11. Continuations in Kotlin suspend fun asyncSum(initial: Int) { } ,11
12. Continuations in Kotlin suspend fun asyncSum(initial: Int) { println("Started execution") val value = asyncValue() println("Sum: ${initial + value}") } ,12
13. Continuations in Kotlin suspend fun asyncSum(initial: Int) { println("Started execution") val value = asyncValue() println("Sum: ${initial + value}") } fun main() { ::asyncSum.startCoroutine(20) println("Back to main") computation.resumeWith(22) } ,13
14. Continuations in Kotlin suspend fun asyncSum(initial: Int) { println("Started execution") val value = asyncValue() println("Sum: ${initial + value}") } fun main() { ::asyncSum.startCoroutine(20) println("Back to main") computation.resumeWith(22) } Started execution ,14
15. Continuations in Kotlin suspend fun asyncSum(initial: Int) { println("Started execution") val value = asyncValue() println("Sum: ${initial + value}") } fun main() { ::asyncSum.startCoroutine(20) println("Back to main") computation.resumeWith(22) } Started execution Back to main ,15
16. Continuations in Kotlin suspend fun asyncSum(initial: Int) { println("Started execution") val value = asyncValue() println("Sum: ${initial + value}") } fun main() { ::asyncSum.startCoroutine(20) println("Back to main") computation.resumeWith(22) } Started execution Back to main Sum: 42 ,16
17. How could it work? ,17
18. Suspension points suspend fun asyncSum(initial: Int) { // State 1 println("Started execution") val value = computeAsync() // State 2 val sum = initial + value println("Sum: $sum") } ,18
19. CPS transformation interface Continuation<in T> { val context: CoroutineContext fun resumeWith(result: Result<T>) } fun asyncSum(..., continuation: Continuation<*>) { println("Started execution") val value = asyncValue() println("Sum: ${initial + value}") } ,19
20. CPS transformation fun asyncSum(initial: Int, callback: () -> Unit) { println("Started execution") asyncValue { value -> println("Sum: ${initial + value}") callback() } } ,20
21. CPS. State machine fun asyncSum(initial: Int, c: Continuation<*>) { } ,21
22. CPS. State machine fun asyncSum(initial: Int, c: Continuation<*>) { val me = c as? AsyncSumCont ?: AsyncSumCont(c) } ,22
23. fun asyncSum(initial: Int, c: Continuation<*>) { val me = c as? AsyncSumCont ?: AsyncSumCont(c) when (me.label) { 0 -> { println("Started execution”) asyncValue(me) } } } CPS. State machine ,23
24. fun asyncSum(initial: Int, c: Continuation<*>) { val me = c as? AsyncSumCont ?: AsyncSumCont(c) when (me.label) { 0 -> { println("Started execution”) asyncValue(me) } 1 -> { val value = me.valueOrThrow println("Sum: ${initial + value}”) me.caller.resumeWith(Unit) } } } CPS. State machine ,24
25. fun asyncSum(initial: Int, c: Continuation<*>) { when(me.label) { 0 -> { println("Started execution”) asyncValue(me) } } } CPS. State manipulation ,25
26. fun asyncSum(initial: Int, c: Continuation<*>) { when(me.label) { 0 -> { println("Started execution") me.label = 1 me.initial = initial asyncValue(me) } } } State manipulation. Spilling ,26
27. class AsyncSumCont() : Continuation<Int> { val caller: Continuation<Unit> val label: Int val initial: Int val value: Int override fun resumeWith(value: Int) { value = value asyncSum(initial, this) } } State manipulation. Resume ,27
28. Example revisited suspend fun computeAsync(): Int { ... } suspend fun asyncSum(initial: Int) { println("Started execution") val value = computeAsync() println("Sum: ${initial + value}") } ,28
29. Example revisited suspend fun computeAsync(): Int { val value = suspendCoroutine<Int> { cont -> computation = cont } return value } suspend fun asyncSum(initial: Int) { println("Started execution") val value = computeAsync() println("Sum: ${initial + value}") } ,29
30. ,30
31. suspend fun <T> CompletableFuture<T>.await(): T CompletableFuture integration ,31
32. suspend fun <T> CompletableFuture<T>.await(): T = suspendCoroutine { cont -> } CompletableFuture integration ,32
33. suspend fun <T> CompletableFuture<T>.await(): T = suspendCoroutine { cont -> whenComplete { value, exception -> } } CompletableFuture integration ,33
34. suspend fun <T> CompletableFuture<T>.await(): T = suspendCoroutine { cont -> whenComplete { value, exception -> if (exception != null) cont.resumeWith(exception) else cont.resumeWith(value) } } CompletableFuture integration ,34
35. CompletableFuture integration suspend fun computeAsync(): CompletableFuture<Int> suspend fun asyncSum(initial: Int) { val future = computeAsync() println("Sum: ${initial + future.await()}") } ,35
36. Callbacks strike back suspend fun asyncSum(count: Int) { var sum = 0 for (i in 0 until count) { sum += computeAsync().await() } println(sum) } ,36
37. Exception in thread "main" java.lang.StackOverflowError at StackOverflowKt.computeAsync(StackOverflow.kt:6) at StackOverflowKt.asyncSum(StackOverflow.kt:11) at StackOverflowKt$asyncSum$2.invoke(StackOverflow.kt:16) at StackOverflowKt$asyncSum$2.invoke(StackOverflow.kt) at StackOverflowKt.await(StackOverflow.kt:4) at StackOverflowKt.asyncSum(StackOverflow.kt:11) at StackOverflowKt$asyncSum$2.invoke(StackOverflow.kt:16) at StackOverflowKt$asyncSum$2.invoke(StackOverflow.kt) at StackOverflowKt.await(StackOverflow.kt:4) at StackOverflowKt.asyncSum(StackOverflow.kt:11) at StackOverflowKt$asyncSum$2.invoke(StackOverflow.kt:16) at StackOverflowKt$asyncSum$2.invoke(StackOverflow.kt) Callbacks strike back ,37
38. Callbacks strike back suspend fun asyncSum(count: Int) { var sum = 0 for (i in 0 until count) { sum += computeAsync().await() } println(sum) } Callstack await whenComplete asyncSum$resumeWith await whenComplete asyncSum$resumeWith ... ,38
39. Calling convention fun asyncSum(cont: Continuation<*>): Unit fun asyncSum(cont: Continuation<*>): Any? fun asyncSum(cont: Continuation<*>): Unit | COROUTINE_SUSPENDED ,39
40. fun asyncSum(initial: Int, c: Continuation<*>) { when (me.label) { 0 -> { println("Started execution") val result = computeAsync(me) if (result == COROUTINE_SUSPENDED) { return COROUTINE_SUSPENDED } me.value = result goto 1 } } } Stack unrolling ,40
41. fun asyncSum(initial: Int, c: Continuation<*>) { when (me.label) { 1 -> { val value = me.valueOrThrow println("Sum: ${initial + value}”) return Unit } } } Stack unrolling ,41
42. Hotspot strikes back ,42
43. Escape analysis fun Vector.length(): Double = sqrt(x * x + y * y) fun length(x: Int, y: Int): Double { val vector = new Vector(x, y) return vector.length() } ,43
44. Escape analysis fun Vector.length(): Double = sqrt(x * x + y * y) fun length(x: Int, y: Int): Double { val vector = new Vector(x, y) return sqrt(v.x * v.x + v.y * v.y) } ,44
45. Escape analysis. Scalarization fun Vector.length(): Double = sqrt(x * x + y * y) fun length(x: Int, y: Int): Double { val vector = new Vector(x, y) return sqrt(x * x + y * y) } ,45
46. Escape analysis fun length(x: Int, y: Int): Double { val v: Vector if (x % 2 == 0) { v = new Vector(x, y) } else { v = new Vector(x / 2, y) } return v.length() } https://www.youtube.com/watch?v=K6c3W6vhQOA ,46
47. Escape analysis fun asyncSum() { val result: Any = asyncValue() if (result == COROUTINE_SUSPENDED) { return COROUTINE_SUSPENDED } else { val value = (result as Int) . . . } } ,47
48. fun asyncSum() { val result: Any = Integer.valueOf(…) if (result == COROUTINE_SUSPENDED) { return COROUTINE_SUSPENDED } else { val value = (result as Int) . . . } } Escape analysis ,48
49. Escape analysis fun asyncSum() { val result: Any = new Int() if (result == COROUTINE_SUSPENDED) { return COROUTINE_SUSPENDED } else { val value = (result as Int) . . . } } ,49
50. •Abstraction •CPS transformation •Cost Synopsis ,50
51. Part III. Effective abstractions ,51
52. Embracing concurrency suspend fun <T> CompletableFuture<T>.await(): T = suspendCoroutine { cont -> whenComplete { value, exception -> if (exception != null) cont.resumeWith(exception) else cont.resumeWith(value) } } kotlin.KotlinNullPointerException: null at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted(ContinuationImpl.kt:117) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:38) at kotlin.coroutines.ContinuationKt.startCoroutine(Continuation.kt:113) at ConcurrencyKt.builder(Concurrency.kt:9) at ConcurrencyLt.main(Concurrency.kt:19) at ConcurrencyKt.main(Concurrency.kt) ,52
53. Thread 1 Thread 2 await whenComplete COROUTINE_SUSPENDED // Saves state Future.complete UniWhenComplete await$resume // Uses state Embracing concurrency ,53
54. Consensus problem ,54
55. Thread safety 2. Add jetbrains.misc.Unsafe and use it in standard library 3. Abstraction 1. Make every continuation thread-safe ,55
56. suspend fun <T> suspendCoroutine(lambda): T = suspendCoroutineOrReturn { continuation -> val safe = SafeContinuation(continuation) lambda(safe) safe.getOrThrow() } Abstracting the problem away ,56
57. suspend fun <T> suspendCoroutine(lambda): T = suspendCoroutineOrReturn { continuation -> val safe = SafeContinuation(continuation) lambda(safe) safe.getOrThrow() } public fun SafeContinuation.resumeWith(result: Result<T>) { while (true) { // lock-free loop val cur = this.result // atomic read when { cur === UNDECIDED -> if (RESULT.compareAndSet(this, UNDECIDED, result.value)) return cur === COROUTINE_SUSPENDED -> if (RESULT.compareAndSet(this, COROUTINE_SUSPENDED, RESUMED)) { delegate.resumeWith(result) return } else -> throw IllegalStateException("Already resumed") } } Abstracting the problem away ,57
58. suspend fun <T> suspendCoroutine(lambda): T = suspendCoroutineOrReturn { continuation -> val safe = SafeContinuation(continuation) block(safe) safe.getOrThrow() } Abstracting the problem away ,58
59. Execution context suspend fun readFromDisk(): String { val future = CompletableFuture.supplyAsync({io()}, ioExecutor) return future.await() } ,59
60. suspend fun <T> CompletableFuture<T>.await( executor: Executor): T = suspendCoroutine { cont -> whenComplete { value, exception -> executor.execute(Runnable { cont.resumeWith(exception) }) } } Execution context ,60
61. suspend fun <T> CompletableFuture<T>.await( executor: Executor): T = suspendCoroutine { cont -> whenComplete { value, exception -> executor.execute(Runnable { cont.resumeWith(exception) }) } } Execution context ,61
62. suspend fun <T> CompletableFuture<T>.await( executor: Executor): T = suspendCoroutine { cont -> whenCompleteAsync(..., executor) } Execution context ,62
63. interface Continuation<in T> { val context: CoroutineContext fun resumeWith(result: Result<T>) } ::computeAsync.startCoroutine(executor.asContext()) Execution context class ExecutorCoroutineContext : CoroutineContext ,63
64. •Composability •Extensibility •Performance Problems ,64
65. Libraries-driven design interface ContinuationInterceptor : ContextElement { fun <T> interceptContinuation( continuation: Continuation<T>): Continuation<T> } ,65
66. object ForkJoinPoolInterceptor : ContinuationInterceptor { /// ... context and key implementation ... override fun <T> interceptContinuation(cont: Continuation<T>): Continuation<T> { return object : Continuation<T>, Runnable { private var savedState: Result<T>? = null override fun resumeWith(result: Result<T>) { savedState = result ForkJoinPool.commonPool().execute(this) } override fun run() { cont.resumeWith(savedState!!) } } } } Libraries-driven design ,66
67. • Always intercept • Unification • Allocation issue • Cache it in the state machine! One more calling convention ,67
68. Synopsis • Power of abstraction • Complexity can be abstracted • Standard library base ,68
69. Further reading • kotlinx.coroutines • KEEP • https://medium.com/@elizarov • Loom proposal ,69