CodeFest 2019. Всеволод Толстопятов (JetBrains) — Корутины изнутри

16b6c87229eaf58768d25ed7b2bbbf52?s=47 CodeFest
April 05, 2019

CodeFest 2019. Всеволод Толстопятов (JetBrains) — Корутины изнутри

Поддержка асинхронного программирования сейчас представлена примерно везде, и Kotlin — не исключение. В Kotlin 1.3 стабилизировались корутины, удобный и прозрачный механизм для асинхронного программирования.

В этом докладе мы посмотрим на них изнутри: как в принципе представить метод в виде приостанавливаемого вычисления и какие грабли разложены на этом пути. Потом пройдемся по оптимизациям, которые пришлось поддержать в компиляторе, узнаем, где и зачем можно экономить на объектах, и как обхитрить Hotspot. Кроме того, увидим, как отвязать потокобезопасную реализацию корутин от спецификации языка, причём здесь задача о консенсусе и что нужно сделать, чтобы написать отладчик для корутин без перекомпилирования кода.

16b6c87229eaf58768d25ed7b2bbbf52?s=128

CodeFest

April 05, 2019
Tweet

Transcript

  1. Coroutines under the hood — Толстопятов Всеволод @qwwdfsad 1

  2. Agenda This talk is about coroutines implementation and its implications

    ,2
  3. Part I. The importance of being asynchronous • Remote calls

    • Microservices • Interactions ,3
  4. Part I. The importance of being asynchronous • Remote calls

    • Microservices • Interactions ,4
  5. Threading • Thread per connection • Footprint (~1M stack, 1M

    VM metadata) • Scalability ,5
  6. Callbacks http://thecodebarbarian.com/2015/03/20/callback-hell-is-a-myth ,6

  7. Reactive programming https://medium.com/netflix-techblog/reactive-programming-in-the-netflix-api-with- rxjava-7811c3a1496a ,7

  8. Reactive programming ,8

  9. Lesser of two evils Project Loom: Fibers and Continuations for

    Java by Alan Bateman ,9
  10. Part II. Continuations A continuation is a representation of the

    computation ,10
  11. Continuations in Kotlin suspend fun asyncSum(initial: Int) { } ,11

  12. Continuations in Kotlin suspend fun asyncSum(initial: Int) { println("Started execution")

    val value = asyncValue() println("Sum: ${initial + value}") } ,12
  13. Continuations in Kotlin suspend fun asyncSum(initial: Int) { println("Started execution")

    val value = asyncValue() println("Sum: ${initial + value}") } fun main() { ::asyncSum.startCoroutine(20) println("Back to main") computation.resumeWith(22) } ,13
  14. Continuations in Kotlin suspend fun asyncSum(initial: Int) { println("Started execution")

    val value = asyncValue() println("Sum: ${initial + value}") } fun main() { ::asyncSum.startCoroutine(20) println("Back to main") computation.resumeWith(22) } Started execution ,14
  15. Continuations in Kotlin suspend fun asyncSum(initial: Int) { println("Started execution")

    val value = asyncValue() println("Sum: ${initial + value}") } fun main() { ::asyncSum.startCoroutine(20) println("Back to main") computation.resumeWith(22) } Started execution Back to main ,15
  16. Continuations in Kotlin suspend fun asyncSum(initial: Int) { println("Started execution")

    val value = asyncValue() println("Sum: ${initial + value}") } fun main() { ::asyncSum.startCoroutine(20) println("Back to main") computation.resumeWith(22) } Started execution Back to main Sum: 42 ,16
  17. How could it work? ,17

  18. Suspension points suspend fun asyncSum(initial: Int) { // State 1

    println("Started execution") val value = computeAsync() // State 2 val sum = initial + value println("Sum: $sum") } ,18
  19. CPS transformation interface Continuation<in T> { val context: CoroutineContext fun

    resumeWith(result: Result<T>) } fun asyncSum(..., continuation: Continuation<*>) { println("Started execution") val value = asyncValue() println("Sum: ${initial + value}") } ,19
  20. CPS transformation fun asyncSum(initial: Int, callback: () -> Unit) {

    println("Started execution") asyncValue { value -> println("Sum: ${initial + value}") callback() } } ,20
  21. CPS. State machine fun asyncSum(initial: Int, c: Continuation<*>) { }

    ,21
  22. CPS. State machine fun asyncSum(initial: Int, c: Continuation<*>) { val

    me = c as? AsyncSumCont ?: AsyncSumCont(c) } ,22
  23. fun asyncSum(initial: Int, c: Continuation<*>) { val me = c

    as? AsyncSumCont ?: AsyncSumCont(c) when (me.label) { 0 -> { println("Started execution”) asyncValue(me) } } } CPS. State machine ,23
  24. fun asyncSum(initial: Int, c: Continuation<*>) { val me = c

    as? AsyncSumCont ?: AsyncSumCont(c) when (me.label) { 0 -> { println("Started execution”) asyncValue(me) } 1 -> { val value = me.valueOrThrow println("Sum: ${initial + value}”) me.caller.resumeWith(Unit) } } } CPS. State machine ,24
  25. fun asyncSum(initial: Int, c: Continuation<*>) { when(me.label) { 0 ->

    { println("Started execution”) asyncValue(me) } } } CPS. State manipulation ,25
  26. fun asyncSum(initial: Int, c: Continuation<*>) { when(me.label) { 0 ->

    { println("Started execution") me.label = 1 me.initial = initial asyncValue(me) } } } State manipulation. Spilling ,26
  27. class AsyncSumCont() : Continuation<Int> { val caller: Continuation<Unit> val label:

    Int val initial: Int val value: Int override fun resumeWith(value: Int) { value = value asyncSum(initial, this) } } State manipulation. Resume ,27
  28. Example revisited suspend fun computeAsync(): Int { ... } suspend

    fun asyncSum(initial: Int) { println("Started execution") val value = computeAsync() println("Sum: ${initial + value}") } ,28
  29. Example revisited suspend fun computeAsync(): Int { val value =

    suspendCoroutine<Int> { cont -> computation = cont } return value } suspend fun asyncSum(initial: Int) { println("Started execution") val value = computeAsync() println("Sum: ${initial + value}") } ,29
  30. ,30

  31. suspend fun <T> CompletableFuture<T>.await(): T CompletableFuture integration ,31

  32. suspend fun <T> CompletableFuture<T>.await(): T = suspendCoroutine { cont ->

    } CompletableFuture integration ,32
  33. suspend fun <T> CompletableFuture<T>.await(): T = suspendCoroutine { cont ->

    whenComplete { value, exception -> } } CompletableFuture integration ,33
  34. suspend fun <T> CompletableFuture<T>.await(): T = suspendCoroutine { cont ->

    whenComplete { value, exception -> if (exception != null) cont.resumeWith(exception) else cont.resumeWith(value) } } CompletableFuture integration ,34
  35. CompletableFuture integration suspend fun computeAsync(): CompletableFuture<Int> suspend fun asyncSum(initial: Int)

    { val future = computeAsync() println("Sum: ${initial + future.await()}") } ,35
  36. Callbacks strike back suspend fun asyncSum(count: Int) { var sum

    = 0 for (i in 0 until count) { sum += computeAsync().await() } println(sum) } ,36
  37. Exception in thread "main" java.lang.StackOverflowError at StackOverflowKt.computeAsync(StackOverflow.kt:6) at StackOverflowKt.asyncSum(StackOverflow.kt:11) at

    StackOverflowKt$asyncSum$2.invoke(StackOverflow.kt:16) at StackOverflowKt$asyncSum$2.invoke(StackOverflow.kt) at StackOverflowKt.await(StackOverflow.kt:4) at StackOverflowKt.asyncSum(StackOverflow.kt:11) at StackOverflowKt$asyncSum$2.invoke(StackOverflow.kt:16) at StackOverflowKt$asyncSum$2.invoke(StackOverflow.kt) at StackOverflowKt.await(StackOverflow.kt:4) at StackOverflowKt.asyncSum(StackOverflow.kt:11) at StackOverflowKt$asyncSum$2.invoke(StackOverflow.kt:16) at StackOverflowKt$asyncSum$2.invoke(StackOverflow.kt) Callbacks strike back ,37
  38. Callbacks strike back suspend fun asyncSum(count: Int) { var sum

    = 0 for (i in 0 until count) { sum += computeAsync().await() } println(sum) } Callstack await whenComplete asyncSum$resumeWith await whenComplete asyncSum$resumeWith ... ,38
  39. Calling convention fun asyncSum(cont: Continuation<*>): Unit fun asyncSum(cont: Continuation<*>): Any?

    fun asyncSum(cont: Continuation<*>): Unit | COROUTINE_SUSPENDED ,39
  40. fun asyncSum(initial: Int, c: Continuation<*>) { when (me.label) { 0

    -> { println("Started execution") val result = computeAsync(me) if (result == COROUTINE_SUSPENDED) { return COROUTINE_SUSPENDED } me.value = result goto 1 } } } Stack unrolling ,40
  41. fun asyncSum(initial: Int, c: Continuation<*>) { when (me.label) { 1

    -> { val value = me.valueOrThrow println("Sum: ${initial + value}”) return Unit } } } Stack unrolling ,41
  42. Hotspot strikes back ,42

  43. Escape analysis fun Vector.length(): Double = sqrt(x * x +

    y * y) fun length(x: Int, y: Int): Double { val vector = new Vector(x, y) return vector.length() } ,43
  44. Escape analysis fun Vector.length(): Double = sqrt(x * x +

    y * y) fun length(x: Int, y: Int): Double { val vector = new Vector(x, y) return sqrt(v.x * v.x + v.y * v.y) } ,44
  45. Escape analysis. Scalarization fun Vector.length(): Double = sqrt(x * x

    + y * y) fun length(x: Int, y: Int): Double { val vector = new Vector(x, y) return sqrt(x * x + y * y) } ,45
  46. Escape analysis fun length(x: Int, y: Int): Double { val

    v: Vector if (x % 2 == 0) { v = new Vector(x, y) } else { v = new Vector(x / 2, y) } return v.length() } https://www.youtube.com/watch?v=K6c3W6vhQOA ,46
  47. Escape analysis fun asyncSum() { val result: Any = asyncValue()

    if (result == COROUTINE_SUSPENDED) { return COROUTINE_SUSPENDED } else { val value = (result as Int) . . . } } ,47
  48. fun asyncSum() { val result: Any = Integer.valueOf(…) if (result

    == COROUTINE_SUSPENDED) { return COROUTINE_SUSPENDED } else { val value = (result as Int) . . . } } Escape analysis ,48
  49. Escape analysis fun asyncSum() { val result: Any = new

    Int() if (result == COROUTINE_SUSPENDED) { return COROUTINE_SUSPENDED } else { val value = (result as Int) . . . } } ,49
  50. •Abstraction •CPS transformation •Cost Synopsis ,50

  51. Part III. Effective abstractions ,51

  52. Embracing concurrency suspend fun <T> CompletableFuture<T>.await(): T = suspendCoroutine {

    cont -> whenComplete { value, exception -> if (exception != null) cont.resumeWith(exception) else cont.resumeWith(value) } } kotlin.KotlinNullPointerException: null at kotlin.coroutines.jvm.internal.ContinuationImpl.releaseIntercepted(ContinuationImpl.kt:117) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:38) at kotlin.coroutines.ContinuationKt.startCoroutine(Continuation.kt:113) at ConcurrencyKt.builder(Concurrency.kt:9) at ConcurrencyLt.main(Concurrency.kt:19) at ConcurrencyKt.main(Concurrency.kt) ,52
  53. Thread 1 Thread 2 await whenComplete COROUTINE_SUSPENDED // Saves state

    Future.complete UniWhenComplete await$resume // Uses state Embracing concurrency ,53
  54. Consensus problem ,54

  55. Thread safety 2. Add jetbrains.misc.Unsafe and use it in standard

    library 3. Abstraction 1. Make every continuation thread-safe ,55
  56. suspend fun <T> suspendCoroutine(lambda): T = suspendCoroutineOrReturn { continuation ->

    val safe = SafeContinuation(continuation) lambda(safe) safe.getOrThrow() } Abstracting the problem away ,56
  57. suspend fun <T> suspendCoroutine(lambda): T = suspendCoroutineOrReturn { continuation ->

    val safe = SafeContinuation(continuation) lambda(safe) safe.getOrThrow() } public fun SafeContinuation.resumeWith(result: Result<T>) { while (true) { // lock-free loop val cur = this.result // atomic read when { cur === UNDECIDED -> if (RESULT.compareAndSet(this, UNDECIDED, result.value)) return cur === COROUTINE_SUSPENDED -> if (RESULT.compareAndSet(this, COROUTINE_SUSPENDED, RESUMED)) { delegate.resumeWith(result) return } else -> throw IllegalStateException("Already resumed") } } Abstracting the problem away ,57
  58. suspend fun <T> suspendCoroutine(lambda): T = suspendCoroutineOrReturn { continuation ->

    val safe = SafeContinuation(continuation) block(safe) safe.getOrThrow() } Abstracting the problem away ,58
  59. Execution context suspend fun readFromDisk(): String { val future =

    CompletableFuture.supplyAsync({io()}, ioExecutor) return future.await() } ,59
  60. suspend fun <T> CompletableFuture<T>.await( executor: Executor): T = suspendCoroutine {

    cont -> whenComplete { value, exception -> executor.execute(Runnable { cont.resumeWith(exception) }) } } Execution context ,60
  61. suspend fun <T> CompletableFuture<T>.await( executor: Executor): T = suspendCoroutine {

    cont -> whenComplete { value, exception -> executor.execute(Runnable { cont.resumeWith(exception) }) } } Execution context ,61
  62. suspend fun <T> CompletableFuture<T>.await( executor: Executor): T = suspendCoroutine {

    cont -> whenCompleteAsync(..., executor) } Execution context ,62
  63. interface Continuation<in T> { val context: CoroutineContext fun resumeWith(result: Result<T>)

    } ::computeAsync.startCoroutine(executor.asContext()) Execution context class ExecutorCoroutineContext : CoroutineContext ,63
  64. •Composability •Extensibility •Performance Problems ,64

  65. Libraries-driven design interface ContinuationInterceptor : ContextElement { fun <T> interceptContinuation(

    continuation: Continuation<T>): Continuation<T> } ,65
  66. object ForkJoinPoolInterceptor : ContinuationInterceptor { /// ... context and key

    implementation ... override fun <T> interceptContinuation(cont: Continuation<T>): Continuation<T> { return object : Continuation<T>, Runnable { private var savedState: Result<T>? = null override fun resumeWith(result: Result<T>) { savedState = result ForkJoinPool.commonPool().execute(this) } override fun run() { cont.resumeWith(savedState!!) } } } } Libraries-driven design ,66
  67. • Always intercept • Unification • Allocation issue • Cache

    it in the state machine! One more calling convention ,67
  68. Synopsis • Power of abstraction • Complexity can be abstracted

    • Standard library base ,68
  69. Further reading • kotlinx.coroutines • KEEP • https://medium.com/@elizarov • Loom

    proposal ,69