$30 off During Our Annual Pro Sale. View Details »

Kotlin Native Concurrency Explained (Kotlinconf 2019)

Kotlin Native Concurrency Explained (Kotlinconf 2019)

Explain the Kotlin native concurrency model, best practices, and tools and libraries you should check out.

Kevin Galligan

December 05, 2019
Tweet

More Decks by Kevin Galligan

Other Decks in Programming

Transcript

  1. Copenhagen
    Denmark
    KOTLIN NATIVE
    CONCURRENCY
    EXPLAINED
    KEVIN GALLIGAN
    @kpgalligan

    View Slide

  2. Touchlab

    View Slide

  3. View Slide

  4. [Your Org Here]
    reach out if interested, obv

    View Slide

  5. View Slide

  6. View Slide

  7. [email protected]
    @kpgalligan

    View Slide

  8. View Slide

  9. View Slide

  10. I still have a talk!
    and then…

    View Slide

  11. multithreaded coroutines!

    View Slide

  12. Intro

    View Slide

  13. What is Kotlin Multiplatform?

    View Slide

  14. kot·lin mul·ti·plat·form
    /ˌkätˈlin məltiˈplatfôrm,ˌkätˈlin məltīˈplatfôrm/
    noun
    noun: kotlin multiplatform
    1.optional, natively-integrated, open-source, code sharing platform, based on the popular,
    modern language kotlin. facilitates non-ui logic availability on many platforms.

    View Slide

  15. modern

    View Slide

  16. Saner Concurrency?
    concurrency is hard

    View Slide

  17. View Slide

  18. Kotlin Native model is controversial
    lots of disagreement

    View Slide

  19. JVM model is controversial
    many other platforms don’t let you do that

    View Slide

  20. JVM model is controversial
    many other platforms don’t let you do that

    View Slide

  21. JVM model is controversial
    many other platforms don’t let you do that

    View Slide

  22. This talk will cover
    •How the model works
    •My thoughts on concurrency and best practices
    •Libraries and future stuff (MT coroutines!)

    View Slide

  23. This talk will not cover
    •Me “selling” the model
    •Me complaining about the model
    •Debating various perspectives from the community

    View Slide

  24. Also…
    no “getting around” rules

    View Slide

  25. You need to understand it
    libraries won’t hide all details

    View Slide

  26. important to relax

    View Slide

  27. Kotlin Native Rules

    View Slide

  28. 1) Mutable State = one thread
    thread confined

    View Slide

  29. 2) Immutable state = many threads
    no changes mean no problems

    View Slide

  30. [email protected]
    @kpgalligan

    View Slide

  31. Internalize The Rules
    mutable = 1/immutable = many

    View Slide

  32. object BigGlobalState {
    var statusString = ""
    }

    View Slide

  33. Don’t manage concurrent state
    no “synchronized”, “volatile”, etc

    View Slide

  34. JVM Trusts You
    native does not

    View Slide

  35. Runtime Verification

    View Slide

  36. Do Things Differently
    not “losing” anything

    View Slide

  37. (jvm strict mode would be nice)

    View Slide

  38. You can break rules too
    just not a great idea

    View Slide

  39. OK. Code please?

    View Slide

  40. Worker
    kn concurrency queue

    View Slide

  41. Similar to ExecutorService
    Handler/MessageQueue/Looper on Android

    View Slide

  42. Worker
    job queue
    thread
    ???
    execute

    View Slide

  43. Worker
    job queue
    thread
    ???
    execute

    View Slide

  44. Worker
    job queue
    thread
    ???
    execute

    View Slide

  45. Worker
    job queue
    thread
    ???

    View Slide

  46. Worker
    job queue
    thread
    ???

    View Slide

  47. Worker
    job queue
    thread
    ???

    View Slide

  48. Worker
    job queue
    thread
    ???

    View Slide

  49. val worker = Worker.start()
    worker.execute(TransferMode.SAFE, {"Hello "}) {
    println(it)
    }

    View Slide

  50. val worker = Worker.start()
    worker.execute(TransferMode.SAFE, {"Hello "}) {
    println(it)
    }

    View Slide

  51. val worker = Worker.start()
    worker.execute(TransferMode.SAFE, {"Hello "}) {
    println(it)
    }

    View Slide

  52. Hello

    View Slide

  53. val worker = Worker.start()
    worker.execute(TransferMode.SAFE, {"Hello "}) {
    println(it)
    }

    View Slide

  54. public fun execute(
    mode: TransferMode,
    producer: () -> T1,
    job: (T1) -> T2):
    Future {
    //...
    }

    View Slide

  55. public fun execute(
    mode: TransferMode,
    producer: () -> T1,
    job: (T1) -> T2):
    Future {
    //...
    }

    View Slide

  56. public fun execute(
    mode: TransferMode,
    producer: () -> T1,
    job: (T1) -> T2):
    Future {
    //...
    }

    View Slide

  57. val worker = Worker.start()
    worker.execute(TransferMode.SAFE, {"Hello "}) {
    println(it)
    }

    View Slide

  58. public fun execute(
    mode: TransferMode,
    producer: () -> T1,
    job: (T1) -> T2):
    Future {
    //...
    }

    View Slide

  59. val worker = Worker.start()
    worker.execute(TransferMode.SAFE, {"Hello "}) {
    println(it)
    }.result

    View Slide

  60. public fun execute(
    mode: TransferMode,
    producer: () -> T1,
    job: (T1) -> T2):
    Future {
    //...
    }

    View Slide

  61. producer: () -> T1

    View Slide

  62. {"Hello "}
    (Unit)->String

    View Slide

  63. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    worker.execute(TransferMode.SAFE, {SomeData("Hello ", 2)}) {
    println(it)
    }.result
    }

    View Slide

  64. data class SomeData(val s:String, val i:Int)
    {SomeData("Hello ", 2)}
    (Unit)->SomeData

    View Slide

  65. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    worker.execute(TransferMode.SAFE, {SomeData("Hello ", 2)}) {
    println(it)
    }.result
    }

    View Slide

  66. fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }

    View Slide

  67. fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }

    View Slide

  68. fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    kotlin.IllegalStateException: Illegal transfer state

    View Slide

  69. fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    execute wants to transfer someData between threads

    View Slide

  70. fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    checks result (and children) for external refs

    View Slide

  71. fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    local is still a ref till the end

    View Slide

  72. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    Mutable state?

    View Slide

  73. freeze()
    runtime immutability designation

    View Slide

  74. public fun T.freeze(): T {...}

    View Slide

  75. Freeze
    • Recursively freezes everything
    • One-way (no “unfreeze”)
    • Data may be shared between threads
    Int
    String
    Float
    Float
    String
    Int
    MoreData
    val strData
    var width
    SomeData
    val moreData
    var count

    View Slide

  76. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }

    View Slide

  77. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData.freeze() }) {
    println(it)
    }.result
    }

    View Slide

  78. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData.freeze() }) {
    println(it)
    }.result
    }

    View Slide

  79. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData.freeze() }) {
    println(it)
    }.result
    }

    View Slide

  80. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    val someData = SomeData("Hello ", 2).freeze()
    worker.execute(TransferMode.SAFE, { }) {
    println(someData)
    }.result
    }
    Worker.execute must take an unbound,
    non-capturing function or lambda

    View Slide

  81. fun background(block:(Unit)->Unit){
    worker.execute(TransferMode.SAFE, {}, block)
    }

    View Slide

  82. fun background(block:(Unit)->Unit){
    worker.execute(TransferMode.SAFE, {}, block)
    }
    Worker.execute must take an unbound,
    non-capturing function or lambda

    View Slide

  83. fun background(block:(Unit)->Unit){
    worker.execute(TransferMode.SAFE, {}, block.freeze())
    }
    Worker.execute must take an unbound,
    non-capturing function or lambda

    View Slide

  84. fun background(block:()->Unit){
    worker.execute(TransferMode.SAFE, {block}) {
    it()
    }
    }
    kotlin.IllegalStateException: Illegal transfer state

    View Slide

  85. fun background(block:()->Unit){
    worker.execute(TransferMode.SAFE, {block.freeze()}) {
    it()
    }
    }

    View Slide

  86. fun printStuffBg() {
    val someData = SomeData("Hello ", 2)
    background {
    println(someData)
    }
    }

    View Slide

  87. fun printStuffBg() {
    val someData = SomeData("Hello ", 2)
    background {
    println(someData)
    }
    }

    View Slide

  88. fun printStuffBg() {
    val someData = SomeData("Hello ", 2)
    background {
    println(someData)
    }
    }

    View Slide

  89. class CounterModel {
    var count = 0
    fun countClicked() {
    count++
    }
    }

    View Slide

  90. class CounterModel {
    var count = 0
    fun countClicked() {
    background { count++ }
    }
    }

    View Slide

  91. class CounterModel {
    var count = 0
    fun countClicked() {
    background { count++ }
    }
    }

    View Slide

  92. class CounterModel {
    var count = 0
    fun countClicked() {
    background { count++ }
    }
    }

    View Slide

  93. class CounterModel {
    var count = 0
    fun countClicked() {
    background { count++ }
    }
    }

    View Slide

  94. class CounterModel {
    var count = 0
    fun countClicked() {
    background { count++ }
    }
    }
    InvalidMutabilityException: mutation attempt of frozen
    sample.CounterModel

    View Slide

  95. Functions Have State
    careful what you capture

    View Slide

  96. Global State
    some special rules

    View Slide

  97. val mainOnly = SomeData("a", 1)

    View Slide

  98. val mainOnly = SomeData("a", 1)
    object GlobalObject {
    val data = SomeData("b", 2)
    }

    View Slide

  99. @ThreadLocal
    val mainOnly = SomeData("a", 1)
    @ThreadLocal
    object GlobalObject {
    val data = SomeData("b", 2)
    }

    View Slide

  100. @SharedImmutable
    val mainOnly = SomeData("a", 1)
    @ThreadLocal
    object GlobalObject {
    val data = SomeData("b", 2)
    }

    View Slide

  101. Why Special Rules?
    you can access it from anywhere

    View Slide

  102. Tips and New Stuff

    View Slide

  103. Debugging Issues
    living with the new model

    View Slide

  104. InvalidMutabilityException
    changing frozen state

    View Slide

  105. class CounterModel {
    init {
    ensureNeverFrozen()
    }
    var count = 0
    fun countClicked() {
    background { count++ }
    }
    }

    View Slide

  106. Rethinking architecture
    intentional mutability

    View Slide

  107. Main Thread
    Other Thread (maybe?)
    Data
    Data
    Data
    Content
    Model
    iOS View
    Model

    View Slide

  108. Atomics
    mutating frozen state

    View Slide

  109. class CounterModel {
    val count = AtomicInt(0)
    fun countClicked() {
    background { count.value++ }
    }
    }

    View Slide

  110. class CounterModel {
    val count = AtomicInt(0)
    fun countClicked() {
    background { count.value++ }
    }
    }

    View Slide

  111. class CounterModel {
    val count = AtomicInt(0)
    fun countClicked() {
    background { count.value++ }
    }
    }

    View Slide

  112. val atom = AtomicReference(SomeData("", 0))

    View Slide

  113. class SomeModel {
    val atom = atomic(SomeData("abc", 123))
    fun update(sd:SomeData){
    atom.value = sd
    }
    init {
    freeze()
    }
    }

    View Slide

  114. class SomeModel {
    val atom = atomic(SomeData("abc", 123))
    fun update(sd:SomeData){
    atom.value = sd
    }
    init {
    freeze()
    }
    }

    View Slide

  115. class SomeModel {
    val atom = atomic(SomeData("abc", 123))
    fun update(sd:SomeData){
    atom.value = sd
    }
    init {
    freeze()
    }
    }

    View Slide

  116. class SomeModel {
    val atom = atomic(SomeData("abc", 123))
    fun update(sd:SomeData){
    atom.value = sd
    }
    init {
    freeze()
    }
    }

    View Slide

  117. Atomic-fu/Stately

    View Slide

  118. Don’t use too many atomics
    AtomicRef can leak memory

    View Slide

  119. DetachedObjectGraph
    transfer state

    View Slide

  120. DetachedObjectGraph
    transfer state

    View Slide

  121. Worker dispatched state
    mutable state thread local

    View Slide

  122. State Thread
    State Worker
    ???
    State Container
    Data

    View Slide

  123. State Thread
    State Worker
    ???
    State Container
    ???
    Data

    View Slide

  124. Multiplatform State
    freeze in common

    View Slide

  125. freeze() is native only

    View Slide

  126. expect fun T.freeze(): T
    expect fun T.isFrozen(): Boolean
    expect fun T.isNativeFrozen(): Boolean
    expect fun Any.ensureNeverFrozen()

    View Slide

  127. expect fun T.freeze(): T
    expect fun T.isFrozen(): Boolean
    expect fun T.isNativeFrozen(): Boolean
    expect fun Any.ensureNeverFrozen()

    View Slide

  128. Multithreaded Coroutines

    View Slide

  129. General Thoughts
    what does this mean?

    View Slide

  130. State Model Clarity
    strict mode is sticking around

    View Slide

  131. Library Development
    was a blocker

    View Slide

  132. KMP is more “real”

    View Slide

  133. View Slide

  134. Disclaimers
    just fyi

    View Slide

  135. View Slide

  136. Overview
    coroutine always bound to a single thread
    create with newSingleThreadContext
    on native that uses Worker
    Default has single background thread
    Main defined for Apple targets
    Windows/Linux ¯\_(ツ)_/¯

    View Slide

  137. Switching Threads
    use withContext (or equivalents)
    data passed in (or captured) is frozen
    data returned is frozen

    View Slide

  138. fun printStuffBg() {
    val someData = SomeData("Hello ", 2)
    background {
    println(someData)
    }
    }

    View Slide

  139. suspend fun printStuffBg() {
    val someData = SomeData("Hello ", 2)
    withContext(workerDispatcher) {
    println(someData)
    }
    }

    View Slide

  140. suspend fun printStuffBg() {
    val someData = SomeData("Hello ", 2)
    withContext(workerDispatcher) {
    println(someData)
    }
    }

    View Slide

  141. suspend fun printStuffForeground() {
    val someData = SomeData("Hello ", 2)
    val someMoreData = withContext(workerDispatcher) {
    SomeData("${someData.s}, Dogs!", someData.i + 1)
    }
    println(someMoreData)
    println("I am frozen? ${someMoreData.isFrozen()}")
    }

    View Slide

  142. suspend fun printStuffForeground() {
    val someData = SomeData("Hello ", 2)
    val someMoreData = withContext(workerDispatcher) {
    SomeData("${someData.s}, Dogs!", someData.i + 1)
    }
    println(someMoreData)
    println("I am frozen? ${someMoreData.isFrozen()}")
    }

    View Slide

  143. suspend fun printStuffForeground() {
    val someData = SomeData("Hello ", 2)
    val someMoreData = withContext(workerDispatcher) {
    SomeData("${someData.s}, Dogs!", someData.i + 1)
    }
    println(someMoreData)
    println("I am frozen? ${someMoreData.isFrozen()}")
    }

    View Slide

  144. SomeData(s=Hello , Dogs!, i=3)
    I am frozen? true

    View Slide

  145. class DbModel(private val dbId:Int){
    fun showDbStuff() = mainScope.launch {
    val sd = withContext(workerDispatcher){
    DB.find(dbId)
    }
    println(sd)
    }
    init {
    ensureNeverFrozen()
    }
    }

    View Slide

  146. class DbModel(private val dbId:Int){
    fun showDbStuff() = mainScope.launch {
    val capturedId = dbId
    val sd = withContext(workerDispatcher){
    DB.find(capturedId)
    }
    println(sd)
    }
    init {
    ensureNeverFrozen()
    }
    }

    View Slide

  147. class DbModel(private val dbId: Int) {
    fun showDbStuff() = mainScope.launch {
    val sd = loadDbInfo(dbId)
    println(sd)
    }
    private suspend fun loadDbInfo(id: Int) =
    withContext(workerDispatcher) {
    DB.find(id)
    }
    init {
    ensureNeverFrozen()
    }
    }

    View Slide

  148. class DbModel(private val dbId: Int) {
    fun showDbStuff() = mainScope.launch {
    val sd = loadDbInfo(dbId)
    println(sd)
    }
    private suspend fun loadDbInfo(id: Int) =
    withContext(workerDispatcher) {
    DB.find(id)
    }
    init {
    ensureNeverFrozen()
    }
    }

    View Slide

  149. class DbModel(private val dbId: Int) {
    fun showDbStuff() = mainScope.launch {
    val sd = loadDbInfo(dbId)
    println(sd)
    }
    private suspend fun loadDbInfo(id: Int) =
    withContext(workerDispatcher) {
    DB.find(id)
    }
    init {
    ensureNeverFrozen()
    }
    }

    View Slide

  150. Flow, Channel, etc
    all freezable

    View Slide

  151. Main Thread
    Other Thread (maybe?)
    Data
    Data
    Data
    Content
    Model
    iOS View
    Model

    View Slide

  152. init {
    ensureNeverFrozen()
    mainScope.launch {
    query.asFlow()
    .map {
    assertNotMainThread()
    extractData(it)
    }
    .flowOn(ServiceRegistry.backgroundDispatcher)
    .collect { vt ->
    view?.let {
    it.update(vt)
    }
    }
    }
    }

    View Slide

  153. init {
    ensureNeverFrozen()
    mainScope.launch {
    query.asFlow()
    .map {
    assertNotMainThread()
    extractData(it)
    }
    .flowOn(ServiceRegistry.backgroundDispatcher)
    .collect { vt ->
    view?.let {
    it.update(vt)
    }
    }
    }
    }

    View Slide

  154. init {
    ensureNeverFrozen()
    mainScope.launch {
    query.asFlow()
    .map {
    assertNotMainThread()
    extractData(it)
    }
    .flowOn(ServiceRegistry.backgroundDispatcher)
    .collect { vt ->
    view?.let {
    it.update(vt)
    }
    }
    }
    }

    View Slide

  155. init {
    ensureNeverFrozen()
    mainScope.launch {
    query.asFlow()
    .map {
    assertNotMainThread()
    extractData(it)
    }
    .flowOn(ServiceRegistry.backgroundDispatcher)
    .collect { vt ->
    view?.let {
    it.update(vt)
    }
    }
    }
    }

    View Slide

  156. init {
    ensureNeverFrozen()
    mainScope.launch {
    query.asFlow()
    .map {
    assertNotMainThread()
    extractData(it)
    }
    .flowOn(ServiceRegistry.backgroundDispatcher)
    .collect { vt ->
    view?.let {
    it.update(vt)
    }
    }
    }
    }

    View Slide

  157. init {
    ensureNeverFrozen()
    mainScope.launch {
    query.asFlow()
    .map {
    assertNotMainThread()
    extractData(it)
    }
    .flowOn(ServiceRegistry.backgroundDispatcher)
    .collect { vt ->
    view?.let {
    it.update(vt)
    }
    }
    }
    }

    View Slide

  158. Community Libraries

    View Slide

  159. Stately?

    View Slide

  160. View Slide

  161. View Slide

  162. Stately v1 (Probably) deprecated
    a lot of it, anyway

    View Slide

  163. @Deprecated
    Stately Collections

    View Slide

  164. @Discouraged
    Stately Collections

    View Slide

  165. January
    that whole “holidays” thing

    View Slide

  166. github.com/Autodesk/coroutineworker

    View Slide

  167. github.com/Badoo/reaktive

    View Slide

  168. Next Steps

    View Slide

  169. go.touchlab.co/knthreads

    View Slide

  170. JetBrains Links
    • https://github.com/JetBrains/kotlin-native/blob/
    master/IMMUTABILITY.md
    • https://github.com/JetBrains/kotlin-native/blob/
    master/CONCURRENCY.md
    • https://www.youtube.com/watch?v=nw6YTfEyfO0

    View Slide

  171. go.touchlab.co/mtco

    View Slide

  172. Kotlin Koans
    for native and state

    View Slide

  173. January
    or whenever MT coroutines are stable(ish)

    View Slide

  174. Try out MT Coroutines?
    some assembly required

    View Slide

  175. Build and deploy local
    assuming no updates from kotlinx.coroutines

    View Slide

  176. > git clone -b native-mt \
    https://github.com/Kotlin/kotlinx.coroutines.git
    > cd kotlinx.coroutines/
    > ./gradlew build publishToMavenLocal

    View Slide

  177. Samples
    from slides
    https://github.com/kpgalligan/MTCoroutines
    Droidcon app
    https://github.com/touchlab/DroidconKotlin/
    (look for the kpg/mt_coroutines branch)

    View Slide

  178. KMP evaluation kit

    View Slide

  179. KaMP-Kit
    go.touchlab.co/KaMP-Kit

    View Slide

  180. Stickers!

    View Slide

  181. #KotlinConf
    THANK YOU
    AND
    REMEMBER
    TO VOTE
    Kevin Galligan @kpgalligan

    View Slide

  182. Thanks!
    @kpgalligan
    touchlab.co

    View Slide

  183. Thanks!
    @kpgalligan
    touchlab.co
    Join the team
    !

    View Slide