Kotlin Native Concurrency Explained (Kotlinconf 2019)

Explain the Kotlin native concurrency model, best practices, and tools and libraries you should check out.

Transcript

1. Copenhagen
   Denmark
   KOTLIN NATIVE
   CONCURRENCY
   EXPLAINED
   KEVIN GALLIGAN
   @kpgalligan
   

   View Slide

2. Touchlab
   

   View Slide

3. View Slide

4. [Your Org Here]
   reach out if interested, obv
   

   View Slide

5. View Slide

6. View Slide

7. kevin@touchlab.co
   @kpgalligan
   
   

   View Slide

8. View Slide

9. View Slide

10. I still have a talk!
    and then…
    

    View Slide

11. multithreaded coroutines!
    

    View Slide

12. Intro
    

    View Slide

13. What is Kotlin Multiplatform?
    

    View Slide

14. kot·lin mul·ti·plat·form
    /ˌkätˈlin məltiˈplatfôrm,ˌkätˈlin məltīˈplatfôrm/
    noun
    noun: kotlin multiplatform
    1.optional, natively-integrated, open-source, code sharing platform, based on the popular,
    modern language kotlin. facilitates non-ui logic availability on many platforms.
    

    View Slide

15. modern
    

    View Slide

16. Saner Concurrency?
    concurrency is hard
    

    View Slide

17. View Slide

18. Kotlin Native model is controversial
    lots of disagreement
    

    View Slide

19. JVM model is controversial
    many other platforms don’t let you do that
    

    View Slide

20. JVM model is controversial
    many other platforms don’t let you do that
    

    View Slide

21. JVM model is controversial
    many other platforms don’t let you do that
    

    View Slide

22. This talk will cover
    •How the model works
    •My thoughts on concurrency and best practices
    •Libraries and future stuff (MT coroutines!)
    

    View Slide

23. This talk will not cover
    •Me “selling” the model
    •Me complaining about the model
    •Debating various perspectives from the community
    

    View Slide

24. Also…
    no “getting around” rules
    

    View Slide

25. You need to understand it
    libraries won’t hide all details
    

    View Slide

26. important to relax
    

    View Slide

27. Kotlin Native Rules
    

    View Slide

28. 1) Mutable State = one thread
    thread confined
    

    View Slide

29. 2) Immutable state = many threads
    no changes mean no problems
    

    View Slide

30. kevin@touchlab.co
    @kpgalligan
    
    

    View Slide

31. Internalize The Rules
    mutable = 1/immutable = many
    

    View Slide

32. object BigGlobalState {
    var statusString = ""
    }
    

    View Slide

33. Don’t manage concurrent state
    no “synchronized”, “volatile”, etc
    

    View Slide

34. JVM Trusts You
    native does not
    

    View Slide

35. Runtime Verification
    

    View Slide

36. Do Things Differently
    not “losing” anything
    

    View Slide

37. (jvm strict mode would be nice)
    

    View Slide

38. You can break rules too
    just not a great idea
    

    View Slide

39. OK. Code please?
    

    View Slide

40. Worker
    kn concurrency queue
    

    View Slide

41. Similar to ExecutorService
    Handler/MessageQueue/Looper on Android
    

    View Slide

42. Worker
    job queue
    thread
    ???
    execute
    

    View Slide

43. Worker
    job queue
    thread
    ???
    execute
    

    View Slide

44. Worker
    job queue
    thread
    ???
    execute
    

    View Slide

45. Worker
    job queue
    thread
    ???
    

    View Slide

46. Worker
    job queue
    thread
    ???
    

    View Slide

47. Worker
    job queue
    thread
    ???
    

    View Slide

48. Worker
    job queue
    thread
    ???
    

    View Slide

49. val worker = Worker.start()
    worker.execute(TransferMode.SAFE, {"Hello "}) {
    println(it)
    }
    

    View Slide

50. val worker = Worker.start()
    worker.execute(TransferMode.SAFE, {"Hello "}) {
    println(it)
    }
    

    View Slide

51. val worker = Worker.start()
    worker.execute(TransferMode.SAFE, {"Hello "}) {
    println(it)
    }
    

    View Slide

52. Hello
    

    View Slide

53. val worker = Worker.start()
    worker.execute(TransferMode.SAFE, {"Hello "}) {
    println(it)
    }
    

    View Slide

54. public fun execute(
    mode: TransferMode,
    producer: () -> T1,
    job: (T1) -> T2):
    Future {
    //...
    }
    

    View Slide

55. public fun execute(
    mode: TransferMode,
    producer: () -> T1,
    job: (T1) -> T2):
    Future {
    //...
    }
    

    View Slide

56. public fun execute(
    mode: TransferMode,
    producer: () -> T1,
    job: (T1) -> T2):
    Future {
    //...
    }
    

    View Slide

57. val worker = Worker.start()
    worker.execute(TransferMode.SAFE, {"Hello "}) {
    println(it)
    }
    

    View Slide

58. public fun execute(
    mode: TransferMode,
    producer: () -> T1,
    job: (T1) -> T2):
    Future {
    //...
    }
    

    View Slide

59. val worker = Worker.start()
    worker.execute(TransferMode.SAFE, {"Hello "}) {
    println(it)
    }.result
    

    View Slide

60. public fun execute(
    mode: TransferMode,
    producer: () -> T1,
    job: (T1) -> T2):
    Future {
    //...
    }
    

    View Slide

61. producer: () -> T1
    

    View Slide

62. {"Hello "}
    (Unit)->String
    

    View Slide

63. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    worker.execute(TransferMode.SAFE, {SomeData("Hello ", 2)}) {
    println(it)
    }.result
    }
    

    View Slide

64. data class SomeData(val s:String, val i:Int)
    {SomeData("Hello ", 2)}
    (Unit)->SomeData
    

    View Slide

65. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    worker.execute(TransferMode.SAFE, {SomeData("Hello ", 2)}) {
    println(it)
    }.result
    }
    

    View Slide

66. fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    

    View Slide

67. fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    

    View Slide

68. fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    kotlin.IllegalStateException: Illegal transfer state
    

    View Slide

69. fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    execute wants to transfer someData between threads
    

    View Slide

70. fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    checks result (and children) for external refs
    

    View Slide

71. fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    local is still a ref till the end
    

    View Slide

72. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    Mutable state?
    

    View Slide

73. freeze()
    runtime immutability designation
    

    View Slide

74. public fun T.freeze(): T {...}
    

    View Slide

75. Freeze
    • Recursively freezes everything
    • One-way (no “unfreeze”)
    • Data may be shared between threads
    Int
    String
    Float
    Float
    String
    Int
    MoreData
    val strData
    var width
    SomeData
    val moreData
    var count
    

    View Slide

76. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData }) {
    println(it)
    }.result
    }
    

    View Slide

77. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData.freeze() }) {
    println(it)
    }.result
    }
    

    View Slide

78. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData.freeze() }) {
    println(it)
    }.result
    }
    

    View Slide

79. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    val someData = SomeData("Hello ", 2)
    worker.execute(TransferMode.SAFE, { someData.freeze() }) {
    println(it)
    }.result
    }
    

    View Slide

80. data class SomeData(val s:String, val i:Int)
    fun printStuff2() {
    val someData = SomeData("Hello ", 2).freeze()
    worker.execute(TransferMode.SAFE, { }) {
    println(someData)
    }.result
    }
    Worker.execute must take an unbound,
    non-capturing function or lambda
    

    View Slide

81. fun background(block:(Unit)->Unit){
    worker.execute(TransferMode.SAFE, {}, block)
    }
    

    View Slide

82. fun background(block:(Unit)->Unit){
    worker.execute(TransferMode.SAFE, {}, block)
    }
    Worker.execute must take an unbound,
    non-capturing function or lambda
    

    View Slide

83. fun background(block:(Unit)->Unit){
    worker.execute(TransferMode.SAFE, {}, block.freeze())
    }
    Worker.execute must take an unbound,
    non-capturing function or lambda
    

    View Slide

84. fun background(block:()->Unit){
    worker.execute(TransferMode.SAFE, {block}) {
    it()
    }
    }
    kotlin.IllegalStateException: Illegal transfer state
    

    View Slide

85. fun background(block:()->Unit){
    worker.execute(TransferMode.SAFE, {block.freeze()}) {
    it()
    }
    }
    

    View Slide

86. fun printStuffBg() {
    val someData = SomeData("Hello ", 2)
    background {
    println(someData)
    }
    }
    

    View Slide

87. fun printStuffBg() {
    val someData = SomeData("Hello ", 2)
    background {
    println(someData)
    }
    }
    

    View Slide

88. fun printStuffBg() {
    val someData = SomeData("Hello ", 2)
    background {
    println(someData)
    }
    }
    

    View Slide

89. class CounterModel {
    var count = 0
    fun countClicked() {
    count++
    }
    }
    

    View Slide

90. class CounterModel {
    var count = 0
    fun countClicked() {
    background { count++ }
    }
    }
    

    View Slide

91. class CounterModel {
    var count = 0
    fun countClicked() {
    background { count++ }
    }
    }
    

    View Slide

92. class CounterModel {
    var count = 0
    fun countClicked() {
    background { count++ }
    }
    }
    

    View Slide

93. class CounterModel {
    var count = 0
    fun countClicked() {
    background { count++ }
    }
    }
    

    View Slide

94. class CounterModel {
    var count = 0
    fun countClicked() {
    background { count++ }
    }
    }
    InvalidMutabilityException: mutation attempt of frozen
    sample.CounterModel
    

    View Slide

95. Functions Have State
    careful what you capture
    

    View Slide

96. Global State
    some special rules
    

    View Slide

97. val mainOnly = SomeData("a", 1)
    

    View Slide

98. val mainOnly = SomeData("a", 1)
    object GlobalObject {
    val data = SomeData("b", 2)
    }
    

    View Slide

99. @ThreadLocal
    val mainOnly = SomeData("a", 1)
    @ThreadLocal
    object GlobalObject {
    val data = SomeData("b", 2)
    }
    

    View Slide

100. @SharedImmutable
     val mainOnly = SomeData("a", 1)
     @ThreadLocal
     object GlobalObject {
     val data = SomeData("b", 2)
     }
     

     View Slide

101. Why Special Rules?
     you can access it from anywhere
     

     View Slide

102. Tips and New Stuff
     

     View Slide

103. Debugging Issues
     living with the new model
     

     View Slide

104. InvalidMutabilityException
     changing frozen state
     

     View Slide

105. class CounterModel {
     init {
     ensureNeverFrozen()
     }
     var count = 0
     fun countClicked() {
     background { count++ }
     }
     }
     

     View Slide

106. Rethinking architecture
     intentional mutability
     

     View Slide

107. Main Thread
     Other Thread (maybe?)
     Data
     Data
     Data
     Content
     Model
     iOS View
     Model
     

     View Slide

108. Atomics
     mutating frozen state
     

     View Slide

109. class CounterModel {
     val count = AtomicInt(0)
     fun countClicked() {
     background { count.value++ }
     }
     }
     

     View Slide

110. class CounterModel {
     val count = AtomicInt(0)
     fun countClicked() {
     background { count.value++ }
     }
     }
     

     View Slide

111. class CounterModel {
     val count = AtomicInt(0)
     fun countClicked() {
     background { count.value++ }
     }
     }
     

     View Slide

112. val atom = AtomicReference(SomeData("", 0))
     

     View Slide

113. class SomeModel {
     val atom = atomic(SomeData("abc", 123))
     fun update(sd:SomeData){
     atom.value = sd
     }
     init {
     freeze()
     }
     }
     

     View Slide

114. class SomeModel {
     val atom = atomic(SomeData("abc", 123))
     fun update(sd:SomeData){
     atom.value = sd
     }
     init {
     freeze()
     }
     }
     

     View Slide

115. class SomeModel {
     val atom = atomic(SomeData("abc", 123))
     fun update(sd:SomeData){
     atom.value = sd
     }
     init {
     freeze()
     }
     }
     

     View Slide

116. class SomeModel {
     val atom = atomic(SomeData("abc", 123))
     fun update(sd:SomeData){
     atom.value = sd
     }
     init {
     freeze()
     }
     }
     

     View Slide

117. Atomic-fu/Stately
     

     View Slide

118. Don’t use too many atomics
     AtomicRef can leak memory
     

     View Slide

119. DetachedObjectGraph
     transfer state
     

     View Slide

120. DetachedObjectGraph
     transfer state
     

     View Slide

121. Worker dispatched state
     mutable state thread local
     

     View Slide

122. State Thread
     State Worker
     ???
     State Container
     Data
     

     View Slide

123. State Thread
     State Worker
     ???
     State Container
     ???
     Data
     

     View Slide

124. Multiplatform State
     freeze in common
     

     View Slide

125. freeze() is native only
     

     View Slide

126. expect fun T.freeze(): T
     expect fun T.isFrozen(): Boolean
     expect fun T.isNativeFrozen(): Boolean
     expect fun Any.ensureNeverFrozen()
     

     View Slide

127. expect fun T.freeze(): T
     expect fun T.isFrozen(): Boolean
     expect fun T.isNativeFrozen(): Boolean
     expect fun Any.ensureNeverFrozen()
     

     View Slide

128. Multithreaded Coroutines
     

     View Slide

129. General Thoughts
     what does this mean?
     

     View Slide

130. State Model Clarity
     strict mode is sticking around
     

     View Slide

131. Library Development
     was a blocker
     

     View Slide

132. KMP is more “real”
     

     View Slide

133. View Slide

134. Disclaimers
     just fyi
     

     View Slide

135. View Slide

136. Overview
     coroutine always bound to a single thread
     create with newSingleThreadContext
     on native that uses Worker
     Default has single background thread
     Main defined for Apple targets
     Windows/Linux ¯\_(ツ)_/¯
     

     View Slide

137. Switching Threads
     use withContext (or equivalents)
     data passed in (or captured) is frozen
     data returned is frozen
     

     View Slide

138. fun printStuffBg() {
     val someData = SomeData("Hello ", 2)
     background {
     println(someData)
     }
     }
     

     View Slide

139. suspend fun printStuffBg() {
     val someData = SomeData("Hello ", 2)
     withContext(workerDispatcher) {
     println(someData)
     }
     }
     

     View Slide

140. suspend fun printStuffBg() {
     val someData = SomeData("Hello ", 2)
     withContext(workerDispatcher) {
     println(someData)
     }
     }
     

     View Slide

141. suspend fun printStuffForeground() {
     val someData = SomeData("Hello ", 2)
     val someMoreData = withContext(workerDispatcher) {
     SomeData("${someData.s}, Dogs!", someData.i + 1)
     }
     println(someMoreData)
     println("I am frozen? ${someMoreData.isFrozen()}")
     }
     

     View Slide

142. suspend fun printStuffForeground() {
     val someData = SomeData("Hello ", 2)
     val someMoreData = withContext(workerDispatcher) {
     SomeData("${someData.s}, Dogs!", someData.i + 1)
     }
     println(someMoreData)
     println("I am frozen? ${someMoreData.isFrozen()}")
     }
     

     View Slide

143. suspend fun printStuffForeground() {
     val someData = SomeData("Hello ", 2)
     val someMoreData = withContext(workerDispatcher) {
     SomeData("${someData.s}, Dogs!", someData.i + 1)
     }
     println(someMoreData)
     println("I am frozen? ${someMoreData.isFrozen()}")
     }
     

     View Slide

144. SomeData(s=Hello , Dogs!, i=3)
     I am frozen? true
     

     View Slide

145. class DbModel(private val dbId:Int){
     fun showDbStuff() = mainScope.launch {
     val sd = withContext(workerDispatcher){
     DB.find(dbId)
     }
     println(sd)
     }
     init {
     ensureNeverFrozen()
     }
     }
     

     View Slide

146. class DbModel(private val dbId:Int){
     fun showDbStuff() = mainScope.launch {
     val capturedId = dbId
     val sd = withContext(workerDispatcher){
     DB.find(capturedId)
     }
     println(sd)
     }
     init {
     ensureNeverFrozen()
     }
     }
     

     View Slide

147. class DbModel(private val dbId: Int) {
     fun showDbStuff() = mainScope.launch {
     val sd = loadDbInfo(dbId)
     println(sd)
     }
     private suspend fun loadDbInfo(id: Int) =
     withContext(workerDispatcher) {
     DB.find(id)
     }
     init {
     ensureNeverFrozen()
     }
     }
     

     View Slide

148. class DbModel(private val dbId: Int) {
     fun showDbStuff() = mainScope.launch {
     val sd = loadDbInfo(dbId)
     println(sd)
     }
     private suspend fun loadDbInfo(id: Int) =
     withContext(workerDispatcher) {
     DB.find(id)
     }
     init {
     ensureNeverFrozen()
     }
     }
     

     View Slide

149. class DbModel(private val dbId: Int) {
     fun showDbStuff() = mainScope.launch {
     val sd = loadDbInfo(dbId)
     println(sd)
     }
     private suspend fun loadDbInfo(id: Int) =
     withContext(workerDispatcher) {
     DB.find(id)
     }
     init {
     ensureNeverFrozen()
     }
     }
     

     View Slide

150. Flow, Channel, etc
     all freezable
     

     View Slide

151. Main Thread
     Other Thread (maybe?)
     Data
     Data
     Data
     Content
     Model
     iOS View
     Model
     

     View Slide

152. init {
     ensureNeverFrozen()
     mainScope.launch {
     query.asFlow()
     .map {
     assertNotMainThread()
     extractData(it)
     }
     .flowOn(ServiceRegistry.backgroundDispatcher)
     .collect { vt ->
     view?.let {
     it.update(vt)
     }
     }
     }
     }
     

     View Slide

153. init {
     ensureNeverFrozen()
     mainScope.launch {
     query.asFlow()
     .map {
     assertNotMainThread()
     extractData(it)
     }
     .flowOn(ServiceRegistry.backgroundDispatcher)
     .collect { vt ->
     view?.let {
     it.update(vt)
     }
     }
     }
     }
     

     View Slide

154. init {
     ensureNeverFrozen()
     mainScope.launch {
     query.asFlow()
     .map {
     assertNotMainThread()
     extractData(it)
     }
     .flowOn(ServiceRegistry.backgroundDispatcher)
     .collect { vt ->
     view?.let {
     it.update(vt)
     }
     }
     }
     }
     

     View Slide

155. init {
     ensureNeverFrozen()
     mainScope.launch {
     query.asFlow()
     .map {
     assertNotMainThread()
     extractData(it)
     }
     .flowOn(ServiceRegistry.backgroundDispatcher)
     .collect { vt ->
     view?.let {
     it.update(vt)
     }
     }
     }
     }
     

     View Slide

156. init {
     ensureNeverFrozen()
     mainScope.launch {
     query.asFlow()
     .map {
     assertNotMainThread()
     extractData(it)
     }
     .flowOn(ServiceRegistry.backgroundDispatcher)
     .collect { vt ->
     view?.let {
     it.update(vt)
     }
     }
     }
     }
     

     View Slide

157. init {
     ensureNeverFrozen()
     mainScope.launch {
     query.asFlow()
     .map {
     assertNotMainThread()
     extractData(it)
     }
     .flowOn(ServiceRegistry.backgroundDispatcher)
     .collect { vt ->
     view?.let {
     it.update(vt)
     }
     }
     }
     }
     

     View Slide

158. Community Libraries
     

     View Slide

159. Stately?
     

     View Slide

160. View Slide

161. View Slide

162. Stately v1 (Probably) deprecated
     a lot of it, anyway
     

     View Slide

163. @Deprecated
     Stately Collections
     

     View Slide

164. @Discouraged
     Stately Collections
     

     View Slide

165. January
     that whole “holidays” thing
     

     View Slide

166. github.com/Autodesk/coroutineworker
     

     View Slide

167. github.com/Badoo/reaktive
     

     View Slide

168. Next Steps
     

     View Slide

169. go.touchlab.co/knthreads
     

     View Slide

170. JetBrains Links
     • https://github.com/JetBrains/kotlin-native/blob/
     master/IMMUTABILITY.md
     • https://github.com/JetBrains/kotlin-native/blob/
     master/CONCURRENCY.md
     • https://www.youtube.com/watch?v=nw6YTfEyfO0
     

     View Slide

171. go.touchlab.co/mtco
     

     View Slide

172. Kotlin Koans
     for native and state
     

     View Slide

173. January
     or whenever MT coroutines are stable(ish)
     

     View Slide

174. Try out MT Coroutines?
     some assembly required
     

     View Slide

175. Build and deploy local
     assuming no updates from kotlinx.coroutines
     

     View Slide

176. > git clone -b native-mt \
     https://github.com/Kotlin/kotlinx.coroutines.git
     > cd kotlinx.coroutines/
     > ./gradlew build publishToMavenLocal
     

     View Slide

177. Samples
     from slides
     https://github.com/kpgalligan/MTCoroutines
     Droidcon app
     https://github.com/touchlab/DroidconKotlin/
     (look for the kpg/mt_coroutines branch)
     

     View Slide

178. KMP evaluation kit
     

     View Slide

179. KaMP-Kit
     go.touchlab.co/KaMP-Kit
     

     View Slide

180. Stickers!
     

     View Slide

181. #KotlinConf
     THANK YOU
     AND
     REMEMBER
     TO VOTE
     Kevin Galligan @kpgalligan
     

     View Slide

182. Thanks!
     @kpgalligan
     touchlab.co
     

     View Slide

183. Thanks!
     @kpgalligan
     touchlab.co
     Join the team
     !
     

     View Slide