Kotlin Native Concurrency Explained (Kotlinconf 2019)

Explain the Kotlin native concurrency model, best practices, and tools and libraries you should check out.

Transcript

1. Copenhagen Denmark KOTLIN NATIVE CONCURRENCY EXPLAINED KEVIN GALLIGAN @kpgalligan
2. Touchlab
4. [Your Org Here] reach out if interested, obv
7. kevin@touchlab.co @kpgalligan
10. I still have a talk! and then…
11. multithreaded coroutines!
12. Intro
13. What is Kotlin Multiplatform?
14. kot·lin mul·ti·plat·form /ˌkätˈlin məltiˈplatfôrm,ˌkätˈlin məltīˈplatfôrm/ noun noun: kotlin multiplatform 1.optional, natively-integrated, open-source, code sharing platform, based on the popular, modern language kotlin. facilitates non-ui logic availability on many platforms.
15. modern
16. Saner Concurrency? concurrency is hard
18. Kotlin Native model is controversial lots of disagreement
19. JVM model is controversial many other platforms don’t let you do that
20. JVM model is controversial many other platforms don’t let you do that
21. JVM model is controversial many other platforms don’t let you do that
22. This talk will cover •How the model works •My thoughts on concurrency and best practices •Libraries and future stuff (MT coroutines!)
23. This talk will not cover •Me “selling” the model •Me complaining about the model •Debating various perspectives from the community
24. Also… no “getting around” rules
25. You need to understand it libraries won’t hide all details
26. important to relax
27. Kotlin Native Rules
28. 1) Mutable State = one thread thread confined
29. 2) Immutable state = many threads no changes mean no problems
30. kevin@touchlab.co @kpgalligan
31. Internalize The Rules mutable = 1/immutable = many
32. object BigGlobalState { var statusString = "" }
33. Don’t manage concurrent state no “synchronized”, “volatile”, etc
34. JVM Trusts You native does not
35. Runtime Verification
36. Do Things Differently not “losing” anything
37. (jvm strict mode would be nice)
38. You can break rules too just not a great idea
39. OK. Code please?
40. Worker kn concurrency queue
41. Similar to ExecutorService Handler/MessageQueue/Looper on Android
42. Worker job queue thread ??? execute
43. Worker job queue thread ??? execute
44. Worker job queue thread ??? execute
45. Worker job queue thread ???
46. Worker job queue thread ???
47. Worker job queue thread ???
48. Worker job queue thread ???
49. val worker = Worker.start() worker.execute(TransferMode.SAFE, {"Hello "}) { println(it) }
50. val worker = Worker.start() worker.execute(TransferMode.SAFE, {"Hello "}) { println(it) }
51. val worker = Worker.start() worker.execute(TransferMode.SAFE, {"Hello "}) { println(it) }
52. Hello
53. val worker = Worker.start() worker.execute(TransferMode.SAFE, {"Hello "}) { println(it) }
54. public fun <T1, T2> execute( mode: TransferMode, producer: () -> T1, job: (T1) -> T2): Future<T2> { //... }
55. public fun <T1, T2> execute( mode: TransferMode, producer: () -> T1, job: (T1) -> T2): Future<T2> { //... }
56. public fun <T1, T2> execute( mode: TransferMode, producer: () -> T1, job: (T1) -> T2): Future<T2> { //... }
57. val worker = Worker.start() worker.execute(TransferMode.SAFE, {"Hello "}) { println(it) }
58. public fun <T1, T2> execute( mode: TransferMode, producer: () -> T1, job: (T1) -> T2): Future<T2> { //... }
59. val worker = Worker.start() worker.execute(TransferMode.SAFE, {"Hello "}) { println(it) }.result
60. public fun <T1, T2> execute( mode: TransferMode, producer: () -> T1, job: (T1) -> T2): Future<T2> { //... }
61. producer: () -> T1
62. {"Hello "} (Unit)->String
63. data class SomeData(val s:String, val i:Int) fun printStuff2() { worker.execute(TransferMode.SAFE, {SomeData("Hello ", 2)}) { println(it) }.result }
64. data class SomeData(val s:String, val i:Int) {SomeData("Hello ", 2)} (Unit)->SomeData
65. data class SomeData(val s:String, val i:Int) fun printStuff2() { worker.execute(TransferMode.SAFE, {SomeData("Hello ", 2)}) { println(it) }.result }
66. fun printStuff2() { val someData = SomeData("Hello ", 2) worker.execute(TransferMode.SAFE, { someData }) { println(it) }.result }
67. fun printStuff2() { val someData = SomeData("Hello ", 2) worker.execute(TransferMode.SAFE, { someData }) { println(it) }.result }
68. fun printStuff2() { val someData = SomeData("Hello ", 2) worker.execute(TransferMode.SAFE, { someData }) { println(it) }.result } kotlin.IllegalStateException: Illegal transfer state
69. fun printStuff2() { val someData = SomeData("Hello ", 2) worker.execute(TransferMode.SAFE, { someData }) { println(it) }.result } execute wants to transfer someData between threads
70. fun printStuff2() { val someData = SomeData("Hello ", 2) worker.execute(TransferMode.SAFE, { someData }) { println(it) }.result } checks result (and children) for external refs
71. fun printStuff2() { val someData = SomeData("Hello ", 2) worker.execute(TransferMode.SAFE, { someData }) { println(it) }.result } local is still a ref till the end
72. data class SomeData(val s:String, val i:Int) fun printStuff2() { val someData = SomeData("Hello ", 2) worker.execute(TransferMode.SAFE, { someData }) { println(it) }.result } Mutable state?
73. freeze() runtime immutability designation
74. public fun <T> T.freeze(): T {...}
75. Freeze • Recursively freezes everything • One-way (no “unfreeze”) • Data may be shared between threads Int String Float Float String Int MoreData val strData var width SomeData val moreData var count
76. data class SomeData(val s:String, val i:Int) fun printStuff2() { val someData = SomeData("Hello ", 2) worker.execute(TransferMode.SAFE, { someData }) { println(it) }.result }
77. data class SomeData(val s:String, val i:Int) fun printStuff2() { val someData = SomeData("Hello ", 2) worker.execute(TransferMode.SAFE, { someData.freeze() }) { println(it) }.result }
78. data class SomeData(val s:String, val i:Int) fun printStuff2() { val someData = SomeData("Hello ", 2) worker.execute(TransferMode.SAFE, { someData.freeze() }) { println(it) }.result }
79. data class SomeData(val s:String, val i:Int) fun printStuff2() { val someData = SomeData("Hello ", 2) worker.execute(TransferMode.SAFE, { someData.freeze() }) { println(it) }.result }
80. data class SomeData(val s:String, val i:Int) fun printStuff2() { val someData = SomeData("Hello ", 2).freeze() worker.execute(TransferMode.SAFE, { }) { println(someData) }.result } Worker.execute must take an unbound, non-capturing function or lambda
81. fun background(block:(Unit)->Unit){ worker.execute(TransferMode.SAFE, {}, block) }
82. fun background(block:(Unit)->Unit){ worker.execute(TransferMode.SAFE, {}, block) } Worker.execute must take an unbound, non-capturing function or lambda
83. fun background(block:(Unit)->Unit){ worker.execute(TransferMode.SAFE, {}, block.freeze()) } Worker.execute must take an unbound, non-capturing function or lambda
84. fun background(block:()->Unit){ worker.execute(TransferMode.SAFE, {block}) { it() } } kotlin.IllegalStateException: Illegal transfer state
85. fun background(block:()->Unit){ worker.execute(TransferMode.SAFE, {block.freeze()}) { it() } }
86. fun printStuffBg() { val someData = SomeData("Hello ", 2) background { println(someData) } }
87. fun printStuffBg() { val someData = SomeData("Hello ", 2) background { println(someData) } }
88. fun printStuffBg() { val someData = SomeData("Hello ", 2) background { println(someData) } }
89. class CounterModel { var count = 0 fun countClicked() { count++ } }
90. class CounterModel { var count = 0 fun countClicked() { background { count++ } } }
91. class CounterModel { var count = 0 fun countClicked() { background { count++ } } }
92. class CounterModel { var count = 0 fun countClicked() { background { count++ } } }
93. class CounterModel { var count = 0 fun countClicked() { background { count++ } } }
94. class CounterModel { var count = 0 fun countClicked() { background { count++ } } } InvalidMutabilityException: mutation attempt of frozen sample.CounterModel
95. Functions Have State careful what you capture
96. Global State some special rules
97. val mainOnly = SomeData("a", 1)
98. val mainOnly = SomeData("a", 1) object GlobalObject { val data = SomeData("b", 2) }
99. @ThreadLocal val mainOnly = SomeData("a", 1) @ThreadLocal object GlobalObject { val data = SomeData("b", 2) }
100. @SharedImmutable val mainOnly = SomeData("a", 1) @ThreadLocal object GlobalObject { val data = SomeData("b", 2) }
101. Why Special Rules? you can access it from anywhere
102. Tips and New Stuff
103. Debugging Issues living with the new model
104. InvalidMutabilityException changing frozen state
105. class CounterModel { init { ensureNeverFrozen() } var count = 0 fun countClicked() { background { count++ } } }
106. Rethinking architecture intentional mutability
107. Main Thread Other Thread (maybe?) Data Data Data Content Model iOS View Model
108. Atomics mutating frozen state
109. class CounterModel { val count = AtomicInt(0) fun countClicked() { background { count.value++ } } }
110. class CounterModel { val count = AtomicInt(0) fun countClicked() { background { count.value++ } } }
111. class CounterModel { val count = AtomicInt(0) fun countClicked() { background { count.value++ } } }
112. val atom = AtomicReference<SomeData>(SomeData("", 0))
113. class SomeModel { val atom = atomic(SomeData("abc", 123)) fun update(sd:SomeData){ atom.value = sd } init { freeze() } }
114. class SomeModel { val atom = atomic(SomeData("abc", 123)) fun update(sd:SomeData){ atom.value = sd } init { freeze() } }
115. class SomeModel { val atom = atomic(SomeData("abc", 123)) fun update(sd:SomeData){ atom.value = sd } init { freeze() } }
116. class SomeModel { val atom = atomic(SomeData("abc", 123)) fun update(sd:SomeData){ atom.value = sd } init { freeze() } }
117. Atomic-fu/Stately
118. Don’t use too many atomics AtomicRef can leak memory
119. DetachedObjectGraph transfer state
120. DetachedObjectGraph transfer state
121. Worker dispatched state mutable state thread local
122. State Thread State Worker ??? State Container Data
123. State Thread State Worker ??? State Container ??? Data
124. Multiplatform State freeze in common
125. freeze() is native only
126. expect fun <T> T.freeze(): T expect fun <T> T.isFrozen(): Boolean expect fun <T> T.isNativeFrozen(): Boolean expect fun Any.ensureNeverFrozen()
127. expect fun <T> T.freeze(): T expect fun <T> T.isFrozen(): Boolean expect fun <T> T.isNativeFrozen(): Boolean expect fun Any.ensureNeverFrozen()
128. Multithreaded Coroutines
129. General Thoughts what does this mean?
130. State Model Clarity strict mode is sticking around
131. Library Development was a blocker
132. KMP is more “real”
134. Disclaimers just fyi
136. Overview coroutine always bound to a single thread create with newSingleThreadContext on native that uses Worker Default has single background thread Main defined for Apple targets Windows/Linux ¯\_(ツ)_/¯
137. Switching Threads use withContext (or equivalents) data passed in (or captured) is frozen data returned is frozen
138. fun printStuffBg() { val someData = SomeData("Hello ", 2) background { println(someData) } }
139. suspend fun printStuffBg() { val someData = SomeData("Hello ", 2) withContext(workerDispatcher) { println(someData) } }
140. suspend fun printStuffBg() { val someData = SomeData("Hello ", 2) withContext(workerDispatcher) { println(someData) } }
141. suspend fun printStuffForeground() { val someData = SomeData("Hello ", 2) val someMoreData = withContext(workerDispatcher) { SomeData("${someData.s}, Dogs!", someData.i + 1) } println(someMoreData) println("I am frozen? ${someMoreData.isFrozen()}") }
142. suspend fun printStuffForeground() { val someData = SomeData("Hello ", 2) val someMoreData = withContext(workerDispatcher) { SomeData("${someData.s}, Dogs!", someData.i + 1) } println(someMoreData) println("I am frozen? ${someMoreData.isFrozen()}") }
143. suspend fun printStuffForeground() { val someData = SomeData("Hello ", 2) val someMoreData = withContext(workerDispatcher) { SomeData("${someData.s}, Dogs!", someData.i + 1) } println(someMoreData) println("I am frozen? ${someMoreData.isFrozen()}") }
144. SomeData(s=Hello , Dogs!, i=3) I am frozen? true
145. class DbModel(private val dbId:Int){ fun showDbStuff() = mainScope.launch { val sd = withContext(workerDispatcher){ DB.find(dbId) } println(sd) } init { ensureNeverFrozen() } }
146. class DbModel(private val dbId:Int){ fun showDbStuff() = mainScope.launch { val capturedId = dbId val sd = withContext(workerDispatcher){ DB.find(capturedId) } println(sd) } init { ensureNeverFrozen() } }
147. class DbModel(private val dbId: Int) { fun showDbStuff() = mainScope.launch { val sd = loadDbInfo(dbId) println(sd) } private suspend fun loadDbInfo(id: Int) = withContext(workerDispatcher) { DB.find(id) } init { ensureNeverFrozen() } }
148. class DbModel(private val dbId: Int) { fun showDbStuff() = mainScope.launch { val sd = loadDbInfo(dbId) println(sd) } private suspend fun loadDbInfo(id: Int) = withContext(workerDispatcher) { DB.find(id) } init { ensureNeverFrozen() } }
149. class DbModel(private val dbId: Int) { fun showDbStuff() = mainScope.launch { val sd = loadDbInfo(dbId) println(sd) } private suspend fun loadDbInfo(id: Int) = withContext(workerDispatcher) { DB.find(id) } init { ensureNeverFrozen() } }
150. Flow, Channel, etc all freezable
151. Main Thread Other Thread (maybe?) Data Data Data Content Model iOS View Model
152. init { ensureNeverFrozen() mainScope.launch { query.asFlow() .map { assertNotMainThread() extractData(it) } .flowOn(ServiceRegistry.backgroundDispatcher) .collect { vt -> view?.let { it.update(vt) } } } }
153. init { ensureNeverFrozen() mainScope.launch { query.asFlow() .map { assertNotMainThread() extractData(it) } .flowOn(ServiceRegistry.backgroundDispatcher) .collect { vt -> view?.let { it.update(vt) } } } }
154. init { ensureNeverFrozen() mainScope.launch { query.asFlow() .map { assertNotMainThread() extractData(it) } .flowOn(ServiceRegistry.backgroundDispatcher) .collect { vt -> view?.let { it.update(vt) } } } }
155. init { ensureNeverFrozen() mainScope.launch { query.asFlow() .map { assertNotMainThread() extractData(it) } .flowOn(ServiceRegistry.backgroundDispatcher) .collect { vt -> view?.let { it.update(vt) } } } }
156. init { ensureNeverFrozen() mainScope.launch { query.asFlow() .map { assertNotMainThread() extractData(it) } .flowOn(ServiceRegistry.backgroundDispatcher) .collect { vt -> view?.let { it.update(vt) } } } }
157. init { ensureNeverFrozen() mainScope.launch { query.asFlow() .map { assertNotMainThread() extractData(it) } .flowOn(ServiceRegistry.backgroundDispatcher) .collect { vt -> view?.let { it.update(vt) } } } }
158. Community Libraries
159. Stately?
162. Stately v1 (Probably) deprecated a lot of it, anyway
163. @Deprecated Stately Collections
164. @Discouraged Stately Collections
165. January that whole “holidays” thing
166. github.com/Autodesk/coroutineworker
167. github.com/Badoo/reaktive
168. Next Steps
169. go.touchlab.co/knthreads
170. JetBrains Links • https://github.com/JetBrains/kotlin-native/blob/ master/IMMUTABILITY.md • https://github.com/JetBrains/kotlin-native/blob/ master/CONCURRENCY.md • https://www.youtube.com/watch?v=nw6YTfEyfO0
171. go.touchlab.co/mtco
172. Kotlin Koans for native and state
173. January or whenever MT coroutines are stable(ish)
174. Try out MT Coroutines? some assembly required
175. Build and deploy local assuming no updates from kotlinx.coroutines
176. > git clone -b native-mt \ https://github.com/Kotlin/kotlinx.coroutines.git > cd kotlinx.coroutines/ > ./gradlew build publishToMavenLocal
177. Samples from slides https://github.com/kpgalligan/MTCoroutines Droidcon app https://github.com/touchlab/DroidconKotlin/ (look for the kpg/mt_coroutines branch)
178. KMP evaluation kit
179. KaMP-Kit go.touchlab.co/KaMP-Kit
180. Stickers!
181. #KotlinConf THANK YOU AND REMEMBER TO VOTE Kevin Galligan @kpgalligan
182. Thanks! @kpgalligan touchlab.co
183. Thanks! @kpgalligan touchlab.co Join the team !