Channels & Flows in Practice

In this talk, Mohit will present on how to use constructs such as channels and flows in various use cases. These use cases are modeling streams that are hot or cold, multicasting, polling, handling errors and testing. A Channel is similar to a blocking queue or a subject from RxJava. There are different types of channels that fit different use cases. On the other hand, a Flow represents a cold stream. We’ll look at how they work and thier APIs for consuming and producing a stream. I also cover how to go by testing your code using the coroutines testing library that is provided. I’ll share various helper extensions that you could use to improve your testing. By the end of this talk, you will have expanded your toolset for solving problems using Coroutines.

Transcript

1. Channels & Flows 
 in Practice Mohit Sarveiya /heyitsmohit twitter.com
2. Channels & Flows in Practice • Channels • Inside Channels • Flows • Implementing & Testing Polling
3. Coroutine A Problem Coroutine B
4. Coroutine A Problem Coroutine B 1 2 3
5. Passing Data • Deferred • Channels • Flows
6. Channels
7. What is a Channel? Item Channel
 
 Buffer (optional) Item
8. What is a Channel? interface Channel<E> : SendChannel<E>, ReceiveChannel<E> interface ReceiveChannel<out E> { suspend fun receive(): E } interface SendChannel<in E> { suspend fun send(element: E) }
9. Channel Types • Rendezvous • Buffered • Unlimited • Conflated • Broadcast Channel
10. Creating Channel fun <E> Channel(capacity: Int = RENDEZVOUS) = when (capacity) { RENDEZVOUS "-> RendezvousChannel() UNLIMITED "-> LinkedListChannel() CONFLATED "-> Con"flatedChannel() BUFFERED "-> ArrayChannel(CHANNEL_DEFAULT_CAPACITY) else "-> ArrayChannel(capacity) }
11. Using Channels runBlocking {
 val channel = Channel<Int>() launch { channel.send(1) } val num = channel.receive() } Consumer Producer
12. Rendezvous Channel • No Buffer • Waits for receiver to send data
13. Rendezvous Channel runBlocking { val channel = Channel<Int>() launch { channel.send(1) channel.send(2) } }
14. Rendezvous Channel runBlocking { val channel = Channel<Int>() launch { channel.send(1) channel.send(2) } } SUSPEND
15. Rendezvous Channel runBlocking { val channel = Channel<Int>() launch { channel.send(1) channel.send(2) }
 val num = channel.receive() } Receiver
16. runBlocking { val channel = Channel<Int>() launch { channel.send(1) channel.send(2) }
 val num = channel.receive() } Rendezvous Channel RESUME Receive value 1
17. runBlocking { val channel = Channel<Int>() launch { channel.send(1) channel.send(2) }
 val num = channel.receive() } Rendezvous Channel SUSPEND Received value 1
18. Buffered Channel • Buffer with specific capacity. • Caches sent items in Array
19. runBlocking { val channel = Channel<Int>(capacity = 3) launch { channel.send(1) channel.send(2) channel.send(3) } } Buffered Channel 1 2 3 Buffer
20. runBlocking { val channel = Channel<Int>(capacity = 3) launch { channel.send(1) channel.send(2) channel.send(3)
 channel.send(4) } } Buffered Channel SUSPEND 1 2 3 Buffer
21. Unlimited Channel runBlocking { val channel = Channel<Int>(Channel.UNLIMITED) launch { channel.send(1) channel.send(2) channel.send(3) } }
22. Broadcast Channel Coroutine A Coroutine B Coroutine C Coroutine D
23. Broadcast Channel • Buffered • Unlimited • Conflated
24. Broadcast Channel runBlocking { val channel = broadcast<Int>(capacity = 10, block = { send(1) }) val consumer1 = channel.openSubscription() val consumer2 = channel.openSubscription() launch { println(consumer1.receive()) "// 1 println(consumer2.receive()) "// 1 }
25. Creating Producers runBlocking {
 val channel = Channel<Int>() launch { for (i in 1"..5) { channel.send(i) } channel.close() } } Producer
26. Producer Builder • Creates Coroutine with Channel • DSL for creating channels
27. Producer Builder fun <E> CoroutineScope.produce( context: CoroutineContext, capacity: Int = 0, block: suspend ProducerScope<E>.() "-> Unit ): ReceiveChannel<E>
28. Creating Producers suspend fun CoroutineScope.producer(): ReceiveChannel<Int> = produce { for (i in 1"..5) { send(i) } close() }
29. Creating Producers runBlocking {
 val channel: ReceiveChannel<Int> = producer() }
 suspend fun CoroutineScope.producer(): ReceiveChannel<Int> = produce { for (i in 1"..5) { send(i) } close() }
30. Consuming Producers runBlocking {
 val channel: ReceiveChannel<Int> = producer() for (num in channel) { println(num) } }
 suspend fun CoroutineScope.producer(): ReceiveChannel<Int> = produce { for (i in 1"..5) { send(i) } close() }
31. Inside Channels
32. Inside Channels
33. Inside Channels 
 val channel = Channel<Int>() What happens under the hood?
34. Inside Channels • Queue (Doubly Linked List) • Buffer (Array)
35. Channel States • Empty Queue • Send Queued • Send Buffered • Closed
36. Channel States EmptyQueue LockFreeLinkedListNode SendBuffered SendQueued Closed
37. Query Channel State • Get most recent state • Size of buffer
38. class AbstractSendChannel { fun toString() = {$queueDebugStateString}$bufferDebugString” } Query Channel State Current Channel State Buffer Size
39. Rendezvous Channel val channel = Channel<Int>() EmptyQueue Queue
40. Rendezvous Channel val channel = Channel<Int>() channel.send(1)
41. Rendezvous Channel class SendElement( val data: Any?, val cont: CancellableContinuation<Unit> ) : LockFreeLinkedListNode()
42. Rendezvous Channel val channel = Channel<Int>() channel.send(1) SendElement(1) Queue
43. Rendezvous Channel val channel = Channel<Int>() channel.send(1)
 println(channel)
 
 Output: 
 RendezvousChannel{SendQueued}
44. Rendezvous Channel val channel = Channel<Int>() channel.send(1)
 channel.receive() EmptyQueue Queue
45. Unlimited Channel runBlocking { val channel = Channel<Int>(Channel.UNLIMITED) launch { channel.send(1) channel.send(2) channel.send(3) } } SendBuffered(1) SendBuffered(2) SendBuffered(3)
46. Consuming Values runBlocking {
 val channel: ReceiveChannel<Int> = producer() for (num in channel) { println(num) } }
47. Coroutine A Consuming Values Coroutine B 1 2 3 x Closed token
48. Channel Iterator interface ChannelIterator<out E> { suspend fun hasNextSuspend(): E fun next(): E }
49. Channel Iterator fun hasNextSuspend() = suspendCancellable { cont "-> while (true) { val result = channel.pollInternal() this.result = result if (result is Closed""<*>) {
 cont.resume(false) } else { cont.resume(true) } …
 } }
50. Coroutine A Channel Iterator Coroutine B EmptyQueue Channel Queue
51. Coroutine A Channel Iterator Coroutine B 1 SendElement(1) Channel Queue hasNext()
52. Coroutine A Channel Iterator Coroutine B 1 SendElement(1) Channel Queue next()
53. Coroutine A Channel Iterator Coroutine B 1 SendElement(1) Channel Queue next()
54. Coroutine A Channel Iterator Coroutine B 1 SendElement(3) Channel Queue next() 2 3
55. Coroutine A Channel Iterator Coroutine B 1 Closed Channel Queue 2 3 x Closed token
56. Coroutine A Channel Iterator Coroutine B 1 2 3 Suspended No Closed token
57. Flows
58. Coroutine A Cold Stream Coroutine B
59. Coroutine A Cold Stream Coroutine B
60. Coroutine A Cold Stream Coroutine B 1 2 3
61. Creating Flows • flowOf(…) • flow { … } • channelFlow { … }
62. Creating Flows runBlocking { val "flow: "Flow<Int> = "flowOf(1, 2, 3) } fun <T> "flowOf(vararg elements: T): "Flow<T>
63. Collecting Flows runBlocking { val "flow: "Flow<Int> = "flowOf(1, 2, 3) "flow.collect { println(it) "// 1 2 3 } }
64. Creating Flows val "flow = "flow { emit(1) emit(2) emit(3) } "flow.collect { println(it) "// 1 2 3 }
65. Creating Flows val "flow = "flow { emit(1) emit(2) emit(3) } "flow.collect { println(it) } "// 1 2 3 "flow.collect { println(it) } "// 1 2 3
66. Terminal Operators • Collect • Single • Reduce • toList
67. Single Operator Collects only one value. val "flow = "flowOf(1) val num: Int = "flow.single()
68. Single Operator Collects only one value. val "flow = "flowOf(1,2,3) val num: Int = "flow.single() java.lang.IllegalStateException: Expected only one element
69. Flow Operators • Map, Zip • Buffer • Retry • Take • delayEach
70. DelayEach runBlocking { val "flow = "flowOf(1,2,3).delayEach(1000) "flow.collect { println(it) } } Operators are not suspending functions
71. DelayEach runBlocking { val "flow = get"Flow() "flow.collect { println(it) } } fun get"Flow() = "flowOf(1,2,3).delayEach(1000)
72. Flow Context runBlocking { val "flow = "flow { emit(1) emit(2) emit(3) } "flow.collect { println(it) } } Which dispatcher does this run on?
73. Flow Context runBlocking { val "flow = "flow { emit(1) emit(2) emit(3) } "flow.collect { println(it) } } BlockingEventLoop BlockingEventLoop
74. Flow Context runBlocking { val "flow = "flow { emit(1) emit(2) emit(3) }
 "flow ."flowOn(Dispatchers.IO) .collect { println(it) } } Dispatchers.IO BlockingEventLoop
75. Flow Constraints • Context Preservation • Exception Handling
76. Flow Constraints Emit from the same coroutine val "flow = "flow { launch(Dispatchers.IO) { emit(1) } } Flow invariant is violated: emission from another coroutine is detected
77. Channel Flow Builder val "flow = channel"Flow { launch(Dispatchers.IO) { send(1) } launch(Dispatchers.IO) { send(2) } } Flow Channel
78. Channel Flow Builder class Channel"FlowBuilder<T>( val block: suspend ProducerScope<T>.() "-> Unit, capacity: Int = BUFFERED ) : Channel"Flow<T>(context, capacity) Uses Buffered Channel
79. Exception Handling val "flow = "flow { try { makeRequest() } catch (e: IOException) { } }
80. Exception Handling fun "Flow<T>.catch(action: suspend "FlowCollector<T>.(cause: Throwable) "-> Unit ): "Flow<T> Emit exception in a wrapper
81. Exception Handling sealed class ApiResponse<T> { data class Success<T>(val data: T): ApiResponse<T>() data class Error<T>(val t: Throwable): ApiResponse<T>() }
82. Exception Handling val "flow = "flow<ApiResponse<Data">> { val data = makeRequest() emit(ApiResponse.Success(data)) }.catch { emit(ApiResponse.Error(it)) }
83. Channel to Flow Bridge runBlocking { val bufferedChannel = Channel<Int>(capacity = 10) val broadcastChannel = broadcast(capacity = 10) { send(1) } val "flow1: "Flow<Int> = bufferedChannel.consumeAs"Flow() val "flow2 = broadcastChannel.as"Flow() }
84. Use Case: Polling
85. Polling 1 second /feed Data
86. Polling interface ApiService { suspend fun getData(): Data }
87. Polling interface Poller { fun poll(delay: Long): "Flow<Data> fun close() }
88. Polling class CoroutinePoller( val apiService: ApiService, val dispatcher: CoroutineDispatcher ): Poller { override fun poll(delay: Long): "Flow<Data> { } override fun close() { } }
89. Polling class CoroutinePoller(…): Poller { override fun poll(delay: Long): "Flow<Data> { return channel"Flow { while (!isClosedForSend) { val data = apiService.getData() send(data) delay(delay) } }."flowOn(dispatcher) } }
90. Polling class CoroutinePoller("..): Poller { override fun cancel() { dispatcher.cancel() } }
91. Polling Usage class MyViewModel(val poller: Poller): ViewModel { fun startPolling() { viewModelScope.launch { val "flow: "Flow<Data> = poller.poll(1000) "flow.collect { data !-> processData(data) } } } }
92. Testing Strategy Create coroutine for test • Run blocking Coroutine
 • Coroutine with test dispatcher and scope
93. Testing
94. Testing • Dispatchers.setMain • runBlockingTest • TestCoroutineDispatcher • advanceTimeBy, pauseDispatcher, resumeDispatcher
95. Testing
96. Testing Strategy • Mock API Service • Advance time every second • Assert on Flow
97. Testing @Test fun `should poll every second`() = runBlockingTest { val data = mock<Data>() val apiService = mock<ApiService>() { onBlocking { getData() } doReturn data } val testDispatcher = TestCoroutineDispatcher() val poller = CoroutinePoller(apiService, testDispatcher) }
98. Testing @Test fun `should poll every second`() = runBlockingTest { val "flow = poller.poll(1_000) launch { val dataList = "flow.take(2).toList() assertEquals(dataList.size, 2) } testDispatcher.advanceTimeBy(2_000) poller.close() }
99. Testing
100. Flow Assert suspend fun <T> "Flow<T>.test( validate: suspend "FlowAssert<T>.() "-> Unit ) { }
101. Flow Assert sealed class Event<out T> { object Complete : Event<Nothing>() data class Error(val t: Throwable) : Event<Nothing>() data class Item<T>(val item: T) : Event<T>() }
102. Flow Assert suspend fun <T> "Flow<T>.test( validate: suspend "FlowAssert<T>.() "-> Unit ) { coroutineScope { val events = Channel<Event<T">>(UNLIMITED) launch { try { collect { item !-> events.send(Event.Item(item)) } Event.Complete } }
103. Flow Assert suspend fun <T> "Flow<T>.test( validate: suspend "FlowAssert<T>.() "-> Unit ) { coroutineScope { val events = Channel<Event<T">>(UNLIMITED) launch { try {
 … } catch (t: Throwable) { Event.Error(t) } if (terminalEvent "!= null) { events.send(terminalEvent) } events.close()
104. Flow Assert class FlowAssert<T>(val events: Channel<Event<T>>) { suspend fun expectItem(): T suspend fun expectComplete() suspend fun expectError(): Throwable … }
105. Flow Assert @Test fun `should poll every second`() = runBlockingTest { val "flow = poller.poll(1_000) launch { "flow.test { assertEquals(expectItem(), data) assertEquals(expectItem(), data) expectComplete() } } testDispatcher.advanceTimeBy(2_000) poller.close()
106. Resources • Kotlinx-coroutines-test
 https://github.com/Kotlin/kotlinx.coroutines/tree/master/kotlinx-coroutines-test • Coroutine JUnit Test Rule
 https://craigrussell.io/2019/11/unit-testing-coroutine-suspend-functions-using-testcoroutinedispatcher/ • Corbind 
 https://github.com/LDRAlighieri/Corbind • FlowAssert 
 
 https://github.com/cashapp/sqldelight/blob/master/extensions/coroutines-extensions/src 
 commonTest/kotlin/com/squareup/sqldelight/runtime/coroutines/FlowAssert.kt
107. Thank you! twitter.com/heyitsmohit