Channels & Flows in Practice

B3f560d34c14a9113e5024bc34ac26a0?s=47 Mohit S
January 16, 2020

Channels & Flows in Practice

In this talk, Mohit will present on how to use constructs such as channels and flows in various use cases. These use cases are modeling streams that are hot or cold, multicasting, polling, handling errors and testing. A Channel is similar to a blocking queue or a subject from RxJava. There are different types of channels that fit different use cases. On the other hand, a Flow represents a cold stream. We’ll look at how they work and thier APIs for consuming and producing a stream. I also cover how to go by testing your code using the coroutines testing library that is provided. I’ll share various helper extensions that you could use to improve your testing. By the end of this talk, you will have expanded your toolset for solving problems using Coroutines.

B3f560d34c14a9113e5024bc34ac26a0?s=128

Mohit S

January 16, 2020
Tweet

Transcript

  1. Channels & Flows 
 in Practice Mohit Sarveiya /heyitsmohit twitter.com

  2. Channels & Flows in Practice • Channels • Inside Channels

    • Flows • Implementing & Testing Polling
  3. Coroutine A Problem Coroutine B

  4. Coroutine A Problem Coroutine B 1 2 3

  5. Passing Data • Deferred • Channels • Flows

  6. Channels

  7. What is a Channel? Item Channel
 
 Buffer (optional) Item

  8. What is a Channel? interface Channel<E> : SendChannel<E>, ReceiveChannel<E> interface

    ReceiveChannel<out E> { suspend fun receive(): E } interface SendChannel<in E> { suspend fun send(element: E) }
  9. Channel Types • Rendezvous • Buffered • Unlimited • Conflated

    • Broadcast Channel
  10. Creating Channel fun <E> Channel(capacity: Int = RENDEZVOUS) = when

    (capacity) { RENDEZVOUS "-> RendezvousChannel() UNLIMITED "-> LinkedListChannel() CONFLATED "-> Con"flatedChannel() BUFFERED "-> ArrayChannel(CHANNEL_DEFAULT_CAPACITY) else "-> ArrayChannel(capacity) }
  11. Using Channels runBlocking {
 val channel = Channel<Int>() launch {

    channel.send(1) } val num = channel.receive() } Consumer Producer
  12. Rendezvous Channel • No Buffer • Waits for receiver to

    send data
  13. Rendezvous Channel runBlocking { val channel = Channel<Int>() launch {

    channel.send(1) channel.send(2) } }
  14. Rendezvous Channel runBlocking { val channel = Channel<Int>() launch {

    channel.send(1) channel.send(2) } } SUSPEND
  15. Rendezvous Channel runBlocking { val channel = Channel<Int>() launch {

    channel.send(1) channel.send(2) }
 val num = channel.receive() } Receiver
  16. runBlocking { val channel = Channel<Int>() launch { channel.send(1) channel.send(2)

    }
 val num = channel.receive() } Rendezvous Channel RESUME Receive value 1
  17. runBlocking { val channel = Channel<Int>() launch { channel.send(1) channel.send(2)

    }
 val num = channel.receive() } Rendezvous Channel SUSPEND Received value 1
  18. Buffered Channel • Buffer with specific capacity. • Caches sent

    items in Array
  19. runBlocking { val channel = Channel<Int>(capacity = 3) launch {

    channel.send(1) channel.send(2) channel.send(3) } } Buffered Channel 1 2 3 Buffer
  20. runBlocking { val channel = Channel<Int>(capacity = 3) launch {

    channel.send(1) channel.send(2) channel.send(3)
 channel.send(4) } } Buffered Channel SUSPEND 1 2 3 Buffer
  21. Unlimited Channel runBlocking { val channel = Channel<Int>(Channel.UNLIMITED) launch {

    channel.send(1) channel.send(2) channel.send(3) } }
  22. Broadcast Channel Coroutine A Coroutine B Coroutine C Coroutine D

  23. Broadcast Channel • Buffered • Unlimited • Conflated

  24. Broadcast Channel runBlocking { val channel = broadcast<Int>(capacity = 10,

    block = { send(1) }) val consumer1 = channel.openSubscription() val consumer2 = channel.openSubscription() launch { println(consumer1.receive()) "// 1 println(consumer2.receive()) "// 1 }
  25. Creating Producers runBlocking {
 val channel = Channel<Int>() launch {

    for (i in 1"..5) { channel.send(i) } channel.close() } } Producer
  26. Producer Builder • Creates Coroutine with Channel • DSL for

    creating channels
  27. Producer Builder fun <E> CoroutineScope.produce( context: CoroutineContext, capacity: Int =

    0, block: suspend ProducerScope<E>.() "-> Unit ): ReceiveChannel<E>
  28. Creating Producers suspend fun CoroutineScope.producer(): ReceiveChannel<Int> = produce { for

    (i in 1"..5) { send(i) } close() }
  29. Creating Producers runBlocking {
 val channel: ReceiveChannel<Int> = producer() }


    suspend fun CoroutineScope.producer(): ReceiveChannel<Int> = produce { for (i in 1"..5) { send(i) } close() }
  30. Consuming Producers runBlocking {
 val channel: ReceiveChannel<Int> = producer() for

    (num in channel) { println(num) } }
 suspend fun CoroutineScope.producer(): ReceiveChannel<Int> = produce { for (i in 1"..5) { send(i) } close() }
  31. Inside Channels

  32. Inside Channels

  33. Inside Channels 
 val channel = Channel<Int>() What happens under

    the hood?
  34. Inside Channels • Queue (Doubly Linked List) • Buffer (Array)

  35. Channel States • Empty Queue • Send Queued • Send

    Buffered • Closed
  36. Channel States EmptyQueue LockFreeLinkedListNode SendBuffered SendQueued Closed

  37. Query Channel State • Get most recent state • Size

    of buffer
  38. class AbstractSendChannel { fun toString() = {$queueDebugStateString}$bufferDebugString” } Query Channel

    State Current Channel State Buffer Size
  39. Rendezvous Channel val channel = Channel<Int>() EmptyQueue Queue

  40. Rendezvous Channel val channel = Channel<Int>() channel.send(1)

  41. Rendezvous Channel class SendElement( val data: Any?, val cont: CancellableContinuation<Unit>

    ) : LockFreeLinkedListNode()
  42. Rendezvous Channel val channel = Channel<Int>() channel.send(1) SendElement(1) Queue

  43. Rendezvous Channel val channel = Channel<Int>() channel.send(1)
 println(channel)
 
 Output:

    
 RendezvousChannel{SendQueued}
  44. Rendezvous Channel val channel = Channel<Int>() channel.send(1)
 channel.receive() EmptyQueue Queue

  45. Unlimited Channel runBlocking { val channel = Channel<Int>(Channel.UNLIMITED) launch {

    channel.send(1) channel.send(2) channel.send(3) } } SendBuffered(1) SendBuffered(2) SendBuffered(3)
  46. Consuming Values runBlocking {
 val channel: ReceiveChannel<Int> = producer() for

    (num in channel) { println(num) } }
  47. Coroutine A Consuming Values Coroutine B 1 2 3 x

    Closed token
  48. Channel Iterator interface ChannelIterator<out E> { suspend fun hasNextSuspend(): E

    fun next(): E }
  49. Channel Iterator fun hasNextSuspend() = suspendCancellable { cont "-> while

    (true) { val result = channel.pollInternal() this.result = result if (result is Closed""<*>) {
 cont.resume(false) } else { cont.resume(true) } …
 } }
  50. Coroutine A Channel Iterator Coroutine B EmptyQueue Channel Queue

  51. Coroutine A Channel Iterator Coroutine B 1 SendElement(1) Channel Queue

    hasNext()
  52. Coroutine A Channel Iterator Coroutine B 1 SendElement(1) Channel Queue

    next()
  53. Coroutine A Channel Iterator Coroutine B 1 SendElement(1) Channel Queue

    next()
  54. Coroutine A Channel Iterator Coroutine B 1 SendElement(3) Channel Queue

    next() 2 3
  55. Coroutine A Channel Iterator Coroutine B 1 Closed Channel Queue

    2 3 x Closed token
  56. Coroutine A Channel Iterator Coroutine B 1 2 3 Suspended

    No Closed token
  57. Flows

  58. Coroutine A Cold Stream Coroutine B

  59. Coroutine A Cold Stream Coroutine B

  60. Coroutine A Cold Stream Coroutine B 1 2 3

  61. Creating Flows • flowOf(…) • flow { … } •

    channelFlow { … }
  62. Creating Flows runBlocking { val "flow: "Flow<Int> = "flowOf(1, 2,

    3) } fun <T> "flowOf(vararg elements: T): "Flow<T>
  63. Collecting Flows runBlocking { val "flow: "Flow<Int> = "flowOf(1, 2,

    3) "flow.collect { println(it) "// 1 2 3 } }
  64. Creating Flows val "flow = "flow { emit(1) emit(2) emit(3)

    } "flow.collect { println(it) "// 1 2 3 }
  65. Creating Flows val "flow = "flow { emit(1) emit(2) emit(3)

    } "flow.collect { println(it) } "// 1 2 3 "flow.collect { println(it) } "// 1 2 3
  66. Terminal Operators • Collect • Single • Reduce • toList

  67. Single Operator Collects only one value. val "flow = "flowOf(1)

    val num: Int = "flow.single()
  68. Single Operator Collects only one value. val "flow = "flowOf(1,2,3)

    val num: Int = "flow.single() java.lang.IllegalStateException: Expected only one element
  69. Flow Operators • Map, Zip • Buffer • Retry •

    Take • delayEach
  70. DelayEach runBlocking { val "flow = "flowOf(1,2,3).delayEach(1000) "flow.collect { println(it)

    } } Operators are not suspending functions
  71. DelayEach runBlocking { val "flow = get"Flow() "flow.collect { println(it)

    } } fun get"Flow() = "flowOf(1,2,3).delayEach(1000)
  72. Flow Context runBlocking { val "flow = "flow { emit(1)

    emit(2) emit(3) } "flow.collect { println(it) } } Which dispatcher does this run on?
  73. Flow Context runBlocking { val "flow = "flow { emit(1)

    emit(2) emit(3) } "flow.collect { println(it) } } BlockingEventLoop BlockingEventLoop
  74. Flow Context runBlocking { val "flow = "flow { emit(1)

    emit(2) emit(3) }
 "flow ."flowOn(Dispatchers.IO) .collect { println(it) } } Dispatchers.IO BlockingEventLoop
  75. Flow Constraints • Context Preservation • Exception Handling

  76. Flow Constraints Emit from the same coroutine val "flow =

    "flow { launch(Dispatchers.IO) { emit(1) } } Flow invariant is violated: emission from another coroutine is detected
  77. Channel Flow Builder val "flow = channel"Flow { launch(Dispatchers.IO) {

    send(1) } launch(Dispatchers.IO) { send(2) } } Flow Channel
  78. Channel Flow Builder class Channel"FlowBuilder<T>( val block: suspend ProducerScope<T>.() "->

    Unit, capacity: Int = BUFFERED ) : Channel"Flow<T>(context, capacity) Uses Buffered Channel
  79. Exception Handling val "flow = "flow { try { makeRequest()

    } catch (e: IOException) { } }
  80. Exception Handling fun "Flow<T>.catch(action: suspend "FlowCollector<T>.(cause: Throwable) "-> Unit ):

    "Flow<T> Emit exception in a wrapper
  81. Exception Handling sealed class ApiResponse<T> { data class Success<T>(val data:

    T): ApiResponse<T>() data class Error<T>(val t: Throwable): ApiResponse<T>() }
  82. Exception Handling val "flow = "flow<ApiResponse<Data">> { val data =

    makeRequest() emit(ApiResponse.Success(data)) }.catch { emit(ApiResponse.Error(it)) }
  83. Channel to Flow Bridge runBlocking { val bufferedChannel = Channel<Int>(capacity

    = 10) val broadcastChannel = broadcast(capacity = 10) { send(1) } val "flow1: "Flow<Int> = bufferedChannel.consumeAs"Flow() val "flow2 = broadcastChannel.as"Flow() }
  84. Use Case: Polling

  85. Polling 1 second /feed Data

  86. Polling interface ApiService { suspend fun getData(): Data }

  87. Polling interface Poller { fun poll(delay: Long): "Flow<Data> fun close()

    }
  88. Polling class CoroutinePoller( val apiService: ApiService, val dispatcher: CoroutineDispatcher ):

    Poller { override fun poll(delay: Long): "Flow<Data> { } override fun close() { } }
  89. Polling class CoroutinePoller(…): Poller { override fun poll(delay: Long): "Flow<Data>

    { return channel"Flow { while (!isClosedForSend) { val data = apiService.getData() send(data) delay(delay) } }."flowOn(dispatcher) } }
  90. Polling class CoroutinePoller("..): Poller { override fun cancel() { dispatcher.cancel()

    } }
  91. Polling Usage class MyViewModel(val poller: Poller): ViewModel { fun startPolling()

    { viewModelScope.launch { val "flow: "Flow<Data> = poller.poll(1000) "flow.collect { data !-> processData(data) } } } }
  92. Testing Strategy Create coroutine for test • Run blocking Coroutine


    • Coroutine with test dispatcher and scope
  93. Testing

  94. Testing • Dispatchers.setMain • runBlockingTest • TestCoroutineDispatcher • advanceTimeBy, pauseDispatcher,

    resumeDispatcher
  95. Testing

  96. Testing Strategy • Mock API Service • Advance time every

    second • Assert on Flow
  97. Testing @Test fun `should poll every second`() = runBlockingTest {

    val data = mock<Data>() val apiService = mock<ApiService>() { onBlocking { getData() } doReturn data } val testDispatcher = TestCoroutineDispatcher() val poller = CoroutinePoller(apiService, testDispatcher) }
  98. Testing @Test fun `should poll every second`() = runBlockingTest {

    val "flow = poller.poll(1_000) launch { val dataList = "flow.take(2).toList() assertEquals(dataList.size, 2) } testDispatcher.advanceTimeBy(2_000) poller.close() }
  99. Testing

  100. Flow Assert suspend fun <T> "Flow<T>.test( validate: suspend "FlowAssert<T>.() "->

    Unit ) { }
  101. Flow Assert sealed class Event<out T> { object Complete :

    Event<Nothing>() data class Error(val t: Throwable) : Event<Nothing>() data class Item<T>(val item: T) : Event<T>() }
  102. Flow Assert suspend fun <T> "Flow<T>.test( validate: suspend "FlowAssert<T>.() "->

    Unit ) { coroutineScope { val events = Channel<Event<T">>(UNLIMITED) launch { try { collect { item !-> events.send(Event.Item(item)) } Event.Complete } }
  103. Flow Assert suspend fun <T> "Flow<T>.test( validate: suspend "FlowAssert<T>.() "->

    Unit ) { coroutineScope { val events = Channel<Event<T">>(UNLIMITED) launch { try {
 … } catch (t: Throwable) { Event.Error(t) } if (terminalEvent "!= null) { events.send(terminalEvent) } events.close()
  104. Flow Assert class FlowAssert<T>(val events: Channel<Event<T>>) { suspend fun expectItem():

    T suspend fun expectComplete() suspend fun expectError(): Throwable … }
  105. Flow Assert @Test fun `should poll every second`() = runBlockingTest

    { val "flow = poller.poll(1_000) launch { "flow.test { assertEquals(expectItem(), data) assertEquals(expectItem(), data) expectComplete() } } testDispatcher.advanceTimeBy(2_000) poller.close()
  106. Resources • Kotlinx-coroutines-test
 https://github.com/Kotlin/kotlinx.coroutines/tree/master/kotlinx-coroutines-test • Coroutine JUnit Test Rule
 https://craigrussell.io/2019/11/unit-testing-coroutine-suspend-functions-using-testcoroutinedispatcher/

    • Corbind 
 https://github.com/LDRAlighieri/Corbind • FlowAssert 
 
 https://github.com/cashapp/sqldelight/blob/master/extensions/coroutines-extensions/src 
 commonTest/kotlin/com/squareup/sqldelight/runtime/coroutines/FlowAssert.kt
  107. Thank you! twitter.com/heyitsmohit