Tweet
Tweet

Slide 1

Slide 1 text

Channels in Kotlin Coroutines* Nikita Koval, Joker 2018 * A look into the future

Slide 2

Slide 2 text

Attention! This talk is about concurrency and algorithms! 2

Slide 3

Slide 3 text

Speaker: Nikita Koval ● Graduated at ITMO University ● Previously worked as developer and researcher at Devexperts ● Teaches concurrent programming at ITMO University ● PhD student at IST Austria ● Researcher at JetBrains 3 @nkoval_

Slide 4

Slide 4 text

What coroutines are ● Lightweight threads, can be suspended and resumed for free ○ You can run millions of coroutines and not die! 4

Slide 5

Slide 5 text

What coroutines are ● Lightweight threads, can be suspended and resumed for free ○ You can run millions of coroutines and not die! ● Support writing an asynchronous code like a synchronous one fun postItem(item: Item) { val token = requestToken() val post = createPost(token, item) processPost(post) } 5 suspend functions

Slide 6

Slide 6 text

“Kotlin Coroutines in Practice” by Roman Elizarov @ KotlinConf 2018 6

Slide 7

Slide 7 text

Producer-consumer problem * Both clients and consumers are coroutines 7 ... Worker 1 Worker M ... Send a task Receive a task Client 1 Client 2 Client N

Slide 8

Slide 8 text

Producer-consumer problem solution 1. Let’s create a channel val tasks = Channel() 8

Slide 9

Slide 9 text

Producer-consumer problem solution 1. Let’s create a channel val tasks = Channel() 2. Clients send tasks to workers through this channel val task = Task(...) tasks.send(task) 9

Slide 10

Slide 10 text

Producer-consumer problem solution 1. Let’s create a channel val tasks = Channel() 2. Clients send tasks to workers through this channel val task = Task(...) tasks.send(task) 3. Workers receive tasks in infinite loop while(true) { val task = tasks.receive() processTask(task) } 10

Slide 11

Slide 11 text

Channel semantics Client 1 val task = Task(...) tasks.send(task) 11 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel()

Slide 12

Slide 12 text

Channel semantics Client 1 val task = Task(...) tasks.send(task) 12 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel() Have to wait for send 1

Slide 13

Slide 13 text

Channel semantics Client 1 val task = Task(...) tasks.send(task) 13 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel() 1

Slide 14

Slide 14 text

Channel semantics Client 1 val task = Task(...) tasks.send(task) 14 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel() 1

Slide 15

Slide 15 text

Channel semantics Client 1 val task = Task(...) tasks.send(task) 15 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel() Rendezvous! 1 2

Slide 16

Slide 16 text

Channel semantics Client 1 val task = Task(...) tasks.send(task) 16 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } 1 val tasks = Channel() 3 2

Slide 17

Slide 17 text

Channel semantics Client 1 val task = Task(...) tasks.send(task) 17 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } 1 val tasks = Channel() 3 2 4 Have to wait for receive

Slide 18

Slide 18 text

Channel semantics Client 1 val task = Task(...) tasks.send(task) 18 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } 1 val tasks = Channel() 3 2 4

Slide 19

Slide 19 text

Channel semantics Client 1 val task = Task(...) tasks.send(task) 19 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } 1 val tasks = Channel() 3 2 4 5 Rendezvous!

Slide 20

Slide 20 text

Rendezvous channel 20

Slide 21

Slide 21 text

Rendezvous channel class Coroutine { var element: Any? ... } fun curCoroutine(): Coroutine { ... } suspend fun suspend(c: Coroutine) { ... } fun resume(c: Coroutine) { ... } 21 Stores an element to be sent Returns the current coroutine Functions to manipulate with coroutines

Slide 22

Slide 22 text

Rendezvous channel class Coroutine { var element: Any? ... } fun curCoroutine(): Coroutine { ... } suspend fun suspend(c: Coroutine) { ... } fun resume(c: Coroutine) { ... } val senders = Queue() val receivers = Queue() 22 Queues of suspended send and receive invocations

Slide 23

Slide 23 text

Rendezvous channel class Coroutine { var element: Any? ... } fun curCoroutine(): Coroutine { ... } suspend fun suspend(c: Coroutine) { ... } fun resume(c: Coroutine) { ... } val senders = Queue() val receivers = Queue() 23 suspend fun send(element: T) { if (receivers.isEmpty()) { val curCor = curCoroutine() curCor.element = element senders.enqueue(curCor) suspend(curCor) } else { val r = receivers.dequeue() r.element = element resume(receiver) } } Checks if there is no receiver and suspends Rendezvous: retrieves the first receiver

Slide 24

Slide 24 text

Rendezvous channel: Golang 24

Slide 25

Slide 25 text

Rendezvous channel: Golang Uses per-channel locks 25 suspend fun send(element: T) = channelLock.withLock { if (receivers.isEmpty()) { val curCor = curCoroutine() curCor.element = element senders.enqueue(curCor) suspend(curCor) } else { val r = receivers.dequeue() r.element = element resume(receiver) } }

Slide 26

Slide 26 text

Rendezvous channel: Golang Uses per-channel locks 26 Non-scalable, no progress guarantee... suspend fun send(element: T) = channelLock.withLock { if (receivers.isEmpty()) { val curCor = curCoroutine() curCor.element = element senders.enqueue(curCor) suspend(curCor) } else { val r = receivers.dequeue() r.element = element resume(receiver) } }

Slide 27

Slide 27 text

Modern queues use Fetch-And-Add... Let’s try to use the same ideas for channels! PPoPP’13 PPoPP’16

Slide 28

Slide 28 text

Concurrent primitives ● Fetch-And-Add(p, val): Long Atomically increments the located by address p register by val and returns the new value ● Compare-And-Swap(p, old, new): Boolean Atomically checks if the located by address p value equals old and replaces it with new 28

Slide 29

Slide 29 text

Rendezvous channel: Kotlin ● Assume we have an atomic 128-bit register ○ That is not true; we will fix this later

Slide 30

Slide 30 text

Rendezvous channel: Kotlin ● Assume we have an atomic 128-bit register ○ That is not true; we will fix this later senders receivers 64 bits 64 bits The total number of send and receive operations 30 sendersAndReceivers

Slide 31

Slide 31 text

Rendezvous channel: Kotlin ● Assume we have an atomic 128-bit register ○ That is not true; we will fix this later senders receivers 64 bits 64 bits The total number of send and receive operations ● Every send and receive increments its counter using FAA ○ send increments the register by (1 << 64) ○ receive increments the register by 1 31 sendersAndReceivers

Slide 32

Slide 32 text

Rendezvous channel: Kotlin ● Each send-receive pair works with an unique cell ● This cell id is either senders or receivers counter after the increment (for send and receive respectively) ● How to understand if we can make a rendezvous? 32

Slide 33

Slide 33 text

Rendezvous channel: Kotlin 33 when { senders < receivers -> // make a rendezvous senders >= receivers -> // suspend } The balance before the send operation

Slide 34

Slide 34 text

Rendezvous channel: Kotlin 34 EMPTY coroutine DONE suspend rendezvous when { senders < receivers -> // make a rendezvous senders >= receivers -> // suspend } Cell life cycle The balance before the send operation

Slide 35

Slide 35 text

Rendezvous channel: Kotlin 35 EMPTY coroutine DONE BROKEN suspend rendezvous failed rendezvous when { senders < receivers -> // make a rendezvous senders >= receivers -> // suspend } The balance before the send operation Cell life cycle This helps not to block

Slide 36

Slide 36 text

Rendezvous channel: Kotlin 1. How to implement an atomic 128-bit counter using 64-bit ones? 2. How to organize the cell storage? 36

Slide 37

Slide 37 text

Rendezvous channel: Kotlin 1. How to implement an atomic 128-bit counter using 64-bit ones? 2. How to organize the cell storage? 37

Slide 38

Slide 38 text

Rendezvous channel: Kotlin 38 senders_L receivers_L 1/0 1/0 senders_H receivers_H 1 bit 31 bits 1 bit 31 bits 32 bits 32 bits We maintain highest and lowest parts separately 0000...001111...11 highest part lowest part L H

Slide 39

Slide 39 text

Rendezvous channel: Kotlin 39 senders_L receivers_L 1/0 1/0 senders_H receivers_H 1 bit 31 bits 1 bit 31 bits 32 bits 32 bits We maintain highest and lowest parts separately Indicates that the lowest part is overflowed L H

Slide 40

Slide 40 text

Rendezvous channel: Kotlin 40 senders_L receivers_L 1/0 1/0 senders_H receivers_H 1 bit 31 bits 1 bit 31 bits 32 bits 32 bits Read-write lock for highest parts H_rwlock L H

Slide 41

Slide 41 text

Rendezvous channel: Kotlin 41 senders_H receivers_H 32 bits 32 bits H_rwlock senders_L receivers_L 1/0 1/0 1 bit 31 bits 1 bit 31 bits Increment algorithm: 1. Acquire H_rwlock for read 2. Read H 3. Inc L by FAA 4. Release the lock L H

Slide 42

Slide 42 text

Rendezvous channel: Kotlin 42 senders_H receivers_H 32 bits 32 bits H_rwlock senders_L receivers_L 1/0 1/0 1 bit 31 bits 1 bit 31 bits Increment algorithm: 1. Acquire H_rwlock for read 2. Read H 3. Inc L by FAA 4. Release the lock L H Just a FAA

Slide 43

Slide 43 text

Rendezvous channel: Kotlin 43 senders_H receivers_H 32 bits 32 bits H_rwlock senders_L receivers_L 1/0 1/0 1 bit 31 bits 1 bit 31 bits Increment algorithm: 1. Acquire H_rwlock for read 2. Read H 3. Inc L by FAA 4. Release the lock 5. If the lowest part is overflowed 5.1. Acquire H_rwlock for write 5.2. Reset the bit 5.3. Inc H 5.4. Release the lock L H

Slide 44

Slide 44 text

Rendezvous channel: Kotlin 1. How to implement an atomic 128-bit counter using 64-bit ones? 2. How to organize the cell storage? 44

Slide 45

Slide 45 text

Rendezvous channel: Kotlin 45 0 N ... 1 N ... K N ... ... Michael-Scott queue of segments with the fixed number of cells in each Segments HEAD TAIL

Slide 46

Slide 46 text

Rendezvous channel: Kotlin 46 0 N ... 1 N ... K N ... ... Michael-Scott queue of segments with the fixed number of cells in each Cells HEAD TAIL

Slide 47

Slide 47 text

Rendezvous channel: Kotlin 47 0 N ... 1 N ... K N ... ... Michael-Scott queue of segments with the fixed number of cells in each Pointer to the next segment HEAD TAIL Segment id

Slide 48

Slide 48 text

Rendezvous channel: Kotlin 48 0 N ... 1 N ... K N ... ... HEAD TAIL 1. Read both HEAD and TAIL 2. Increment the counter

Slide 49

Slide 49 text

Rendezvous channel: Kotlin 49 0 N ... 1 N ... K N ... ... HEAD TAIL 1. Read both HEAD and TAIL 2. Increment the counter 3. Either make a rendezvous 3.1. Find the cell starting from the head 3.2. Move HEAD forward if needed

Slide 50

Slide 50 text

Rendezvous channel: Kotlin 50 0 N ... 1 N ... K N ... ... HEAD TAIL 1. Read both HEAD and TAIL 2. Increment the counter 3. Either make a rendezvous 3.1. Find the cell starting from the head 3.2. Move HEAD forward if needed 4. or suspend 4.1. Find the cell starting from the tail 4.2. Create new segments if needed

Slide 51

Slide 51 text

Rendezvous channel: Kotlin vs Golang 51 4 x 2 x Intel Xeon Gold 6150 (Skylake) 2.7GHz = (144 virtual cores)

Slide 52

Slide 52 text

Producer-consumer problem: buffering 52 Client 1 Client 2 Client N ... Worker 1 Worker M ... Send a task Receive a task We don’t want to wait on every send...

Slide 53

Slide 53 text

Producer-consumer problem: buffering 53 Client 1 Client 2 Client N ... Worker 1 Worker M ... Send a task Receive a task Let’s use a fixed-size buffer!

Slide 54

Slide 54 text

Buffered channel semantics Client 1 val task = Task(...) tasks.send(task) 54 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel(capacity = 1) One element can be sent without suspension

Slide 55

Slide 55 text

Buffered channel semantics Client 1 val task = Task(...) tasks.send(task) 55 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel(capacity = 1) 1 Does not suspend!

Slide 56

Slide 56 text

Buffered channel semantics Client 1 val task = Task(...) tasks.send(task) 56 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel(capacity = 1) 1 The buffer is full, has to suspend 2

Slide 57

Slide 57 text

Buffered channel semantics Client 1 val task = Task(...) tasks.send(task) 57 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel(capacity = 1) 1 Receives the buffered element at first 2 3

Slide 58

Slide 58 text

Buffered channel semantics Client 1 val task = Task(...) tasks.send(task) 58 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel(capacity = 1) 1 Makes a rendezvous with the 2nd client 2 3 4

Slide 59

Slide 59 text

Buffered channel: Golang ● Maintains an additional fixed-size buffer ○ Tries to send to this buffer instead of suspending ● Performs all operations under the channel lock 59

Slide 60

Slide 60 text

Buffered channel: Kotlin Rendezvous channel when { senders - receivers < 0 -> // make a rendezvous senders - receivers >= 0 -> // suspend } 60 The balance before the send operation

Slide 61

Slide 61 text

Buffered channel: Kotlin Rendezvous channel when { senders - receivers < 0 -> // make a rendezvous senders - receivers >= 0 -> // suspend } Buffered channel when { senders - receivers < 0 -> // make a rendezvous 0 <= senders - receivers < capacity -> // send the element without suspension senders - receivers >= capacity -> // suspend } 61

Slide 62

Slide 62 text

Buffered channel: Kotlin vs Golang 62 4 x 2 x Intel Xeon Gold 6150 (Skylake) 2.7GHz = (144 virtual cores)

Slide 63

Slide 63 text

The select expression 63 Client val task = Task(...) tasks.send(task) Suspends here

Slide 64

Slide 64 text

The select expression 64 Client val task = Task(...) tasks.send(task)

Slide 65

Slide 65 text

The select expression 65 Client val task = Task(...) tasks.send(task) The client was interrupted while waiting for a worker

Slide 66

Slide 66 text

The select expression 66 Client val task = Task(...) tasks.send(task) The client was interrupted while waiting for a worker Do we need to process the task anymore?

Slide 67

Slide 67 text

The select expression 67 Client val task = Task(...) tasks.send(task) The client was interrupted while waiting for a worker Do we need to process the task anymore? It would be better to cancel the request and detect this

Slide 68

Slide 68 text

The select expression Client val task = Task(...) val cancelled = Channel() 68 Unit is sent to this channel if the client is interrupted

Slide 69

Slide 69 text

The select expression Client val task = Task(...) val cancelled = Channel() select { tasks.onSend(task) { println("Task has been sent") } cancelled.onReceive { println("Cancelled") } } 69 Waits simultaneously, at most one clause is selected atomically.

Slide 70

Slide 70 text

The select expression: Golang ● Fine-grained locking ● Acquires all involved channels locks to register into the queues ○ Uses hierarchical order to avoid deadlocks ● Acquires all these locks again to resume the coroutine ○ Otherwise, two select clauses could interfere 70

Slide 71

Slide 71 text

The select expression: Kotlin 71 SelectInstance state PENDING or SELECTED alternatives

Slide 72

Slide 72 text

For each channel The select expression: Kotlin 72 SelectInstance alternatives state PENDING or SELECTED Get the counters snapshot Try to make a rendezvous Try to store the SelectInstance Waiting phase Remove the stored SelectInstance-s senders < receivers senders >= receivers

Slide 73

Slide 73 text

The select expression: Kotlin EMPTY coroutine DONE BROKEN suspend rendezvous failed rendezvous Cell life cycle Select Desc rendezvous try to make a rendezvous rendezvous failed 73

Slide 74

Slide 74 text

The select expression: Kotlin EMPTY coroutine DONE BROKEN suspend rendezvous failed rendezvous Cell life cycle Select Desc rendezvous try to make a rendezvous rendezvous failed Does not update counters here! Increment the counter on success 74

Slide 75

Slide 75 text

The select expression: Kotlin EMPTY coroutine DONE BROKEN suspend rendezvous failed rendezvous Cell life cycle Select Desc rendezvous try to make a rendezvous rendezvous failed Does not update counters here! Increment the counter on success Tries to increment the counter from the snapshot by CAS 75

Slide 76

Slide 76 text

No content

Slide 77

Slide 77 text

The select expression: Kotlin A rendezvous between two selects: 77 SelectInstance alternatives state PENDING or SELECTED SelectInstance alternatives state PENDING or SELECTED Need to update them atomically Let’s update them similarly to the Harris multiword CAS* * ”A practical multi-word compare-and-swap operation” by Harris et al. @ DISC’02

Slide 78

Slide 78 text

The select expression: Kotlin vs Golang 78 4 x 2 x Intel Xeon Gold 6150 (Skylake) 2.7GHz = (144 virtual cores)

Slide 79

Slide 79 text

Cancellation in Kotlin ● Cancellation is a built-in feature in Kotlin ○ However, the previous pattern is widely used in Golang val job = GlobalScope.launch { ... } job.cancel() 79 Removes the coroutine from all channels as well

Slide 80

Slide 80 text

Cancellation in Kotlin ● Cancellation is a built-in feature in Kotlin ○ However, the previous pattern is widely used in Golang val job = GlobalScope.launch { ... } job.cancel() GlobalScope.launch { withTimeout(time = 1, unit = TimeUnit.SECONDS) { ... } } 80 Invokes job.cancel() after the timeout

Slide 81

Slide 81 text

There are more message passing primitives ● BroadcastChannel Sends to multiple receivers ● ConflatedChannel Receivers always get the most recently sent element ● ConflatedBroadcastChannel Mix of the previous ones ● Mutex 81

Slide 82

Slide 82 text

Industry Academia