Сказ о том, как мы алгоритм каналов в Kotlin Coroutines делали

https://jpoint.ru/talks/6tqzgwcm9atqfq4nkazsd9/

Transcript

1. Сказ о том, как мы алгоритм каналов в Kotlin Coroutines делали Никита Коваль, JPoint 2019
2. Attention! This talk is about concurrency and algorithms! 2
3. Speaker: Nikita Koval • Graduated @ ITMO University • Previously worked as developer and researcher @ Devexperts • Teaching concurrent programming course @ ITMO University • Researcher @ JetBrains • PhD student @ IST Austria 3 @nkoval_
4. What coroutines are • Lightweight threads, can be suspended and resumed for free ◦ You can run millions of coroutines and not die! 4
5. What coroutines are • Lightweight threads, can be suspended and resumed for free ◦ You can run millions of coroutines and not die! • Support writing an asynchronous code like a synchronous one suspend fun dbRequest(c: Client, r: Request) { val token = requestToken(c) val result = doDbRequest(token, r) processResult(result) } 5 suspend functions
6. * “Kotlin Coroutines in Practice” by Roman Elizarov @ KotlinConf 2018 6
7. Producer-Consumer Problem * Both clients and workers are coroutines 7 ... Worker 1 Worker M ... Send a task Receive a task Client 1 Client 2 Client N
8. Producer-Consumer Problem Solution 1. Let’s create a channel val tasks = Channel<Task>() 8
9. Producer-Consumer Problem Solution 1. Let’s create a channel val tasks = Channel<Task>() 2. Clients send tasks to workers through this channel val task = Task(...) tasks.send(task) 9
10. Producer-Consumer Problem Solution 1. Let’s create a channel val tasks = Channel<Task>() 2. Clients send tasks to workers through this channel val task = Task(...) tasks.send(task) 3. Workers receive tasks in an infinite loop while(true) { val task = tasks.receive() processTask(task) } 10
11. Rendezvous Channel Semantics Client 1 val task = Task(...) tasks.send(task) 11 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel<Task>()
12. Rendezvous Channel Semantics Client 1 val task = Task(...) tasks.send(task) 12 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel<Task>() Have to wait for send 1
13. Rendezvous Channel Semantics Client 1 val task = Task(...) tasks.send(task) 13 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel<Task>() 1
14. Rendezvous Channel Semantics Client 1 val task = Task(...) tasks.send(task) 14 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel<Task>() 1
15. Rendezvous Channel Semantics Client 1 val task = Task(...) tasks.send(task) 15 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel<Task>() Rendezvous! 1 2
16. Rendezvous Channel Semantics Client 1 val task = Task(...) tasks.send(task) 16 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } 1 val tasks = Channel<Task>() 3 2
17. Rendezvous Channel Semantics Client 1 val task = Task(...) tasks.send(task) 17 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } 1 val tasks = Channel<Task>() 3 2 4 Have to wait for receive
18. Rendezvous Channel Semantics Client 1 val task = Task(...) tasks.send(task) 18 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } 1 val tasks = Channel<Task>() 3 2 4
19. Rendezvous Channel Semantics Client 1 val task = Task(...) tasks.send(task) 19 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } 1 val tasks = Channel<Task>() 3 2 4 5 Rendezvous!
20. Coroutines Management 20
21. Coroutines Management class Coroutine { var element: Any? ... } fun curCoroutine(): Coroutine { ... } suspend fun suspend(c: Coroutine) { ... } fun resume(c: Coroutine) { ... } 21 Element to be sent Returns the current coroutine Functions to manipulate with coroutines
22. Sequential Rendezvous Channel Implementation class Coroutine { var element: Any? ... } fun curCoroutine(): Coroutine { ... } suspend fun suspend(c: Coroutine) { ... } fun resume(c: Coroutine) { ... } val senders = Queue<Coroutine>() val receivers = Queue<Coroutine>() 22 Queues of suspended send and receive invocations
23. Sequential Rendezvous Channel Implementation class Coroutine { var element: Any? ... } fun curCoroutine(): Coroutine { ... } suspend fun suspend(c: Coroutine) { ... } fun resume(c: Coroutine) { ... } val senders = Queue<Coroutine>() val receivers = Queue<Coroutine>() 23 suspend fun send(element: T) { if (receivers.isEmpty()) { val curCor = curCoroutine() curCor.element = element senders.enqueue(curCor) suspend(curCor) } else { val r = receivers.dequeue() r.element = element resume(r) } } Check if there is no receiver and suspends Rendezvous: retrieve the first receiver
24. Sequential Rendezvous Channel Implementation 24 suspend fun send(element: T) { if (receivers.isEmpty()) { val curCor = curCoroutine() curCor.element = element senders.enqueue(curCor) suspend(curCor) } else { val r = receivers.dequeue() r.element = element resume(r) } } suspend fun receive(): T { if (senders.isEmpty()) { val curCor = curCoroutine() receivers.enqueue(curCor) suspend(curCor) return curCor.element } else { val s = senders.dequeue() val res = s.element resume(s) return res } }
25. Rendezvous Channel: Golang 25
26. Rendezvous Channel: Golang Uses per-channel locks 26 suspend fun send(element: T) = channelLock.withLock { if (receivers.isEmpty()) { val curCor = curCoroutine() curCor.element = element senders.enqueue(curCor) suspend(curCor) } else { val r = receivers.dequeue() r.element = element resume(receiver) } }
27. Rendezvous Channel: Golang Uses per-channel locks 27 Non-scalable, no progress guarantee... suspend fun send(element: T) = channelLock.withLock { if (receivers.isEmpty()) { val curCor = curCoroutine() curCor.element = element senders.enqueue(curCor) suspend(curCor) } else { val r = receivers.dequeue() r.element = element resume(receiver) } }
28. Rendezvous Channel: Java 28 PPoPP’06 “Our synchronous queues have been adopted for inclusion in Java 6” j.u.c.SynchronousQueue
29. Rendezvous Channel: Java Based on Michael-Scott lock-free queue algorithm the simplest known lock-free queue, j.u.c.ConcurrentLinkedQueue 29
30. Rendezvous Channel: Java Based on Michael-Scott lock-free queue algorithm the simplest known lock-free queue, j.u.c.ConcurrentLinkedQueue 30 Either senders or receivers are in the queue!
31. Rendezvous Channel: Java Based on Michael-Scott lock-free queue algorithm the simplest known lock-free queue, j.u.c.ConcurrentLinkedQueue 31 HEAD N TAIL Stores both the element to be sent (RECEIVE_EL for receive) and the coroutine C “1” dummy N N C “2”
32. Rendezvous Channel: Java Based on Michael-Scott lock-free queue algorithm the simplest known lock-free queue, j.u.c.ConcurrentLinkedQueue 32 HEAD N TAIL C “1” dummy N N C “2” dequeue updates HEAD enqueue updates TAIL and NEXT
33. Rendezvous Channel: Java Based on Michael-Scott lock-free queue algorithm the simplest known lock-free queue, j.u.c.ConcurrentLinkedQueue 33 HEAD N TAIL C “1” dummy N N C “2” send(x): t := TAIL h := HEAD if t == h || t.isSender() { enqueueAndSuspend(t, x) } else { dequeueAndResume(h) }
34. Rendezvous Channel: Java Pros: • Clear and simple algorithm • Guarantees lock-freedom for the registration phase Cons: • Сreates a new node on each suspend • Cancellation works in O(N) • Non-scalable 34
35. Rendezvous Channel: First Solution Let’s store multiple waiters in node! 35
36. Rendezvous Channel: First Solution • Each node stores K waiters ◦ More cache-efficient ◦ More GC-efficient • Node removing works in O(1) • The select expression support via descriptors ◦ Will be discussed a bit later 36
37. Rendezvous Channel: First Solution 37 Less is better!
38. Rendezvous Channel: First Solution 38 Less is better! How to achieve more performance?
39. Modern queues use Fetch-And-Add... Let’s try to use the same ideas for channels! PPoPP’13 PPoPP’16
40. Rendezvous Channel: Second Solution Assume we have an atomic array and an atomic 128-bit register 40 ... senders receivers 64 bits 64 bits sendersAndReceivers senders = cell for the next send receivers = cell for the next receive arr
41. Rendezvous Channel: Second Solution Assume we have an atomic array and an atomic 128-bit register 41 ... senders receivers 64 bits 64 bits sendersAndReceivers send(x): s, r := incSenders() if s >= r { arr[s] = Waiter{curCor(), x} } else { resume(arr[s], x) } arr senders = cell for the next send receivers = cell for the next receive
42. Rendezvous Channel: Second Solution Assume we have an atomic array and an atomic 128-bit register 42 ... senders receivers 64 bits 64 bits sendersAndReceivers send(x): s, r := incSenders() if s >= r { arr[s] = Waiter{curCor(), x} } else { resume(arr[s], x) } arr send(1): receive():
43. Rendezvous Channel: Second Solution Assume we have an atomic array and an atomic 128-bit register 43 ... senders receivers 64 bits 64 bits sendersAndReceivers send(x): s, r := incSenders() if s >= r { arr[s] = Waiter{curCor(), x} } else { resume(arr[s], x) } arr send(1): receive(): 1. Inc receivers
44. Rendezvous Channel: Second Solution Assume we have an atomic array and an atomic 128-bit register 44 C ... senders receivers 64 bits 64 bits sendersAndReceivers send(x): s, r := incSenders() if s >= r { arr[s] = Waiter{curCor(), x} } else { resume(arr[s], x) } arr send(1): receive(): 1. Inc receivers 2. Store the coroutine
45. Rendezvous Channel: Second Solution Assume we have an atomic array and an atomic 128-bit register 45 C ... senders receivers 64 bits 64 bits sendersAndReceivers send(x): s, r := incSenders() if s >= r { arr[s] = Waiter{curCor(), x} } else { resume(arr[s], x) } arr send(1): 3. Inc senders receive(): 1. Inc receivers 2. Store the coroutine
46. Rendezvous Channel: Second Solution Assume we have an atomic array and an atomic 128-bit register 46 C ... senders receivers 64 bits 64 bits sendersAndReceivers send(x): s, r := incSenders() if s >= r { arr[s] = Waiter{curCor(), x} } else { resume(arr[s], x) } arr send(1): 3. Inc senders 4. Make a rendezvous receive(): 1. Inc receivers 2. Store the coroutine
47. Rendezvous Channel: Second Solution Assume we have an atomic array and an atomic 128-bit register 47 C ... senders receivers 64 bits 64 bits sendersAndReceivers send(x): s, r := incSenders() if s >= r { arr[s] = Waiter{curCor(), x} } else { resume(arr[s], x) } arr send(1): 3. Inc senders 4. Make a rendezvous receive(): 1. Inc receivers 2. Store the coroutine Any problem with this solution?
48. Rendezvous Channel: Second Solution Assume we have an atomic array and an atomic 128-bit register 48 ... senders receivers 64 bits 64 bits sendersAndReceivers send(x): s, r := incSenders() if s >= r { arr[s] = Waiter{curCor(), x} } else { resume(arr[s], x) } arr send(1): 2. Inc senders 3. Make a rendezvous? receive(): 1. Inc receivers The cell is empty!
49. Rendezvous Channel: Second Solution Assume we have an atomic array and an atomic 128-bit register 49 ... senders receivers 64 bits 64 bits sendersAndReceivers arr send(1): 2. Inc senders 3. Make a rendezvous? receive(): 1. Inc receivers The cell is empty! EMPTY coroutine DONE BROKEN suspend rendezvous Cell life cycle rendezvous failed, try the operation again
50. Rendezvous Channel: Second Solution Assume we have an atomic array and an atomic 128-bit register 50 ... senders receivers 64 bits 64 bits sendersAndReceivers arr send(1): 2. Inc senders 3. Make a rendezvous? receive(): 1. Inc receivers The cell is empty! EMPTY coroutine DONE BROKEN suspend rendezvous failed, try the operation again rendezvous Cell life cycle Do not need this BROKEN state in practice, can just wait
51. Rendezvous Channel: Second Solution • Each send-receive pair works with an unique cell • This cell id is either senders or receivers counter after the increment (for send and receive respectively) 51
52. Rendezvous Channel: Second Solution • Each send-receive pair works with an unique cell • This cell id is either senders or receivers counter after the increment (for send and receive respectively) • How to implement an atomic 128-bit counter using 64-bit ones? • How to organize the cell storage? 52
53. Second Solution: Counters 53 senders_L receivers_L 1/0 1/0 senders_H receivers_H 1 bit 31 bits 1 bit 31 bits 32 bits 32 bits We maintain highest and lowest parts separately 0000...001111...11 highest part lowest part L H
54. Second Solution: Counters 54 senders_L receivers_L 1/0 1/0 1 bit 31 bits 1 bit 31 bits 32 bits 32 bits We maintain highest and lowest parts separately Indicates that the lowest part is overflowed L H senders_H receivers_H
55. Second Solution: Counters 55 senders_L receivers_L 1/0 1/0 1 bit 31 bits 1 bit 31 bits 32 bits 32 bits Read-write lock for highest parts H_rwlock L H senders_H receivers_H
56. Second Solution: Counters 56 32 bits 32 bits H_rwlock senders_L receivers_L 1/0 1/0 1 bit 31 bits 1 bit 31 bits Increment algorithm: 1. Acquire H_rwlock for read 2. Read H 3. Inc L by FAA 4. Release the lock L H senders_H receivers_H
57. Second Solution: Counters 57 32 bits 32 bits H_rwlock senders_L receivers_L 1/0 1/0 1 bit 31 bits 1 bit 31 bits Increment algorithm: 1. Acquire H_rwlock for read 2. Read H 3. Inc L by FAA 4. Release the lock L H Just a FAA senders_H receivers_H
58. Second Solution: Counters 58 32 bits 32 bits H_rwlock senders_L receivers_L 1/0 1/0 1 bit 31 bits 1 bit 31 bits Increment algorithm: 1. Acquire H_rwlock for read 2. Read H 3. Inc L by FAA 4. Release the lock 5. If the lowest part is overflowed 5.1. Acquire H_rwlock for write 5.2. Reset the bit 5.3. Inc H 5.4. Release the lock L H senders_H receivers_H
59. Second Solution: Cell Storage 59 0 N ... 1 N ... K N ... ... HEAD TAIL Lock-free Michael-Scott queue of segments
60. Second Solution: Cell Storage 60 0 N ... 1 N ... K N ... ... HEAD TAIL 1. Read both HEAD and TAIL 2. Increment the counter
61. Second Solution: Cell Storage 61 0 N ... 1 N ... K N ... ... HEAD TAIL 1. Read both HEAD and TAIL 2. Increment the counter 3. Either make a rendezvous 3.1. Find the cell starting from the head 3.2. Move HEAD forward if needed
62. Second Solution: Cell Storage 62 0 N ... 1 N ... K N ... ... HEAD TAIL 1. Read both HEAD and TAIL 2. Increment the counter 3. Either make a rendezvous 3.1. Find the cell starting from the head 3.2. Move HEAD forward if needed 4. or suspend 4.1. Find the cell starting from the tail 4.2. Create new segments if needed
63. Rendezvous Channel: Second Solution 63 Less is better!
64. Buffered Channel Semantics Client 1 val task = Task(...) tasks.send(task) 64 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel<Task>(capacity = 1) One element can be sent without suspension
65. Buffered Channel Semantics Client 1 val task = Task(...) tasks.send(task) 65 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel<Task>(capacity = 1) 1 Does not suspend!
66. Buffered Channel Semantics Client 1 val task = Task(...) tasks.send(task) 66 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel<Task>(capacity = 1) 1 The buffer is full, suspends 2
67. Buffered Channel Semantics Client 1 val task = Task(...) tasks.send(task) 67 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel<Task>(capacity = 1) 1 Receives the buffered element, resumes the 2nd client, and moves its task to the buffer 3 2
68. Buffered Channel Semantics Client 1 val task = Task(...) tasks.send(task) 68 Client 2 val task = Task(...) tasks.send(task) Worker while(true) { val task = tasks.receive() processTask(task) } val tasks = Channel<Task>(capacity = 1) 1 Retrieves the 2nd task, no waiters to resume 2 3 4
69. Buffered Channel: Golang • Maintains an additional fixed-size buffer ◦ Tries to send to this buffer instead of suspending • Performs all operations under the channel lock 69
70. Buffered Channel: Our Solution Channel with capacity = 1 70 ... senders receivers
71. Buffered Channel: Our Solution Channel with capacity = 1 71 1 ... senders receivers send(1): DONE
72. Buffered Channel: Our Solution Channel with capacity = 1 72 1 S ... senders receivers send(1): DONE send(2): SUSPENDED
73. Buffered Channel: Our Solution Channel with capacity = 1 73 1 2 ... senders receivers send(1): DONE send(2): DONE receive(): 1
74. Buffered Channel: Our Solution Channel with capacity = 1 74 1 2 ... senders receivers send(1): DONE send(2): DONE receive(): 1 Can we use only senders and receivers counters to define the current buffer?
75. Buffered Channel: Our Solution Two counters are not enough! 75 1 S ... senders receivers send(1): DONE send(2): SUSPENDED
76. Buffered Channel: Our Solution Two counters are not enough! 76 1 S ... S senders receivers send(1): DONE send(2): SUSPENDED send(3): SUSPENDED
77. Buffered Channel: Our Solution Two counters are not enough! 77 1 X ... S senders receivers send(1): DONE send(2): CANCELLED send(3): SUSPENDED
78. Buffered Channel: Our Solution Two counters are not enough! 78 1 X ... 2? senders receivers send(1): DONE send(2): CANCELLED send(3): DONE??? receive(): 1 We have to find the first non-cancelled send request to resume (put into the buffer)
79. Buffered Channel: Our Solution Two counters are not enough! 79 1 X ... 2? senders receivers send(1): DONE send(2): CANCELLED send(3): DONE??? receive(): 1 We have to find the first non-cancelled send request to resume (put into the buffer) Works in O(N)
80. Buffered Channel: Our Solution Let’s use three counters! 80 ... senders receivers buffer_end Specifies the last send to be buffered
81. Buffered Channel: Our Solution Let’s use three counters! 81 ... senders receivers buffer_end send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ }
82. Buffered Channel: Our Solution Let’s use three counters! 82 ... senders receivers buffer_end send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure
83. Buffered Channel: Our Solution Let’s use three counters! 83 ... senders receivers buffer_end send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure
84. Buffered Channel: Our Solution Let’s use three counters! 84 R ... senders receivers buffer_end receive(): SUSPENDED send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure
85. Buffered Channel: Our Solution Let’s use three counters! 85 R ... senders receivers buffer_end receive(): 1 send(1): DONE send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure
86. Buffered Channel: Our Solution Let’s use three counters! 86 R 2 ... senders receivers buffer_end receive(): 1 send(1): DONE send(2): DONE send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure
87. Buffered Channel: Our Solution Let’s use three counters! 87 ... senders receivers buffer_end send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure
88. Buffered Channel: Our Solution Let’s use three counters! 88 1 ... senders receivers buffer_end send(1): DONE send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure
89. Buffered Channel: Our Solution Let’s use three counters! 89 1 S ... senders receivers buffer_end send(1): DONE send(2): SUSPEND send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure
90. Buffered Channel: Our Solution Let’s use three counters! 90 1 2 ... senders receivers buffer_end send(1): DONE send(2): DONE receive(): 1 send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure
91. Buffered Channel: Our Solution Let’s use three counters! 91 ... senders receivers buffer_end send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure
92. Buffered Channel: Our Solution Let’s use three counters! 92 1 ... senders receivers buffer_end send(1): DONE send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure
93. Buffered Channel: Our Solution Let’s use three counters! 93 1 S ... receivers buffer_end send(1): DONE send(2): SUSPEND send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure senders
94. Buffered Channel: Our Solution Let’s use three counters! 94 1 S ... S receivers buffer_end send(1): DONE send(2): SUSPEND send(3): SUSPEND send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure senders
95. Buffered Channel: Our Solution Let’s use three counters! 95 1 X ... S receivers buffer_end send(1): DONE send(2): CANCELLED send(3): SUSPEND send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure senders
96. Buffered Channel: Our Solution Let’s use three counters! 96 1 X ... 3 receivers buffer_end send(1): DONE send(2): CANCELLED send(3): DONE receive(): 1 send(x): senders++, receivers, buffer_end if senders >= receivers { if senders < buffer_end { storeElement(senders, x) // buffering! } else { /* suspend */ } } else { /* rendezvous */ } receive(): senders, receivers++, buffer_end++ receiveImpl(senders, receivers) makeBuffered(buffer_end) // inc buffer_end // again on failure senders
97. Buffered Channel: Our Solution (capacity = 32) 97
98. Buffered Channel: Our Solution (capacity = 128) 98
99. The select Expression 99 Client val task = Task(...) tasks.send(task) Suspends here
100. The select Expression 100 Client val task = Task(...) tasks.send(task)
101. The select Expression 101 Client val task = Task(...) tasks.send(task) The client was interrupted while waiting for a worker
102. The select Expression 102 Client val task = Task(...) tasks.send(task) The client was interrupted while waiting for a worker Do we need to process the task anymore?
103. The select Expression 103 Client val task = Task(...) tasks.send(task) The client was interrupted while waiting for a worker Do we need to process the task anymore? It would be better to cancel the request and detect this
104. The select Expression Client val task = Task(...) val cancelled = Channel<Unit>() 104 Unit is sent to this channel if the client is interrupted
105. The select Expression Client val task = Task(...) val cancelled = Channel<Unit>() select<Unit> { tasks.onSend(task) { println("Task has been sent") } cancelled.onReceive { println("Cancelled") } } 105 Waits simultaneously, at most one clause is selected atomically.
106. The select Expression: Golang • Fine-grained locking • Acquires all involved channels locks to register into the queues ◦ Uses hierarchical order to avoid deadlocks • Acquires all these locks again to resume the coroutine ◦ Otherwise, two select clauses could interfere 106
107. The First Solution Operation Descriptors
108. The select Expression: Second Solution 108 SelectOp state alternatives Each alternative contains: • element to be sent (RECEIVE_EL for receive) • channel • action Progress state of this select instance
109. The select Expression: Second Solution 109 SelectOp state alternatives For each alternative Increment the corresponding counter Try to make a rendezvous Try to store the SelectOp Waiting phase Remove the stored SelectOp-s
110. The select Expression: Second Solution 110 SelectOp state alternatives For each alternative Increment the corresponding counter Try to make a rendezvous Try to store the SelectOp Waiting phase Remove the stored SelectOp-s How to make a rendezvous with this select instance?
111. The select Expression: Second Solution 111 SelectOp state alternatives REG CHANNEL WAITING DONE CAS Rendezvous during the registration phase Registered into all channels Another request makes a rendezvous Get both the element and the channel For each alternative Increment the corresponding counter Try to make a rendezvous Try to store the SelectOp Waiting phase Remove the stored SelectOp-s
112. The select Expression: Second Solution Client: select<Unit> { tasks.onSend(task) { println("Task has been sent") } cancelled.onReceive { println("Cancelled") } } Worker: val task = tasks.receive() processTask(task)
113. The select Expression: Second Solution Client: select<Unit> { tasks.onSend(task) { println("Task has been sent") } cancelled.onReceive { println("Cancelled") } } Worker: val task = tasks.receive() processTask(task) ... tasks ... cancelled SelectOp state: REG
114. The select Expression: Second Solution Client: select<Unit> { tasks.onSend(task) { println("Task has been sent") } cancelled.onReceive { println("Cancelled") } } Worker: val task = tasks.receive() processTask(task) SI ... tasks ... cancelled SelectOp state: REG C: Register in tasks
115. The select Expression: Second Solution Client: select<Unit> { tasks.onSend(task) { println("Task has been sent") } cancelled.onReceive { println("Cancelled") } } Worker: val task = tasks.receive() processTask(task) SI ... tasks ... cancelled SelectOp state: REG C: Register in tasks W: Rendezvous attempt in tasks, wait for state != REG
116. The select Expression: Second Solution Client: select<Unit> { tasks.onSend(task) { println("Task has been sent") } cancelled.onReceive { println("Cancelled") } } Worker: val task = tasks.receive() processTask(task) SI ... tasks SI ... cancelled SelectOp state: REG C: Register in tasks W: Rendezvous attempt in tasks, wait for state != REG C: Register in cancelled
117. The select Expression: Second Solution Client: select<Unit> { tasks.onSend(task) { println("Task has been sent") } cancelled.onReceive { println("Cancelled") } } Worker: val task = tasks.receive() processTask(task) SI ... tasks SI ... cancelled SelectOp state: WAITING C: Register in tasks W: Rendezvous attempt in tasks, wait for state != REG C: Register in cancelled C: Change state to WAITING
118. The select Expression: Second Solution Client: select<Unit> { tasks.onSend(task) { println("Task has been sent") } cancelled.onReceive { println("Cancelled") } } Worker: val task = tasks.receive() processTask(task) SI ... tasks SI ... cancelled SelectOp state: tasks C: Register in tasks W: Rendezvous attempt in tasks, wait for state != REG C: Register in cancelled C: Change state to WAITING W: Change state to tasks, the rendezvous done
119. The select Expression: Second Solution Client: select<Unit> { tasks.onSend(task) { println("Task has been sent") } cancelled.onReceive { println("Cancelled") } } Worker: val task = tasks.receive() processTask(task) SI ... tasks X ... cancelled SelectOp state: DONE C: Register in tasks W: Rendezvous attempt in tasks, wait for state != REG C: Register in cancelled C: Change state to WAITING W: Change state to tasks, the rendezvous done C: Selected, change state to DONE
120. The select Expression: Deadlock Avoidance Coroutine 1: select<Unit> { chan_1.onSend(task) { ... } chan_2.onReceive { ... } } Coroutine 2: select<Unit> { chan_2.onSend(task) { ... } chan_1.onReceive { ... } }
121. The select Expression: Deadlock Avoidance Coroutine 1: select<Unit> { chan_1.onSend(task) { ... } chan_2.onReceive { ... } } Coroutine 2: select<Unit> { chan_2.onSend(task) { ... } chan_1.onReceive { ... } } ... chan_1 ... SelectOp 1 state: REG SelectOp 2 state: REG chan_2
122. The select Expression: Deadlock Avoidance Coroutine 1: select<Unit> { chan_1.onSend(task) { ... } chan_2.onReceive { ... } } Coroutine 2: select<Unit> { chan_2.onSend(task) { ... } chan_1.onReceive { ... } } SI 1 ... chan_1 ... SelectOp 1 state: REG SelectOp 2 state: REG chan_2 1. C1: Register in chan_1
123. The select Expression: Deadlock Avoidance Coroutine 1: select<Unit> { chan_1.onSend(task) { ... } chan_2.onReceive { ... } } Coroutine 2: select<Unit> { chan_2.onSend(task) { ... } chan_1.onReceive { ... } } SI 1 ... chan_1 SI 2 ... SelectOp 1 state: REG SelectOp 2 state: REG chan_2 1. C1: Register in chan_1 2. C2: Register in chan_2
124. The select Expression: Deadlock Avoidance Coroutine 1: select<Unit> { chan_1.onSend(task) { ... } chan_2.onReceive { ... } } Coroutine 2: select<Unit> { chan_2.onSend(task) { ... } chan_1.onReceive { ... } } SI 1 ... chan_1 SI 2 ... SelectOp 1 state: REG SelectOp 2 state: REG chan_2 1. C1: Register in chan_1 3. C1: Rendezvous attempt in chan_2, wait for state != REG 2. C2: Register in chan_2
125. The select Expression: Deadlock Avoidance Coroutine 1: select<Unit> { chan_1.onSend(task) { ... } chan_2.onReceive { ... } } Coroutine 2: select<Unit> { chan_2.onSend(task) { ... } chan_1.onReceive { ... } } SI 1 ... chan_1 SI 2 ... SelectOp 1 state: REG SelectOp 2 state: REG chan_2 1. C1: Register in chan_1 3. C1: Rendezvous attempt in chan_2, wait for state != REG 2. C2: Register in chan_2 4. C2: Rendezvous attempt in chan_1, wait for state != REG Deadlock!
126. The select Expression: Deadlock Avoidance SI 1 ... chan_1 SI 2 ... SelectOp 1 state: REG SelectOp 2 state: REG chan_2 1. C1: Register in chan_1 3. C1: Rendezvous attempt in chan_2, wait for state != REG 2. C2: Register in chan_2 4. C2: Rendezvous attempt in chan_1, wait for state != REG 1. Each select instance has unique id 2. Change the state of the select instance of minimal id in a waiting cycle from REG to WAITING
127. The select Expression: Deadlock Avoidance SI 1 ... chan_1 SI 2 ... SelectOp 1 state: WAITING SelectOp 2 state: REG chan_2 1. C1: Register in chan_1 3. C1: Rendezvous attempt in chan_2, wait for state != REG 5. C1: Deadlock, change state to WAITING 2. C2: Register in chan_2 4. C2: Rendezvous attempt in chan_1, wait for state != REG 1. Each select instance has unique id 2. Change the state of the select instance of minimal id in a waiting cycle from REG to WAITING
128. The select Expression: Deadlock Avoidance SI 1 ... chan_1 X ... SelectOp 1 state: chan_1 SelectOp 2 state: DONE chan_2 1. C1: Register in chan_1 3. C1: Rendezvous attempt in chan_2, wait for state != REG 5. C1: Deadlock, change state to WAITING 2. C2: Register in chan_2 4. C2: Rendezvous attempt in chan_1, wait for state != REG 6. C2: Change 1st state to chan_1, rendezvous done 1. Each select instance has unique id 2. Change the state of the select instance of minimal id in a waiting cycle from REG to WAITING
129. The select Expression: Deadlock Avoidance SI 1 ... chan_1 X ... SelectOp 1 state: DONE SelectOp 2 state: DONE chan_2 1. C1: Register in chan_1 3. C1: Rendezvous attempt in chan_2, wait for state != REG 5. C1: Deadlock, change state to WAITING 7. C1: Selected, change state to DONE 2. C2: Register in chan_2 4. C2: Rendezvous attempt in chan_1, wait for state != REG 6. C2: Change 1st state to chan_1, rendezvous done 1. Each select instance has unique id 2. Change the state of the select instance of minimal id in a waiting cycle from REG to WAITING
130. The select Expression (capacity = 0) 130
131. The select Expression (capacity = 32) 131
132. The select Expression (capacity = 128) 132
133. Instead of Summary • Locks != bad • Non-blocking != scalable • Nowadays concurrent programming is full of trade-offs Channels in Kotlin Coroutines are the best in the world https://github.com/Kotlin/kotlinx.coroutines/tree/channels 133
134. 134 Questions