Understanding Channels

69c2f55e7b157c112c0d988ddba7484d?s=47 kavya
July 14, 2017

Understanding Channels

Channels provide a simple mechanism for goroutines to communicate, and a powerful construct to build sophisticated concurrency patterns. We will delve into the inner workings of channels and channel operations, including how they're supported by the runtime scheduler.

69c2f55e7b157c112c0d988ddba7484d?s=128

kavya

July 14, 2017
Tweet

Transcript

  1. Understanding channels @kavya719

  2. kavya

  3. inner workings of channels

  4. goroutines to execute tasks independently, potentially in parallel. channels for

    communication, synchronization between goroutines. concurrency features
  5. func main() { tasks := getTasks() // Process each task.

    for _, task := range tasks { process(task) }
 
 ... } hellaTasks
  6. task queue func worker(ch) { for {
 // Get a

    task. task := <-taskCh
 process(task) } } func main() { // Buffered channel. ch := make(chan Task, 3) // Run fixed number of workers. for i := 0; i < numWorkers; i++ { go worker(ch) } // Send tasks to workers. hellaTasks := getTasks() for _, task := range hellaTasks { taskCh <- task } ... }
  7. task queue func worker(ch) { for {
 // Receive task.

    task := <-taskCh
 process(task) } } func main() { // Buffered channel. ch := make(chan Task, 3) // Run fixed number of workers. for i := 0; i < numWorkers; i++ { go worker(ch) } // Send tasks to workers. hellaTasks := getTasks() for _, task := range hellaTasks { taskCh <- task } ... }
  8. task queue func worker(ch) { for {
 // Receive task.

    task := <-taskCh
 process(task) } } func main() { // Buffered channel. ch := make(chan Task, 3) // Run fixed number of workers. for i := 0; i < numWorkers; i++ { go worker(ch) } // Send tasks to workers. hellaTasks := getTasks() for _, task := range hellaTasks { taskCh <- task } ... }
  9. goroutine-safe. store and pass values between goroutines. provide FIFO semantics.

    can cause goroutines to block and unblock. channels are inherently interesting
  10. making channels the hchan struct stepping back: design considerations sends

    and receives goroutine scheduling
  11. making channels

  12. unbuffered channel ch := make(chan int) buffered channel ch :=

    make(chan Task, 3) make chan
  13. goroutine-safe stores up to capacity elements,
 and provides FIFO semantics

    sends values between goroutines can cause them to block, unblock buffered channel ch := make(chan Task, 3)
  14. buf lock ... recvx circular queue mutex send index receive

    index sendx hchan buffered channel ch := make(chan Task, 3)
  15. buf lock ... recvx circular queue mutex send index receive

    index sendx hchan 0 0 2 1 0 empty buffered channel ch := make(chan Task, 3)
  16. buf lock ... recvx circular queue mutex send index receive

    index sendx hchan 0 1 buffered channel ch := make(chan Task, 3) an enqueue 2 1 0
  17. buf lock ... recvx circular queue mutex send index receive

    index sendx hchan 0 0 buffered channel ch := make(chan Task, 3) two more; so, full 2 1 0
  18. buf lock ... recvx circular queue mutex send index receive

    index hchan sendx 1 0 buffered channel ch := make(chan Task, 3) a dequeue 2 1 0
  19. make chan heap stack high addr low addr program memory

    buffered channel ch := make(chan Task, 3) allocates an hchan struct on the heap. initializes it. returns a pointer to it.
  20. make chan heap allocates an hchan struct on the heap.

    initializes it. returns a pointer to it. ch buffered channel ch := make(chan Task, 3) This is why we can pass channels between functions, don’t need to pass 
 pointers to channels.
  21. sends and receives

  22. task queue func main() { ... for _, task :=

    range tasks { taskCh <- task } ... } func worker() { for { task := <-taskCh
 process(task) } } G2 G1
  23. ch <- task0 buf ... lock G1

  24. ch <- task0 G1 buf ... lock 1. acquire

  25. ch <- task0 buf ... lock 2. enqueue task0 copy

    G1
  26. ch <- task0 buf ... lock 3. release task0 copy

    G1
  27. t := <-ch G2 buf ... lock task0 copy

  28. buf ... lock 1. acquire task0 copy t := <-ch

    G2
  29. buf ... lock 2. dequeue task0 copy’ t := <-ch

    G2
  30. buf ... lock 3. release task0 copy’ t := <-ch

    G2
  31. no shared memory
 (except hchan) copies

  32. “Do not communicate by sharing memory; 
 instead, share memory

    by communicating.” no shared memory
 (except hchan) copies
  33. buf ... lock G1 G2

  34. buf ... lock ch <- task1 G1 G2

  35. buf ... lock ch <- task1 ch <- task2 G1

    G2
  36. buf ... lock ch <- task1 ch <- task2 ch

    <- task3 G1 G2
  37. buf ... lock ch <- task1 ch <- task2 ch

    <- task3 ch <- task4 G1 G2
  38. buf ... lock ch <- task1 ch <- task2 ch

    <- task3 ch <- task4 ruh-roh, channel is full! G1’s execution is paused, resumed after a receive. G1 G2
  39. buf ... lock ch <- task1 ch <- task2 ch

    <- task3 ch <- task4 ? ruh-roh, channel is full! G1’s execution is paused, resumed after a receive. G1 G2
  40. goroutines are user-space threads. created and managed by the Go

    runtime, not the OS. lightweight compared to OS threads. the runtime scheduler schedules them onto OS threads. g5 g1 g4 g3 OS thread2 M:N scheduling OS thread1 g1 g2 g6 g2 interlude: the runtime scheduler
  41. Go’s M:N scheduling is described using three structures: M: OS

    thread G: goroutine P: context for scheduling. M
  42. M G Go’s M:N scheduling is described using three structures:

    M: OS thread G: goroutine P: context for scheduling.
  43. M P G Go’s M:N scheduling is described using three

    structures: M: OS thread G: goroutine P: context for scheduling.
  44. Ps hold the runqueues. In order to run goroutines (G),


    a thread (M) must hold a context (P). M P runQ G G G runnable Go’s M:N scheduling is described using three structures: M: OS thread G: goroutine P: context for scheduling. }
  45. Ps hold the runqueues. In order to run goroutines (G),


    a thread (M) must hold a context (P). M P runQ G G G runnable current G running Go’s M:N scheduling is described using three structures: M: OS thread G: goroutine P: context for scheduling. }
  46. pausing goroutines ch <- task4 send on a full channel

    }
  47. ch <- task4 calls into the scheduler gopark M P

    G1 G G runQ runnable current G running } G1
  48. M P G1 G G runnable runQ current G waiting

    ch <- task4 sets G1 to waiting calls into the scheduler gopark } G1
  49. ch <- task4 sets G1 to waiting calls into the

    scheduler removes association between G1, M gopark M P G G G1 waiting runQ runnable } G1
  50. G1 waiting M P G G runQ runnable current G

    ch <- task4 schedules a runnable G 
 from the runqueue sets G1 to waiting calls into the scheduler removes association between G1, M “returns” 
 to a different G } G1 gopark
  51. This is neat. G1 is blocked as needed, but not

    the OS thread.
  52. This is neat. G1 is blocked as needed, but not

    the OS thread. …great, but how do we resume the blocked goroutine? after a channel receive, and the channel is no longer full
  53. the hchan struct stores waiting senders, receivers as well. ...

    hchan sendq recvq waiting senders waiting receivers buf lock G elem sudog ... waiting G elem to send/ recv resuming goroutines represents a G in a waiting list stored as a sudog
  54. G elem sudog ... G1 task4 ch <- task4 G1

    creates a sudog for itself, puts it in the sendq happens before calling into the scheduler.
  55. ... hchan sendq recvq G elem sudog ... G1 task4

    ch <- task4 G1 creates a sudog for itself, puts it in the sendq happens before calling into the scheduler. receiver uses it to resume G1.
  56. ... hchan sendq recvq G elem sudog ... G1 task4

    ch <- task4 G1 creates a sudog for itself, puts it in the sendq happens before calling into the scheduler. receiver uses it to resume G1.
  57. sendq ... g elem ... buf lock task4 task1 G1

    t := <-ch full buffer waiting sender { { G2 waiting to send task4 ||
  58. sendq ... g elem ... buf lock task4 t :=

    <-ch G1 task1 dequeue G2
  59. t := <-ch task1 sendq ... g elem ... buf

    lock task4 G1 pops off sudog G2
  60. task4 task1 sendq ... g elem ... buf lock G1

    t := <-ch enqueues the sudog’s elem: G2
  61. need to set G1 to runnable task1 task4 sendq ...

    g elem ... buf lock G1 t := <-ch G2
  62. t := <-ch calls into the scheduler G1 waiting M

    P G2 G runQ current G G2 goready
 (G1)
  63. goready
 (G1) t := <-ch sets G1 to runnable calls

    into the scheduler G1 runnable M P G2 G runQ current G G2
  64. t := <-ch sets G1 to runnable calls into the

    scheduler puts it on runqueue G1 M P G2 G runQ current G returns to G2 G2 }runnable goready
 (G1)
  65. sends and receives when the receiver comes first

  66. recvq ... buf lock t := <-ch G2 receive on

    an empty channel:
 G2’s execution is paused resumed after a send. empty buffer {
  67. receive on an empty channel:
 G2’s execution is paused resumed

    after a send. recvq ... buf lock G2 t := <-ch empty buffer {
  68. receive on an empty channel:
 G2’s execution is paused resumed

    after a send. recvq ... buf lock G2 t := <-ch empty buffer { set up state for resumption, and pause put a sudog in the recvq gopark G2.
  69. recvq ... buf lock G elem ... G2 t empty

    buffer waiting receiver { { G2 t := <-ch set up state for resumption, and pause put a sudog in the recvq gopark G2. waiting to receive to t ||
  70. recvq ... buf lock elem ... G t G2 ch

    <- task G1
  71. recvq ... buf lock elem ... G t G2 enqueue

    task in the buffer, goready(G2) could: ch <- task G1
  72. recvq ... buf lock elem ... G t G2 or

    we can be smarter. ch <- task G1
  73. recvq ... buf lock elem ... G t G2 ch

    <- task G1 or we can be smarter. the memory location for the receive
  74. recvq ... buf lock elem ... G t G2 ch

    <- task G1 G1 writes to t directly. ! the memory location for the receive
  75. } per-goroutine stacks heap stack G1 stack G2 stack t

    only operations in runtime where this happens. G1 writes to G2’s stack! direct send
  76. This is clever. On resuming, G2 does not need to

    acquire channel lock and manipulate the buffer. 
 Also, one fewer memory copy.
  77. goroutine-safe hchan mutex store values, pass in FIFO. copying into

    and out of hchan buffer can cause goroutines to pause and resume. hchan sudog queues calls into the runtime scheduler 
 (gopark, goready) we now understand channels (sorta)…
  78. a note (or two)… unbuffered channels
 unbuffered channels always work

    like the “direct send” case: receiver first —> sender writes to receiver’s stack. sender first —> receiver receives directly from the sudog.
 select (general-case) all channels locked. a sudog is put in the sendq /recvq queues of all channels. channels unlocked, and the select-ing G is paused. CAS operation so there’s one winning case. resuming mirrors the pause sequence.
  79. stepping back…

  80. simplicity and performance

  81. queue with a lock preferred to lock-free implementation: “The performance

    improvement does not materialize from the air, it comes with code complexity increase.” — dvyokov simplicity
  82. calling into the runtime scheduler: OS thread remains unblocked. cross-goroutine

    stack reads and writes. goroutine wake-up path is lockless, potentially fewer memory copies need to account for memory management: 
 garbage collection, stack-shrinking performance }
  83. simplicity and performance astute trade-offs between

  84. “The noblest pleasure is the joy of understanding.” - Leonardo

    da Vinci @kavya719 speakerdeck.com/kavya719/understanding-channels
  85. Railgun: CloudFlare’s web proxy We chose to use Go because

    Railgun is inherently highly concurrent… Railgun makes extensive use of goroutines and channels. Docker: software container platform …Go is geared for distributed computing. It has many built-in features to support concurrency… Doozer: Heroku’s distributed data store Fortunately, Go’s concurrency primitives made the task much easier. Kubernetes: Google’s container orchestration platform Built in concurrency. Building distributed systems in Go is helped tremendously by being able to fan out …
  86. unbuffered channels unbuffered channel ch := make(chan int) That’s how

    unbuffered channels (always) work too.
 If receiver is first: > puts a sudog on the recvq and is paused. > subsequent sender will “send directly”, and resume the receiver. If sender is first: 
 > puts a sudog for itself on the sendq and is paused.
 > receiver then “receives directly” from the sudog and 
 resumes the sender.
  87. func worker() { for { select { case task :=

    <-taskCh: process(task) case cmd := <-cmdCh: execute(cmd) } } } var taskCh = make(chan Task, 3) var cmdCh = make(chan Command) selects