Understanding Channels

Channels provide a simple mechanism for goroutines to communicate, and a powerful construct to build sophisticated concurrency patterns. We will delve into the inner workings of channels and channel operations, including how they're supported by the runtime scheduler.

Transcript

1. Understanding
   channels
   @kavya719
   

   View Slide

2. kavya
   

   View Slide

3. inner workings of
   channels
   

   View Slide

4. goroutines
   to execute tasks independently,
   potentially in parallel.
   channels
   for communication, synchronization
   between goroutines.
   concurrency features
   

   View Slide

5. func main() {
   tasks := getTasks()
   // Process each task.
   for _, task := range tasks {
   process(task)
   }

   

   ...
   }
   hellaTasks
   

   View Slide

6. task queue
   func worker(ch) {
   for {

   // Get a task.
   task := <-taskCh

   process(task)
   }
   }
   func main() {
   // Buffered channel.
   ch := make(chan Task, 3)
   // Run fixed number of workers.
   for i := 0; i < numWorkers; i++ {
   go worker(ch)
   }
   // Send tasks to workers.
   hellaTasks := getTasks()
   for _, task := range hellaTasks {
   taskCh <- task
   }
   ...
   }
   

   View Slide

7. task queue
   func worker(ch) {
   for {

   // Receive task.
   task := <-taskCh

   process(task)
   }
   }
   func main() {
   // Buffered channel.
   ch := make(chan Task, 3)
   // Run fixed number of workers.
   for i := 0; i < numWorkers; i++ {
   go worker(ch)
   }
   // Send tasks to workers.
   hellaTasks := getTasks()
   for _, task := range hellaTasks {
   taskCh <- task
   }
   ...
   }
   

   View Slide

8. task queue
   func worker(ch) {
   for {

   // Receive task.
   task := <-taskCh

   process(task)
   }
   }
   func main() {
   // Buffered channel.
   ch := make(chan Task, 3)
   // Run fixed number of workers.
   for i := 0; i < numWorkers; i++ {
   go worker(ch)
   }
   // Send tasks to workers.
   hellaTasks := getTasks()
   for _, task := range hellaTasks {
   taskCh <- task
   }
   ...
   }
   

   View Slide

9. goroutine-safe.
   store and pass values between goroutines.
   provide FIFO semantics.
   can cause goroutines to block and unblock.
   channels are inherently interesting
   

   View Slide

10. making channels
    the hchan struct
    stepping back:
    design considerations
    sends and receives
    goroutine scheduling
    

    View Slide

11. making channels
    

    View Slide

12. unbuffered channel
    ch := make(chan int)
    buffered channel
    ch := make(chan Task, 3)
    make chan
    

    View Slide

13. goroutine-safe
    stores up to capacity elements,

    and provides FIFO semantics
    sends values between goroutines
    can cause them to block, unblock
    buffered channel
    ch := make(chan Task, 3)
    

    View Slide

14. buf
    lock
    ...
    recvx
    circular queue
    mutex
    send index
    receive index
    sendx
    hchan
    buffered channel
    ch := make(chan Task, 3)
    

    View Slide

15. buf
    lock
    ...
    recvx
    circular queue
    mutex
    send index
    receive index
    sendx
    hchan
    0
    0
    2 1 0
    empty
    buffered channel
    ch := make(chan Task, 3)
    

    View Slide

16. buf
    lock
    ...
    recvx
    circular queue
    mutex
    send index
    receive index
    sendx
    hchan
    0
    1
    buffered channel
    ch := make(chan Task, 3)
    an enqueue
    2 1 0
    

    View Slide

17. buf
    lock
    ...
    recvx
    circular queue
    mutex
    send index
    receive index
    sendx
    hchan
    0
    0
    buffered channel
    ch := make(chan Task, 3)
    two more;
    so, full
    2 1 0
    

    View Slide

18. buf
    lock
    ...
    recvx
    circular queue
    mutex
    send index
    receive index
    hchan
    sendx
    1
    0
    buffered channel
    ch := make(chan Task, 3)
    a dequeue
    2 1 0
    

    View Slide

19. make chan
    heap
    stack
    high addr
    low addr
    program memory
    buffered channel
    ch := make(chan Task, 3)
    allocates an hchan struct on the heap.
    initializes it.
    returns a pointer to it.
    

    View Slide

20. make chan
    heap
    allocates an hchan struct on the heap.
    initializes it.
    returns a pointer to it.
    ch
    buffered channel
    ch := make(chan Task, 3)
    This is why we can pass
    channels between functions,
    don’t need to pass 

    pointers to channels.
    

    View Slide

21. sends and receives
    

    View Slide

22. task queue
    func main() {
    ...
    for _, task := range tasks {
    taskCh <- task
    }
    ...
    }
    func worker() {
    for {
    task := <-taskCh

    process(task)
    }
    }
    G2
    G1
    

    View Slide

23. ch <- task0
    buf
    ...
    lock
    G1
    

    View Slide

24. ch <- task0
    G1
    buf
    ...
    lock
    1. acquire
    

    View Slide

25. ch <- task0
    buf
    ...
    lock
    2. enqueue
    task0
    copy
    G1
    

    View Slide

26. ch <- task0
    buf
    ...
    lock
    3. release
    task0
    copy
    G1
    

    View Slide

27. t := <-ch
    G2
    buf
    ...
    lock
    task0
    copy
    

    View Slide

28. buf
    ...
    lock
    1. acquire
    task0
    copy
    t := <-ch
    G2
    

    View Slide

29. buf
    ...
    lock
    2. dequeue
    task0
    copy’
    t := <-ch
    G2
    

    View Slide

30. buf
    ...
    lock
    3. release
    task0
    copy’
    t := <-ch
    G2
    

    View Slide

31. no shared memory

    (except hchan)
    copies
    

    View Slide

32. “Do not communicate by sharing memory; 

    instead, share memory by communicating.”
    no shared memory

    (except hchan)
    copies
    

    View Slide

33. buf
    ...
    lock
    G1 G2
    

    View Slide

34. buf
    ...
    lock
    ch <- task1
    G1 G2
    

    View Slide

35. buf
    ...
    lock
    ch <- task1
    ch <- task2
    G1 G2
    

    View Slide

36. buf
    ...
    lock
    ch <- task1
    ch <- task2
    ch <- task3
    G1 G2
    

    View Slide

37. buf
    ...
    lock
    ch <- task1
    ch <- task2
    ch <- task3
    ch <- task4
    G1 G2
    

    View Slide

38. buf
    ...
    lock
    ch <- task1
    ch <- task2
    ch <- task3
    ch <- task4
    ruh-roh, channel is full!
    G1’s execution is paused,
    resumed after a receive.
    G1 G2
    

    View Slide

39. buf
    ...
    lock
    ch <- task1
    ch <- task2
    ch <- task3
    ch <- task4
    ?
    ruh-roh, channel is full!
    G1’s execution is paused,
    resumed after a receive.
    G1 G2
    

    View Slide

40. goroutines are user-space threads.
    created and managed by the Go runtime, not the OS.
    lightweight compared to OS threads.
    the runtime scheduler schedules them onto OS threads.
    g5
    g1 g4
    g3
    OS thread2
    M:N scheduling
    OS thread1
    g1
    g2 g6
    g2
    interlude: the runtime scheduler
    

    View Slide

41. Go’s M:N scheduling is described using three structures:
    M: OS thread
    G: goroutine
    P: context for scheduling.
    M
    

    View Slide

42. M
    G
    Go’s M:N scheduling is described using three structures:
    M: OS thread
    G: goroutine
    P: context for scheduling.
    

    View Slide

43. M
    P
    G
    Go’s M:N scheduling is described using three structures:
    M: OS thread
    G: goroutine
    P: context for scheduling.
    

    View Slide

44. Ps hold the runqueues.
    In order to run goroutines (G),

    a thread (M) must hold a context (P).
    M
    P
    runQ
    G
    G
    G
    runnable
    Go’s M:N scheduling is described using three structures:
    M: OS thread
    G: goroutine
    P: context for scheduling.
    }
    

    View Slide

45. Ps hold the runqueues.
    In order to run goroutines (G),

    a thread (M) must hold a context (P).
    M
    P
    runQ
    G
    G
    G
    runnable
    current G
    running
    Go’s M:N scheduling is described using three structures:
    M: OS thread
    G: goroutine
    P: context for scheduling.
    }
    

    View Slide

46. pausing goroutines
    ch <- task4
    send on a full channel
    }
    

    View Slide

47. ch <- task4
    calls into the scheduler
    gopark
    M
    P
    G1
    G
    G
    runQ
    runnable
    current G
    running
    }
    G1
    

    View Slide

48. M
    P
    G1
    G
    G
    runnable
    runQ
    current G
    waiting
    ch <- task4
    sets G1 to waiting
    calls into the scheduler
    gopark
    }
    G1
    

    View Slide

49. ch <- task4
    sets G1 to waiting
    calls into the scheduler
    removes association
    between G1, M
    gopark
    M
    P
    G
    G
    G1
    waiting
    runQ
    runnable
    }
    G1
    

    View Slide

50. G1
    waiting
    M
    P
    G
    G
    runQ
    runnable
    current G
    ch <- task4
    schedules a runnable G 

    from the runqueue
    sets G1 to waiting
    calls into the scheduler
    removes association
    between G1, M
    “returns” 

    to a
    different G
    }
    G1
    gopark
    

    View Slide

51. This is neat.
    G1 is blocked as needed, but not the OS thread.
    

    View Slide

52. This is neat.
    G1 is blocked as needed, but not the OS thread.
    …great, but how do we resume
    the blocked goroutine?
    after a channel receive, and
    the channel is no longer full
    

    View Slide

53. the hchan struct stores waiting senders, receivers as well.
    ...
    hchan
    sendq
    recvq
    waiting senders
    waiting receivers
    buf
    lock
    G
    elem
    sudog
    ...
    waiting G
    elem to send/ recv
    resuming goroutines
    represents a G
    in a waiting list
    stored as a sudog
    

    View Slide

54. G
    elem
    sudog
    ...
    G1
    task4
    ch <- task4
    G1
    creates a sudog for itself, puts it in the sendq
    happens before calling into the scheduler.
    

    View Slide

55. ...
    hchan
    sendq
    recvq
    G
    elem
    sudog
    ...
    G1
    task4
    ch <- task4
    G1
    creates a sudog for itself, puts it in the sendq
    happens before calling into the scheduler.
    receiver
    uses it to
    resume G1.
    

    View Slide

56. ...
    hchan
    sendq
    recvq
    G
    elem
    sudog
    ...
    G1
    task4
    ch <- task4
    G1
    creates a sudog for itself, puts it in the sendq
    happens before calling into the scheduler.
    receiver
    uses it to
    resume G1.
    

    View Slide

57. sendq
    ...
    g
    elem
    ...
    buf
    lock
    task4
    task1
    G1
    t := <-ch
    full buffer
    waiting
    sender
    {
    {
    G2
    waiting to send
    task4
    ||
    

    View Slide

58. sendq
    ...
    g
    elem
    ...
    buf
    lock
    task4
    t := <-ch
    G1
    task1
    dequeue G2
    

    View Slide

59. t := <-ch
    task1
    sendq
    ...
    g
    elem
    ...
    buf
    lock
    task4
    G1
    pops off sudog
    G2
    

    View Slide

60. task4
    task1
    sendq
    ...
    g
    elem
    ...
    buf
    lock
    G1
    t := <-ch
    enqueues the sudog’s elem:
    G2
    

    View Slide

61. need to set G1 to runnable
    task1
    task4
    sendq
    ...
    g
    elem
    ...
    buf
    lock
    G1
    t := <-ch
    G2
    

    View Slide

62. t := <-ch
    calls into the scheduler
    G1
    waiting
    M
    P
    G2
    G
    runQ
    current G
    G2
    goready

    (G1)
    

    View Slide

63. goready

    (G1)
    t := <-ch
    sets G1 to runnable
    calls into the scheduler
    G1
    runnable
    M
    P
    G2
    G
    runQ
    current G
    G2
    

    View Slide

64. t := <-ch
    sets G1 to runnable
    calls into the scheduler
    puts it on runqueue
    G1
    M
    P
    G2
    G
    runQ
    current G
    returns to G2
    G2
    }runnable
    goready

    (G1)
    

    View Slide

65. sends and receives
    when the receiver
    comes first
    

    View Slide

66. recvq
    ...
    buf
    lock
    t := <-ch
    G2
    receive on an empty channel:

    G2’s execution is paused
    resumed after a send.
    empty buffer
    {
    

    View Slide

67. receive on an empty channel:

    G2’s execution is paused
    resumed after a send.
    recvq
    ...
    buf
    lock
    G2
    t := <-ch
    empty buffer
    {
    

    View Slide

68. receive on an empty channel:

    G2’s execution is paused
    resumed after a send.
    recvq
    ...
    buf
    lock
    G2
    t := <-ch
    empty buffer
    {
    set up state for resumption, and pause
    put a sudog in the recvq gopark G2.
    

    View Slide

69. recvq
    ...
    buf
    lock
    G
    elem
    ...
    G2
    t
    empty buffer
    waiting
    receiver
    {
    {
    G2
    t := <-ch
    set up state for resumption, and pause
    put a sudog in the recvq gopark G2.
    waiting to receive
    to t
    ||
    

    View Slide

70. recvq
    ...
    buf
    lock
    elem
    ...
    G
    t
    G2
    ch <- task
    G1
    

    View Slide

71. recvq
    ...
    buf
    lock
    elem
    ...
    G
    t
    G2
    enqueue task in the buffer,
    goready(G2)
    could:
    ch <- task
    G1
    

    View Slide

72. recvq
    ...
    buf
    lock
    elem
    ...
    G
    t
    G2
    or we can be smarter.
    ch <- task
    G1
    

    View Slide

73. recvq
    ...
    buf
    lock
    elem
    ...
    G
    t
    G2
    ch <- task
    G1
    or we can be smarter.
    the memory location
    for the receive
    

    View Slide

74. recvq
    ...
    buf
    lock
    elem
    ...
    G
    t
    G2
    ch <- task
    G1
    G1 writes to t directly.
    !
    the memory location
    for the receive
    

    View Slide

75. }
    per-goroutine stacks heap
    stack
    G1 stack
    G2 stack
    t
    only operations in runtime where this happens.
    G1 writes to G2’s stack!
    direct send
    

    View Slide

76. This is clever.
    On resuming, G2 does not need to acquire channel lock
    and manipulate the buffer.
    

    Also, one fewer memory copy.
    

    View Slide

77. goroutine-safe
    hchan mutex
    store values, pass in FIFO.
    copying into and out of hchan buffer
    can cause goroutines to pause and resume.
    hchan sudog queues
    calls into the runtime scheduler 

    (gopark, goready)
    we now understand channels (sorta)…
    

    View Slide

78. a note (or two)…
    unbuffered channels

    unbuffered channels always work like the “direct send” case:
    receiver first —> sender writes to receiver’s stack.
    sender first —> receiver receives directly from the sudog.

    select (general-case)
    all channels locked.
    a sudog is put in the sendq /recvq queues of all channels.
    channels unlocked, and the select-ing G is paused.
    CAS operation so there’s one winning case.
    resuming mirrors the pause sequence.
    

    View Slide

79. stepping back…
    

    View Slide

80. simplicity and performance
    

    View Slide

81. queue with a lock preferred to lock-free implementation:
    “The performance improvement does not materialize from the air,
    it comes with code complexity increase.” — dvyokov
    simplicity
    

    View Slide

82. calling into the runtime scheduler:
    OS thread remains unblocked.
    cross-goroutine stack reads and writes.
    goroutine wake-up path is lockless,
    potentially fewer memory copies
    need to account for
    memory management:
    

    garbage collection,
    stack-shrinking
    performance
    }
    

    View Slide

83. simplicity and performance
    astute trade-offs
    between
    

    View Slide

84. “The noblest pleasure is
    the joy of understanding.”
    - Leonardo da Vinci
    @kavya719
    speakerdeck.com/kavya719/understanding-channels
    

    View Slide

85. Railgun: CloudFlare’s web proxy
    We chose to use Go because Railgun is inherently
    highly concurrent…
    Railgun makes extensive use of goroutines and channels.
    Docker: software container platform
    …Go is geared for distributed computing.
    It has many built-in features to support concurrency…
    Doozer: Heroku’s distributed data store
    Fortunately, Go’s concurrency primitives made the task
    much easier.
    Kubernetes: Google’s container orchestration platform
    Built in concurrency. Building distributed systems in Go is
    helped tremendously by being able to fan out …
    

    View Slide

86. unbuffered channels
    unbuffered channel
    ch := make(chan int)
    That’s how unbuffered channels (always) work too.

    If receiver is first:
    > puts a sudog on the recvq and is paused.
    > subsequent sender will “send directly”, and resume the receiver.
    If sender is first: 

    > puts a sudog for itself on the sendq and is paused.

    > receiver then “receives directly” from the sudog and 

    resumes the sender.
    

    View Slide

87. func worker() {
    for {
    select {
    case task := <-taskCh:
    process(task)
    case cmd := <-cmdCh:
    execute(cmd)
    }
    }
    }
    var taskCh = make(chan Task, 3)
    var cmdCh = make(chan Command)
    selects
    

    View Slide