$30 off During Our Annual Pro Sale. View Details »

Understanding Channels

kavya
July 14, 2017

Understanding Channels

Channels provide a simple mechanism for goroutines to communicate, and a powerful construct to build sophisticated concurrency patterns. We will delve into the inner workings of channels and channel operations, including how they're supported by the runtime scheduler.

kavya

July 14, 2017
Tweet

More Decks by kavya

Other Decks in Programming

Transcript

  1. Understanding
    channels
    @kavya719

    View Slide

  2. kavya

    View Slide

  3. inner workings of
    channels

    View Slide

  4. goroutines
    to execute tasks independently,
    potentially in parallel.
    channels
    for communication, synchronization
    between goroutines.
    concurrency features

    View Slide

  5. func main() {
    tasks := getTasks()
    // Process each task.
    for _, task := range tasks {
    process(task)
    }


    ...
    }
    hellaTasks

    View Slide

  6. task queue
    func worker(ch) {
    for {

    // Get a task.
    task := <-taskCh

    process(task)
    }
    }
    func main() {
    // Buffered channel.
    ch := make(chan Task, 3)
    // Run fixed number of workers.
    for i := 0; i < numWorkers; i++ {
    go worker(ch)
    }
    // Send tasks to workers.
    hellaTasks := getTasks()
    for _, task := range hellaTasks {
    taskCh <- task
    }
    ...
    }

    View Slide

  7. task queue
    func worker(ch) {
    for {

    // Receive task.
    task := <-taskCh

    process(task)
    }
    }
    func main() {
    // Buffered channel.
    ch := make(chan Task, 3)
    // Run fixed number of workers.
    for i := 0; i < numWorkers; i++ {
    go worker(ch)
    }
    // Send tasks to workers.
    hellaTasks := getTasks()
    for _, task := range hellaTasks {
    taskCh <- task
    }
    ...
    }

    View Slide

  8. task queue
    func worker(ch) {
    for {

    // Receive task.
    task := <-taskCh

    process(task)
    }
    }
    func main() {
    // Buffered channel.
    ch := make(chan Task, 3)
    // Run fixed number of workers.
    for i := 0; i < numWorkers; i++ {
    go worker(ch)
    }
    // Send tasks to workers.
    hellaTasks := getTasks()
    for _, task := range hellaTasks {
    taskCh <- task
    }
    ...
    }

    View Slide

  9. goroutine-safe.
    store and pass values between goroutines.
    provide FIFO semantics.
    can cause goroutines to block and unblock.
    channels are inherently interesting

    View Slide

  10. making channels
    the hchan struct
    stepping back:
    design considerations
    sends and receives
    goroutine scheduling

    View Slide

  11. making channels

    View Slide

  12. unbuffered channel
    ch := make(chan int)
    buffered channel
    ch := make(chan Task, 3)
    make chan

    View Slide

  13. goroutine-safe
    stores up to capacity elements,

    and provides FIFO semantics
    sends values between goroutines
    can cause them to block, unblock
    buffered channel
    ch := make(chan Task, 3)

    View Slide

  14. buf
    lock
    ...
    recvx
    circular queue
    mutex
    send index
    receive index
    sendx
    hchan
    buffered channel
    ch := make(chan Task, 3)

    View Slide

  15. buf
    lock
    ...
    recvx
    circular queue
    mutex
    send index
    receive index
    sendx
    hchan
    0
    0
    2 1 0
    empty
    buffered channel
    ch := make(chan Task, 3)

    View Slide

  16. buf
    lock
    ...
    recvx
    circular queue
    mutex
    send index
    receive index
    sendx
    hchan
    0
    1
    buffered channel
    ch := make(chan Task, 3)
    an enqueue
    2 1 0

    View Slide

  17. buf
    lock
    ...
    recvx
    circular queue
    mutex
    send index
    receive index
    sendx
    hchan
    0
    0
    buffered channel
    ch := make(chan Task, 3)
    two more;
    so, full
    2 1 0

    View Slide

  18. buf
    lock
    ...
    recvx
    circular queue
    mutex
    send index
    receive index
    hchan
    sendx
    1
    0
    buffered channel
    ch := make(chan Task, 3)
    a dequeue
    2 1 0

    View Slide

  19. make chan
    heap
    stack
    high addr
    low addr
    program memory
    buffered channel
    ch := make(chan Task, 3)
    allocates an hchan struct on the heap.
    initializes it.
    returns a pointer to it.

    View Slide

  20. make chan
    heap
    allocates an hchan struct on the heap.
    initializes it.
    returns a pointer to it.
    ch
    buffered channel
    ch := make(chan Task, 3)
    This is why we can pass
    channels between functions,
    don’t need to pass 

    pointers to channels.

    View Slide

  21. sends and receives

    View Slide

  22. task queue
    func main() {
    ...
    for _, task := range tasks {
    taskCh <- task
    }
    ...
    }
    func worker() {
    for {
    task := <-taskCh

    process(task)
    }
    }
    G2
    G1

    View Slide

  23. ch <- task0
    buf
    ...
    lock
    G1

    View Slide

  24. ch <- task0
    G1
    buf
    ...
    lock
    1. acquire

    View Slide

  25. ch <- task0
    buf
    ...
    lock
    2. enqueue
    task0
    copy
    G1

    View Slide

  26. ch <- task0
    buf
    ...
    lock
    3. release
    task0
    copy
    G1

    View Slide

  27. t := <-ch
    G2
    buf
    ...
    lock
    task0
    copy

    View Slide

  28. buf
    ...
    lock
    1. acquire
    task0
    copy
    t := <-ch
    G2

    View Slide

  29. buf
    ...
    lock
    2. dequeue
    task0
    copy’
    t := <-ch
    G2

    View Slide

  30. buf
    ...
    lock
    3. release
    task0
    copy’
    t := <-ch
    G2

    View Slide

  31. no shared memory

    (except hchan)
    copies

    View Slide

  32. “Do not communicate by sharing memory; 

    instead, share memory by communicating.”
    no shared memory

    (except hchan)
    copies

    View Slide

  33. buf
    ...
    lock
    G1 G2

    View Slide

  34. buf
    ...
    lock
    ch <- task1
    G1 G2

    View Slide

  35. buf
    ...
    lock
    ch <- task1
    ch <- task2
    G1 G2

    View Slide

  36. buf
    ...
    lock
    ch <- task1
    ch <- task2
    ch <- task3
    G1 G2

    View Slide

  37. buf
    ...
    lock
    ch <- task1
    ch <- task2
    ch <- task3
    ch <- task4
    G1 G2

    View Slide

  38. buf
    ...
    lock
    ch <- task1
    ch <- task2
    ch <- task3
    ch <- task4
    ruh-roh, channel is full!
    G1’s execution is paused,
    resumed after a receive.
    G1 G2

    View Slide

  39. buf
    ...
    lock
    ch <- task1
    ch <- task2
    ch <- task3
    ch <- task4
    ?
    ruh-roh, channel is full!
    G1’s execution is paused,
    resumed after a receive.
    G1 G2

    View Slide

  40. goroutines are user-space threads.
    created and managed by the Go runtime, not the OS.
    lightweight compared to OS threads.
    the runtime scheduler schedules them onto OS threads.
    g5
    g1 g4
    g3
    OS thread2
    M:N scheduling
    OS thread1
    g1
    g2 g6
    g2
    interlude: the runtime scheduler

    View Slide

  41. Go’s M:N scheduling is described using three structures:
    M: OS thread
    G: goroutine
    P: context for scheduling.
    M

    View Slide

  42. M
    G
    Go’s M:N scheduling is described using three structures:
    M: OS thread
    G: goroutine
    P: context for scheduling.

    View Slide

  43. M
    P
    G
    Go’s M:N scheduling is described using three structures:
    M: OS thread
    G: goroutine
    P: context for scheduling.

    View Slide

  44. Ps hold the runqueues.
    In order to run goroutines (G),

    a thread (M) must hold a context (P).
    M
    P
    runQ
    G
    G
    G
    runnable
    Go’s M:N scheduling is described using three structures:
    M: OS thread
    G: goroutine
    P: context for scheduling.
    }

    View Slide

  45. Ps hold the runqueues.
    In order to run goroutines (G),

    a thread (M) must hold a context (P).
    M
    P
    runQ
    G
    G
    G
    runnable
    current G
    running
    Go’s M:N scheduling is described using three structures:
    M: OS thread
    G: goroutine
    P: context for scheduling.
    }

    View Slide

  46. pausing goroutines
    ch <- task4
    send on a full channel
    }

    View Slide

  47. ch <- task4
    calls into the scheduler
    gopark
    M
    P
    G1
    G
    G
    runQ
    runnable
    current G
    running
    }
    G1

    View Slide

  48. M
    P
    G1
    G
    G
    runnable
    runQ
    current G
    waiting
    ch <- task4
    sets G1 to waiting
    calls into the scheduler
    gopark
    }
    G1

    View Slide

  49. ch <- task4
    sets G1 to waiting
    calls into the scheduler
    removes association
    between G1, M
    gopark
    M
    P
    G
    G
    G1
    waiting
    runQ
    runnable
    }
    G1

    View Slide

  50. G1
    waiting
    M
    P
    G
    G
    runQ
    runnable
    current G
    ch <- task4
    schedules a runnable G 

    from the runqueue
    sets G1 to waiting
    calls into the scheduler
    removes association
    between G1, M
    “returns” 

    to a
    different G
    }
    G1
    gopark

    View Slide

  51. This is neat.
    G1 is blocked as needed, but not the OS thread.

    View Slide

  52. This is neat.
    G1 is blocked as needed, but not the OS thread.
    …great, but how do we resume
    the blocked goroutine?
    after a channel receive, and
    the channel is no longer full

    View Slide

  53. the hchan struct stores waiting senders, receivers as well.
    ...
    hchan
    sendq
    recvq
    waiting senders
    waiting receivers
    buf
    lock
    G
    elem
    sudog
    ...
    waiting G
    elem to send/ recv
    resuming goroutines
    represents a G
    in a waiting list
    stored as a sudog

    View Slide

  54. G
    elem
    sudog
    ...
    G1
    task4
    ch <- task4
    G1
    creates a sudog for itself, puts it in the sendq
    happens before calling into the scheduler.

    View Slide

  55. ...
    hchan
    sendq
    recvq
    G
    elem
    sudog
    ...
    G1
    task4
    ch <- task4
    G1
    creates a sudog for itself, puts it in the sendq
    happens before calling into the scheduler.
    receiver
    uses it to
    resume G1.

    View Slide

  56. ...
    hchan
    sendq
    recvq
    G
    elem
    sudog
    ...
    G1
    task4
    ch <- task4
    G1
    creates a sudog for itself, puts it in the sendq
    happens before calling into the scheduler.
    receiver
    uses it to
    resume G1.

    View Slide

  57. sendq
    ...
    g
    elem
    ...
    buf
    lock
    task4
    task1
    G1
    t := <-ch
    full buffer
    waiting
    sender
    {
    {
    G2
    waiting to send
    task4
    ||

    View Slide

  58. sendq
    ...
    g
    elem
    ...
    buf
    lock
    task4
    t := <-ch
    G1
    task1
    dequeue G2

    View Slide

  59. t := <-ch
    task1
    sendq
    ...
    g
    elem
    ...
    buf
    lock
    task4
    G1
    pops off sudog
    G2

    View Slide

  60. task4
    task1
    sendq
    ...
    g
    elem
    ...
    buf
    lock
    G1
    t := <-ch
    enqueues the sudog’s elem:
    G2

    View Slide

  61. need to set G1 to runnable
    task1
    task4
    sendq
    ...
    g
    elem
    ...
    buf
    lock
    G1
    t := <-ch
    G2

    View Slide

  62. t := <-ch
    calls into the scheduler
    G1
    waiting
    M
    P
    G2
    G
    runQ
    current G
    G2
    goready

    (G1)

    View Slide

  63. goready

    (G1)
    t := <-ch
    sets G1 to runnable
    calls into the scheduler
    G1
    runnable
    M
    P
    G2
    G
    runQ
    current G
    G2

    View Slide

  64. t := <-ch
    sets G1 to runnable
    calls into the scheduler
    puts it on runqueue
    G1
    M
    P
    G2
    G
    runQ
    current G
    returns to G2
    G2
    }runnable
    goready

    (G1)

    View Slide

  65. sends and receives
    when the receiver
    comes first

    View Slide

  66. recvq
    ...
    buf
    lock
    t := <-ch
    G2
    receive on an empty channel:

    G2’s execution is paused
    resumed after a send.
    empty buffer
    {

    View Slide

  67. receive on an empty channel:

    G2’s execution is paused
    resumed after a send.
    recvq
    ...
    buf
    lock
    G2
    t := <-ch
    empty buffer
    {

    View Slide

  68. receive on an empty channel:

    G2’s execution is paused
    resumed after a send.
    recvq
    ...
    buf
    lock
    G2
    t := <-ch
    empty buffer
    {
    set up state for resumption, and pause
    put a sudog in the recvq gopark G2.

    View Slide

  69. recvq
    ...
    buf
    lock
    G
    elem
    ...
    G2
    t
    empty buffer
    waiting
    receiver
    {
    {
    G2
    t := <-ch
    set up state for resumption, and pause
    put a sudog in the recvq gopark G2.
    waiting to receive
    to t
    ||

    View Slide

  70. recvq
    ...
    buf
    lock
    elem
    ...
    G
    t
    G2
    ch <- task
    G1

    View Slide

  71. recvq
    ...
    buf
    lock
    elem
    ...
    G
    t
    G2
    enqueue task in the buffer,
    goready(G2)
    could:
    ch <- task
    G1

    View Slide

  72. recvq
    ...
    buf
    lock
    elem
    ...
    G
    t
    G2
    or we can be smarter.
    ch <- task
    G1

    View Slide

  73. recvq
    ...
    buf
    lock
    elem
    ...
    G
    t
    G2
    ch <- task
    G1
    or we can be smarter.
    the memory location
    for the receive

    View Slide

  74. recvq
    ...
    buf
    lock
    elem
    ...
    G
    t
    G2
    ch <- task
    G1
    G1 writes to t directly.
    !
    the memory location
    for the receive

    View Slide

  75. }
    per-goroutine stacks heap
    stack
    G1 stack
    G2 stack
    t
    only operations in runtime where this happens.
    G1 writes to G2’s stack!
    direct send

    View Slide

  76. This is clever.
    On resuming, G2 does not need to acquire channel lock
    and manipulate the buffer.

    Also, one fewer memory copy.

    View Slide

  77. goroutine-safe
    hchan mutex
    store values, pass in FIFO.
    copying into and out of hchan buffer
    can cause goroutines to pause and resume.
    hchan sudog queues
    calls into the runtime scheduler 

    (gopark, goready)
    we now understand channels (sorta)…

    View Slide

  78. a note (or two)…
    unbuffered channels

    unbuffered channels always work like the “direct send” case:
    receiver first —> sender writes to receiver’s stack.
    sender first —> receiver receives directly from the sudog.

    select (general-case)
    all channels locked.
    a sudog is put in the sendq /recvq queues of all channels.
    channels unlocked, and the select-ing G is paused.
    CAS operation so there’s one winning case.
    resuming mirrors the pause sequence.

    View Slide

  79. stepping back…

    View Slide

  80. simplicity and performance

    View Slide

  81. queue with a lock preferred to lock-free implementation:
    “The performance improvement does not materialize from the air,
    it comes with code complexity increase.” — dvyokov
    simplicity

    View Slide

  82. calling into the runtime scheduler:
    OS thread remains unblocked.
    cross-goroutine stack reads and writes.
    goroutine wake-up path is lockless,
    potentially fewer memory copies
    need to account for
    memory management:

    garbage collection,
    stack-shrinking
    performance
    }

    View Slide

  83. simplicity and performance
    astute trade-offs
    between

    View Slide

  84. “The noblest pleasure is
    the joy of understanding.”
    - Leonardo da Vinci
    @kavya719
    speakerdeck.com/kavya719/understanding-channels

    View Slide

  85. Railgun: CloudFlare’s web proxy
    We chose to use Go because Railgun is inherently
    highly concurrent…
    Railgun makes extensive use of goroutines and channels.
    Docker: software container platform
    …Go is geared for distributed computing.
    It has many built-in features to support concurrency…
    Doozer: Heroku’s distributed data store
    Fortunately, Go’s concurrency primitives made the task
    much easier.
    Kubernetes: Google’s container orchestration platform
    Built in concurrency. Building distributed systems in Go is
    helped tremendously by being able to fan out …

    View Slide

  86. unbuffered channels
    unbuffered channel
    ch := make(chan int)
    That’s how unbuffered channels (always) work too.

    If receiver is first:
    > puts a sudog on the recvq and is paused.
    > subsequent sender will “send directly”, and resume the receiver.
    If sender is first: 

    > puts a sudog for itself on the sendq and is paused.

    > receiver then “receives directly” from the sudog and 

    resumes the sender.

    View Slide

  87. func worker() {
    for {
    select {
    case task := <-taskCh:
    process(task)
    case cmd := <-cmdCh:
    execute(cmd)
    }
    }
    }
    var taskCh = make(chan Task, 3)
    var cmdCh = make(chan Command)
    selects

    View Slide