Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Illustrated Guide To Kubernetes Networking

Tim Hockin
September 21, 2016

Illustrated Guide To Kubernetes Networking

A short walk through of some ideas around container networking.

Tim Hockin

September 21, 2016
Tweet

More Decks by Tim Hockin

Other Decks in Technology

Transcript

  1. Google Cloud Platform
    An Illustrated Guide to
    Kubernetes Networking
    Tim Hockin
    Senior Staff Software Engineer
    @thockin

    View Slide

  2. Google Cloud Platform
    Layer 2: ethernet

    View Slide

  3. Google Cloud Platform
    node-a
    node-c
    node-b
    node-d
    192.168.1.1/16
    11:22:33:44:55:01
    192.168.1.2/16
    01:23:45:67:89:02
    192.168.1.3/16
    11:22:33:44:55:03
    192.168.1.4/16
    01:23:45:67:89:04
    L2
    switch

    View Slide

  4. Google Cloud Platform
    node-a
    node-c
    node-b
    node-d
    L2
    to: 192.168.1.3
    from: 192.168.1.1
    GET / 192.168.1.1/16
    11:22:33:44:55:01
    192.168.1.2/16
    01:23:45:67:89:02
    192.168.1.3/16
    11:22:33:44:55:03
    192.168.1.4/16
    01:23:45:67:89:04

    View Slide

  5. Google Cloud Platform
    node-d
    node-b
    node-a
    node-c
    L2
    to:
    from: 11:22:33:44:55:01
    who has 192.168.1.3?
    to: 192.168.1.3
    from: 192.168.1.1
    GET / 192.168.1.1/16
    11:22:33:44:55:01
    192.168.1.2/16
    01:23:45:67:89:02
    192.168.1.3/16
    11:22:33:44:55:03
    192.168.1.4/16
    01:23:45:67:89:04
    “ARP request”

    View Slide

  6. Google Cloud Platform
    node-a
    node-c
    node-b
    node-d
    L2
    to: 192.168.1.3
    from: 192.168.1.1
    GET / 192.168.1.1/16
    11:22:33:44:55:01
    192.168.1.2/16
    01:23:45:67:89:02
    192.168.1.3/16
    11:22:33:44:55:03
    192.168.1.4/16
    01:23:45:67:89:04
    to: 11:22:33:44:55:01
    from: 11:22:33:44:55:03
    I have 192.168.1.3
    “ARP response”

    View Slide

  7. Google Cloud Platform
    node-a
    node-c
    node-b
    node-d
    L2
    to: 192.168.1.3
    via: 11:22:33:44:55:03
    from: 192.168.1.1
    GET /
    192.168.1.1/16
    11:22:33:44:55:01
    192.168.1.2/16
    01:23:45:67:89:02
    192.168.1.3/16
    11:22:33:44:55:03
    192.168.1.4/16
    01:23:45:67:89:04

    View Slide

  8. Google Cloud Platform
    node-a
    root netns
    eth0: 192.168.1.1/16
    11:22:33:44:55:01
    L2 with containers
    cbr0: 10.0.1.1/24
    ctr-1
    eth0: 10.0.1.2/24
    ctr-2
    eth0: 10.0.1.3/24
    ctr-3
    eth0: 10.0.1.4/24

    View Slide

  9. Google Cloud Platform
    node-a
    192.168.1.1/16
    11:22:33:44:55:01
    node-c
    node-b
    node-d
    192.168.1.2/16
    01:23:45:67:89:02
    192.168.1.3/16
    11:22:33:44:55:03
    192.168.1.4/16
    01:23:45:67:89:04
    L2
    ctr-1 10.0.1.2
    aa:bb:cc:dd:e1:01
    ctr-2 10.0.3.2
    aa:bb:cc:dd:e3:02

    View Slide

  10. Google Cloud Platform
    node-a
    192.168.1.1/16
    11:22:33:44:55:01
    node-c
    node-b
    node-d
    192.168.1.2/16
    01:23:45:67:89:02
    192.168.1.3/16
    11:22:33:44:55:03
    192.168.1.4/16
    01:23:45:67:89:04
    L2
    ctr-1 10.0.1.2
    aa:bb:cc:dd:e1:01
    ctr-2 10.0.3.2
    aa:bb:cc:dd:e3:02
    to: 10.0.3.2
    from: 10.0.1.2
    GET /

    View Slide

  11. Google Cloud Platform
    node-a
    192.168.1.1/16
    11:22:33:44:55:01
    node-c
    node-b
    node-d
    192.168.1.2/16
    01:23:45:67:89:02
    192.168.1.3/16
    11:22:33:44:55:03
    192.168.1.4/16
    01:23:45:67:89:04
    L2
    ctr-1 10.0.1.2
    aa:bb:cc:dd:e1:01
    ctr-2 10.0.3.2
    aa:bb:cc:dd:e3:02
    to: 10.0.3.2
    from: 10.0.1.2
    GET /
    to:
    from: aa:bb:cc:dd:e1:01
    who has 10.0.3.2?
    “ARP request”

    View Slide

  12. Google Cloud Platform
    node-a
    192.168.1.1/16
    11:22:33:44:55:01
    node-c
    node-b
    node-d
    192.168.1.2/16
    01:23:45:67:89:02
    192.168.1.3/16
    11:22:33:44:55:03
    192.168.1.4/16
    01:23:45:67:89:04
    L2
    ctr-1 10.0.1.2
    aa:bb:cc:dd:e1:01
    ctr-2 10.0.3.2
    aa:bb:cc:dd:e3:02
    to: 10.0.3.2
    from: 10.0.1.2
    GET /
    to: aa:bb:cc:dd:e1:01
    from: 11:22:33:44:55:03
    I have 10.0.3.2
    “proxy ARP
    response”

    View Slide

  13. Google Cloud Platform
    node-a
    192.168.1.1/16
    11:22:33:44:55:01
    node-c
    node-b
    node-d
    192.168.1.2/16
    01:23:45:67:89:02
    192.168.1.3/16
    11:22:33:44:55:03
    192.168.1.4/16
    01:23:45:67:89:04
    L2
    ctr-1 10.0.1.2
    aa:bb:cc:dd:e1:01
    ctr-2 10.0.3.2
    aa:bb:cc:dd:e3:02
    to: 10.0.3.2
    via: 11:22:33:44:55:03
    from: 10.0.1.2
    GET /

    View Slide

  14. Google Cloud Platform
    Layer 3 - IP

    View Slide

  15. Google Cloud Platform
    node-a
    node-c
    node-b
    node-d
    192.168.1.1/32 192.168.1.2/32
    192.168.1.3/32 192.168.1.4/32
    L3
    gateway

    View Slide

  16. Google Cloud Platform
    node-a
    node-c
    node-b
    node-d
    192.168.1.1/32 192.168.1.2/32
    192.168.1.3/32 192.168.1.4/32
    L3
    to: 192.168.1.3
    from: 192.168.1.1
    GET /

    View Slide

  17. Google Cloud Platform
    node-a
    node-c
    node-b
    node-d
    192.168.1.1/32 192.168.1.2/32
    192.168.1.3/32 192.168.1.4/32
    L3
    to: 192.168.1.3
    from: 192.168.1.1
    GET /

    View Slide

  18. Google Cloud Platform
    node-a
    node-c
    node-b
    node-d
    192.168.1.1/32 192.168.1.2/32
    192.168.1.3/32 192.168.1.4/32
    L3
    to: 192.168.1.3
    from: 192.168.1.1
    GET /
    routing decision,
    static or learned
    (e.g. BGP)

    View Slide

  19. Google Cloud Platform
    node-a
    192.168.1.1/32
    node-c
    node-b
    node-d
    192.168.1.2/32
    192.168.1.3/32 192.168.1.4/32
    L3
    ctr-1 10.0.1.2
    ctr-2 10.0.3.2

    View Slide

  20. Google Cloud Platform
    node-a
    192.168.1.1/32
    node-c
    node-b
    node-d
    192.168.1.2/32
    192.168.1.3/32 192.168.1.4/32
    L3
    ctr-1 10.0.1.2
    ctr-2 10.0.3.2
    to: 10.0.3.2
    from: 10.0.1.2
    GET /

    View Slide

  21. Google Cloud Platform
    node-a
    192.168.1.1/32
    node-c
    node-b
    node-d
    192.168.1.2/32
    192.168.1.3/32 192.168.1.4/32
    L3
    ctr-1 10.0.1.2
    ctr-2 10.0.3.2
    to: 10.0.3.2
    from: 10.0.1.2
    GET /

    View Slide

  22. Google Cloud Platform
    node-a
    192.168.1.1/32
    node-c
    node-b
    node-d
    192.168.1.2/32
    192.168.1.3/32 192.168.1.4/32
    L3
    ctr-1 10.0.1.2
    ctr-2 10.0.3.2
    to: 10.0.3.2
    from: 10.0.1.2
    GET /
    routing decision,
    static or learned
    (e.g. BGP)

    View Slide

  23. Google Cloud Platform
    node-a
    192.168.1.1/32
    node-c
    node-b
    node-d
    192.168.1.2/32
    192.168.1.3/32 192.168.1.4/32
    L3
    ctr-1 10.0.1.2
    ctr-2 10.0.3.2
    to: 10.0.3.2
    from: 10.0.1.2
    GET /

    View Slide

  24. Google Cloud Platform
    Overlays
    Q: When should I use an overlay?
    A: When nothing else works, or when you have
    specific reasons to want it (e.g. the added value
    of management)

    View Slide

  25. Google Cloud Platform
    node-a
    root netns
    eth0: 192.168.1.1/16
    Overlay (e.g. flannel, weave)
    cbr0: 10.0.1.1/24
    ctr-1
    eth0: 10.0.1.2/24
    ctr-2
    eth0: 10.0.1.3/24
    ctr-3
    eth0: 10.0.1.4/24
    flannel0: 10.0.1.254/16

    View Slide

  26. Google Cloud Platform
    node-a
    root netns
    eth0: 192.168.1.1/16
    cbr0: 10.0.1.1/24
    ctr-1
    eth0: 10.0.1.2/24
    ctr-2
    eth0: 10.0.1.3/24
    ctr-3
    eth0: 10.0.1.4/24
    flannel0: 10.0.1.254/16
    to: 10.0.3.2
    from: 10.0.1.2
    GET /
    Overlay (e.g. flannel, weave)

    View Slide

  27. Google Cloud Platform
    node-a
    root netns
    eth0: 192.168.1.1/16
    cbr0: 10.0.1.1/24
    ctr-1
    eth0: 10.0.1.2/24
    ctr-2
    eth0: 10.0.1.3/24
    ctr-3
    eth0: 10.0.1.4/24
    flannel0: 10.0.1.254/16
    to: 10.0.3.2
    from: 10.0.1.2
    GET /
    Overlay (e.g. flannel, weave)

    View Slide

  28. Google Cloud Platform
    node-a
    root netns
    eth0: 192.168.1.1/16
    cbr0: 10.0.1.1/24
    ctr-1
    eth0: 10.0.1.2/24
    ctr-2
    eth0: 10.0.1.3/24
    ctr-3
    eth0: 10.0.1.4/24
    flannel0: 10.0.1.254/16
    to: 10.0.3.2
    from: 10.0.1.2
    GET /
    Overlay (e.g. flannel, weave)

    View Slide

  29. Google Cloud Platform
    node-a
    root netns
    eth0: 192.168.1.1/16
    cbr0: 10.0.1.1/24
    ctr-1
    eth0: 10.0.1.2/24
    ctr-2
    eth0: 10.0.1.3/24
    ctr-3
    eth0: 10.0.1.4/24
    flannel0: 10.0.1.254/16
    to: 192.168.1.3
    from: 192.168.1.1
    encap:
    to: 10.0.3.2
    from: 10.0.1.2
    GET /
    Overlay (e.g. flannel, weave)

    View Slide

  30. Google Cloud Platform
    node-a
    192.168.1.1/16
    node-c
    node-b
    node-d
    192.168.1.2/16
    192.168.1.3/16 192.168.1.4/16
    ctr-1 10.0.1.2
    ctr-2 10.0.3.2
    Overlay (e.g. flannel, weave)

    View Slide

  31. Google Cloud Platform
    node-c
    root netns
    eth0: 192.168.1.3/16
    cbr0: 10.0.3.1/24
    ctr-4
    eth0: 10.0.3.2/24
    ctr-5
    eth0: 10.0.3.3/24
    ctr-6
    eth0: 10.0.3.4/24
    flannel0: 10.0.3.254/16
    to: 192.168.1.3
    from: 192.168.1.1
    encap:
    to: 10.0.3.2
    from: 10.0.1.2
    GET /
    Overlay (e.g. flannel, weave)

    View Slide

  32. Google Cloud Platform
    node-c
    root netns
    eth0: 192.168.1.3/16
    cbr0: 10.0.3.1/24
    ctr-4
    eth0: 10.0.3.2/24
    ctr-5
    eth0: 10.0.3.3/24
    ctr-6
    eth0: 10.0.3.4/24
    flannel0: 10.0.3.254/16
    to: 10.0.3.2
    from: 10.0.1.2
    GET /
    Overlay (e.g. flannel, weave)

    View Slide

  33. Google Cloud Platform
    node-c
    root netns
    eth0: 192.168.1.3/16
    cbr0: 10.0.3.1/24
    ctr-4
    eth0: 10.0.3.2/24
    ctr-5
    eth0: 10.0.3.3/24
    ctr-6
    eth0: 10.0.3.4/24
    flannel0: 10.0.3.254/16
    to: 10.0.3.2
    from: 10.0.1.2
    GET /
    Overlay (e.g. flannel, weave)

    View Slide

  34. Google Cloud Platform
    Overlays - the hard part

    View Slide

  35. Google Cloud Platform
    node-a
    192.168.1.1/16
    node-c
    non-node
    node-d
    192.168.1.2/16
    192.168.1.3/16 192.168.1.4/16
    ctr-1 10.0.1.2
    ctr-2 10.0.3.2
    Overlay (e.g. flannel, weave)

    View Slide

  36. Google Cloud Platform
    node-a
    192.168.1.1/16
    node-c
    non-node
    node-d
    192.168.1.2/16
    192.168.1.3/16 192.168.1.4/16
    ctr-1 10.0.1.2
    ctr-2 10.0.3.2
    to: 10.0.3.2
    from: 192.168.1.2
    GET /
    Overlay (e.g. flannel, weave)

    View Slide

  37. Google Cloud Platform
    node-a
    192.168.1.1/16
    node-c
    non-node
    node-d
    192.168.1.2/16
    192.168.1.3/16 192.168.1.4/16
    ctr-1 10.0.1.2
    ctr-2 10.0.3.2
    to: 10.0.3.2
    from: 192.168.1.2
    GET /
    Overlay (e.g. flannel, weave)

    View Slide

  38. Google Cloud Platform
    node-a
    192.168.1.1/16
    node-c
    non-node
    node-d
    192.168.1.2/16
    192.168.1.3/16 192.168.1.4/16
    ctr-1 10.0.1.2
    ctr-2 10.0.3.2
    to: 10.0.3.2
    from: 192.168.1.2
    GET /
    ?!?!
    Overlay (e.g. flannel, weave)

    View Slide

  39. Google Cloud Platform
    We need a bridge between the
    physical and overlay networks...

    View Slide

  40. Google Cloud Platform
    We need a bridge between the
    physical and overlay networks...
    • could: route to nodes
    • could: route to 1 or more bridge machines
    • could: run flannel on client machines

    View Slide

  41. Google Cloud Platform
    We need a bridge between the
    physical and overlay networks...
    • could: route to nodes
    • could: route to 1 or more bridge machines
    • could: run flannel on client machines
    • see “When should I use an overlay?”

    View Slide