Slide 1

Slide 1 text

Betting on Container Networking Lee Calcote July 27th, 2016 Overlay Underlay http://calcotestudios.com/over-under

Slide 2

Slide 2 text

http://calcotestudios.com/oubcn Lee Calcote clouds, containers, infrastructure, applications and their management linkedin.com/in/leecalcote @lcalcote blog.gingergeek.com [email protected]

Slide 3

Slide 3 text

Show of Hands @lcalcote

Slide 4

Slide 4 text

Container Networking ...it's complicated.

Slide 5

Slide 5 text

Preset Expectations Experience & Management Reliability & Performance same demands and measurements developer-friendly and application-driven simple to use and deploy for developers and operators better or at least on par with their existing virtualized data center networking

Slide 6

Slide 6 text

Container Networking Specifications

Slide 7

Slide 7 text

Very interesting but no need to actually know these @lcalcote

Slide 8

Slide 8 text

(CNM) Container Network Model ...is a specification proposed by Docker, adopted by projects such as Plugins built by projects such as , , and libnetwork Weave Project Calico Kuryr (CNI) Container Network Interface ...is a specification proposed by CoreOS and adopted by projects such as , , , , and Plugins created by projects such as , , and rkt Kurma Kubernetes Cloud Foundry Apache Mesos Weave Project Calico Contiv Networking @lcalcote Container Networking Specifications

Slide 9

Slide 9 text

Container Network Model Specification @lcalcote Remote Drivers Local Drivers

Slide 10

Slide 10 text

Container Network Model Topology @lcalcote Network Sandbox Endpoint Backend Network Docker Container Network Sandbox Endpoint Docker Container Network Sandbox Endpoint Docker Container Endpoint Frontend Network

Slide 11

Slide 11 text

@lcalcote Container Network Interface (CNI)

Slide 12

Slide 12 text

@lcalcote Container Network Interface Flow 1. Container runtime needs to: 1. allocate a network namespace to the container and assign a container ID 2. pass along a number of parameters (CNI config) to network driver. 2. Network driver attaches container to a network and then reports the assigned IP address back to the container runtime (via JSON schema)

Slide 13

Slide 13 text

@lcalcote CNI Network (JSON) { " n a m e " : " m y n e t " , " t y p e " : " b r i d g e " , " b r i d g e " : " c n i 0 " , " i s G a t e w a y " : t r u e , " i p M a s q " : t r u e , " i p a m " : { " t y p e " : " h o s t - l o c a l " , " s u b n e t " : " 1 0 . 2 2 . 0 . 0 / 1 6 " , " r o u t e s " : [ { " d s t " : " 0 . 0 . 0 . 0 / 0 " } ] }

Slide 14

Slide 14 text

CNI and CNM Similar in that each... ...are driver-based, and therefore democratize the selection of which type of container networking ...allow multiple network drivers to be active and used concurrently each provide a one-to-one mapping of network to that network’s driver ...allow containers to join one or more networks. ...allow the container runtime to launch the network in its own namespace segregate the application/business logic of connecting the container to the network to the network driver. @lcalcote

Slide 15

Slide 15 text

CNI and CNM Different in that... CNI supports any container runtime CNM only support Docker runtime CNI is simpler, has adoption beyond its creator CNM acts as a broker for conflict resolution CNI is still considering its approach to arbitration @lcalcote

Slide 16

Slide 16 text

Types of Container Networking None Links and Ambassadors Container-mapped Bridge Host Overlay Underlay MACvlan IPvlan Direct Routing Point-to-Point Fan Networking @lcalcote

Slide 17

Slide 17 text

None @lcalcote container receives a network stack, but lacks an external network interface. it does, however, receive a loopback interface.

Slide 18

Slide 18 text

Links facilitate single host connectivity "discovery" via /etc/hosts or env vars @lcalcote Ambassadors facilitate multi-host connectivity uses a tcp port forwarder (socat) Web Host MySQL Ambassador PHP DB Host PHP Ambassador MySQL link link

Slide 19

Slide 19 text

Container-Mapped one container reuses (maps to) the networking namespace of another container. @lcalcote may only be invoked when running a docker container (cannot be defined in Dockerfile): - - n e t = c o n t a i n e r = s o m e _ c o n t a i n e r _ n a m e _ o r _ i d

Slide 20

Slide 20 text

Bridge Ah, yes, docker0 default networking for Docker uses a host-internal network leverages iptables for network address translation (NAT) and port-mapping @lcalcote

Slide 21

Slide 21 text

Host container created shares its network namespace with the host default Mesos networking mode better performance easy to understand and troubleshoot suffers port conflicts secure? @lcalcote

Slide 22

Slide 22 text

Overlay use networking tunnels to delivery communication across hosts Most useful in hybrid cloud scenarios or when shadow IT is needed Many tunneling technologies exist VXLAN being the most commonly used Requires distributed key-value store @lcalcote

Slide 23

Slide 23 text

K/V Store for Overlay Networking Docker - requires K/V store (built-in as experimental as of 1.12) WeaveMesh - does not require K/V store WeaveNet - limited to single network; requires K/V store Flannel - requires K/V store Plumgrid - requires K/V store; built-in and not pluggable Midokura - requires K/V store; built-in and not pluggable Calico - requires K/V store @lcalcote

Slide 24

Slide 24 text

Underlays expose host interfaces (i.e. the physical network interface at eth0) directly to containers running on the host MACvlan IPvlan Direct Routing @lcalcote not necessarily public cloud friendly

Slide 25

Slide 25 text

Point-to-Point Default rkt networking mode Uses NAT (IPMASQ) by default Creates a virtual ethernet pair placing one on the host and the other into the container pod leverages iptables to provide port-forwarding for inbound traffic to the pod internal communication between other containers in the pod over the loopback interface @lcalcote Internet

Slide 26

Slide 26 text

MACvlan allows creation of multiple virtual network interfaces behind the host’s single physical interface Each virtual interface has unique MAC and IP addresses assigned with restriction: the IP address needs to be in the same broadcast domain as the physical interface eliminates the need for the Linux bridge, NAT and port- mapping allowing you to connect directly to physical interface @lcalcote

Slide 27

Slide 27 text

IPvlan allows creation of multiple virtual network interfaces behind the host’s single physical interface Each virtual interface has unique IP addresses assigned Same MAC address used for all containers L2-mode containers must be on same network as host (similar to MACvlan) L3-mode containers must be on different network than host Network advertisement and redistribution into the network still needs to be done. @lcalcote

Slide 28

Slide 28 text

MACvlan and IPvlan While multiple modes of networking are supported on a given host, MACvlan and IPvlan can’t be used on the same physical interface concurrently. ARP and broadcast traffic, the L2 modes of these underlay drivers operate just as a server connected to a switch does by flooding and learning using 802.1d packets IPvlan L3-mode - No multicast or broadcast traffic is allowed in. In short, if you’re used to running trunks down to hosts, L2 mode is for you. If scale is a primary concern, L3 has the potential for massive scale. @lcalcote

Slide 29

Slide 29 text

Direct Routing Benefits of pushing past L2 to L3 resonates with network engineers leverage existing network infrastructure use routing protocols for connectivity; easier to interoperate with existing data center across VMs and bare metal servers Better scaling More granular control over filtering and isolating network traffic Easier traffic engineering for quality of service Easier to diagnose network issues @lcalcote

Slide 30

Slide 30 text

Fan Networking a way of gaining access to many more IP addresses, expanding from one assigned IP address to 250 more IP addresses “address expansion” - multiplies the number of available IP addresses on the host, providing an extra 253 usable addresses for each host IP Fan addresses are assigned as subnets on a virtual bridge on the host, IP addresses are mathematically mapped between networks uses IP-in-IP tunneling; high performance particularly useful when running containers in a public cloud where a single IP address is assigned to a host and spinning up additional networks is prohibitive or running another load-balancer instance is costly @lcalcote

Slide 31

Slide 31 text

Fan Networking @lcalcote

Slide 32

Slide 32 text

Network Capabilities and Services IPAM, multicast, broadcast, IPv6, load-balancing, service discovery, policy, quality of service, advanced filtering and performance are all additional considerations to account for when selecting networking that fits your needs. @lcalcote

Slide 33

Slide 33 text

IPv6 and IPAM IPv6 lack of support for IPv6 in the top public clouds reinforces the need for other networking types (overlays and fan networking) some tier 2 public cloud providers offer support for IPv6 IPAM most container runtime engines default to host-local for assigning addresses to containers as they are connected to networks. Host-local IPAM involves defining a fixed block of IP addresses to be selected. DCHP is universally supported across the container networking projects. CNM and CNI both have IPAM built-in and plugin frameworks for integration with IPAM systems @lcalcote

Slide 34

Slide 34 text

Text @lcalcote Docker 1.12 (Load-balancing)

Slide 35

Slide 35 text

Lee Calcote clouds, containers, infrastructure, applications and their management linkedin.com/in/leecalcote @lcalcote blog.gingergeek.com [email protected] Thank you! Questions?