Container Network Interface: Network Abstraction for Containers

CNI simplifies the interface for container executors and cluster orchestrators to configure the array of networks required for today's environments. (Open Networking Summit '16, Santa Clara)

Transcript

1. Container Network Interface (CNI): Network Abstraction for Containers Josh Wood DocOps at CoreOS j@coreos.com @joshixisjosh9
2. CoreOS is running the world’s containers We’re hiring: careers@coreos.com sales@coreos.com 90+ Projects on GitHub, 1,000+ Contributors coreos.com Support plans, training and more OPEN SOURCE ENTERPRISE
4. What’s the problem? SDN, NFV is great -- all of them: Container engines and images, likewise...
5. SDN: So Darnmany Networks? linux-bridge macvlan ipvlan Open vSwitch Weave Project Calico flannel
6. What’s the problem(s)? Cloud, on-prem wiring: AWS GCP Azure Rackspace
7. How to allocate IP addresses? - From a fixed block on a host - DHCP - IPAM system backed by SQL database - SDN assigned: e.g. Weave, flannel - Provider policy and API
8. How do you mix and match? (macvlan | ipvlan) + (DHCP | Calico policy | [...]) (AWS + premises) | (GCP + …)
9. Order matters! - macvlan + DHCP ◦ Create macvlan device ◦ Use the device to DHCP ◦ Configure device with allocated IP - Routed + IPAM ◦ Ask IPAM for an IP ◦ Create veth and routes on host and/or fabric ◦ Configure device with allocated IP
10. App Container (appc) github.com/appc appc-dev@googlegroups.com
11. Container Network Interface (CNI) Part of the appc effort: • Open source • Open specification • Composable, tools philosophy • CNI == Kubernetes network plugin model
12. Container Runtime (Kubernetes, rkt, ...) veth macvlan ipvlan OVS Container Networking Interface (CNI)
13. CNI: A VFS for Container Networking - Abstraction layer for network configuration - Single API to multiple, extensible networks - Narrow, simple API - Plugins for third-party implementations
14. Container Network Interface - Container can join multiple networks - Network described by JSON config - Plugin supports two commands - Add container to the network - Remove container from the network
15. CNI Network Configuration $ cat /etc/cni/net.d/10-mynet.conf { "name": "mynet", "type": "bridge", "ipam": { "type": "host-local", "subnet": "10.10.0.0/16" } }
16. CNI: Step 1 Container runtime creates network namespace and gives it a name: $ cd /var/lib/cni $ touch myns $ unshare -n mount --bind /proc/self/ns/net myns
17. CNI: Step 2 Container runtime invokes the CNI plugin $ export CNI_COMMAND=ADD $ export CNI_NETNS=/var/lib/cni/myns $ export CNI_CONTAINERID=5248e9f8-3c91-11e5-... $ export CNI_IFNAME=eth0 $ $CNI_PATH/bridge </etc/cni/net.d/10-mynet.conf
18. CNI: Step 3 Inside the bridge plugin (1): $ brctl addbr mynet $ ip link add veth123 type veth peer name $CNI_IFNAME $ brctl addif mynet veth123 $ ip link set $CNI_IFNAME netns $CNI_IFNAME $ ip link set veth123 up
19. CNI: Step 3 Inside the bridge plugin (2): $ IPAM_PLUGIN=host-local # from network conf $ echo $IPAM_PLUGIN { "ip4": { "ip": "10.10.5.9/16", "gateway": "10.10.0.1" } }
20. CNI: Step 3 Inside the bridge plugin (3): # switch to container namespace $ ip addr add 10.0.5.9/16 dev $CNI_IFNAME # Finally, print IPAM result JSON to stdout
21. App Container (appc) github.com/appc appc-dev@googlegroups.com
22. rkt A modern, secure container runtime Simple CLI tool - exorcism (no daemon) Composable with systemd, standard init systems
23. rkt • Implements appc spec • Implements CNI • Cryptographic image validation, detached signatures • Logs to tamper-evident TPM facility on DTC systems
24. rkt Stage-based architecture similar to boot chainloading Interposable stages: cgroups, LKVM, … LXC? Alternate Kubernetes container executor
26. rkt run • Isolates containers with the linux container primitives (cgroups, ns), systemd-nspawn • Container apps in a machine slice PID namespace • Manage with standard init tools: systemd
27. $ rkt run --net=host quay.io/josh_wood/caddy rkt: using image from local store for image name coreos.com/rkt/stage1-coreos:0.15.0 rkt: using image from local store for image name quay.io/josh_wood/caddy [ 1161.330635] caddy[4]: Activating privacy features... done. [ 1161.333482] caddy[4]: :2015 $ rkt run
29. Kubernetes + CNI + Docker - k8s starts "pause" container to create netns - k8s invokes CNI driver - CNI executes a CNI plugin - CNI plugin joins "pause" container to network - Pod containers use "pause" container netns
30. Kubernetes + rkt - rkt natively supports CNI - Kubernetes delegates to rkt to invoke CNI plugins
31. Kubernetes + rkt - rkt natively supports CNI - Kubernetes delegates to rkt to invoke CNI plugins - Example: Emulating Docker-style NAT
32. Host veth0 rkt-br-hl-nat eth0 lo NAT / Routing Internet 10.0.0.0/24 Pod 1 veth0 lo 2 1 Pod 1 veth0 lo 3 veth1
33. CNI Network Configuration $ cat /etc/cni/net.d/10-br-host-local-nat.conf { "name": "br-host-local-nat", "type": "bridge", "bridge": "rkt-br-hl-nat", "ipMasq": true, "isGateway": true, "ipam": { "type": "host-local", "subnet": "10.0.0.0/24", "routes": [ { "dst": "0.0.0.0/0" } ]
34. CNI: Step 2 Container runtime invokes the CNI plugin $ export CNI_COMMAND=ADD $ export CNI_NETNS=/var/lib/cni/br-host-local-nat $ export CNI_CONTAINERID=5248e9f8-3c91-11e5-... $ export CNI_IFNAME=eth0 $ $CNI_PATH/bridge </etc/cni/net.d/10-br-host-local- nat.conf
35. rkt and Kubernetes on CoreOS • rkt fly executes kubelet: packaging and distribution of containers, ns at host level • rkt is container execution engine, runs cluster work • Pod :: Pod • CNI networking: Native • Write plugins!
36. See also: • coreos.com/rkt • github.com/appc/cni • blog.kubernetes.io/2016/01/why-Kubernetes- doesnt-use-libnetwork.html • https://docs.google. com/presentation/d/1aFraZcWg5kXKTpuWwDs m4XBM__BVcP70P0X1sNGUTi0/edit? usp=sharing
37. May 9 & 10, 2016 | Berlin, Germany coreos.com/fest @coreosfest
38. j@coreos.com @joshixisjosh9 github.com/joshix QUESTIONS? Thanks! We’re hiring: coreos.com/careers Let’s talk! IRC More events: coreos.com/community LONGER CHAT?