Networking for Developers: flannel, Calico, and Canal Tom Denham @_tomdee 

Every pod needs an IP address Physical / Cloud Network 

Developers don’t want to worry about servers 

Secure your pods with network policy 

Networking: flannel ● What is flannel? ○ Maintains IP address range per node Writes IP address range to file (for integration with Docker or CNI) ○ Pluggable “backend” for networking Usually vxlan - also udp, host-gw, gce, aws etc... 

Networking: flannel Progress in the last year ● Kubernetes integration ● 5 releases - v0.8.0 coming soon ○ >100 PRs merged with commits from 41 different authors ● Docs overhaul 

Network Policy : Calico ● Calico implements Kubernetes Network Policy API ○ Felix agent (golang) on each node ● Also extends with richer policy capabilities ○ Insert policy via calicoctl kind: NetworkPolicy metadata: name: user-auth spec: podSelector: svc: user-auth ingress: - from: - podSelector: matchLabels: svc: user-login ports: - port: 80 

Network Policy : Calico Progress in the last year ● K8s datastore support ● 5 major releases and many patch releases ○ Commits from >70 new people ● 4x slack community growth (>1000 people now) ● K8s NetworkPolicy API 

Putting them together: Canal ● projectcalico/canal project on github ● K8s manifest for deploying Canal policy with flannel networking 

Demo - flannel networking and Calico policy 

Hosts My Laptop Vagrant/Virtualbox n3 n1 (master) n2 

kubectl apply -f canal.yaml 

From this... Frontend Frontend Frontend Client Redis 

Client Redis Frontend Frontend Frontend To this... 

How to contribute Standard Github process, with Slack available for interactive discussions ● Sign up: slack.projectcalico.org (#flannel, #calico-dev) Ideas for ways to contribute ○ Packaging ○ Platforms ○ Testing, raising (good) bugs ○ Documentation ○ “Help wanted” labels on Github More ways to contribute than code For big things (e.g. a new dataplane driver), engage with community before coding! 

Future work Everything from the previous slide, in particular ● Documentation ● Release cadence and communication ● Fixing all the issues! Istio service mesh 

Working with the flannel code Easy in principle but distributed systems have challenges ● Made easier with Minikube and kubeadm Use my new “extension” backend for dataplane prototyping 

minikube-start: minikube start --network-plugin cni minikube-build-image: CGO_ENABLED=1 go build -v -o dist/flanneld-amd64 sh -c 'eval $$(minikube docker-env) && docker build -f Dockerfile.amd64 -t flannel/minikube .' minikube-deploy-flannel: kubectl apply -f Documentation/minikube.yml minikube-remove-flannel: kubectl delete -f Documentation/minikube.yml minikube-restart-pod: # Use this to pick up a new image kubectl delete pods -l app=flannel --grace-period=0 kubernetes-logs: kubectl logs `kubectl get po -l app=flannel -o=custom-columns=NAME:metadata.name --no-headers=true` -c kube-flannel -f 

Extension backend - host-gw clone { "Network": "10.244.0.0/16", "Backend": { "Type": "extension", "SubnetAddCommand": "ip route add $SUBNET via $PUBLIC_IP", "SubnetRemoveCommand": "ip route del $SUBNET via $PUBLIC_IP" } } 

{ "Network": "10.50.0.0/16", "Backend": { "Type": "extension", "PreStartupCommand": "export VNI=1; export IF_NAME=flannel-vxlan; ip link del $IF_NAME 2>/dev/null; ip link add $IF_NAME type vxlan id $VNI dstport 8472 nolearning && ip link set mtu 1450 dev $IF_NAME && cat /sys/class/net/$IF_NAME/address", "PostStartupCommand": "export IF_NAME=flannel-vxlan; export SUBNET_IP=`echo $SUBNET | cut -d'/' -f 1`; ip addr add $SUBNET_IP/32 dev $IF_NAME && ip link set $IF_NAME up", "ShutdownCommand": "export IF_NAME=flannel-vxlan; ip link del $IF_NAME", "SubnetAddCommand": "export SUBNET_IP=`echo $SUBNET | cut -d'/' -f 1`; export IF_NAME=flannel-vxlan; read VTEP; ip route add $SUBNET nexthop via $SUBNET_IP dev $IF_NAME onlink && arp -s $SUBNET_IP $VTEP dev $IF_NAME && bridge fdb add $VTEP dev $IF_NAME self dst $PUBLIC_IP" } } https://github.com/coreos/flannel/blob/master/Documentation/extension.md#complex-example-vxlan 

Making changes to Calico code ● Generally the same challenges as flannel for testing ● More components than flannel - check docs.projectcalico.org for guidance ● Let’s look at CNI in detail ○ Invoke plugin directly echo '{"cniVersion": "0.3.2","type":"IGNORED", "name": "a","ipam": {"type": "host-local", "subnet":"10.1.2.3/24"}}' | sudo CNI_COMMAND=ADD CNI_NETNS=a CNI_PATH=a CNI_IFNAME=a CNI_CONTAINERID=a CNI_VERSION=0.3.1 dist/calico ○ Or use Docker to easily try out different versions echo '{"cniVersion": "0.3.2","type":"IGNORED", "name": "a","ipam": {"type": "host-local", "subnet":"10.1.2.3/24"}}' | docker run -e CNI_COMMAND=VERSION -e CNI_NETNS=a -e CNI_PATH=a -e CNI_IFNAME=a -i calico/cni:v1.6.0 ./host-local ○ What about trying it for real under kubernetes Let’s try changing a log 

For Calico and flannel support slack.projectcalico.org https://github.com/tomdee/kubeadm-vagrant https://github.com/tomdee/kube-demos/tree/coreosfest-2017 We’re Hiring! @tomdee @_tomdee [email protected]