Simplifying and Securing your OpenShift Network with Project Calico
This was presented as part of Red Hat's OpenShift Commons briefing series, hosted by Diane Mueller. There is a recording here: https://youtu.be/hqzUfefL1ek
of Cloud-Native: Scale & Churn <0.1x Median lifespan >10x Workloads per host 100+x Churn per host First-generation, centralized SDN controller Traditional security appliance
the network, by removing unnecessary layers of complexity What’s Required? … implemented in a scale-out, distributed architecture … SECURE workloads with fine-grained policy rules, leveraging orchestrator
Simplify the Network ☑ Flat IP network (pods are endpoints too) ☑ No overlay by default ⇒ zero packet overhead ☑ Routed model — one hop to the kernel, another hop to the destination (remote kernel or local pod) ☑ Leverages Linux’s built-in, efficient network stack ☑ Maximum performance, simplest to troubleshoot
Architecture: Routing Physical fabric (L2 or L3) or public cloud SDN (e.g. Amazon VPC / subnet) Cloud OS / Orchestration System Cloud OS / Orchestration System Compute Node Compute Node Compute Node kernel Cloud OS / Orchestration System Calico- node Routes Pod Eth0 Calico Plugin Compute Node Compute Node Compute Node kernel Calico- node Routes Pod Eth0 Control plane (etcd / Raft + BGP) Data plane (IP)
Architecture: Policy Enforcement Physical fabric (L2 or L3) or public cloud SDN (e.g. Amazon VPC / subnet) Cloud OS / Orchestration System Cloud OS / Orchestration System Compute Node Compute Node Compute Node kernel Cloud OS / Orchestration System Calico- node Routes ACLs Pod Eth0 Calico Plugin Compute Node Compute Node Compute Node kernel Calico- node Routes ACLs Pod Eth0 Control plane (etcd / Raft + BGP) Data plane (IP)
of a Calico Network Policy apiVersion: v1 kind: policy metadata: name: allow-tcp-6379 spec: selector: role == 'database' ingress: - action: allow protocol: tcp source: selector: role == 'frontend' destination: ports: - 6379 egress: - action: allow Name of this policy Which pods does it apply to? Who can talk to those pods (with which protocols?) To whom can those pods talk (with which protocols?) $ calicoctl apply -f mypolicy.yaml API version Yes, this looks a lot like a Kubernetes Network Policy… Calico can enforce k8s policy or this extended model
Architecture: Policy Enforcement Revisited Cloud OS / Orchestration System Cloud OS / Orchestration System Compute Node Compute Node Compute Node kernel Cloud OS / Orchestration System Calico- node Routes ACLs Pod Eth0 Calico Plugin Compute Node Compute Node Compute Node kernel Calico- node Routes ACLs Pod Eth0 ▪ Policy rendering to ACLs is distributed to calico agents ▪ Each node efficiently calculates what it needs & programs iptables ▪ At scale, <10ms to first ping
Comparison OVS-based (e.g. OpenShift SDN) Project Calico One subnet per host Dynamic allocation of IP address ranges to host as additional containers scheduled (reduces wasted addresses without imposing an upper limit on # containers) Pods connected to OVS Bridge (br0) Pods connected into Linux kernel routing engine (no bridge, single routed hop, same path intra/inter node) Access to pods on remote nodes via VXLAN tunnel (tun0) Tunnel possible but not required — pods have real IPs on underlying network — no double-encapsulation when running on underlying SDN (e.g. public cloud or OpenStack) Connectivity outside cluster via NAT NAT not required by default to outside world, since pods have real IPs Network isolation enforced in OVS via tenant separation (separate ovs-multitenant plug-in) or Kubernetes network policy with ovs-subnet Network isolation (including multi-tenant) enforced via ingress + egress policy rules encoded into iptables rules in Linux kernel OVS in control and data path Calico in control path only (data path = traditional Linux kernel L3 forwarding & filtering)
for other SDN solutions (Some) Other Networking Solutions Project Calico Centralized controller calculates rules for each node All policy calculations / rendering Must replace internal service routing — not compatible with Kube-proxy Fully compatible with standard Kube-proxy Must use own external load balancing — not compatible with OpenShift Router Fully compatible with OpenShift Router and any other regular IP networking mechanisms (it’s just IP)
Calico with Flannel Networking A collaboration between Tigera and CoreOS to apply Calico policy to flannel overlay networks More: http://github.com/projectcalico/canal
of recipes Calico + Kubernetes ▪ E.g. AWS Quick Start, Stack Point Cloud, kops, ... Users have deployed with OpenShift ▪ “Roll-your-own” installation until recently Tigera / Red Hat collaborating on supported integration and certification for OpenShift ▪ Integration was working - but broken by OCP 3.4. Addressing a few minor remaining issues. ▪ “Watch this space” - by signing up to the Project Calico Slack (http://slack.projectcalico.org), joining the #openshift channel, and let us know you’re interested! Calico-OpenShift Integration &