Kubernetes Network Policies

66dc269635b1dbaaf2751ca5a5728525?s=47 Eloy Coto
October 11, 2019

Kubernetes Network Policies

66dc269635b1dbaaf2751ca5a5728525?s=128

Eloy Coto

October 11, 2019
Tweet

Transcript

  1. 2.

    @eloycoto - Software engineer at Red Hat - Working now

    in 3Scale API Management - Former Isovalent software engineer - Working full time on Cilium - Former VoIP software dev at Foehn Coms.
  2. 7.

    Kubernetes architecture Kubelet KubeProxy Scheduler CNI DS APIServer Kubelet KubeProxy

    Kubelet KubeProxy Kubelet KubeProxy K8S1 Master K8SN K8SN CNI DS CNI DS CNI DS
  3. 8.

    Network communication: Direct routing K8S2 10.10.2.10 DR POD: BA 10.10.2.11

    POD: BB 10.10.2.12 POD: BC 10.10.2.13 K8S1 10.10.1.10 DR POD: BA 10.10.1.11 POD: BB 10.10.1.12 POD: BC 10.10.1.13
  4. 9.

    Network communication: Encapsulation K8S2 192.168.1.12 DR POD: BA 10.10.2.11 POD:

    BB 10.10.2.12 POD: BC 10.10.2.13 K8S1 192.168.1.11 DR POD: BA 10.10.1.11 POD: BB 10.10.1.12 POD: BC 10.10.1.13 VXLAN/GENEVE
  5. 11.

    - Native BPF/XDP from start - Overlay (VXLan or Geneve)

    or direct routing - Identity based network policies - L3/L4/L7 Policies - Egress DNS policies - Use a lot of kernel features (Some kernel maintainers working on the project) - Socket acceleration for sidecars proxies. - A lot of traction in the community. - IPSEC communication between endpoints - Multi cluster communication Cilium
  6. 12.

    - The main feature is BGP peering connections - Also

    provides a way to use VXLan - New version will have BPF and socket acceleration support. - L3/L4 policies based on IPtables - L7 policies using Istio - Network policies based on SPIFFE - Integration with Openstack/Mesos Calico
  7. 13.

    Weave - VXLan encapsulation - L3/L4 policies based on IPtables

    - Use Kernel fast datapath features - IPSec encryption between pods.
  8. 14.

    BPF hooks Pod TCP Network Veth Veth Iptables Normal connect

    flow Connect() LLVM Bytecode + verifier + JIT Source code (c) Pod Connect() BPF LSM Hook EACCESS BPF connect() hook
  9. 15.
  10. 16.

    Ethernet Ethernet Socket TCP/IP Ethernet Loopback Ethernet Socket TCP/IP IPtables

    Ethernet POD Loopback Socket acceleration Socket TCP/IP Ethernet Socket TCP/IP IPtables Ethernet ETH0 Socket TCP/IP IPtables Ethernet ETH0 Socket TCP/IP IPtables Ethernet POD
  11. 17.

    Socket acceleration Socket Socket TCP/IP IPtables Ethernet ETH0 Socket TCP/IP

    IPtables Ethernet ETH0 Socket Loopback Socket POD Socket POD Cilium Cilium
  12. 19.

    $ iptables -A INPUT -p tcp \ -d 10.10.1.10/32 --dport

    22 \ -s 10.10.2.10/32 \ -j ACCEPT
  13. 20.

    Right now we have 8576 entries. Before removing them the

    first time we had >30k entries. While observing the iptables we learned that every 1-2 seconds a new rule is added.
  14. 21.

    Kubernetes network policy apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: test-network-policy

    namespace: default spec: podSelector: matchLabels: role: db policyTypes: - Ingress ingress: - from: - podSelector: matchLabels: role: backend ports: - protocol: TCP port: 6379
  15. 22.

    Network policy spec EGRESS ports (port, protocol) to ipBlock namespaceSelector

    podSelector INGRESS ports (port, protocol) from ipBlock namespaceSelector podSelector POD podSelector
  16. 23.

    Kubernetes Network policies Backend (Role: backend) Frontend (Role: frontend) Auth

    (Role: auth) Identity (Role: identity) DB (role: db) Egress
  17. 24.

    Namespace isolation NS: billing NS: monitoring NS: backend Service Service

    Service Pod Pod Pod Pod Pod Pod Pod Pod Pod Compromised
  18. 25.

    Namespace isolation kind: NetworkPolicy spec: namespaceSelector: matchLabels: project: myproject policyTypes:

    - Ingress - Egress ingress: - from: - namespaceSelector: matchLabels: project: myproject egress: - to: - namespaceSelector: matchLabels: project: myproject
  19. 26.

    Namespace isolation NS: billing NS: monitoring NS: backend Service Service

    Service Pod Pod Pod Pod Pod Pod Pod Pod Pod Compromised
  20. 28.

    Cilium L7 policies to rescue! apiVersion: "cilium.io/v2" kind: CiliumNetworkPolicy Metadata:

    {name: "rule1"} spec: endpointSelector: matchLabels: {app: "service"} ingress: - fromEndpoints: - matchLabels: env: prod toPorts: - ports: - port: "80" Protocol: tcp rules: http: - method: "GET" path: "/public"
  21. 29.

    Cilium L7 policies traffic flow! TCP: L3/L4 Client pod Server

    pod TCP: L7 GET /private GET /foo GET /public
  22. 30.

    Cilium L7 policies toPorts: - ports: - port: "9092" protocol:

    TCP rules: kafka: - role: "consume" topic: "empire-announce" Kafka Protocol toPorts: - ports: - port: "9042" protocol: TCP rules: l7proto: cassandra l7: - query_action: "select" query_table: "system\\..*" Cassandra Protocol Memcached Protocol AWS metadata
  23. 31.

    Cilium Egress DNS policies egress: - toPorts: - ports: {

    port: 53, protocol: ANY} rules: dns: - matchName: "cilium.io" - matchPattern: "*.cilium.io" - matchPattern: "*.api.cilium.io" - toFQDNs: - matchName: "cilium.io" - matchName: "service1.api.cilium.io" - matchPattern: "special*service.api.cilium.io" toPorts: - ports: - port: "80" protocol: TCP
  24. 32.
  25. 33.
  26. 34.
  27. 37.

    Future reading Cilium BPF and XDP: Thomas Graf slides https://www.slideshare.net/ThomasGraf5

    Cilium blog: https://cilium.io/blog Istio: https://istio.io/docs/reference/config/istio.authentication.v1alpha1/ SPIFFE: https://spiffe.io/ Community days recording: https://www.youtube.com/watch?v=H5IlmYmEDKk