CNI Introduction - Speaker Deck

Slide 1

Slide 1 text

CNI Introduction Date: 2018/12/06 Place: ITRI Presenter: Samina (Shan-Jung Fu) Prepare Hands-On Environment: http://bit.ly/2QtUMyZ

Slide 2

Slide 2 text

Outline ● CNI Overview ● CNI Plugins ● Hands-on: CNI Plugins with Network Namespace ● CNI Network Controller Intro ● Deep Dive Into K8s Pod & CNI ● Hands-on: Playing with POD Networking 2

Slide 3

Slide 3 text

● CNI Overview ○ Preface ○ What is CNI ○ CNI Introduction ● CNI Plugins ● Hands-on: CNI Plugins with Network Namespace ● CNI Network Controller Intro ● Deep Dive Into K8s Pod & CNI ● Hands-on: Playing with POD Networking Outline 3

Slide 4

Slide 4 text

● Preface ● What is CNI ● CNI Introduction Preface How the container setup network? Let’s take a look at Docker with default network mode - bridge. 4

Slide 5

Slide 5 text

● Preface ● What is CNI ● CNI Introduction 1. Create a linux bridge 2. Create a linux network namespace 3. Create a veth pair 4. Attach veth into namespace 5. Attach veth into the linux bridge 6. Setup IP address 7. Setup route rules 8. Setup iptables 5 Linux Host Linux Host br0 Linux Host br0 ns1 Linux Host br0 ns1 veth0 veth1 Linux Host br0 ns1 veth0 veth1 Linux Host br0 Create a container & setup network ns1 veth0 veth1

Slide 6

Slide 6 text

1. Create a linux bridge 2. Create a linux network namespace 3. Create a veth pair 4. Attach veth into namespace 5. Attach veth into the linux bridge 6. Setup IP address 7. Setup route rules 8. Setup iptables 6 Linux Host Linux Host br0 Linux Host br0 ns1 Linux Host br0 ns1 veth0 veth1 Linux Host br0 ns1 veth0 veth1 Linux Host br0 ns1 veth0 veth1 ● Preface ● What is CNI ● CNI Introduction Create a container & setup network

Slide 7

Slide 7 text

1. Create a linux bridge 2. Create a linux network namespace 3. Create a veth pair 4. Attach veth into namespace 5. Attach veth into the linux bridge 6. Setup IP address 7. Setup route rules 8. Setup iptables 7 Linux Host Linux Host br0 Linux Host br0 ns1 Linux Host br0 ns1 veth0 veth1 Linux Host br0 ns1 veth0 veth1 Linux Host br0 ns1 veth0 veth1 ● Preface ● What is CNI ● CNI Introduction Create a container & setup network

Slide 8

Slide 8 text

1. Create a linux bridge 2. Create a linux network namespace 3. Create a veth pair 4. Attach veth into namespace 5. Attach veth into the linux bridge 6. Setup IP address 7. Setup route rules 8. Setup iptables 8 Linux Host Linux Host br0 Linux Host br0 ns1 Linux Host br0 ns1 veth0 veth1 Linux Host br0 ns1 veth0 veth1 Linux Host br0 ns1 veth0 veth1 ● Preface ● What is CNI ● CNI Introduction Create a container & setup network

Slide 9

Slide 9 text

1. Create a linux bridge 2. Create a linux network namespace 3. Create a veth pair 4. Attach veth into namespace 5. Attach veth into the linux bridge 6. Setup IP address 7. Setup route rules 8. Setup iptables 9 Linux Host Linux Host br0 Linux Host br0 ns1 Linux Host br0 ns1 veth0 veth1 Linux Host br0 ns1 veth0 veth1 Linux Host br0 ns1 veth0 veth1 ● Preface ● What is CNI ● CNI Introduction Create a container & setup network

Slide 10

Slide 10 text

10 Linux Host Linux Host br0 Linux Host br0 ns1 Linux Host br0 ns1 veth0 veth1 Linux Host br0 ns1 veth0 veth1 Linux Host br0 ns1 veth0 veth1 1. Create a linux bridge 2. Create a linux network namespace 3. Create a veth pair 4. Attach veth into namespace 5. Attach veth into the linux bridge 6. Setup IP address 7. Setup route rules 8. Setup iptables ● Preface ● What is CNI ● CNI Introduction Create a container & setup network

Slide 11

Slide 11 text

Preface (Cont.) Problem: ● Many container technical (Docker, rkt, LXC...) ● Many container cluster orchestration (K8s, Mesos...) ● Many network setup ways (OVS, layer3 routing, overlay network...) Solve: ● Provide an interface ● Let everyone can write its own network service 11 Container Network Interface(CNI) ● Preface ● What is CNI ● CNI Introduction

Slide 12

Slide 12 text

● Container Network Interface ● The simplest possible interface between container runtime & network implementation ● CNCF (Cloud Native Computing Foundation) Project What is CNI 12 ● Preface ● What is CNI ● CNI Introduction

Slide 13

Slide 13 text

What is CNI (Cont.) 13 ● Preface ● What is CNI ● CNI Introduction Two main Projects: ● containernetworking/cni Libraries for writing plugins to configure network interfaces ● containernetworking/plugins Additional CNI network plugins

Slide 14

Slide 14 text

What is CNI (Cont.) ● CNI Specification ○ The API between runtimes and network plugins ● Conventions ○ Extensions to the API that are not required for all plugins ● Library ○ A Go implementation of the CNI specification that plugins and runtimes can use 14 ● Preface ● What is CNI ● CNI Introduction

Slide 15

Slide 15 text

Container Runtime Net Plugin Config CNI commands & network config Configures Network Ref: https://github.com/sufuf3/network-study-notes/blob/master/Container-Network/cni.md CNI Introduction 15 ● Preface ● What is CNI ● CNI Introduction

Slide 16

Slide 16 text

CNI Introduction (Cont.) 16 ● Preface ● What is CNI ● CNI Introduction Container runtime ● A software ● Executes containers ● Manages container images on a node ● Ecosystem ○ Docker ○ Rkt ○ Containerd ○ lxd

Slide 17

Slide 17 text

CNI Introduction (Cont.) 17 ● Preface ● What is CNI ● CNI Introduction { "cniVersion": "0.6.0", "name": "mynet", "type": "my-plugin", "some-parameter": "foo", "ipam": { "type": "host-local", "subnet": "10.42.0.0/24", } } A json document defining the network my-config

Slide 18

Slide 18 text

CNI Introduction (Cont.) 18 ● Preface ● What is CNI ● CNI Introduction CNI_COMMAND=ADD CNI_CONTAINERID=$id \ CNI_NETNS=/proc/$pid/ns/net \ CNI_PATH=/opt/cni/bin \ CNI_IFNAME=net0 \ my-plugin < my-config CNI_COMMAND=ADD \ CNI_CONTAINERID=ns1 \ CNI_NETNS=/var/run/netns/ns1 \ CNI_IFNAME=eth2 \ bridge

Slide 19

Slide 19 text

CNI_COMMAND=ADD CNI_CONTAINERID=$id \ CNI_NETNS=/proc/$pid/ns/net \ CNI_PATH=/opt/cni/bin \ CNI_IFNAME=net0 \ my-plugin < my-config CNI Introduction (Cont.) 19 ● Preface ● What is CNI ● CNI Introduction https://github.com/containernetworking/cni/blob/maste r/SPEC.md#parameters ● ADD: Add container to network ● DEL: Delete container from network ● VERSION: Report version

Slide 20

Slide 20 text

CNI Introduction (Cont.) 20 ● Preface ● What is CNI ● CNI Introduction CNI_COMMAND=ADD \ CNI_CONTAINERID=ns1 \ CNI_NETNS=/var/run/netns/ns1 \ CNI_IFNAME=eth2 \ bridge

Slide 21

Slide 21 text

Container Runtime Net Plugin Config CNI commands & network config Configures Network Ref: https://github.com/sufuf3/network-study-notes/blob/master/Container-Network/cni.md CNI Introduction (Cont.) 21 ● Preface ● What is CNI ● CNI Introduction

Slide 22

Slide 22 text

Outline 22 ● CNI Overview ● CNI Plugins ○ network plugins of containernetworking ○ 3rd Party Plugins ● Hands-on: CNI Plugins with Network Namespace ● CNI Network Controller Intro ● Deep Dive Into K8s Pod & CNI ● Hands-on: Playing with POD Networking

Slide 23

Slide 23 text

CNI Plugins 23 ● CNI Plugins ○ Containernetworking ○ 3rd Party Plugins

Slide 24

Slide 24 text

Network Plugins of containernetworking ● Main: interface-creating ● IPAM: IP address allocation ● Meta: other plugins ● Sample 24 ● CNI Plugins ○ Containernetworking ○ 3rd Party Plugins

Slide 25

Slide 25 text

● Main: interface-creating ○ bridge: Creates a bridge, adds the host & the container to it. ○ ipvlan: Adds an ipvlan interface in the container. ○ loopback: Set the state of loopback interface to up. ○ macvlan: Creates a new MAC address, forwards all traffic to that to the container. ○ ptp: Creates a veth pair. ○ vlan: Allocates a vlan device. ○ host-device: Move an already-existing device into a container. Network Plugins of containernetworking (Cont.) 25 ● CNI Plugins ○ Containernetworking ○ 3rd Party Plugins

Slide 26

Slide 26 text

● Main: interface-creating ● IPAM: IP address allocation ○ dhcp: Runs a daemon on the host to make DHCP requests on behalf of the container ○ host-local: Maintains a local database of allocated IPs ○ static: Allocate a static IPv4/IPv6 addresses to container and it's useful in debugging purpose. ● Meta: other plugins ● Sample Network Plugins of containernetworking (Cont.) 26 ● CNI Plugins ○ Containernetworking ○ 3rd Party Plugins

Slide 27

Slide 27 text

● Main: interface-creating ● IPAM: IP address allocation ● Meta: other plugins ○ flannel: Generates an interface corresponding to a flannel config file ○ tuning: Tweaks sysctl parameters of an existing interface ○ portmap: An iptables-based port mapping plugin. Maps ports from the host's address space to the container. ○ bandwidth: Allows bandwidth-limiting through use of traffic control tbf (ingress/egress). ● Sample Network Plugins of containernetworking (Cont.) 27 ● CNI Plugins ○ Containernetworking ○ 3rd Party Plugins

Slide 28

Slide 28 text

● Main: interface-creating ● IPAM: IP address allocation ● Meta: other plugins ● Sample ○ Provides an example for building your own plugin Network Plugins of containernetworking (Cont.) 28 ● CNI Plugins ○ Containernetworking ○ 3rd Party Plugins

Slide 29

Slide 29 text

Calico ● Provides secure network connectivity workloads ● Creates and manages a flat layer 3 network ● Assigning each workload a fully routable IP address More 3rd party plugins: https://github.com/containernetworking/cni#3rd-party-plugins 29 ● CNI Plugins ○ Containernetworking ○ 3rd Party Plugins 3rd Party Plugins

Slide 30

Slide 30 text

Outline ● CNI Overview ● CNI Plugins ● Hands-on: CNI Plugins with Network Namespace ● CNI Network Controller Intro ● Deep Dive Into K8s Pod & CNI ● Hands-on: Playing with POD Networking 30

Slide 31

Slide 31 text

Host Hands-on Using bridge CNI Plugin with Network Namespaces 31 Network Namespace 0 (net0) Network Namespace 1 (net1) Veth0 Veth1 Bridge cni0 10.23.0.X/24 10.23.0.Y/24 Network config file cd ~/cni-hands-on/ mybridge.conf.tmp

Slide 32

Slide 32 text

Answer of Hands-on 32 sudo ip netns add ns1 sudo ip netns add ns2 cat > mybridge.conf <<"EOF" { "cniVersion": "0.2.0", "name": "mynet", "type": "bridge", "bridge": "cni0", "isGateway": true, "ipMasq": true, "ipam": { "type": "host-local", "subnet": "10.23.0.0/24", "routes": [ { "dst": "0.0.0.0/0" } ] } } EOF cat > loopback.conf <<"EOF" { "cniVersion": "0.2.0", "type": "loopback" } EOF

Slide 33

Slide 33 text

sudo CNI_COMMAND=ADD CNI_CONTAINERID=ns1 CNI_NETNS=/var/run/netns/ns1 CNI_IFNAME=lo CNI_PATH=`pwd` ./loopback < loopback.conf sudo CNI_COMMAND=ADD CNI_CONTAINERID=ns2 CNI_NETNS=/var/run/netns/ns2 CNI_IFNAME=lo CNI_PATH=`pwd` ./loopback < loopback.conf sudo CNI_COMMAND=ADD CNI_CONTAINERID=ns1 CNI_NETNS=/var/run/netns/ns1 CNI_IFNAME=net0 CNI_PATH=`pwd` ./bridge

Slide 34

Slide 34 text

Answer of Hands-on (Cont.) 34 sudo ip --all netns exec ip a sudo ip netns exec ns2 ping 10.23.0.2

Slide 35

Slide 35 text

Outline 35 ● CNI Overview ● CNI Plugins ● Hands-on: CNI Plugins with Network Namespace ● CNI Network Controller Introduction ● Deep Dive Into K8s Pod & CNI ● Hands-on: Playing with POD Networking

Slide 36

Slide 36 text

CNI Network Controller Introduction ● Create multiple network interface of pod at init-container 36

Slide 37

Slide 37 text

CNI Network Controller Introduction 37

Slide 38

Slide 38 text

Slide 39

Slide 39 text

● CNI Overview ● CNI Plugins ● Hands-on: CNI Plugins with Network Namespace ● CNI Network Controller Intro ● Deep Dive Into K8s Pod & CNI ○ Install pod network method ○ About CNI path of kubernetes ○ K8s pod with CNI ● Hands-on: Playing with POD Networking Outline 39

Slide 40

Slide 40 text

● Install pod network ● CNI path ● pod with CNI Install pod network method 40

Slide 41

Slide 41 text

● Install pod network ● CNI path ● pod with CNI About CNI path of K8s ● /etc/cni/net.d ○ netconf file ● /opt/cni/bin ○ CNI plugins binary file These files are on every K8s nodes. 41

Slide 42

Slide 42 text

● Install pod network ● CNI path ● pod with CNI ● User post Pod config to master ● Master find a node to deploy the pod ● Create a Pause container ● Load CNI network config ● Execute the CNI plugin ● Create target containers ● Attach to Pause container K8s pod with CNI 42

Slide 43

Slide 43 text

● User post Pod config to master ● Master find a node to deploy the pod ● Create a Pause container ● Load CNI network config ● Execute the CNI plugin ● Create target containers ● Attach to Pause container K8s pod with CNI 43 Linux Host ● Install pod network ● CNI path ● pod with CNI

Slide 44

Slide 44 text

● User post Pod config to master ● Master find a node to deploy the pod ● Create a Pause container ● Load CNI network config ● Execute the CNI plugin ● Create target containers ● Attach to Pause container K8s pod with CNI 44 Linux Host Pause container ● Install pod network ● CNI path ● pod with CNI

Slide 45

Slide 45 text

● User post Pod config to master ● Master find a node to deploy the pod ● Create a Pause container ● Load CNI network config ● Execute the CNI plugin ● Create target containers ● Attach to Pause container K8s pod with CNI 45 Linux Host Pause container /etc/cni/net.d/ xxx.config Call /opt/cni/bin/CNI ● Install pod network ● CNI path ● pod with CNI

Slide 46

Slide 46 text

● User post Pod config to master ● Master find a node to deploy the pod ● Create a Pause container ● Load CNI network config ● Execute the CNI plugin ● Create target containers ● Attach to Pause container K8s pod with CNI 46 Linux Host Pause container Network Connect ● Install pod network ● CNI path ● pod with CNI

Slide 47

Slide 47 text

K8s pod with CNI 47 Linux Host Pause container Network Connect ● User post Pod config to master ● Master find a node to deploy the pod ● Create a Pause container ● Load CNI network config ● Execute the CNI plugin ● Create target containers ● Attach to Pause container container1 container2 container3 ● Install pod network ● CNI path ● pod with CNI

Slide 48

Slide 48 text

K8s pod with CNI 48 Linux Host Pause container Network Connect container1 container2 container3 ● User post Pod config to master ● Master find a node to deploy the pod ● Create a Pause container ● Load CNI network config ● Execute the CNI plugin ● Create target containers ● Attach to Pause container ● Install pod network ● CNI path ● pod with CNI

Slide 49

Slide 49 text

K8s pod with CNI ● User post Pod config to master ● Master find a node to deploy the pod ● Create a Pause container ● Load CNI network config ● Execute the CNI plugin ● Create target containers ● Attach to Pause container 49 container1 Linux Host Pause container Network Connect container1 container2 container3 Pod ● Install pod network ● CNI path ● pod with CNI

Slide 50

Slide 50 text

● CNI Overview ● CNI Plugins ● Hands-on: CNI Plugins with Network Namespace ● CNI Network Controller Intro ● Deep Dive Into K8s Pod & CNI ● Hands-on: Playing with POD Networking Outline 50

Slide 51

Slide 51 text

Network namespace path: /proc//ns/net 1. Find containerID via kubectl 2. Find PID via docker command 3. monitor POD network Playing with POD Networking 51 kubectl get po -o jsonpath='{.status.containerStatuses[0].containerID} ' | cut -c 10-21 PID=$(sudo docker inspect c94df80d080e --format '{{ .State.Pid }}') sudo nsenter -t ${PID} -n ip addr

Slide 52

Slide 52 text

Playing with POD Networking (Cont.) 52 $ $ $ $ $ kubectl create -f https://k8s.io/examples/application/shell-demo.yaml kubectl get po -o wide kubectl get po shell-demo -o jsonpath='{.status.containerStatuses[0].containerID }' | cut -c 10-21 c94df80d080e PID=$(sudo docker inspect c94df80d080e --format '{{ .State.Pid }}') sudo nsenter -t ${PID} -n ip addr

Slide 53

Slide 53 text

References ● https://github.com/containernetworking/cni ● https://www.slideshare.net/hongweiqiu/introduction -to-cni-container-network-interface 53

Slide 54

Slide 54 text

Thanks for your attention. Slides: http://bit.ly/cni1206 GitHub, Telegram: @sufuf3 Twitter: @sufuf3149 54