Kubernetes Advanced Resource Features - Episode 1 Date: 2018/12/06 Place: ITRI Presenter: Samina (Shan-Jung Fu) Prepare Hands-On Environment: http://bit.ly/2zLHggi All Lab base on K8s@1.11

● Overview ● What Happens When We Type kubectl run ● Namespace ● Kubernetes QoS Outline 2

● Overview ○ Introduction ○ Setup ○ Architecture ○ Components ○ Resources ● What Happens When We Type kubectl run ● Namespace ● Kubernetes QoS Outline 3

● Introduction ● Setup ● Architecture ● Components ● Resources Kubernetes Introduction ● Kubernetes is Greek for captain or pilot ● Aka K8s, replace “ubernete” with 8 ● Experiences from Google and design by Google ● An open-source system ● A container management system 4

● Introduction ● Setup ● Architecture ● Components ● Resources ● Local-machine solutions ● Hosted solutions ● Turnkey cloud solutions ● On-premises turnkey cloud solutions ● Custom solutions ● etc. Kubernetes Setup Solutions 5

● Local-machine solutions A local, single-node Kubernetes cluster for development and testing ○ Minikube ○ microk8s ○ etc. ● Hosted solutions ● Turnkey cloud solutions ● On-premises turnkey cloud solutions ● Custom solutions Kubernetes Setup Solutions (Cont.) ● Introduction ● Setup ● Architecture ● Components ● Resources 6

● Local-machine solutions ● Hosted solutions Maintain more machines and higher availability ○ OpenShift ○ VMware Cloud PKS ○ etc. ● Turnkey cloud solutions ● On-premises turnkey cloud solutions ● Custom solutions ● ... Kubernetes Setup Solutions (Cont.) ● Introduction ● Setup ● Architecture ● Components ● Resources 7

Kubernetes Setup Solutions (Cont.) ● Local-machine solutions ● Hosted solutions ● Turnkey cloud solutions Create K8s clusters on a range of Cloud IaaS providers with only a few commands ○ AWS ○ Google Compute Engine (GCE) ○ etc. ● On-psremises turnkey cloud solutions ● Custom solutions ● Introduction ● Setup ● Architecture ● Components ● Resources 8

● Local-machine solutions ● Hosted solutions ● Turnkey cloud solutions ● On-premises turnkey cloud solutions Create K8s clusters on your internal, secure, cloud network with only a few commands ○ GKE On-Prem | Google Cloud ○ SUSE CaaS Platform ○ etc. ● Custom solutions Kubernetes Setup Solutions (Cont.) ● Introduction ● Setup ● Architecture ● Components ● Resources 9

Kubernetes Setup Solutions (Cont.) ● Local-machine solutions ● Hosted solutions ● Turnkey cloud solutions ● On-premises turnkey cloud solutions ● Custom solutions ○ kubeadm ○ Kubespray ○ Kubernetes The Hard Way ○ etc. ● Introduction ● Setup ● Architecture ● Components ● Resources 10

Kubernetes Architecture ● Introduction ● Setup ● Architecture ● Components ● Resources apiserver etcd scheduler controller kubelet kubelet kubelet API CLI UI Users Control plane Nodes 11

Node ● A worker machine in K8s ● Can be a VM or physical machine ● also called "minions" ● Node components do: ○ Run on each node ○ Maintain running pods ○ Provid K8s runtime ENV Kubernetes Architecture (Cont.) ● Introduction ● Setup ● Architecture ● Components ● Resources ● Components: ○ Kubelet ○ Kube-proxy ○ Container Runtime 12

● K8s logic (its "brains") is a collection of services: ○ API server (our point of entry to everything!) ○ core services like scheduler & controller manager ○ etcd (a highly available key/value store; "database" of K8s) ● Together, these services form the control plane of our cluster ● Also called the "master" Kubernetes Architecture (Cont.) ● Introduction ● Setup ● Architecture ● Components ● Resources 13

Kubernetes Components ● Introduction ● Setup ● Architecture ● Components ● Resources 14

Kubernetes Components (Cont.) ● Introduction ● Setup ● Architecture ● Components ● Resources 15

● The Kubernetes API defines a lot of objects called resources ● These resources are organized by type, or Kind (in the API) Kubernetes Resources ● Introduction ● Setup ● Architecture ● Components ● Resources 16

● A few common resource types are: ○ node (a machine - physical or virtual - in our cluster) ○ pod (group of containers running together on a node) ○ service (stable network endpoint to connect to one or multiple containers) ○ namespace (more-or-less isolated group of things) And much more! Kubernetes Resources (Cont.) ● Introduction ● Setup ● Architecture ● Components ● Resources 17

Kubernetes Resources (Cont.) ● Introduction ● Setup ● Architecture ● Components ● Resources ● We can see the full list by running kubectl api-resources (In Kubernetes 1.10 and prior, the command to list API resources was kubectl get) 18

● Overview ● What Happens When We Type kubectl run ● Namespace ● Kubernetes QoS Outline 19

What Happens When We Type kubectl run https://github.com/jamiehannaford/what-happens-when-k8s From: https://github.com/kubernetes/website/blob/master/static/images/docs/architecture.png 20

● Overview ● What Happens When We Type kubectl run ● Namespace ○ Namespace Resource Introduction ○ Working with Namespaces ○ Not All Objects are in a Namespace ● Kubernetes QoS Outline 21

Namespace Introduction ● Virtual clusters ● Functionalities ○ Help pod-to-pod communication using the same namespace ○ Can sit on top of the same physical cluster ○ Provide logical separation between the teams and their environments 22

Create a namespace(ns) $ kubectl create -f ns-file-name.yml Working with Namespace apiVersion: v1 kind: Namespace metadata: name: my-ns 23

Working with Namespace (Cont.) Viewing namespaces(ns) $ kubectl get namespaces NAME STATUS AGE default Active 1d kube-system Active 1d kube-public Active 1d 24

NAME READY STATUS RESTARTS AGE pod/coredns-78fcdf6894-cp6hs 1/1 Running 0 4m pod/coredns-78fcdf6894-l6sx7 1/1 Running 0 4m pod/etcd-k8slab 1/1 Running 0 3m pod/kube-apiserver-k8slab 1/1 Running 0 3m pod/kube-controller-manager-k8slab 1/1 Running 0 3m pod/kube-flannel-ds-kzt8n 1/1 Running 0 4m pod/kube-proxy-96srn 1/1 Running 0 4m pod/kube-scheduler-k8slab 1/1 Running 0 3m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kube-dns ClusterIP 10.96.0.10 53/UDP,53/TCP 4m ... Working with Namespace (Cont.) Viewing resources under namespace(ns) $ kubectl get all -n kube-system 25

Working with Namespace (Cont.) Using Namespace in pod - Example apiVersion: v1 kind: Pod metadata: name: nginx-demo namespace: my-ns spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80 26

Not All Objects are in a Namespace # In a namespace $ kubectl api-resources --namespaced=true # Not in a namespace $ kubectl api-resources --namespaced=false 27

● Overview ● What Happens When We Type kubectl run ● Namespace ● Kubernetes QoS ○ QoS Classes in K8s ○ QoS of Guaranteed ○ QoS of Burstable ○ QoS of BestEffort Outline 28

QoS Classes in K8s ● K8s provides different levels of Quality of Service(QoS) to pods ● For each resource, containers specify ○ Request: system guarantees to the amount of the resource for the container ○ Limit: system allows the container to use the maximum quantity ● Defining resource constraints for pods ○ CPU ○ Memory ● QoS Classes ● Guaranteed ● Burstable ● BestEffort 29

QoS Classes in K8s (Cont.) QoS classes to the Pod: ● Guaranteed ● Burstable ● BestEffort In decreasing order of priority. ● QoS Classes ● Guaranteed ● Burstable ● BestEffort 30

● Pods are ○ Considered Top-priority ○ Guaranteed to not be killed until pods exceed pods limits ● Every Container in the Pod have: ○ The same memory limit & memory request ○ The same CPU limit & CPU request QoS of Guaranteed ● QoS Classes ● Guaranteed ● Burstable ● BestEffort 31

kubectl create namespace qos-example kubectl create -f https://k8s.io/examples/pods/qos/qos-p od.yaml --namespace=qos-example kubectl get pod qos-demo --namespace=qos-example --output=yaml ... qosClass: Guaranteed kubectl delete pod qos-demo --namespace=qos-example apiVersion: v1 kind: Pod metadata: name: qos-demo namespace: qos-example spec: containers: - name: qos-demo-ctr image: nginx resources: limits: memory: "200Mi" cpu: "700m" requests: memory: "200Mi" cpu: "700m" QoS of Guaranteed (Cont.) ● QoS Classes ● Guaranteed ● Burstable ● BestEffort $ $ $ $ 32

QoS of Burstable ● Pods have some form of minimal resource guarantee ● Can use more resources when available ● Pod can be killed once they exceed their requests ● A Pod is given a QoS class of Burstable if: ○ Pod doesn’t meet the criteria for QoS class Guaranteed. ○ At least one Container in the Pod has a memory or CPU request. ● QoS Classes ● Guaranteed ● Burstable ● BestEffort 33

kubectl create namespace qos-example kubectl create -f https://k8s.io/examples/pods/qos/qos-p od-2.yaml --namespace=qos-example kubectl get pod qos-demo-2 --namespace=qos-example --output=yaml ... qosClass: Burstable kubectl delete pod qos-demo-2 --namespace=qos-example apiVersion: v1 kind: Pod metadata: name: qos-demo-2 namespace: qos-example spec: containers: - name: qos-demo-2-ctr image: nginx resources: limits: memory: "200Mi" requests: memory: "100Mi" QoS of Burstable (Cont.) ● QoS Classes ● Guaranteed ● Burstable ● BestEffort $ $ $ $ 34

kubectl create namespace qos-example kubectl create -f https://k8s.io/examples/pods/qos/qos-p od-4.yaml --namespace=qos-example kubectl get pod qos-demo-4 --namespace=qos-example --output=yaml ... qosClass: Burstable kubectl delete pod qos-demo-4 --namespace=qos-example apiVersion: v1 kind: Pod metadata: name: qos-demo-4 namespace: qos-example spec: containers: - name: qos-demo-4-ctr-1 image: nginx resources: requests: memory: "200Mi" - name: qos-demo-4-ctr-2 image: redis QoS of Burstable (Cont.) ● QoS Classes ● Guaranteed ● Burstable ● BestEffort $ $ $ $ 35

● Pods are ○ Considered lowest priority ○ The first to get killed if the system runs out of memory ● These containers can use any amount of free memory in the node Pod must NOT have any memory or CPU limits or requests. QoS of BestEffort ● QoS Classes ● Guaranteed ● Burstable ● BestEffort 36

kubectl create namespace qos-example kubectl create -f https://k8s.io/examples/pods/qos/qos-p od-3.yaml --namespace=qos-example kubectl get pod qos-demo-3 --namespace=qos-example --output=yaml ... qosClass: BestEffort kubectl delete pod qos-demo-3 --namespace=qos-example apiVersion: v1 kind: Pod metadata: name: qos-demo-3 namespace: qos-example spec: containers: - name: qos-demo-3-ctr image: nginx QoS of BestEffort (Cont.) ● QoS Classes ● Guaranteed ● Burstable ● BestEffort $ $ $ $ 37

Defined in terms of Request and Limit ● Guaranteed: highest protection ○ request > 0 && limit == request ● Burstable: medium protection ○ request > 0 && limit > request ● Best Effort: lowest protection ○ request == 0 QoS Classes in K8s 38

References ● https://kubernetes.io ● https://github.com/jamiehannaford/what-happens-when-k8s ● https://qconsf2018.container.training ● https://medium.com/google-cloud/quality-of-service-class-qos-in-kubernetes -bb76a89eb2c6 ● https://drive.google.com/file/d/1iOsAa4HwXrNMfkkTJFA1mHt6glgpOYbL/vie w ● https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod /#qos-classes 39

Thanks for your attention. Slides: http://bit.ly/k8s1206 GitHub, Telegram: @sufuf3 Twitter: @sufuf3149 40