Slide 1

Slide 1 text

kubectl (et al) Tips and Tricks KubernetesUG Singapore - August 2017

Slide 2

Slide 2 text

Overview - Miscellaneous setup - Use Abbreviations - Formatting Output - Use Explain - Proxying and Forwarding - Explore API Groups and Resources - Recommended Auxiliary tools

Slide 3

Slide 3 text

Misc Setup - shell autocompletion - Enable shell autocompletion For OSX & Bash… (see link for Linux / zsh / …) brew install bash-completion kubectl completion bash > $(brew --prefix)/etc/bash_completion.d/kubectl echo "source $(brew --prefix)/etc/bash_completion" >> $HOME/.bash_profile

Slide 4

Slide 4 text

Misc Setup - shell autocompletion Or use kube-prompt ...

Slide 5

Slide 5 text

Misc Setup - merging configurations - KUBECONFIG file describes (~/.kube/config ) - clusters (set-cluster) - users (set-user), and - contexts (set-context / use-context / current-context) = (user,cluster,namespace) - KUBECONFIG environment variable - merges a list of config file paths

Slide 6

Slide 6 text

Misc Setup - generating configurations - Use the KUBECONFIG env variable to generate config file I.e.: generate config script - Or for Google Cloud clusters: KUBECONFIG=./ws01/config gcloud container clusters get-credentials c01

Slide 7

Slide 7 text

Misc Setup - inspecting configurations - View config has multiple output options (see later) - View has --minify option kubectl config view --minify clusters: - cluster: certificate-authority: /Users/m121-hb/.minikube/ca.crt server: https://192.168.64.3:8443 name: minikube contexts: - context: cluster: minikube user: minikube name: minikube current-context: minikube Users: ...

Slide 8

Slide 8 text

Misc Setup - inspecting configurations - Get overview of available contexts kubectl config get-contexts CURRENT NAME CLUSTER AUTHINFO NAMESPACE * minikube minikube minikube prod production admin-production default staging staging admin-staging default

Slide 9

Slide 9 text

Misc Setup - add context and namespace to prompt - Use powerline-kubernetes - Or with simple shell functions

Slide 10

Slide 10 text

Misc Setup - Use kubectx - Use kubectx to switch contexts and namespaces easily $ kctx minikube prod Staging $ kctx staging Switched to context "staging". $ kctx - Switched to context "minikube". $ kns default kube-public kube-system $ kns kube-system Context "minikube" modified. Active namespace is "kube-system". $ kns - Context "minikube" modified. Active namespace is "default".

Slide 11

Slide 11 text

Use Abbreviations - Most commonly used - Node no - Pod po - Deployment deploy - Service svc - Namespace ns - ReplicaSet rs - ConfigMap cm - Ingress ing - Daemonset ds

Slide 12

Slide 12 text

Formatting output - Output yaml / json kubectl get po authn-dex-1709624687-gxmr1 -o yaml apiVersion: v1 kind: Pod metadata: Annotations: ... labels: app: dex release: authn name: authn-dex-1709624687-gxmr1 namespace: default spec: containers:...

Slide 13

Slide 13 text

Formatting output - Output wide kubectl get no -o wide NAME STATUS AGE VERSION EXTERNAL-IP OS-IMAGE KERNEL-VERSION minikube Ready 12d v1.6.4 Buildroot 2017.02 4.9.13

Slide 14

Slide 14 text

Formatting output - Explore output with jid kubectl get no -o json | jid -q | pbcopy

Slide 15

Slide 15 text

Formatting output - Advanced jq queries... .items[] for each item select apply filter [ … ] box into array @tsv convert array to tab separated output kubectl get no -o json | jq -r '.items[] \ | select(.spec.unschedulable!=true) \ | [.metadata.name,.spec.externalID] | @tsv' ip-172-10-10-139.ap-southeast-1.compute.internal i-10e8a7c3ba512909f ip-172-10-10-172.ap-southeast-1.compute.internal i-01276ef5c4716745c ip-172-10-10-29.ap-southeast-1.compute.internal i-1d8c68b7524d978f9

Slide 16

Slide 16 text

Formatting output - Golang Templates kubectl get no -o go-template='{{range .items}}{{if not .spec.unschedulable}}{{.metadata.name}} {{.spec.externalID}}{{"\n"}}{{end}}{{end}}' ip-172-10-10-139.ap-southeast-1.compute.internal i-10e8a7c3ba512909f ip-172-10-10-172.ap-southeast-1.compute.internal i-01276ef5c4716745c ip-172-10-10-29.ap-southeast-1.compute.internal i-1d8c68b7524d978f9

Slide 17

Slide 17 text

Formatting output - or Custom Columns... kubectl get no -o=custom-columns=NAME:.metadata.name,AWS-INSTANCE:.spec.externalID NAME AWS-INSTANCE ip-172-10-10-139.ap-southeast-1.compute.internal i-10e8a7c3ba512909f ip-172-10-10-172.ap-southeast-1.compute.internal i-01276ef5c4716745c ...

Slide 18

Slide 18 text

Formatting output - Advanced jq queries... [ … ] box into array join join array of strings kubectl get --all-namespaces svc -o json | jq -r '.items[] \ | [.metadata.name,([.spec.ports[].nodePort | tostring ] | join("|"))] \ | @csv' "vault-sample-app","null" "kafka-1","31789|30938" "Kafka-2","32650|31374" ... "zookeeper-1","31829|32495|30224"

Slide 19

Slide 19 text

Formatting output - Advanced jq queries... [ … ] box into array map for each element apply function and return as new array kubectl get pods --all-namespaces -o json | jq '.items \ | map({podName: .metadata.name, nodeName: .spec.nodeName}) \ | group_by(.nodeName) \ | map({nodeName: .[0].nodeName, pods: map(.podName)})' { "nodeName": "ip-172-10-11-47.ap-southeast-1.compute.internal", "pods": [ "kube-registry-proxy-qss54", "Datadog-agent-datadog-h8l46", ] }

Slide 20

Slide 20 text

Formatting output - Jsonpath - List all image on all nodes .. recursively return all fields named image kubectl get pods --all-namespaces -o jsonpath="{..image}" \ | tr -s '[[:space:]]' '\n' \ | sort \ | uniq -c 2 alpine:3.5 2 gcr.io/google-containers/kube-addon-manager:v6.4-beta.1 2 gcr.io/google_containers/defaultbackend:1.2 2 gcr.io/google_containers/kubernetes-dashboard-amd64:v1.6.3 2 gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.11 2 gcr.io/kubernetes-helm/tiller:v2.5.1 6 nginx:1.10-alpine 2 quay.io/coreos/dex:v2.6.0 2 quay.io/vincentdesmet/dex-app:0.0.1 2 registry:2.6.1 2 so0k/kuar-inspector:1.0.0

Slide 21

Slide 21 text

Formatting output - Jsonpath - read docker registry secret secret_name=myregistry kubectl get secret ${secret_name} -o jsonpath="{['data']['\.dockercfg']}" \ | base64 -D | jq . { "registry.honestbee.com": { "username": "kube", "password": "kubeCuddles", "email": "[email protected]", "auth": "a3ViZTprdWJlQ2VlVGVlTAo=" } }

Slide 22

Slide 22 text

Use Explain - Explain can be used to review allowed values for fields kubectl get svc -n kube-system kubernetes-dashboard -o yaml # what are the possible service types? kubectl explain svc.spec.type

Slide 23

Slide 23 text

Generate manifest YAML with kubectl - Ref kubectl get svc -n kube-system kubernetes-dashboard -o yaml # what are the possible service types? kubectl explain svc.spec.type

Slide 24

Slide 24 text

Proxying and Port Forwarding i.e. Use kube proxy with visualizer $ kubectl proxy --www=. --www-prefix=/visualizer &

Slide 25

Slide 25 text

Proxying and Port Forwarding - Target specific Pods $ kubectl run kuar --image=so0k/kuar-inspector:1.0.0 deployment "kuar" created $ kubectl get pod NAME READY kuar-3186028377-lzq77 0/1 $ kubectl port-forward kuar-3186028377-lzq77 8080:80 Forwarding from 127.0.0.1:8080 -> 80 Forwarding from [::1]:8080 -> 80 Handling connection for 8080 Handling connection for 8080 https://github.com/kelseyhightower/inspector

Slide 26

Slide 26 text

Proxying and Port Forwarding - Debugging:

Slide 27

Slide 27 text

Proxying and Port Forwarding Or watch logs for all pods with stern

Slide 28

Slide 28 text

Proxying and Port Forwarding Accessing API post 1.3 (pre go-sdk) using Service Account Token - Before 1.6, Mounted by default: - In 1.6+ use: apiVersion: v1 kind: ServiceAccount metadata: name: build-robot automountServiceAccountToken: false

Slide 29

Slide 29 text

Explore API Groups and Resources - Authorization requires in depth knowledge of API groups and resources: - The “core” (oftentimes called “legacy”, due to not having an explicit group name) group, is at REST path /api/v1 (empty string is also “core”) kind: Role apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: namespace: default name: pod-reader rules: - apiGroups: [""] # "" indicates the core API group resources: ["pods"] verbs: ["get", "watch", "list"]

Slide 30

Slide 30 text

Explore API Groups and Resources - The “core” is at REST path /api/v1 Use openAPI (Swagger) spec... list REST Paths of resource types: kubectl proxy & curl -sL localhost:8001/swagger.json | jq -r '.paths | keys[]' | less /api/v1/namespaces/{name} /api/v1/nodes /api/v1/persistentvolumeclaims /api/v1/persistentvolumes /api/v1/pods /api/v1/secrets /api/v1/serviceaccounts ... /apis/apps/v1beta1/deployments /apis/extensions/v1beta1/deployments /apis/extensions/v1beta1/ingresses

Slide 31

Slide 31 text

Detach without killing shell

Slide 32

Slide 32 text

Recommended Auxiliary Tools ● kube-prompt - interactive kube shell ● kubectx - switch contexts and namespaces easily ● stern - follow multiple pods More Tips: - Kubectl cheat sheet in Kubernetes docs - CoreOS blog series

Slide 33

Slide 33 text

The End

Slide 34

Slide 34 text

Wait, one more…. team Kyoob Cuddle!