kubectl (et al) Tips and Tricks KubernetesUG Singapore - August 2017

Overview - Miscellaneous setup - Use Abbreviations - Formatting Output - Use Explain - Proxying and Forwarding - Explore API Groups and Resources - Recommended Auxiliary tools

Misc Setup - shell autocompletion - Enable shell autocompletion For OSX & Bash… (see link for Linux / zsh / …) brew install bash-completion kubectl completion bash > $(brew --prefix)/etc/bash_completion.d/kubectl echo "source $(brew --prefix)/etc/bash_completion" >> $HOME/.bash_profile

Misc Setup - shell autocompletion Or use kube-prompt ...

Misc Setup - merging configurations - KUBECONFIG file describes (~/.kube/config ) - clusters (set-cluster) - users (set-user), and - contexts (set-context / use-context / current-context) = (user,cluster,namespace) - KUBECONFIG environment variable - merges a list of config file paths

Misc Setup - generating configurations - Use the KUBECONFIG env variable to generate config file I.e.: generate config script - Or for Google Cloud clusters: KUBECONFIG=./ws01/config gcloud container clusters get-credentials c01

Misc Setup - inspecting configurations - View config has multiple output options (see later) - View has --minify option kubectl config view --minify clusters: - cluster: certificate-authority: /Users/m121-hb/.minikube/ca.crt server: https://192.168.64.3:8443 name: minikube contexts: - context: cluster: minikube user: minikube name: minikube current-context: minikube Users: ...

Misc Setup - inspecting configurations - Get overview of available contexts kubectl config get-contexts CURRENT NAME CLUSTER AUTHINFO NAMESPACE * minikube minikube minikube prod production admin-production default staging staging admin-staging default

Misc Setup - add context and namespace to prompt - Use powerline-kubernetes - Or with simple shell functions

Misc Setup - Use kubectx - Use kubectx to switch contexts and namespaces easily $ kctx minikube prod Staging $ kctx staging Switched to context "staging". $ kctx - Switched to context "minikube". $ kns default kube-public kube-system $ kns kube-system Context "minikube" modified. Active namespace is "kube-system". $ kns - Context "minikube" modified. Active namespace is "default".

Use Abbreviations - Most commonly used - Node no - Pod po - Deployment deploy - Service svc - Namespace ns - ReplicaSet rs - ConfigMap cm - Ingress ing - Daemonset ds

Formatting output - Output yaml / json kubectl get po authn-dex-1709624687-gxmr1 -o yaml apiVersion: v1 kind: Pod metadata: Annotations: ... labels: app: dex release: authn name: authn-dex-1709624687-gxmr1 namespace: default spec: containers:...

Formatting output - Output wide kubectl get no -o wide NAME STATUS AGE VERSION EXTERNAL-IP OS-IMAGE KERNEL-VERSION minikube Ready 12d v1.6.4 Buildroot 2017.02 4.9.13

Formatting output - Explore output with jid kubectl get no -o json | jid -q | pbcopy

Formatting output - Advanced jq queries... .items[] for each item select apply filter [ … ] box into array @tsv convert array to tab separated output kubectl get no -o json | jq -r '.items[] \ | select(.spec.unschedulable!=true) \ | [.metadata.name,.spec.externalID] | @tsv' ip-172-10-10-139.ap-southeast-1.compute.internal i-10e8a7c3ba512909f ip-172-10-10-172.ap-southeast-1.compute.internal i-01276ef5c4716745c ip-172-10-10-29.ap-southeast-1.compute.internal i-1d8c68b7524d978f9

Formatting output - Golang Templates kubectl get no -o go-template='{{range .items}}{{if not .spec.unschedulable}}{{.metadata.name}} {{.spec.externalID}}{{"\n"}}{{end}}{{end}}' ip-172-10-10-139.ap-southeast-1.compute.internal i-10e8a7c3ba512909f ip-172-10-10-172.ap-southeast-1.compute.internal i-01276ef5c4716745c ip-172-10-10-29.ap-southeast-1.compute.internal i-1d8c68b7524d978f9

Formatting output - or Custom Columns... kubectl get no -o=custom-columns=NAME:.metadata.name,AWS-INSTANCE:.spec.externalID NAME AWS-INSTANCE ip-172-10-10-139.ap-southeast-1.compute.internal i-10e8a7c3ba512909f ip-172-10-10-172.ap-southeast-1.compute.internal i-01276ef5c4716745c ...

Formatting output - Advanced jq queries... [ … ] box into array join join array of strings kubectl get --all-namespaces svc -o json | jq -r '.items[] \ | [.metadata.name,([.spec.ports[].nodePort | tostring ] | join("|"))] \ | @csv' "vault-sample-app","null" "kafka-1","31789|30938" "Kafka-2","32650|31374" ... "zookeeper-1","31829|32495|30224"

Formatting output - Advanced jq queries... [ … ] box into array map for each element apply function and return as new array kubectl get pods --all-namespaces -o json | jq '.items \ | map({podName: .metadata.name, nodeName: .spec.nodeName}) \ | group_by(.nodeName) \ | map({nodeName: .[0].nodeName, pods: map(.podName)})' { "nodeName": "ip-172-10-11-47.ap-southeast-1.compute.internal", "pods": [ "kube-registry-proxy-qss54", "Datadog-agent-datadog-h8l46", ] }

Formatting output - Jsonpath - List all image on all nodes .. recursively return all fields named image kubectl get pods --all-namespaces -o jsonpath="{..image}" \ | tr -s '[[:space:]]' '\n' \ | sort \ | uniq -c 2 alpine:3.5 2 gcr.io/google-containers/kube-addon-manager:v6.4-beta.1 2 gcr.io/google_containers/defaultbackend:1.2 2 gcr.io/google_containers/kubernetes-dashboard-amd64:v1.6.3 2 gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.11 2 gcr.io/kubernetes-helm/tiller:v2.5.1 6 nginx:1.10-alpine 2 quay.io/coreos/dex:v2.6.0 2 quay.io/vincentdesmet/dex-app:0.0.1 2 registry:2.6.1 2 so0k/kuar-inspector:1.0.0

Formatting output - Jsonpath - read docker registry secret secret_name=myregistry kubectl get secret ${secret_name} -o jsonpath="{['data']['\.dockercfg']}" \ | base64 -D | jq . { "registry.honestbee.com": { "username": "kube", "password": "kubeCuddles", "email": "cattle@kube.com", "auth": "a3ViZTprdWJlQ2VlVGVlTAo=" } }

Use Explain - Explain can be used to review allowed values for fields kubectl get svc -n kube-system kubernetes-dashboard -o yaml # what are the possible service types? kubectl explain svc.spec.type

Generate manifest YAML with kubectl - Ref kubectl get svc -n kube-system kubernetes-dashboard -o yaml # what are the possible service types? kubectl explain svc.spec.type

Proxying and Port Forwarding i.e. Use kube proxy with visualizer $ kubectl proxy --www=. --www-prefix=/visualizer &

Proxying and Port Forwarding - Target specific Pods $ kubectl run kuar --image=so0k/kuar-inspector:1.0.0 deployment "kuar" created $ kubectl get pod NAME READY kuar-3186028377-lzq77 0/1 $ kubectl port-forward kuar-3186028377-lzq77 8080:80 Forwarding from 127.0.0.1:8080 -> 80 Forwarding from [::1]:8080 -> 80 Handling connection for 8080 Handling connection for 8080 https://github.com/kelseyhightower/inspector

Proxying and Port Forwarding - Debugging:

Proxying and Port Forwarding Or watch logs for all pods with stern

Proxying and Port Forwarding Accessing API post 1.3 (pre go-sdk) using Service Account Token - Before 1.6, Mounted by default: - In 1.6+ use: apiVersion: v1 kind: ServiceAccount metadata: name: build-robot automountServiceAccountToken: false

Explore API Groups and Resources - Authorization requires in depth knowledge of API groups and resources: - The “core” (oftentimes called “legacy”, due to not having an explicit group name) group, is at REST path /api/v1 (empty string is also “core”) kind: Role apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: namespace: default name: pod-reader rules: - apiGroups: [""] # "" indicates the core API group resources: ["pods"] verbs: ["get", "watch", "list"]

Explore API Groups and Resources - The “core” is at REST path /api/v1 Use openAPI (Swagger) spec... list REST Paths of resource types: kubectl proxy & curl -sL localhost:8001/swagger.json | jq -r '.paths | keys[]' | less /api/v1/namespaces/{name} /api/v1/nodes /api/v1/persistentvolumeclaims /api/v1/persistentvolumes /api/v1/pods /api/v1/secrets /api/v1/serviceaccounts ... /apis/apps/v1beta1/deployments /apis/extensions/v1beta1/deployments /apis/extensions/v1beta1/ingresses

Detach without killing shell

Recommended Auxiliary Tools ● kube-prompt - interactive kube shell ● kubectx - switch contexts and namespaces easily ● stern - follow multiple pods More Tips: - Kubectl cheat sheet in Kubernetes docs - CoreOS blog series

The End

Wait, one more…. team Kyoob Cuddle!