Building kubectl plugins like a pro Ahmet Alp Balkan @ahmetb

Developed a few kubectl tools: ● kubectx / kubens ● krew ● kubectl tree ● kubectl pods-on ● kubectl foreach ● kubectl cond About me Sr. Staff Engineer at LinkedIn. Prev: ● Twitter ● Google Cloud ● Microsoft Azure

Outline ● kubectl plugin machinery ● Krew plugin manager ● Writing idiomatic kubectl CLIs ○ Generic CLI flags ○ Resource builder ● Bonus: Favorite kubectl plugins

Is this talk just about kubectl? Many non-kubectl CLIs interact with Kubernetes API! ● helm ● istioctl ● karmadactl ● OpenKruise ● openebsctl ● kn (Knative) ● …

kubectl plugin machinery

kubectl plugins primer ● Introduced in Kubernetes as alpha v1.8 (2017) ● Allows adding new plugins to kubectl ○ Can’t replace existing commands with a plugin ● Place an executable in $PATH named: kubectl-foo ⇒ kubectl foo kubectl-foo-bar ⇒ kubectl foo bar (or kubectl foo-bar) kubectl-foo_bar ⇒ kubectl foo-bar

Normally you can’t shadow the builtin “create” command. But now: kubectl-create-couchbase ⇒ kubectl create couchbase Caveat: “kubectl create -h” won’t show your plugin New in v1.31: “create” plugins

Krew plugin manager

Krew (krew.sigs.k8s.io) ● Started as an intern project at Google in 2018 ● Accepted into SIG CLI in 2019 ● Maintains a curated plugin list (krew-index) ● Distributes 275+ plugins in OSS ● Looking for maintainers and plugin curators!

No content

Developing idiomatic Kubernetes CLIs

Look and feel like kubectl 1. Same set of CLI options (flags) ○ -n/--namespace/-A/--all-namespaces ○ -l/--selector 2. Configuration and context awareness ○ --context flag, reading from the right $KUBECONFIG file (merging, overrides…) 3. Subcommand organization: kubectl 4. Common way of specifying resources: ○ deploy/foo == deployment foo == deployments.v1.apps/foo ○ -f deployment.yaml Writing idiomatic kubectl plugins

Generic CLI flags Add all “kubectl options” to your command

Initializing a typed Kubernetes client

resource.Builder from k8s.io/cli-runtime/pkg/resource! resource.NewBuilder(configFlags) Use positional args (e.g. “pod/foo”, “pods foo”): builder.ResourceTypeOrNameArgs(true, os.Args[1:]) or a specific type of resource: builder.ResourceNames("pods", podNames...) Selecting resources

Working with builtin vs dynamic types If you’re only using builtin types: builder.WithScheme(scheme.Scheme, …). If you’re working with custom resources: builder.Unstructured() …or register your own scheme.

Multiple namespace sources: 1. -n/--namespace: defaults to “” ○ but in Kubernetes API “” means all namespaces! 2. --all-namespaces/-A: need to define yourself 3. namespace configured for context (kubens) in the ~/.kube/config file (cli-runtime doesn’t help you with this part) Working with namespace flags

Working with printers The usual -o=yaml|json|custom-columns|name|... flag you’re familiar with to print objects from your plugin.

Server-side printing As of v1.15 the columns printed and their values are decided on the API server. You can get server side printing in your CLIs with request header: Accept: application/json;as=Table;v=v1;g=meta.k8s.io,application/json

Left as an exercise to the listener ● Select resources via -f/-k/-R ● Select resources via -l/--selector

More tips ● If you make a lot of requests, adjust client-side rate-limiting: ○ restConfig.QPS ○ restConfig.Burst ● If you make large list calls, use pagination. ○ builder.RequestChunksOf(1000).[...]

Recommended resources Ivan Velichko (@iximiuz)’s blog + https://github.com/iximiuz/client-go-examples

Recommended resources ● Read the cli-runtime/builder pkg ● Demo code: https://github.com/ahmetb/rejekts-kubectl-demo ● Server printer example: “kubectl pods-on” ● Resource builder example: “kubectl cond”

Useful kubectl plugins

kubectl who-can

kubectl get-all Queries all API groups

kubectl tree / lineage Show ownerResource hierarchy

kubectl foreach / mc Run a command against multiple clusters in parallel.

kubectl images

kubectl neat / eksporter Tidy up a resource for saving the manifest locally

kubectl cond Render resource conditions as table.

kubectl blame / fields Show which controller/actor updated which field/when

Thank you! tw/gh: @ahmetb 🦋: @ahmet.dev