Building kubectl plugins like a pro

Let’s talk about how to build idiomatic CLI tools that look and feel like kubectl, using the libraries and techniques from SIG CLI. In this talk, we’ll also talk about Krew kubectl plugin manager (a SIG CLI project) and showcase some useful popular plugins.

For almost a decade now, Kubectl has supported a plugin extension mechanism. Today, the Krew plugin manager distributes over 275 open source kubectl plugins that the community uses every day to take their Kubernetes experience to the next level.
In this talk, we will
* Show some techniques and libraries to build your own CLI tools that look and feel like kubectl itself, using some building blocks that SIG CLI provides (that you probably didn’t know about)
* Talk about idioms to pay attention to while building and distributing your own plugins
* Some new features in kubectl plugin machinery introduced in Kubernetes v1.31
* Talk about the brief history of the Krew project, how it got started and became a SIG CLI subproject, how it grew to 275+ plugins
* Showcase some of the cool kubectl plugins from the open source.

Transcript

1. Building kubectl plugins like a pro Ahmet Alp Balkan @ahmetb
2. Developed a few kubectl tools: • kubectx / kubens • krew • kubectl tree • kubectl pods-on • kubectl foreach • kubectl cond About me Sr. Staff Engineer at LinkedIn. Prev: • Twitter • Google Cloud • Microsoft Azure
3. Outline • kubectl plugin machinery • Krew plugin manager • Writing idiomatic kubectl CLIs ◦ Generic CLI flags ◦ Resource builder • Bonus: Favorite kubectl plugins
4. Is this talk just about kubectl? Many non-kubectl CLIs interact with Kubernetes API! • helm • istioctl • karmadactl • OpenKruise • openebsctl • kn (Knative) • …
5. kubectl plugin machinery
6. kubectl plugins primer • Introduced in Kubernetes as alpha v1.8 (2017) • Allows adding new plugins to kubectl ◦ Can’t replace existing commands with a plugin • Place an executable in $PATH named: kubectl-foo ⇒ kubectl foo kubectl-foo-bar ⇒ kubectl foo bar (or kubectl foo-bar) kubectl-foo_bar ⇒ kubectl foo-bar
7. Normally you can’t shadow the builtin “create” command. But now: kubectl-create-couchbase ⇒ kubectl create couchbase Caveat: “kubectl create -h” won’t show your plugin New in v1.31: “create” plugins
8. Krew plugin manager
9. Krew (krew.sigs.k8s.io) • Started as an intern project at Google in 2018 • Accepted into SIG CLI in 2019 • Maintains a curated plugin list (krew-index) • Distributes 275+ plugins in OSS • Looking for maintainers and plugin curators!
11. Developing idiomatic Kubernetes CLIs
12. Look and feel like kubectl 1. Same set of CLI options (flags) ◦ -n/--namespace/-A/--all-namespaces ◦ -l/--selector 2. Configuration and context awareness ◦ --context flag, reading from the right $KUBECONFIG file (merging, overrides…) 3. Subcommand organization: kubectl <verb> <resource> 4. Common way of specifying resources: ◦ deploy/foo == deployment foo == deployments.v1.apps/foo ◦ -f deployment.yaml Writing idiomatic kubectl plugins
13. Generic CLI flags Add all “kubectl options” to your command
14. Initializing a typed Kubernetes client
15. resource.Builder from k8s.io/cli-runtime/pkg/resource! resource.NewBuilder(configFlags) Use positional args (e.g. “pod/foo”, “pods foo”): builder.ResourceTypeOrNameArgs(true, os.Args[1:]) or a specific type of resource: builder.ResourceNames("pods", podNames...) Selecting resources
16. Working with builtin vs dynamic types If you’re only using builtin types: builder.WithScheme(scheme.Scheme, …). If you’re working with custom resources: builder.Unstructured() …or register your own scheme.
17. Multiple namespace sources: 1. -n/--namespace: defaults to “” ◦ but in Kubernetes API “” means all namespaces! 2. --all-namespaces/-A: need to define yourself 3. namespace configured for context (kubens) in the ~/.kube/config file (cli-runtime doesn’t help you with this part) Working with namespace flags
18. Working with printers The usual -o=yaml|json|custom-columns|name|... flag you’re familiar with to print objects from your plugin.
19. Server-side printing As of v1.15 the columns printed and their values are decided on the API server. You can get server side printing in your CLIs with request header: Accept: application/json;as=Table;v=v1;g=meta.k8s.io,application/json
20. Left as an exercise to the listener • Select resources via -f/-k/-R • Select resources via -l/--selector
21. More tips • If you make a lot of requests, adjust client-side rate-limiting: ◦ restConfig.QPS ◦ restConfig.Burst • If you make large list calls, use pagination. ◦ builder.RequestChunksOf(1000).[...]
22. Recommended resources Ivan Velichko (@iximiuz)’s blog + https://github.com/iximiuz/client-go-examples
23. Recommended resources • Read the cli-runtime/builder pkg • Demo code: https://github.com/ahmetb/rejekts-kubectl-demo • Server printer example: “kubectl pods-on” • Resource builder example: “kubectl cond”
24. Useful kubectl plugins
25. kubectl who-can
26. kubectl get-all Queries all API groups
27. kubectl tree / lineage Show ownerResource hierarchy
28. kubectl foreach / mc Run a command against multiple clusters in parallel.
29. kubectl images
30. kubectl neat / eksporter Tidy up a resource for saving the manifest locally
31. kubectl cond Render resource conditions as table.
32. kubectl blame / fields Show which controller/actor updated which field/when
33. Thank you! tw/gh: @ahmetb 🦋: @ahmet.dev