Kubernetes Operators in Python with Kopf

Slide 1

Slide 1 text

Kubernetes Operators in Python with Kopf Sergey Vasilyev Twitter: @nolar DevNation Day, 2020-09-15

Slide 2

Slide 2 text

About me ● My name is Sergey Vasilyev ○ https://twitter.com/nolar ● A Python developer from Siberia. Now in Berlin, Germany. ● ~20 years of experience in Software Engineering. ● In love with Python since 2009. ● Sr. Backend Engineer, Zalando SE. ● Running ML apps & infra on Kubernetes for Zalando Pricing & Forecasting.

Slide 3

Slide 3 text

Express Introduction to Kubernetes (from Software Developers’ point of view)

Slide 4

Slide 4 text

“Kubernetes is a container orchestrator”

Slide 5

Slide 5 text

Kubernetes under the hood

Slide 6

Slide 6 text

Extending Kubernetes: Custom Resource Definitions

Slide 7

Slide 7 text

Defining a resource: YAML-driven development ● Required: group/version. ● Required: kind/plural/singular names. ● Required: scope (“Namespaced”). ● Optional: short names (aliases). ● Optional: list formatting columns. ● Optional: categories. ● Optional: schema. $ kubectl apply -f crd.yaml $ kubectl apply -f obj.yaml Resource definition Resource itself

Slide 8

Slide 8 text

Extending Kubernetes: Controllers/Operators

Slide 9

Slide 9 text

Common problem: infrastructure code hassle

Slide 10

Slide 10 text

MAKE A FRAMEWORK!

Slide 11

Slide 11 text

WILLKOMMEN, KOPF!

Slide 12

Slide 12 text

The simplest Kubernetes operator

Slide 13

Slide 13 text

The simplest Kubernetes operator And here we are! Creating: {'duration': '1m', 'field': 'value', 'items': ['item1', 'item2']} [2019-02-25 14:06:54,742] kopf.reactor.handlin [INFO ] [default/kopf-example-1] Handler create_fn succeeded. [2019-02-25 14:06:54,856] kopf.reactor.handlin [INFO ] [default/kopf-example-1] All handlers succeeded for creation. Name: kopf-example-1 ... Status: create_fn: Message: hello world Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Success 81s kopf Handler create_fn succeeded. $ kopf run scripts.py [--verbose] $ kubectl apply -f obj.yaml $ kubectl describe -f obj.yaml

Slide 14

Slide 14 text

Resource-watching handlers ● As often, as the events arrive from K8s API. ● Raw data, no interpretation. ● Fire-and-forget, errors are ignored. ● Similar to `kubectl get --watch`

Slide 15

Slide 15 text

Change-detection handlers

Slide 16

Slide 16 text

Error handling ● A handler — an atomic unit of work. ● Retried forever (until succeeded of failed permanently).

Slide 17

Slide 17 text

Timers ● Tick every N seconds. ● Idle for the resource to be stable for some time.

Slide 18

Slide 18 text

Daemons — accompany the resource ● Accompany the resource during its lifetime. ● Stop once deleted.

Slide 19

Slide 19 text

Debugging & breakpoints

Slide 20

Slide 20 text

Other things ● Python logging → Kubernetes Events. ● Custom authentication → @on.login ● Configuration → @on.startup / @on.cleanup ● Embeddable into applications. ● Natively asyncio-based. ● Resource hierarchies. ● Testing toolkit. ● And more… ● Other requests? Create a feature request!

Slide 21

Slide 21 text

Kubernetes Operator Patterns (and ideas)

Slide 22

Slide 22 text

Children resources & status gathering

Slide 23

Slide 23 text

Children resources & status gathering

Slide 24

Slide 24 text

Kubernetes as an orchestrator of everything

Slide 25

Slide 25 text

Kubernetes is an orchestrator of everything

Slide 26

Slide 26 text

Application-specific auto-scaling https://github.com/zalando-incubator/es-operator

Slide 27

Slide 27 text

Application-specific auto-scaling

Slide 28

Slide 28 text

Cross-cluster connectivity

Slide 29

Slide 29 text

Cross-cluster connectivity

Slide 30

Slide 30 text

State-Driven Design (of User Experience)

Slide 31

Slide 31 text

The end? [no]

Slide 32

Slide 32 text

ROADMAP (3 major tracks) ● Features: ○ Cluster/resource awareness. ○ Admission hooks: validation/mutation. ○ More patterns: cross-resource handlers. ● Community: ○ Support Kopf-based operators. ○ Build a community of contributors. ● SDK: ○ YAML from Python — CRDs, RBAC, Deployments.

Slide 33

Slide 33 text

SUMMARY ● Kubernetes operators can be easy. ● Kubernetes operators can be ad-hoc. ● Kubernetes operators can be Pythonic. ● Kubernetes is an orchestrator of everything. ● Focus on the domain, not on the infrastructure. ● Simplicity & human-friendliness as the #1 priority. ● Python community as a huge leverage for Kubernetes. ● Use Kopf. ● Spread the word!

Slide 34

Slide 34 text

LINKS ● Source: https://github.com/nolar/kopf (support it with a ⭐) ● Examples: https://github.com/nolar/kopf/tree/master/examples ● Documentation: https://kopf.readthedocs.io/ ● Twitter: @nolar ● LinkedIn: https://www.linkedin.com/in/sergeyvasilyev/ ● Slides: https://speakerdeck.com/nolar/kubernetes-operators-in-python-with-kopf

Slide 35

Slide 35 text

Questions? Kubernetes Operators in Python with Kopf Sergey Vasilyev Twitter: @nolar DevNation Day, 2020-09-15