Share
Embed
Transcript
80 slides
-
EXTENDING K8S WITH ELIXIR* 1 — *or any other language
you are comfortable with -
GOALS OF THIS TALK ๏ explain why K8S plays such
a critical role nowadays ๏ share interesting and surprising facts about K8S ๏ encourage to play with K8S ⚽ 3 -
DISCLAIMER Due to time constaints (20 mins) and the broad
scope of the topic there will be quite some hand-waving involved. ⾠ There will be information gaps and unclarities. ⾠ That's OK. The goal is to trigger your curiosity! 4 -
OUTLINE ๏ WHY? ๏ K8S history ๏ K8S architecture -
Basic principles ๏ K8S Operator pattern ๏ Elixir tooling 6 -
Kubernetes eliminates infrastructure lock-in by providing core capabilities for containers
without imposing restrictions. 12 -
In the future, choosing a cloud provider could be just
a matter of which one offers the best performance/cost 13 -
2003-2004: BIRTH OF THE BORG SYSTEM @ GOOGLE ๏ large-scale
internal cluster management system ๏ hundreds of thousands of jobs ๏ from many thousands of applications ๏ across many clusters, each with up to tens of thousands of machines 16 — https://blog.risingstack.com/the-history-of-kubernetes/ -
2013: FROM BORG TO OMEGA ๏ cluster management system ๏
flexible, scalable scheduler for large compute clusters 17 — https://blog.risingstack.com/the-history-of-kubernetes/ -
2014: GOOGLE INTRODUCES KUBERNETES ๏ June 7: Initial release -
first github commit for Kubernetes ๏ July 10: Microsoft, RedHat, IBM, Docker joins the Kubernetes community. 18 — https://blog.risingstack.com/the-history-of-kubernetes/ -
K8S V0.1 WAS... ๏ a lean yet powerful open-source container
manager ๏ deploys containers into a fleet of machines ๏ provides health management and replication capabilities ๏ makes it easy for containers to connect to one another and the outside world 19 — https://cloudplatform.googleblog.com/2014/06/an-update-on-container-support-on-google-cloud-platform.html -
Additionally, Kubernetes is not a mere orchestration system. 21 —
https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ -
In fact, it eliminates the need for orchestration. 22 —
https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ -
The technical definition of orchestration is execution of a defined
workflow: first do A, then B, then C. 23 — https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ -
In contrast, Kubernetes is comprised of a set of independent,
composable control processes that continuously drive the current state towards the provided desired state. 24 — https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ -
It shouldn’t matter how you get from A to C.
25 — https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ -
This results in a system that is easier to use
and more powerful, robust, resilient, and extensible. 27 — https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ -
If you start thinking about Kubernetes as a fully event-driven
system, there's answers to so many "Why"'s 33 — https://twitter.com/embano1/status/1067537816324845569 -
Think of the API server as an immutable (replicated) log
(queue) and stream of events. 35 — https://twitter.com/embano1/status/1067537816324845569 -
Events are facts that can be causally related ("happened-before") or
not related at all (then we say they happened "concurrently"). etcd is an implementation detail. 36 — https://twitter.com/embano1/status/1067537816324845569 -
All controllers, e.g. the scheduler, deployment controller, endpoint controller, Kubelet,
etc. can be understood as consumers and/or producers (consumers can be producers as well, and vice versa). 37 — https://twitter.com/embano1/status/1067537816324845569 -
Consumers specify the objects (and optionally namespace) they want to
receive events from the API server. 38 — https://twitter.com/embano1/status/1067537816324845569 -
This is called a "watch" in Kubernetes. Think of the
combination of object+namespace as a dedicated (virtual) event queue the API server handles. 39 — https://twitter.com/embano1/status/1067537816324845569 -
Consumers and producers don't know about each other as they're
fully decoupled (by the log) and autonomous. 40 — https://twitter.com/embano1/status/1067537816324845569 -
This makes the whole system extremely scalable, robust and extensible
(flexible). 41 — https://twitter.com/embano1/status/1067537816324845569 -
Information takes time to propagate from consumer(s) to producer(s). The
picture above shows this where the HPA hasn't caught up with the metrics. 43 -
There's NO guarantee that the system will converge to the
desired state (even if you got an "OK"/ACK from the control plane). 45 — https://twitter.com/embano1/status/1067537816324845569 -
Eg: kubectl scale deploy --replicas n (with cpu requests >
cluster_capacity). You'll get a HTTP 200, even though there is no capacity left. 46 — https://twitter.com/embano1/status/1067537816324845569 -
For efficiency and speed received events are placed in an
in-memory buffer (typically also modelled as a queue) per controller. 48 — https://twitter.com/embano1/status/1067537816324845569 -
What if a controller crashes as it does not persist
state? 49 — https://twitter.com/embano1/status/1067537816324845569 -
This is not needed! The event-driven design will replay all
(appropriate) events when the controller starts, also known as event-sourcing. 50 — https://twitter.com/embano1/status/1067537816324845569 -
This is also very useful as events are "at most
once" delivered, i.e. could be lost during transmission. 51 — https://twitter.com/embano1/status/1067537816324845569 -
Kubernetes is level-triggered. If an event gets lost during transmission
(e.g. network issue), next time there's a sync you're going to receive the desired state 52 — https://twitter.com/embano1/status/1067537816324845569 -
Since information can get delivered more than once (e.g. after
failure, resync, etc.) and controllers don't talk to each other directly, ... 53 — https://twitter.com/embano1/status/1067537816324845569 -
... there's a potential of race conditions when state is
to be changed (e.g. a write to same object from different controllers). 54 — https://twitter.com/embano1/status/1067537816324845569 -
This is called optimistic concurrency and needs to be handled
in the application layer (i.e. in each controller logic). 55 — https://twitter.com/embano1/status/1067537816324845569 -
Idempotency and compare and set (based on monotonically increasing resource
versions) are patterns used here. 56 — https://twitter.com/embano1/status/1067537816324845569 -
The Control Plane maintains a record of all of the
Kubernetes Objects in the system, and runs continuous control loops to manage those objects’ state. 59 — https://kubernetes.io/docs/concepts/#kubernetes-control-plane -
At any given time, the Control Plane’s control loops will
respond to changes in the cluster and work to make the actual state of all the objects in the system match the desired state that you provided. 60 — https://kubernetes.io/docs/concepts/#kubernetes-control-plane -
CONTROL LOOPS ๏ respond to changes (watch-ing) ๏ mutate the
"world" towards desired state ๏ cause any kind of side-effects ! 61 -
Kubernetes 1.7 has added an important feature called Custom Controllers
64 — https://blog.couchbase.com/kubernetes-operators-game-changer/ -
CUSTOM CONTROLLERS ENABLE DEVELOPERS TO ๏ extend and add new
functionalities ๏ replace existent ones (like replacing kube-proxy for instance) ๏ automate administration tasks as if they were a native Kubernetes component 65 — https://blog.couchbase.com/kubernetes-operators-game-changer/ -
An Operator is nothing more than a set of application-specific
custom controllers. So, why is it a game changer? 66 -
which means they can monitor the cluster, change pods/services, scale
up/ down and call endpoints of the running applications, all according to custom rules written inside those controllers. 68 -
That is the real power of Operators, they allow you
to write an application to fully manage another 69 -
SIMILARITIES BETWEEN ELIXIR/ERLANG AND K8S ๏ best practices for reliable
/ highly available apps (OTP) ๏ used as control plane ๏ control loops == Erlang processes / GenServers ? 70 -
coryodaniel/k8s ๏ A Kubernetes library and HTTP client in Elixir
(since Jan 11, 2019) ๏ sympathetic newcomer ๏ quite capable and written with the specific goal for writing K8S operators 72 -
obmarg/kazan ๏ Kubernetes API client for Elixir (since 26 Dec
2016) ๏ generates Modules via macros, hard (impossible?) to use with CRD 73 -
NOT IMPLEMENTED ๏ Custom resources ๏ Other forms of authentication
๏ Patching with application/json-patch+json or application/ strategic-merge-patch+jsoncontent types ๏ (... and more) 74 -
coryodaniel/bonny ๏ Kubernetes Operator SDK written in Elixir. Extend the
Kubernetes API and implement CustomResourceDefinitions in Elixir ๏ very interesting, but quite bleeding-edge 75 -
https://metacontroller.app/ ๏ 100% language agnostic ๏ real framework ๏ HTTP
+ JSON reading / generating to start shipping operators ๏ would be my first recommendation to get your feet wet and quickly ship POC 76