Introduction to Custom Resources Pauline Lallinec Workday Dublin Docker Meetup, March 2018 

Software Engineer II - DevOps N - @plallin Workday Public Cloud Engineering We’re hiring! Best logo ever (or so I tell my director) -> 

● Core concepts: Kubernetes, Helm, CRDs ● Overview of the unicorn app ● What are CRDs? ● What are custom controllers? ● Demo of CRDs + Custom controller using unicorns ● Implementation of the Helm Release controller ● More demo using unicorns No Agenda No Attenda 

Before we start: Kubernetes “Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications.” Long story short: it deploys and manages your (Docker) containers for you. 

Before we start: Helm ● Package manager for Kubernetes ● Helm chart: a set of Kubernetes resources ● Helm release: a version of a Helm Chart ● 2 parts: The Helm client (Helm), and the Helm server (Tiller) 

Custom Resources A way to create custom objects that live within your cluster, and are handled by a custom controller running a logic of your own. (Ideally) CRDs responds to CRUD events (Create, Read, Update, Delete) and allow you to implement your own declarative API. 

Custom Resources Example CRDs in Workday S3Bucket a CRD to create, delete, update and retrieve data from S3 Buckets HelmRelease a CRD to install, upgrade, delete, and store information on Helm releases 

I A very simple app! The app One single HTML page showing a unicorn, serviced by Python’s SimpleHTTPServer Kubernetes resources One deployment, with only 1 container containing the Unicorn app 3 Helm charts - Pink unicorn - Blue unicorn - Green unicorn 

I : D 

CRDs: why the interest? “In the future there should be nothing we [Kubernetes Developers] can do that you [Kubernetes users] can’t” Tim Hockin (@thockin) Kubernetes co-founder / pioneer & Co-chair of Kubernetes Network SIG “CRDs aren’t just for add-ons anymore - painting a picture for the future”, Lightning Talk, Kubecon North America 2018 

Standalone CRDs ● Custom object with their own API endpoint ● Store / retrieve structured data CRDs + Custom controllers ● Declarative API What are CRDs? 

What are CRDs? apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: helmreleases.samplecontroller.k8s.io spec: group: samplecontroller.k8s.io version: v1alpha1 names: kind: HelmRelease plural: helmreleases scope: Namespaced Custom object with their own API endpoint 

$ kubectl create -f helm_release_crd.yaml customresourcedefinition.apiextensions.k8s.io/helmreleases.sample controller.k8s.io created $ kubectl get crd NAME CREATED AT helmreleases.samplecontroller.k8s.io 2019-03-23T05:21:43Z Custom object with their own API endpoint What are CRDs? 

apiVersion: samplecontroller.k8s.io/v1alpha1 kind: HelmRelease metadata: name: unicorn-release spec: releaseVersion: pink releaseName: unicorn Store / retrieve structured data What are CRDs? 

$ kubectl create -f unicorn-release-pink.yaml helmrelease.samplecontroller.k8s.io/unicorn-release created $ kubectl get helmreleases NAME AGE unicorn-release 36s Store / retrieve structured data What are CRDs? 

$ kubectl describe helmrelease unicorn-release Name: unicorn-release Namespace: default API Version: samplecontroller.k8s.io/v1alpha1 Kind: HelmRelease Metadata: ... Spec: Release Name: unicorn Release Version: pink Events: Store / retrieve structured data What are CRDs? 

● Listen to any resource type ● Ensure desired state = existing state ● If desired state ≠ existing state, will take action to make existing state = desired state ● This is implemented using your own logic! Clone kubernetes/sample-controller from GitHub for an example of a sample controller Custom controllers 

Helm Release Controller ● Listen to CRDs of type HelmRelease ● Ensures all desired Helm releases are installed / upgraded ● Will install / upgrade the Helm release if not already installed / not upgraded to desired version Custom controllers 

(No) Helm Release Controller ● Current situation: run a script to handle installation / upgrade / rollback of Helm resources ● Need a way (i.e. Jenkins job) to run that script to install / upgrade / rollback Helm charts ● Imperative rather than declarative Custom controllers 

Helm Release Controller ● Cluster logic remains within the cluster ● Declarative API: let the cluster manage itself ● No need for additional script / Jenkins job Custom controllers 

Helm Release Controller ● Automated rollback according to a logic of our own ● Allow for custom business logic ● No need to install / maintain the Helm Client on different servers Custom controllers 

U CRD’ 

(Unicorn) HelmRelease New Helm Release? Y: Install Helm Release N: Upgrade Helm Release U CRD’ 

U CRD’ : 

Clone of the existing Sample Controller from Kubernetes No update done to listeners, informers, event handlers, etc. Focus on SyncHandlers() which is responsible for ensuring that desired state = existing state kubernetes/sample-controller: https://github.com/kubernetes/sample-controller Helm Release Controller: the implementation 

Receive a CRD of type Helm Release Helm Release Controller: the implementation 

Get information on CRD of type HelmRelease Helm Release Controller: the implementation 

Get information about the current CRD (= desired state) Helm Release Controller: the implementation 

Install Helm release if it doesn’t already exist (= match desired state) Helm Release Controller: the implementation 

Check existing deployment (= existing state) Helm Release Controller: the implementation 

Upgrade existing release if necessary (= match desired state) Helm Release Controller: the implementation 

Update Helm Release status Helm Release Controller: the implementation 

Update Helm Release status Helm Release Controller: the implementation 

If an error happens, re-enqueue the event and retry later Helm Release Controller: the implementation 

Finally, return successful sync event Helm Release Controller: the implementation 

● Choice of programming language ● Can enforce validation (example to follow) ● Can support /status and /scale subresources (and maybe /exec and /log in the future) CRDs + Custom controllers: Other benefits 

apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: ... spec: ... validation: openAPIV3Schema: properties: spec: properties: releaseVersion: type: string pattern: "(pink|blue)" E 

HelmRelease Y: New Helm Release? Y: Install Helm Release N: Upgrade Helm Release Is this unicorn release pink or blue? N: reject create / update event E 

E : 

Thank you! Join my team! Apply to “[Senior] Devops Engineer - DevOps in Cloud Orchestration” on Workday (Link: https://lnkd.in/gkMHwMC), or directly introduce yourself to Dave Doran (he’s here tonight!). You will be able to work with trendy technologies AND join the karaoke club. How cool is that? Follow me on Twitter at @plallin to hear me brag about how much I walk (a lot!), how much I lift (really not a lot), and laugh about my struggles with Arch Linux. 

Pictures! All pictures downloaded from Pixabay https://pixabay.com/ Ship of containers: https://pixabay.com/photos/hamburg-port-of-hamburg-3021820/ Ending ship of containers: https://pixabay.com/photos/container-container-ship-port-1611490/ Unicorns: https://pixabay.com/vectors/unicorn-unicorn-crown-flower-crown-3392560/ Picture of a Helm on a blue background: https://pixabay.com/photos/steering-wheel-blue-twist-sea-981439/ Agenda ship sailing in the sun: https://pixabay.com/photos/sea-sailing-vessel-boat-ship-701079/