Machine Learning for Large Scale Code Analysis Extending Kubernetes With Go Alternative title: Kubernetes Operators with cute Gophers 

“Kubernetes is just a framework for containers” 2 

3 What is an Operator? As seen by CoreOS An Operator is a method of packaging, deploying and managing a Kubernetes application. A Kubernetes application is an application that is both deployed on Kubernetes and managed using the Kubernetes APIs and kubectl tooling. You can think of Operators as the runtime that manages this type of application on Kubernetes. 

And we all know one… Even though it isn’t officially an operator 4 

5 Deployments Seen as an example of operators* “A Deployment controller provides declarative updates for Pods and ReplicaSets.” Deployment ReplicaSet Pod Pod Pod 

6 A Kubernetes Controller An overview ● Lives in a Pod ● Uses in cluster API access ● Watched resources, takes action ● Stores all data inside Kubernetes ● Only 1 instance! (redundancy? Use locks) 

7 Operators vs Controllers A hot discussion topic “An operator is a specific type of controller” ● Implements CRD ● App focused (prometheus, etcd, …) ● Term coined by CoreOS More context: https://github.com/kubeflow/tf-operator/issues/300 

Building a simple controller 8 

The Toolchain kubernetes/client-go 9 

The Toolchain ● 2nd class citizens ● Does not include utils In other languages 10 

Getting started with client-go 11 

Connecting package main import ( "k8s.io/client-go/kubernetes" "k8s.io/client-go/rest" ) func main() { config, err := rest.InClusterConfig() clientset, err := kubernetes.NewForConfig(config) } In cluster 12 

13 k8s.io/client-go/rest ● Underlying REST library ● Most important: contains a Config object ● Responsible for talking to the APIs ● Can be used directly, but you should not Fun fact: still contains very Java like patterns like Factory from the old days. See https://archive.fosdem.org/2019/schedule/event/kubernetesclusterfuck/ 

14 k8s.io/client-go/kubernetes ● Does 1 thing: ClientSets! ● Representing objects in code apiVersion: apps/v1 kind: Deployment metadata: name: [...] clientset.AppsV1().Deployments("namespace").Get([...]) clientset.AppsV1().Deployments("namespace").List([...]) clientset.AppsV1().Deployments("namespace").Create([...]) clientset.AppsV1().Deployments("namespace").Delete([...]) 

15 rest.InClusterConfig() How to see you’re in a cluster ● KUBERNETES_SERVICE_HOST && KUBERNETES_SERVICE_PORT ● Mounted service tokens 

Connecting package main import ( "k8s.io/client-go/kubernetes" "k8s.io/client-go/rest" ) func main() { config, err := rest.InClusterConfig() clientset, err := kubernetes.NewForConfig(config) } In cluster 16 And how to test locally??? 

Connecting import ( "k8s.io/client-go/kubernetes" "k8s.io/client-go/tools/clientcmd" ) func main() { config, err := clientcmd.NewNonInteractiveDeferredLoadingClientConfig( clientcmd.NewDefaultClientConfigLoadingRules(), &clientcmd.ConfigOverrides{ CurrentContext: "minikube", }, ).ClientConfig() clientset, err := kubernetes.NewForConfig(config) } Using a config file 17 

18 k8s.io/client-go/tools/clientcmd The tool for kubeconfig “Package clientcmd provides one stop shopping for building a working client from a fixed config, from a .kubeconfig file, from command line flags, or from any merged combination.” 

Watching objects func main() { config, _ := rest.InClusterConfig() // Where is err? I never said this was production code clientset, _ := kubernetes.NewForConfig(config) watcher, err := clientset.CoreV1().Pods("").Watch(metav1.ListOptions{}) for { event := <-watcher.ResultChan() if event.Type == watch.Added { fmt.Printf("Found a new pod: %s\n", (event.Object.(*corev1.Pod).GetName())) } } } Let’s do something useful? 19 

Watching objects watcher, err := clientset. CoreV1().Pods("").Watch(metav1.ListOptions{}) for { event := <-watcher. ResultChan () if event.Type == watch.Added { fmt. Printf("Found a new pod: %s \n", (event.Object.(*corev1.Pod). GetName())) } } Can you guess the output? 20 All namespaces A B C 

Watching objects watcher, err := clientset. CoreV1().Pods("").Watch(metav1.ListOptions{}) for { event := <-watcher. ResultChan () if event.Type == watch.Added { fmt. Printf("Found a new pod: %s \n", (event.Object.(*corev1.Pod). GetName())) } } Can you guess the output? 21 All namespaces A B C 

22 .Watch() Simple resource watcher ● Gives all existing on first call ● “Low” level ● Error prone (eg. errors on proxy timeouts) ● Will repeat all data every x hours ○ Inefficient with complex code 

Cache Magic! 23 

24 Cache In Kubernetes client-go ● Provided in k8s.io/client-go/tools/cache ● 2 “Stores”: cache and FIFO queue ● Very complex and hard to implement 

25 Informers Cache enhanced watchers ● Implements a cache ● Deals with watcher errors ● Does re-sync to not make mistakes ● Uses functions for event handling 

Informers func main() { [...] podInformer := coreinformers.NewPodInformer(clientSet,”namespace”, time.Minute, cache.Indexers{}) podInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{ AddFunc: func(obj interface{}) {}, // ability to handle events here UpdateFunc: func(oldObj interface{}, newObj interface{}) {}, DeleteFunc: func(obj interface{}) {}, }) stop := make(chan struct{}) defer close(stop) go podInformer.Run(stop) // runs informer in thread with ability to stop it } Cache enhanced watchers 26 

27 Writing Objects All the options! ● Create() ● Patch() ● Update() ● UpdateStatus() 

28 Writing Objects All the options! ● Create() ● Patch() ● Update() ● UpdateStatus() 

Writing Objects import ( core_v1 "k8s.io/api/core/v1" meta_v1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) func () setUpService(){ service, err = client.CoreV1().Services(pod.GetNamespace()).Create(&core_v1.Service{ ObjectMeta: meta_v1.ObjectMeta{ Name: "Test", Annotations: map[string]string{"heritage": "srcd-operator"}, }, Spec: core_v1.ServiceSpec{ Type: core_v1.ServiceTypeClusterIP, Ports: []core_v1.ServicePort{core_v1.ServicePort{Port: 80}}, }, }) } Simple core v1 service example 29 

30 k8s.io/api Has Go structs for all specs 

31 client-go takeaways ● Offers simple APIs to do CRUD ● Offers feature complete tools ● One stop for all Kubernetes interactions 

Operator Framework 32 

Operator Framework CoreOS’ answer for Kubernetes deployments (now community maintained) 33 

34 Operator SDK Go edition ● Specifically for “operators” ● Does the heavy lifting for you ● Manages CRDs ● Creates Deployment ● Framework handles watchers and actions 

35 Operator SDK Sample AppService // newPodForCR returns a busybox pod with the same name/namespace as the cr func newPodForCR (cr *testv1alpha1.AppService) *corev1.Pod { labels := map[string]string{ "app": cr.Name, } return &corev1.Pod{ ObjectMeta: metav1.ObjectMeta{ Name: cr.Name + "-pod", Namespace: cr.Namespace, Labels: labels, }, Spec: corev1.PodSpec{ Containers: []corev1.Container{ { Name: "busybox", Image: "busybox", Command: [] string{"sleep", "3600"}, }, }, }, } 

36 So much more… That’s for a later talk 

Coding Challenge 37 

38 Boring Wozniak Steve Wozniak is not boring // GetRandomName generates a random name from the list of adjectives and surnames in this package func GetRandomName(retry int) string { begin: name := fmt.Sprintf("%s_%s", left[rand.Intn(len(left))], right[rand.Intn(len(right))]) if name == "boring_wozniak" /* Steve Wozniak is not boring */ { goto begin } if retry > 0 { name = fmt.Sprintf("%s%d", name, rand.Intn(10)) } return name } 

39 Boring Wozniak Steve Wozniak is not boring Write a controller that makes sure all boring wozniak deployments get replaced. Instructions & Manifests at: https://github.com/meyskens/boring-wozniak-controller 

Machine Learning for Large Scale Code Analysis sourced.tech blog.sourced.tech github.com/src-d Thank you! Gopher by Renee French Gopher artwork by Ashley McNamara, Egon Elbre and Miguel Molina