Deploying Kubernetes on Bare Metal Using the Cluster API

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Who Are We? Jason DeTiberus Senior Member of Technical Staff @VMware @detiber David Watson Senior Software Engineer @Samsung @davidewatson

Slide 3

Slide 3 text

- Background - Motivation - Admission Webhooks - Demo - Custom Webhooks - Future of Cluster API Overview

Slide 4

Slide 4 text

Background

Slide 5

Slide 5 text

What is Cluster API? - Subproject of Kubernetes SIG Cluster Lifecycle - The Cluster API is a Kubernetes project to bring declarative, Kubernetes-style APIs to cluster creation, conﬁguration, and management. It provides optional, additive functionality on top of core Kubernetes. - https://github.com/kubernetes-sigs/cluster-api

Slide 6

Slide 6 text

What is Cluster API?

Slide 7

Slide 7 text

Pod ReplicaSet Deployment Machine MachineSet MachineDeployment StorageClass MachineClass What is Cluster API?

Slide 8

Slide 8 text

What is Cluster API? Management Cluster Provider Controller Manager Cluster Controller Machine Controller Cluster Actuator Kubebuilder Cluster API Provider Machine Actuator Cluster API CM MachineSets MachineDeployments Reconcile Delete Create Delete Update Exists

Slide 9

Slide 9 text

Motivation

Slide 10

Slide 10 text

Motivation - Bare metal infrastructure varies widely - No standard APIs or tooling. - Current extension model is too coarse (for developers and users) - Every Provider has to re-implement Kubernetes software provisioning - New Providers end up copy/pasting existing implementations - Cluster Actuator is responsible for Load Balancing, Firewall, Networking - Controllers are not universally understood (by developers) - Synchronous model may be more familiar - Current extension model requires the use of Go - Non-Go implementations have to implement full replacement controllers

Slide 11

Slide 11 text

Motivation Management Cluster Provider Controller Manager Cluster Controller Machine Controller Cluster Actuator Kubebuilder Cluster API Provider Machine Actuator Cluster API CM MachineSets MachineDeployments Reconcile Delete Create Delete Update Exists

Slide 12

Slide 12 text

Management Cluster Provider Webhooks Clusters Machines Kubebuilder Cluster API Provider Cluster API CM MachineSets MachineDeployments Create Delete Update Exists Motivation

Slide 13

Slide 13 text

Admission Webhooks

Slide 14

Slide 14 text

Admission Webhooks - Webhooks are an existing k8s extension mechanism - Allow requests to be validated and/or mutated before persisting in etcd - Allows controllers to be developed out-of-tree - Conﬁgured at runtime

Slide 15

Slide 15 text

Admission Webhooks func (h *MachineCreateDeleteHandler) Handle(ctx context.Context, req types.Request) types.Response { obj := &clusterv1.Machine{} copy := obj.DeepCopy() allowed, reason, err := h.mutatngMachineFn(ctx, copy) if !allowed { return admission.ValidationResponse(allowed, reason) } return admission.PatchResponse(obj, copy) }

Slide 16

Slide 16 text

Admission Webhooks func (h *MachineCreateDeleteHandler) mutatngMachineFn(ctx context.Context, obj *clusterv1.Machine) (bool, string, error) { response, err := h.MAASClient.Create(ctx, &v1alpha1.MachineCreateRequest{MachineID: obj.Name}) if err != nil { return false, "webhook error prevents admission", err } obj.Spec.ProviderID = response.ProviderID obj.Spec.IPAddresses = response.IPAddresses return true, "allowed to be admitted", nil }

Slide 17

Slide 17 text

Demo

Slide 18

Slide 18 text

Demo - Metal as a Service (MAAS) - PXE - cloud-init/ssh - Packer/MAAS images - Admission webhooks apiVersion: cluster.k8s.io/v1alpha1 kind: Machine metadata: labels: controller-tools.k8s.io: "1.0" cluster.k8s.io/cluster-name: cluster01 name: machine01 spec: providerSpec: kubelet: v1.14.0

Slide 19

Slide 19 text

Video

Slide 20

Slide 20 text

- Controller can not inject data before the webhook sees the request - Idempotency or garbage collection required - Status is not available (by default) - Another entity which complicates mental model Admission Webhooks - Limitations

Slide 21

Slide 21 text

Custom Webhooks

Slide 22

Slide 22 text

Custom Webhooks - Not limited by admission webhook request/response types - Can be initiated from within a common controller - Ensure better consistency of CAPI controller interfaces for the end user

Slide 23

Slide 23 text

Future of Cluster API

Slide 24

Slide 24 text

Future of Cluster API - Separation of bootstrap from infrastructure provider implementations - Control Plane management - Uniﬁed image building - Data Model - Embedded raw blobs are going away - Extension Mechanism - No consensus yet - Independent Controllers - Webhooks - Both

Slide 25

Slide 25 text

Resources - https://github.com/cattlek8s/cluster-api-provider-generic - https://github.com/davidewatson/cluster-api-webhooks-maas - Cluster API Oﬃce Hours - Weekly - Wednesdays at 10:00 am US Paciﬁc Time - Meeting Agenda/Notes