Deploying Kubernetes on Bare Metal Using the Cluster API

1. Deploying Kubernetes on Bare Metal Using the Cluster API
2. Who Are We? Jason DeTiberus Senior Member of Technical Staff @VMware @detiber David Watson Senior Software Engineer @Samsung @davidewatson
3. - Background - Motivation - Admission Webhooks - Demo - Custom Webhooks - Future of Cluster API Overview
4. Background
5. What is Cluster API? - Subproject of Kubernetes SIG Cluster Lifecycle - The Cluster API is a Kubernetes project to bring declarative, Kubernetes-style APIs to cluster creation, configuration, and management. It provides optional, additive functionality on top of core Kubernetes. - https://github.com/kubernetes-sigs/cluster-api
6. What is Cluster API?
7. Pod ReplicaSet Deployment Machine MachineSet MachineDeployment StorageClass MachineClass What is Cluster API?
8. What is Cluster API? Management Cluster Provider Controller Manager Cluster Controller Machine Controller Cluster Actuator Kubebuilder Cluster API Provider Machine Actuator Cluster API CM MachineSets MachineDeployments Reconcile Delete Create Delete Update Exists
9. Motivation
10. Motivation - Bare metal infrastructure varies widely - No standard APIs or tooling. - Current extension model is too coarse (for developers and users) - Every Provider has to re-implement Kubernetes software provisioning - New Providers end up copy/pasting existing implementations - Cluster Actuator is responsible for Load Balancing, Firewall, Networking - Controllers are not universally understood (by developers) - Synchronous model may be more familiar - Current extension model requires the use of Go - Non-Go implementations have to implement full replacement controllers
11. Motivation Management Cluster Provider Controller Manager Cluster Controller Machine Controller Cluster Actuator Kubebuilder Cluster API Provider Machine Actuator Cluster API CM MachineSets MachineDeployments Reconcile Delete Create Delete Update Exists
12. Management Cluster Provider Webhooks Clusters Machines Kubebuilder Cluster API Provider Cluster API CM MachineSets MachineDeployments Create Delete Update Exists Motivation
13. Admission Webhooks
14. Admission Webhooks - Webhooks are an existing k8s extension mechanism - Allow requests to be validated and/or mutated before persisting in etcd - Allows controllers to be developed out-of-tree - Configured at runtime
15. Admission Webhooks func (h *MachineCreateDeleteHandler) Handle(ctx context.Context, req types.Request) types.Response { obj := &clusterv1.Machine{} copy := obj.DeepCopy() allowed, reason, err := h.mutatngMachineFn(ctx, copy) if !allowed { return admission.ValidationResponse(allowed, reason) } return admission.PatchResponse(obj, copy) }
16. Admission Webhooks func (h *MachineCreateDeleteHandler) mutatngMachineFn(ctx context.Context, obj *clusterv1.Machine) (bool, string, error) { response, err := h.MAASClient.Create(ctx, &v1alpha1.MachineCreateRequest{MachineID: obj.Name}) if err != nil { return false, "webhook error prevents admission", err } obj.Spec.ProviderID = response.ProviderID obj.Spec.IPAddresses = response.IPAddresses return true, "allowed to be admitted", nil }
17. Demo
18. Demo - Metal as a Service (MAAS) - PXE - cloud-init/ssh - Packer/MAAS images - Admission webhooks apiVersion: cluster.k8s.io/v1alpha1 kind: Machine metadata: labels: controller-tools.k8s.io: "1.0" cluster.k8s.io/cluster-name: cluster01 name: machine01 spec: providerSpec: kubelet: v1.14.0
19. Video
20. - Controller can not inject data before the webhook sees the request - Idempotency or garbage collection required - Status is not available (by default) - Another entity which complicates mental model Admission Webhooks - Limitations
21. Custom Webhooks
22. Custom Webhooks - Not limited by admission webhook request/response types - Can be initiated from within a common controller - Ensure better consistency of CAPI controller interfaces for the end user
23. Future of Cluster API
24. Future of Cluster API - Separation of bootstrap from infrastructure provider implementations - Control Plane management - Unified image building - Data Model - Embedded raw blobs are going away - Extension Mechanism - No consensus yet - Independent Controllers - Webhooks - Both
25. Resources - https://github.com/cattlek8s/cluster-api-provider-generic - https://github.com/davidewatson/cluster-api-webhooks-maas - Cluster API Office Hours - Weekly - Wednesdays at 10:00 am US Pacific Time - Meeting Agenda/Notes