$30 off During Our Annual Pro Sale. View Details »

Custom Resources for Cloud Native DevOps

Jelmer Snoeck
September 26, 2018

Custom Resources for Cloud Native DevOps

In this talk Jelmer will share with you what Kubernetes' Custom Resources exactly are and how they can help reduce YAML boilerplate. You'll learn how to create an Operator from scratch which you can use to automate DevOps tasks in a CloudNative manner.

Jelmer Snoeck

September 26, 2018
Tweet

More Decks by Jelmer Snoeck

Other Decks in Technology

Transcript

  1. Custom Resources for
    Cloud Native DevOps

    View Slide

  2. FIND ME
    github.com/jelmersnoeck
    twitter.com/jelmersnoeck
    Jelmer Snoeck
    ABOUT ME
    - OX Tech Lead at manifold.co
    - Kubernetes
    - Terraform
    - Kubelist Editor

    View Slide

  3. $ tree custom-resources-cloud-native-devops/
    - What are Custom Resources
    - How to build Custom Resources
    - Publishing Custom Resources

    View Slide

  4. Custom Resource
    (Definitions)
    Photo by The Roaming Platypus on Unsplash

    View Slide

  5. A custom resource is an extension of the Kubernetes API that is not necessarily available on
    every Kubernetes cluster. In other words, it represents a customization of a particular Kubernetes
    installation.

    View Slide

  6. Custom Resource
    Definition (CRD)

    View Slide

  7. The CustomResourceDefinition API resource allows you to define custom resources. Defining a
    CRD object creates a new custom resource with a name and schema that you specify. The
    Kubernetes API serves and handles the storage of your custom resource.

    View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. Validation

    View Slide

  12. Open API Schema

    View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. Admission Webhooks

    View Slide

  17. Printer Columns

    View Slide

  18. View Slide

  19. View Slide

  20. Finalizers

    View Slide

  21. Controller vs Operator

    View Slide

  22. Controller <> Operator
    - Controller purely interacts with k8s
    objects (even custom ones)
    - Operator knows about a different domain
    outside of k8s (etcd, prometheus, …)

    View Slide

  23. Some CRDs in the wild

    View Slide

  24. - cert-manager
    - prometheus-operator
    - ingress-nginx-controller
    - contour
    - ingress-monitor
    - istio
    Examples

    View Slide

  25. What’s under the hood
    Photo by Wade Lambert on Unsplash

    View Slide

  26. Defining APIs

    View Slide

  27. API Definitions

    View Slide

  28. View Slide

  29. Generated runtime.Object code

    View Slide

  30. type Object interface {
    GetObjectKind() schema.ObjectKind
    DeepCopyObject() Object
    }

    View Slide

  31. View Slide

  32. Scheme Definitions

    View Slide

  33. View Slide

  34. View Slide

  35. Generating Clients

    View Slide

  36. k8s.io/code-generator/generate-groups.sh

    View Slide

  37. View Slide

  38. ./vendor/k8s.io/code-generator/generate-groups.sh \
    all \
    github.com/jelmersnoeck/ingress-monitor/pkg/client/generated \
    github.com/jelmersnoeck/ingress-monitor/apis \
    ingressmonitor:v1alpha1 \
    --go-header-file boilerplate.go.txt

    View Slide

  39. View Slide

  40. View Slide

  41. View Slide

  42. Controller

    View Slide

  43. Cache Informers

    View Slide

  44. View Slide

  45. View Slide

  46. Event Handlers

    View Slide

  47. type ResourceEventHandler interface {
    OnAdd(obj interface{})
    OnUpdate(oldObj, newObj interface{})
    OnDelete(obj interface{})
    }

    View Slide

  48. View Slide

  49. View Slide

  50. Workqueue

    View Slide

  51. Built in queue features
    ● Fair: items processed in the order in which they are added.
    ● Stingy: a single item will not be processed multiple times concurrently, and
    if an item is added multiple times before it can be processed, it will only
    be processed once.
    ● Multiple consumers and producers. In particular, it is allowed for an item to
    be reenqueued while it is being processed.
    ● Shutdown notifications.

    View Slide

  52. View Slide

  53. View Slide

  54. View Slide

  55. Label Selectors

    View Slide

  56. Labels

    View Slide

  57. View Slide

  58. View Slide

  59. View Slide

  60. View Slide

  61. View Slide

  62. View Slide

  63. Survive Rescheduling

    View Slide

  64. View Slide

  65. Testing

    View Slide

  66. Testing
    - Fake Clients
    - Add wrappers to test informers
    - Use of testing.Fake Actions

    View Slide

  67. Publishing
    Photo by chuttersnap on Unsplash

    View Slide

  68. RBAC

    View Slide

  69. Survive Rescheduling

    View Slide

  70. Validation

    View Slide

  71. Resource Utilization

    View Slide

  72. Application Structuring

    View Slide

  73. Observability

    View Slide

  74. Thanks!
    Questions?
    @jelmersnoeck
    manifold.co

    View Slide