Custom Resources for Cloud Native DevOps

In this talk Jelmer will share with you what Kubernetes' Custom Resources exactly are and how they can help reduce YAML boilerplate. You'll learn how to create an Operator from scratch which you can use to automate DevOps tasks in a CloudNative manner.

Transcript

1. Custom Resources for
   Cloud Native DevOps
   

   View Slide

2. FIND ME
   github.com/jelmersnoeck
   twitter.com/jelmersnoeck
   Jelmer Snoeck
   ABOUT ME
   - OX Tech Lead at manifold.co
   - Kubernetes
   - Terraform
   - Kubelist Editor
   

   View Slide

3. $ tree custom-resources-cloud-native-devops/
   - What are Custom Resources
   - How to build Custom Resources
   - Publishing Custom Resources
   

   View Slide

4. Custom Resource
   (Definitions)
   Photo by The Roaming Platypus on Unsplash
   

   View Slide

5. A custom resource is an extension of the Kubernetes API that is not necessarily available on
   every Kubernetes cluster. In other words, it represents a customization of a particular Kubernetes
   installation.
   

   View Slide

6. Custom Resource
   Definition (CRD)
   

   View Slide

7. The CustomResourceDefinition API resource allows you to define custom resources. Defining a
   CRD object creates a new custom resource with a name and schema that you specify. The
   Kubernetes API serves and handles the storage of your custom resource.
   

   View Slide

8. View Slide

9. View Slide

10. View Slide

11. Validation
    

    View Slide

12. Open API Schema
    

    View Slide

13. View Slide

14. View Slide

15. View Slide

16. Admission Webhooks
    

    View Slide

17. Printer Columns
    

    View Slide

18. View Slide

19. View Slide

20. Finalizers
    

    View Slide

21. Controller vs Operator
    

    View Slide

22. Controller <> Operator
    - Controller purely interacts with k8s
    objects (even custom ones)
    - Operator knows about a different domain
    outside of k8s (etcd, prometheus, …)
    

    View Slide

23. Some CRDs in the wild
    

    View Slide

24. - cert-manager
    - prometheus-operator
    - ingress-nginx-controller
    - contour
    - ingress-monitor
    - istio
    Examples
    

    View Slide

25. What’s under the hood
    Photo by Wade Lambert on Unsplash
    

    View Slide

26. Defining APIs
    

    View Slide

27. API Definitions
    

    View Slide

28. View Slide

29. Generated runtime.Object code
    

    View Slide

30. type Object interface {
    GetObjectKind() schema.ObjectKind
    DeepCopyObject() Object
    }
    

    View Slide

31. View Slide

32. Scheme Definitions
    

    View Slide

33. View Slide

34. View Slide

35. Generating Clients
    

    View Slide

36. k8s.io/code-generator/generate-groups.sh
    

    View Slide

37. View Slide

38. ./vendor/k8s.io/code-generator/generate-groups.sh \
    all \
    github.com/jelmersnoeck/ingress-monitor/pkg/client/generated \
    github.com/jelmersnoeck/ingress-monitor/apis \
    ingressmonitor:v1alpha1 \
    --go-header-file boilerplate.go.txt
    

    View Slide

39. View Slide

40. View Slide

41. View Slide

42. Controller
    

    View Slide

43. Cache Informers
    

    View Slide

44. View Slide

45. View Slide

46. Event Handlers
    

    View Slide

47. type ResourceEventHandler interface {
    OnAdd(obj interface{})
    OnUpdate(oldObj, newObj interface{})
    OnDelete(obj interface{})
    }
    

    View Slide

48. View Slide

49. View Slide

50. Workqueue
    

    View Slide

51. Built in queue features
    ● Fair: items processed in the order in which they are added.
    ● Stingy: a single item will not be processed multiple times concurrently, and
    if an item is added multiple times before it can be processed, it will only
    be processed once.
    ● Multiple consumers and producers. In particular, it is allowed for an item to
    be reenqueued while it is being processed.
    ● Shutdown notifications.
    

    View Slide

52. View Slide

53. View Slide

54. View Slide

55. Label Selectors
    

    View Slide

56. Labels
    

    View Slide

57. View Slide

58. View Slide

59. View Slide

60. View Slide

61. View Slide

62. View Slide

63. Survive Rescheduling
    

    View Slide

64. View Slide

65. Testing
    

    View Slide

66. Testing
    - Fake Clients
    - Add wrappers to test informers
    - Use of testing.Fake Actions
    

    View Slide

67. Publishing
    Photo by chuttersnap on Unsplash
    

    View Slide

68. RBAC
    

    View Slide

69. Survive Rescheduling
    

    View Slide

70. Validation
    

    View Slide

71. Resource Utilization
    

    View Slide

72. Application Structuring
    

    View Slide

73. Observability
    

    View Slide

74. Thanks!
    Questions?
    @jelmersnoeck
    manifold.co
    

    View Slide