Orchestructure: Building Kubernetes Operators with Ansible Talk

An Operator is a method of packaging, deploying and automating the complete lifecycle management activities of a Kubernetes application. Put more simply, an Operator encodes human operational knowledge. It is designed to watch and respond to the resources in your cluster and enable your application to run as desired.

While powerful, Operators are usually written in Go and requires expertise in the advanced libraries and patterns used to write Kubernetes controllers. Ansible can help. It is a first class citizen in the Operator SDK providing a means of automating the deployment and management of Kubernetes applications on a cluster in an Ansible-native way. Ansible-based Operators provide a lower barrier to entry, faster iterations, and the power of Ansible and its ecosystem.

Combining Ansible and Kubernetes frees up application engineers to minimize the new skill sets required and maximize time to delivery. Furthermore, using the same tried and trusted Ansible tooling lets you automate and orchestrate your applications across both new and existing platforms with one simple language.

This talk introduces Operators with Ansible and demonstrates how you develop and deploy them to automate the management of complex Kubernetes application on a cluster.

Transcript

1. Building Kubernetes
   Operators with
   Ansible
   CHRIS SHORT
   PRINCIPAL TECHNICAL MARKETING
   MANAGER, CLOUD PLATFORMS
   @ChrisShort
   

   View Slide

2. The Ansible Operator SDK makes it
   easier to deploy and manage Kubernetes
   applications with native Ansible support
   

   View Slide

3. Manage how containerized apps interact
   with other apps or the outside world
   Run distributed systems resiliently across
   a cluster of nodes
   Perform health checks
   Kubernetes Operators with Ansible
   What does Kubernetes help do?
   Scale your services up or down
   Perform graceful rolling updates
   Networking and routing of traffic
   Manages ephemeral and persistent
   storage volumes
   

   View Slide

4. Pod
   Deployment
   DaemonSet
   ReplicaSet
   StatefulSet
   Job
   CronJob
   Kubernetes Operators with Ansible
   Kubernetes objects are persistent entities that represent the state
   of your cluster that you can mange with the K8s API
   EndPoints
   Ingress
   Service
   ConfigMap
   Secret
   StorageClass
   Volume
   Namespace
   NetworkPolicy
   PersistentVolume
   Role
   RoleBinding
   ServiceAccount
   Understanding Kubernetes objects
   

   View Slide

5. Kubernetes Operators with Ansible
   apiVersion: v1
   kind: Pod
   metadata:
   name: example-app
   labels:
   app: example-app
   spec:
   containers:
   - name: example
   image: companyname/example:v1.2.0
   ports:
   - containerPort: 8000
   apiVersion: v1
   kind: Service
   metadata:
   name: example-service
   spec:
   selector:
   app: example-app
   ports:
   - protocol: TCP
   port: 80
   targetPort: 8000
   Kubernetes Object Definitions
   

   View Slide

6. Extending Kubernetes with a
   Custom Resource Definition (CRD)
   apiVersion:
   cache.example.com/v1alpha1
   kind: Memcached
   metadata:
   name: example-memcached
   spec:
   size: 3
   Custom resources definition (CRD) is a
   powerful feature introduced into Kubernetes
   which enables users to add their own/custom
   objects to the Kubernetes cluster and use it
   like any other native Kubernetes objects.
   Kubernetes Operators with Ansible
   

   View Slide

7. Ansible fits naturally into a
   Kubernetes environment
   

   View Slide

8. Kubernetes and Ansible
   Kubernetes Operators with Ansible
   ● Both help make hard things easier through automation and orchestration
   ● Both are very active and widely used open source projects
   ● Both have vibrant communities working to solve common problems
   ● Both use YAML to describe the desired state of the world
   

   View Slide

9. Kubernetes Operators with Ansible
   apiVersion: v1
   kind: ConfigMap
   metadata:
   name: foo
   namespace: default
   data:
   color: red
   - name: create foo configmap
   k8s:
   definition:
   apiVersion: v1
   kind: ConfigMap
   metadata:
   name: foo
   namespace: default
   data:
   color: “{{ color }}”
   YAML to describe the desired state of the world
   KUBERNETES/KUBECTL ANSIBLE
   

   View Slide

10. Templating Kubernetes resource definitions
    with Ansible
    ---
    - name: create foo configmap
    k8s:
    definition: "{{ lookup('template', '/foo.yml') | from_yaml
    }}"
    

    View Slide

11. What are
    Kubernetes Operators?
    

    View Slide

12. Stateless is easy,
    Stateful is hard
    

    View Slide

13. ● Encode human operational knowledge
    ● Automatically patch, upgrade, recover, and tune
    container-based apps and services
    ● Kubernetes-native
    ● Purpose-built for a specific application or service
    ● Enable “day 2” management
    Kubernetes Operators with Ansible
    Kubernetes Operators
    Operators simplify management of complex applications
    on Kubernetes
    

    View Slide

14. Encoding and automating Ops knowledge
    WITH OPERATORS: PROACTIVE
    Continually adjusts to optimal state
    Automatically acts in milliseconds
    WITHOUT OPERATORS: REACTIVE
    Continually checks for anomalies
    Alert humans for response
    Requires manual change to fix
    Kubernetes Operators with Ansible
    

    View Slide

15. K8S API CUSTOM
    RESOURCE & MY
    APP STATE
    WATCH
    EVENTS
    RECONCILE MY K8S
    APPLICATION
    The Operator Pattern
    Controller
    Kubernetes Operators with Ansible
    

    View Slide

16. github.com/operator-framework
    Install, update, and manage
    Operators and their
    dependencies
    An open source toolkit to manage application instances
    on Kubernetes in an automated, scalable way
    Build Operators without
    specialized knowledge of
    the Kubernetes API
    Enable usage reporting
    for Operators
    Operator Framework
    Kubernetes Operators with Ansible
    

    View Slide

17. Why build Operators
    with Ansible?
    

    View Slide

18. Making it easier to deploy and manage Kubernetes apps in an Ansible-native way
    Ansible Operator SDK
    Kubernetes Operators with Ansible
    Use the Operator SDK to
    create a new skeleton
    Operator.
    operator-sdk new Add Ansible Content
    Use Ansible Roles and
    playbooks to manage
    lifecycle events for your
    containerized applications.
    operator-sdk build
    Use the Operator SDK to
    build and deploy your
    Operator to Kubernetes.
    

    View Slide

19. EXISTING SKILLS & ECOSYSTEM
    Same tried & trusted Ansible tooling
    Utilize existing skills
    Supports cloud-native & traditional IT automation
    with one simple language
    Leverages vibrant existing ecosystem
    Why build Operators with Ansible?
    LOWER BARRIER OF ENTRY
    No programming required
    Faster iterations and easier maintenance
    Declarative state definitions like K8s
    Templating of resources
    Abstraction layer & helpers that reduces
    necessary K8s API experience
    Kubernetes Operators with Ansible
    

    View Slide

20. Operator capability level
    Kubernetes Operators with Ansible
    Phase I Phase II Phase III Phase IV Phase V
    Basic Install
    Automated application
    provisioning and
    configuration management
    Seamless Upgrades
    Patch and minor version
    upgrades supported
    Full Lifecycle
    App lifecycle, storage
    lifecycle (backup, failure
    recovery)
    Deep Insights
    Metrics, alerts, log
    processing and workload
    analysis
    Auto Pilot
    Horizontal/vertical scaling,
    auto config tuning, abnormal
    detection, scheduling tuning
    

    View Slide

21. K8S API MY K8S
    APPLICATION
    K8s Operator with Ansible
    WATCH
    FILE
    PLAYBOOK
    OR ROLE
    OPERATOR
    SDK
    BINARY
    Design overview
    Kubernetes Operators with Ansible
    CUSTOM
    RESOURCE & MY
    APP STATE
    

    View Slide

22. K8S API
    CUSTOM
    RESOURCE
    MY K8S
    APPLICATION
    automatically
    updates resource
    status
    OPERATOR SDK BINARY
    K8s Operator with Ansible
    WATCHES.YML
    ANSIBLE
    RUNNER
    executes
    & reports
    Reconciliation with K8s Cluster
    event monitoring
    manages
    Kubernetes Operators with Ansible
    INFORMER
    REVERSE
    PROXY
    reads
    

    View Slide

23. Developing your first
    Operator with Ansible
    

    View Slide

24. Developing your first Operator with Ansible
    ● Initialize Your Operator With Ansible
    ○ $ operator-sdk new foo-operator
    --api-version=cache.example.com/v1alpha1 --kind=Foo --type=ansible
    ● Automate With Ansible
    ○ Create new roles and playbooks or reuse an existing one
    ● Define a watches file
    ○ Map a Kubernetes object to your Ansible content
    ● Build Your Operator
    ○ $ operator-sdk build foo-operator:v0.0.1
    ● Deploy Your Operator to a Kubernetes Cluster
    Kubernetes Operators with Ansible
    

    View Slide

25. Initialize your Operator with Ansible
    $ operator-sdk new memcached-operator
    --api-version=cache.example.com/v1alpha1
    --kind=Memcached --type=ansible
    The new command in the Operator SDK using
    type=ansible will create an Ansible Role
    skelton, watches.yaml mapping file, CRD,
    deploy manifest for the Operator and basic
    tests using Molecule.
    Kubernetes Operators with Ansible
    

    View Slide

26. Automate with Ansible
    ● Develop Ansible playbook or roles to reconcile your Kubernetes application
    and manage its lifecycle
    ● Data from the associated Custom Resource will be passed into the Ansible run
    by extra_vars for use with your tasks, templates, conditionals etc.
    ● Typically your automation will leverage the Ansible k8s module and other
    associated ones to interact with the Kubernetes cluster itself
    Kubernetes Operators with Ansible
    

    View Slide

27. Ansible Roles
    Roles are a package of closely related
    Ansible content that can be shared more
    easily than plays alone:
    Improves readability & maintainability of complex plays
    Eases sharing, reuse and standardization of automation
    processes
    Enables Ansible content to exist independently of
    playbooks, projects -- even organizations
    Provides functional conveniences such as file path
    resolution and default values
    memcached/
    ├── defaults
    │ └── main.yml
    ├── files
    ├── handlers
    │ └── main.yml
    ├── meta
    │ └── main.yml
    ├── tasks
    │ └── main.yml
    ├── templates
    ├── tests
    └── vars
    └── main.yml
    Kubernetes Operators with Ansible
    

    View Slide

28. Mapping Kubernetes events to Ansible automation
    Maps a Kubernetes Group Version Kind (GVK) to a role or playbook
    # watches.yaml
    ---
    version: v1alpha1
    group: cache.example.com
    kind: Memcached
    playbook:
    /path/to/playbook
    Kubernetes Operators with Ansible
    The “watches” file (watches.yaml)
    maps a Kubernetes object to your
    Ansible automation
    Associates the Kubernetes Group, Version, Kind
    (GVK) to an Ansible Role or Playbook
    The Operator SDK binary watches the cluster for
    matching events defined in the watches.yml
    Executes the associated Ansible content when an
    event occurs
    

    View Slide

29. Custom resource spec to Ansible extra variables
    apiVersion:
    kind:
    metadata:
    name:
    spec:
    :
    ….
    status:
    :
    ….
    spec: values will be translated to
    Ansible extra_vars.
    status: will be a generic status
    recorded by the Ansible Operator
    SDK operator. This will use
    ansible-runner output to generate
    meaningful information for the user.
    Kubernetes Operators with Ansible
    

    View Slide

30. Anatomy of Ansible-enabled Operator image
    your Operator image
    Ansible Role
    Ansible Role
    watches.yaml
    base Ansible Operator SDK image
    Operator SDK Binary
    ansible-runner
    Ansible
    Python + Libraries
    Kubernetes Operators with Ansible
    

    View Slide

31. Build your Operator image
    $ operator-sdk build memcached-operator:v0.0.1
    …
    ….
    Digest: sha256:6d3b4e6c9eca300277655f5bd2dcbcc33d12c3ac297a71aff4d5f723e09d606a
    Status: Downloaded newer image for quay.io/water-hole/ansible-operator:latest
    ---> ff7b5533bfd2
    Step 2/3 : COPY roles/ ${HOME}/roles/
    ---> d6ec895c8a6d
    Removing intermediate container eac3560f3dab
    Step 3/3 : COPY watches.yaml ${HOME}/watches.yaml
    ---> 060f91e9a843
    Removing intermediate container f680881ec972
    Successfully built 060f91e9a843
    Kubernetes Operators with Ansible
    

    View Slide

32. Deploy your Operator
    ● Create RBAC and Service Account objects for the Operator
    ● Create the Operator Custom Resource Definition (CRD)
    ● Deploy the Operator to the cluster
    ● Create a Custom Resource (CR) to initialize the Operator
    

    View Slide

33. Explore Operators
    OperatorHub.io is a home for
    the Kubernetes community to
    share Operators.
    Find an existing Operator or
    list your own today.
    Kubernetes Operators with Ansible
    

    View Slide

34. Advanced
    Operator patterns
    

    View Slide

35. Get started with Ansible: Get started with Operators:
    ansible.com/get-started
    ansible.com/community
    github.com/operator-framework/getting-started
    ansible.com/operators
    Next steps
    

    View Slide

36. etcd Operator
    A great example of a sophisticated Kubernetes
    Operator using Ansible:
    github.com/water-hole/etcd-ansible-operator
    Memcached Operator
    Simple walkthrough for building an Operators using the
    Ansible Operator SDK and Kubernetes CRDs
    github.com/operator-framework/operator-sdk-sample
    s/tree/master/memcached-operator
    More resources
    

    View Slide

37. Red Hat Certified Operators
    Red Hat Partners that certify their
    Operators have special status due to
    their backing of the Operator(s) with
    formal support, testing, and joint
    go-to-market activities.
    Kubernetes Operators with Ansible
    ● Register to Red Hat Connect
    ● Join TSANet
    ● For Each Operator Version Release:
    ○ Test
    ○ Submit
    ○ Release
    ○ Publish
    See the Operator Developer Guide for Red Hat
    Partners for more details on the Red Hat
    Certified Operators program.
    

    View Slide

38. linkedin.com/company/red-hat
    youtube.com/user/RedHatVideos
    facebook.com/redhatinc
    twitter.com/RedHat
    Thank you
    

    View Slide

39. A SMARTER KUBERNETES PLATFORM
    Automated installation, patching, and updates from the OS on up*
    APPLICATIONS AND SERVICES
    ISV Operators
    Custom Operators (built w/Operator SDK)
    PLATFORM AND CLUSTER MANAGEMENT
    Automated updates for Kubernetes,
    monitoring, security, registry and more
    LINUX HOST
    Atomic, over-the-air updates for
    Red Hat CoreOS
    ANY INFRASTRUCTURE
    *coming soon
    ACROSS HYBRID / MULTI CLOUD DEPLOYMENTS
    Automated operations
    

    View Slide

40. ● Supported model for running
    Playbooks in an Operator fashion
    ● Great for Ops teams that aren’t
    traditional devs
    ● Takes the human out of the loop
    ● Connects the playbooks to
    Kubernetes events like Node failures Tomcat Object
    apiVersion: apache.org/v1
    kind: Tomcat
    metadata:
    name: example-app
    Namespace: production
    spec:
    replicaCount: 10
    maxActiveSessions: 500
    Container Build
    Ansible
    Playbooks
    Operator Container
    v1.2.7 Tomcat Object
    apiVersion: apache.org/v1
    kind: Tomcat
    metadata:
    name: prod-1.2.7
    Namespace: production
    spec:
    replicaCount: 10
    maxActiveSessions: 1000
    Running Operator
    v1.2.7
    $ oc get Tomcats --all-namespaces
    NAMESPACE NAME READY STATUS RESTARTS AGE
    production prod-1.2.7 1/1 Running 0 4d
    staging staging-v1.2.8 1/1 Running 1 2h
    ANSIBLE OPERATOR SDK Playbook
    Mapping
    

    View Slide