GitOps Guide to the Galaxy: MachineConfigs and GitOps

Join Christian Hernandez, GitOps Extraordinaire, for a journey through how to achieve GitOps in any number of ways. The occasional Red Hatters and special guest will join us too.

Transcript

1. MachineConfigs and GitOps
   Working machineconfig and machineconfigpools in GitOps
   GitOps Happy Hour
   1
   Christian Hernandez
   Cloud Platforms BU
   

   View Slide

2. Future
   OpenShift Cluster Management
   2
   Cloud API
   Machine
   Deployment
   Controller
   MachineDeployment
   Machine Set
   Controller
   MachineSet
   Machine
   Controller
   Machine
   Cloud
   Instance
   NodeLink
   Controller
   Node
   Bootstrap
   

   View Slide

3. OpenShift Cluster Management | Machine Configuration
   3
   OS configuration is stored and applied across the
   cluster via the Machine Config Operator.
   ● Subset of ignition modules applicable post
   provisioning
   ○ SSH keys
   ○ Files
   ○ systemd units
   ○ kernel arguments
   ● Standard k8s YAML/JSON manifests
   ● Desired state of nodes is checked/fixed
   regularly
   ● Can be paused to suspend operations
   Machine Config Operator
   A Kube-native way to configure hosts
   # test.yaml
   apiVersion: machineconfiguration.openshift.io/v1
   kind: MachineConfig
   metadata:
   labels:
   machineconfiguration.openshift.io/role: worker
   name: test-file
   spec:
   config:
   storage:
   files:
   - contents:
   source: data:,hello%20world%0A
   verification: {}
   filesystem: root
   mode: 420
   path: /etc/test
   

   View Slide

4. Node
   Machine Config
   Daemon
   Node
   Machine Config
   Daemon
   Operator/Operand Relationships
   OpenShift Cluster Management | Machine Configuration
   Node
   4
   Machine Config
   Daemon
   Machine
   Config
   Operator
   Machine
   Config
   Controller
   Machine
   Config
   Server
   

   View Slide

5. OpenShift Cluster Management | Machine Configuration
   5
   Machine Config and Machine Config Pool
   Inheritance-based mapping of configuration to nodes
   50-kargs
   role:worker
   5-chrony
   role:worker
   50-motd
   role:worker
   Rendered config:
   rendered-worker-
   

   View Slide

6. Pool:
   role:worker
   OpenShift Cluster Management | Machine Configuration
   6
   Custom Machine Config Pools
   Hierarchical/layered configuration rendering
   50-args
   /etc/args
   role:worker
   5-chrony
   /etc/ntp.conf
   role:worker
   50-motd
   /etc/motd
   role:worker
   Pool:
   role:highperf
   60-args
   /etc/args
   role:highperf
   5-other
   /etc/other.conf
   role:highperf
   51-motd
   /etc/motd
   role:worker
   files:
   5-chrony:
   /etc/ntp.conf
   5-other:
   /etc/other.conf
   50-args:
   /etc/args
   50-motd:
   /etc/motd
   51-motd:
   /etc/motd
   60-args:
   /etc/args
   rendered-highperf-
   

   View Slide

7. OpenShift Cluster Management | Machine Configuration
   7
   Machine Config Server
   Providing Ignition configuration for provisioning
   rendered-worker-
   {.spec.config}
   VM / Server
   Ignition
   “worker.ign” RHCOS Image
   Machine
   Config
   Server
   Instance Metadata:
   https://api-int.xxx.local:22623/config/worker
   

   View Slide

8. OpenShift Cluster Management | Machine Configuration
   8
   Machine Config Server
   Identical nodes at massive scale
   New Workers
   …….
   Existing Workers
   rendered-worker-
   {.spec.config}
   Machine
   Config
   Server
   

   View Slide

9. OpenShift Cluster Management | Machine Configuration
   9
   Machine Config Daemon
   Preventing drift
   Machine
   Config
   Daemon
   50-registries
   role:worker
   5-chrony
   role:worker
   50-motd
   role:worker
   Rendered config:
   rendered-worker-
   /etc/containers/registries.conf
   /etc/chrony.conf
   /etc/motd
   

   View Slide

10. OpenShift Cluster Management | Machine Configuration
    10
    Machine Config Daemon
    Acting on drift
    The MCO coordinates with the MCD to perform the
    following actions, in a rolling manner, when OS updates
    and/or configuration changes are applied:
    ● Cordon / uncordons nodes
    ● Drain pods
    ● Stage node changes
    ○ OS upgrade
    ○ config changes
    ○ systemd units
    ● Reboot
    1. Validates node state matches
    desired state
    2. Validate cluster state & policy to
    apply change
    3. Change is rolled across cluster
    OS_VERSION
    =
    !=
    MaxUnavailable
    = 1
    

    View Slide

11. OpenShift Cluster Management | Machine Configuration
    11
    Transactional updates ensure that RHEL
    CoreOS is never altered during runtime. Rather
    it is booted directly into an always “known
    good” version.
    ● Each OS update is versioned and tested
    as a complete image.
    ● OS binaries (/usr) are read-only
    ● OS updates encapsulated in container
    images
    ● file system and package layering
    available for hotfixes and debugging
    Transactional updates with rpm-ostree
    

    View Slide

12. OpenShift Cluster Management
    12
    Over-the-air updates: Cluster Components
    ...
    some-component
    ...
    ...
    Cluster
    Version
    Operator
    Machine
    Config
    Operator
    Machine
    Config
    Operator
    Operands
    Some
    Operator
    Release Payload Info
    Upgrade
    Process
    

    View Slide

13. OpenShift Cluster Management
    13
    Over-the-air updates: Nodes
    ...
    machine-config-operator
    machine-os-content
    ...
    Cluster
    Version
    Operator
    Machine
    Config
    Operator
    Machine
    Config
    Operator
    Machine
    Config
    Daemons
    Machine
    Config
    Operator
    Rolling
    Machine
    Config
    Daemon
    Download and
    mount update
    content into host
    Machine
    Config
    Daemon
    Update host
    using mounted
    content
    Release Payload Info
    

    View Slide

14. Let’s Explore!
    Hacking away at it until it works.
    GitOps Happy Hour
    14
    Keyboard time!
    

    View Slide