Talk from Configuration Management Camp, a rerun and rebranding of the talk I gave at KubeCon. All about testing and linting tools for Kubernetes configs, taking inspiration from the various configuration management tools.
Infrastructure as code is the process of managing and provisioning computers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools https://en.wikipedia.org/wiki/Infrastructure_as_Code
Validate directly using the schemas $ jsonschema -F "{error.message}" -i hello-nginx.json 1.6.6-standalone/deployment.json u'template' is a required property
A nice CLI for validating K8S configurations $ kubeval --help Validate a Kubernetes YAML file against the relevant schema Usage: kubeval [file...] [flags] Flags: -h, --help help for kubeval -v, --kubernetes-version string Version of Kubernetes to validate against --openshift Use OpenShift schemas instead of upstream Kubernetes --schema-location string Base URL used to download schemas. Can also be specified with the environment variable KUBEVAL_SCHEMA_LOCATION --version Display the kubeval version information and exit
Validate multiple resources in the same file $ kubeval multi.yaml The document fixtures/multi.yaml contains a valid Service The document fixtures/multi.yaml contains an invalid Deployment --> spec.template.spec.containers.0.env.0.value: Invalid type. Expected: string, given: integer The document fixtures/multi.yaml contains an invalid ReplicationController --> spec.replicas: Invalid type. Expected: integer, given: string The document fixtures/multi.json contains a valid Deployment The document fixtures/multi.yaml contains a valid ReplicationController
Run tests against your configurations $ kubetest rc.yaml --verbose INFO rc.yaml should not use latest images WARN rc.yaml ReplicationController should have at least 4 replicas
Skylark is a dialect of Python. It is an untyped dynamic language with high-level data types, first-class functions with lexical scope, and garbage collection
A Skylark interpreter is typically embedded within an application which may define additional domain-specific functions and data types beyond those provided by the core language
Example tests for kubetest #// vim: set ft=python: def test_for_latest_image(): if spec["kind"] == "ReplicationController": for container in spec["spec"]["template"]["spec"]["containers"]: tag = container["image"].split(":")[-1] assert_not_equal(tag, "latest", "should not use latest images") def test_minimum_replicas(): if spec["kind"] == "ReplicationController": test = spec["spec"]["replicas"] >= 4 assert_true(test, "ReplicationController should have at least 4 replicas") test_for_latest_image() test_minimum_replicas()
Tests enforcing a team label #// vim: set ft=python: def test_for_team_label(): if spec["kind"] == "Deployment": labels = spec["spec"]["template"]["metadata"]["labels"] assert_contains(labels, "team", "should indicate which team owns the deployment") test_for_team_label()
Using kubeval with Helm $ git clone https://github.com/kubernetes/helm.git $ cd helm/docs/examples $ ls alpine nginx README.md $ helm template nginx | kubeval The document stdin contains a valid Secret The document stdin contains a valid ConfigMap The document stdin contains a valid Service The document stdin contains a valid Pod The document stdin contains a valid Deployment The document stdin contains a valid Job
Tests for our Helm Chart #// vim: set ft=python: def test_for_latest_image(): if spec["kind"] in ["Job", "Deployment"]: for container in spec["spec"]["template"]["spec"]["containers"]: tag = container["image"].split(":")[-1] assert_not_equal(tag, "latest", spec["kind"] + " should not use latest images") test_for_latest_image()
Using kubetest with Helm $ helm template nginx | kubetest --verbose INFO stdin Deployment should not use latest images INFO stdin Job should not use latest images
Using kubeval with ksonnet $ cat deployment.jsonnet local k = import "ksonnet.beta.2/k.libsonnet"; local deployment = k.apps.v1beta1.deployment; local container = deployment.mixin.spec.template.spec.containersType; local containerPort = container.portsType; // Create nginx container with container port 80 open. local nginxContainer = container.new("nginx", "nginx:1.13.0") + container.ports(containerPort.newNamed("http", 80)); // Create default Deployment object from nginx container. deployment.new("nginx", 5, nginxContainer, {app: "nginx"})
Using kubetest with ksonnet $ jsonnet deployment.jsonnet | kubetest --verbose INFO stdin should not use latest images WARN stdin ReplicationController should have at least 7 replicas
Using kubeval with Puppet $ puppet kubernetes convert --manifest guestbook.pp | kubeval The document stdin contains a valid ReplicationController The document stdin contains a valid Service The document stdin contains a valid ReplicationController The document stdin contains a valid Service The document stdin contains a valid ReplicationController The document stdin contains a valid Service
Using kubetest with Puppet $ puppet kubernetes convert --manifest guestbook.pp | kubetest --verbose INFO stdin should not use latest images INFO stdin should not use latest images INFO stdin should not use latest images