Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Unit Testing Your Kubernetes Configuration with Open Policy Agent

Unit Testing Your Kubernetes Configuration with Open Policy Agent

Talk from KubeCon EU 2019, focused on using Open Policy Agent to write tests for structured configuration.

Gareth Rushgrove

May 21, 2019
Tweet

More Decks by Gareth Rushgrove

Other Decks in Technology

Transcript

  1. Using Open Policy Agent
    Gareth Rushgrove | Director, Product Management | Snyk
    snyk.io
    Unit testing your
    Kubernetes configuration
    May, 2019

    View Slide

  2. snyk.io

    View Slide

  3. snyk.io
    - A quick introduction to Open Policy Agent
    - Shift-left testing
    - Introducing conftest
    - Rego as a programming language
    - Portability between different Kubernetes solutions
    - Not just Kubernetes
    Agenda

    View Slide

  4. Open Policy Agent
    snyk.io
    A policy enforcement engine for configuration

    View Slide

  5. What is Open Policy Agent?

    View Slide

  6. Policy
    Data
    OPA
    Service
    How Open Policy Agent works

    View Slide

  7. A growing ecosystem

    View Slide

  8. View Slide

  9. Shift left
    snyk.io
    Faster feedback

    View Slide

  10. Shift-left testing is an approach to software testing and
    system testing in which testing is performed earlier in the
    lifecycle (i.e., moved left on the project timeline).
    Wikipedia

    View Slide

  11. Development cycle
    Fast Slow Slower
    Cluster
    Local
    development
    Continuous
    integration

    View Slide

  12. Open Policy Agent is
    normally used here
    Development cycle
    Cluster
    Local
    development
    Continuous
    integration

    View Slide

  13. Cluster
    What if we could use Open
    Policy Agent here as well?
    Development cycle
    Local
    development
    Continuous
    integration

    View Slide

  14. “Everything as code”
    Said several times at the CDF event

    View Slide

  15. “Would you write code
    without tests?”

    View Slide

  16. Introducing conftest
    snyk.io
    Test your configuration locally, and in CI

    View Slide

  17. Introducing conftest

    View Slide

  18. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-kubernetes
    spec:
    replicas: 3
    selector:
    matchLabels:
    app: hello-kubernetes
    template:
    metadata:
    labels:
    app: hello-kubernetes
    spec:
    containers:
    - name: hello-kubernetes
    image: paulbouwer/hello-kubernetes:1.5
    ports:
    Given a deployment
    snyk.io

    View Slide

  19. package main
    deny[msg] {
    input.kind = "Deployment"
    not input.spec.template.spec.securityContext.runAsNonRoot = true
    msg = "Containers must not run as root"
    }
    deny[msg] {
    input.kind = "Deployment"
    not input.spec.selector.matchLabels.app
    msg = "Containers must provide app label for pod selectors"
    }
    Write your policies
    snyk.io

    View Slide

  20. deny[msg] {
    input.kind = "Deployment"
    not input.spec.template.spec.securityContext.runAsNonRoot = true
    msg = "Containers must not run as root"
    }
    Explaining what we just wrote
    snyk.io
    We should deny any input for which
    Deployment is the value for kind
    and
    When runAsNonRoot is set to false

    View Slide

  21. $ conftest test deployment.yaml
    deployment.yaml
    Containers must not run as root
    $ echo $status
    1
    Run tests locally with conftest
    snyk.io

    View Slide

  22. snyk.io
    Demo

    View Slide

  23. Rego as a language
    snyk.io
    The usual pros and cons of a DSL

    View Slide

  24. Good documentation

    View Slide

  25. The Rego playground

    View Slide

  26. package main
    test_deployment_without_security_context {
    deny["Containers must not run as root"] with input as {"kind": "Deployment"}
    }
    test_deployment_with_security_context {
    no_violations with input as {"kind": "Deployment", "spec": {
    "selector": { "matchLabels": { "app": "something", "release": "something" }},
    "template": { "spec": { "securityContext": { "runAsNonRoot": true }}}}}
    }
    test_services_not_denied {
    no_violations with input as {"kind": "Service"}
    }
    test_services_issue_warning {
    warn["Services are not allowed"] with input as {"kind": "Service"}
    Built-in testing tools

    View Slide

  27. $ opa test --verbose .
    data.main.test_deployment_without_security_context: PASS (1.029µs)
    data.main.test_deployment_with_security_context: PASS (1.058µs)
    data.main.test_services_not_denied: PASS (701ns)
    data.main.test_services_issue_warning: PASS (614ns)
    ---------------------------------------------------------------------
    PASS: 4/4
    Open Policy Agent test runner
    snyk.io

    View Slide

  28. In:path .rego extension:rego
    546 results
    Not much public rego code yet

    View Slide

  29. Open Policy Agent Bundles

    View Slide

  30. package main
    # has_field returns whether an object has a field
    has_field(object, field) {
    object[field]
    }
    # False is a tricky special case, as false responses would create an undefined
    # document unless they are explicitly tested for
    has_field(object, field) {
    object[field] == false
    }
    has_field(object, field) = false {
    General helpers

    View Slide

  31. package main
    empty(value) {
    count(value) == 0
    }
    no_violations {
    empty(deny)
    }
    no_warnings {
    empty(warn)
    }
    Test helpers
    snyk.io

    View Slide

  32. package kubernetes
    is_service {
    input.kind = "Service"
    }
    is_deployment {
    input.kind = "Deployment"
    }
    Domain specific helpers
    snyk.io

    View Slide

  33. Reusing OCI registries

    View Slide

  34. Proposed OCI media types of OPA

    View Slide

  35. $ ls
    kubernetes.rego
    $ conftest push instrumenta.azurecr.io/kubernetes-helpers
    $ ls policy
    $ conftest pull instrumenta.azurecr.io/kubernetes-helpers:latest
    $ ls policy
    kubernetes.rego
    Using conftest to share policy
    snyk.io

    View Slide

  36. Bundle Bar

    View Slide

  37. Portability
    snyk.io
    Helping to move between different Kubernetes tools

    View Slide

  38. snyk.io
    Demo

    View Slide

  39. $ kustomize build | conftest test -
    Kustomize
    snyk.io

    View Slide

  40. $ helm template | conftest test -
    Helm
    snyk.io

    View Slide

  41. import {Pod} from 'kubernetes-types/core/v1'
    import {ObjectMeta} from 'kubernetes-types/meta/v1'
    import * as yaml from 'js-yaml'
    let metadata: ObjectMeta = {name: 'example', labels: {}}
    let pod: Pod = {
    apiVersion: 'v1',
    kind: 'Pod',
    metadata,
    spec: {
    containers: [
    /* ... */
    ],
    },
    }
    console.log(yaml.safeDump(pod))
    Typescript

    View Slide

  42. $ npx ts-node pod.ts | conftest test -
    Typescript
    snyk.io

    View Slide

  43. $ kubectl get all -o json \
    | jq -cj '.items[] | tostring+"\u0000"' \
    | xargs -n1 -0 -I@ bash -c "echo '@' | conftest test -"
    Kubectl (look away now)
    snyk.io

    View Slide

  44. $ kubectl krew install conftest
    $ kubectl conftest deployment some-deployment
    Kubectl plugin
    snyk.io

    View Slide

  45. package kubernetes
    deployment "hello-kubernetes": {
    apiVersion: "apps/v1"
    spec: {
    replicas: 3
    template spec containers: [{
    image: "paulbouwer/hello-kubernetes:1.5"
    ports: [{
    containerPort: 8080
    }]
    }]
    }
    }
    Cue
    snyk.io

    View Slide

  46. package kubernetes
    import "encoding/yaml"
    command test: {
    task conftest: {
    kind: "exec"
    cmd: "conftest test -"
    stdin: yaml.MarshalStream(objects)
    }
    }
    Cue
    snyk.io

    View Slide

  47. $ cue test
    Containers must not run as root
    --- .
    command "conftest test -" failed: exit status 1
    terminating because of errors
    Cue
    snyk.io

    View Slide

  48. Not just Kubernetes
    snyk.io
    Lots of other configurations to care about

    View Slide

  49. apiVersion: kind: extension:yaml
    1,054,453 results
    Lots of Kubernetes-like docs

    View Slide

  50. in:path .yml language:YAML
    69,822,306 results
    Lots more YAML

    View Slide

  51. snyk.io
    Demo

    View Slide

  52. $ terraform plan -out config.tfplan
    $ terraform show -json config.tfplan | configtest test -
    Terraform
    snyk.io

    View Slide

  53. package main
    blacklist = [
    "google_iam",
    "google_container"
    ]
    deny[msg] {
    check_resources(input.resource_changes, blacklist)
    banned := concat(", ", blacklist)
    msg = sprintf("Terraform plan will change prohibited resources in: %v", [banned])
    }
    # Checks whether the plan will cause resources with certain prefixes to change
    check_resources(resources, disallowed_prefixes) {
    startswith(resources[_].type, disallowed_prefixes[_])
    }
    Terraform
    snyk.io

    View Slide

  54. service: aws-python-scheduled-cron
    frameworkVersion: ">=1.2.0 <2.0.0"
    provider:
    name: aws
    runtime: python2.7
    tags:
    author: "this field is required"
    functions:
    cron:
    handler: handler.run
    runtime: python2.7
    events:
    - schedule: cron(0/2 * ? * MON-FRI *)
    Serverless framework
    snyk.io

    View Slide

  55. package main
    deny[msg] {
    input.provider.runtime = "python2.7"
    msg = "Python 2.7 cannot be the default provider runtime"
    }
    runtime[name] {
    input.functions[i].runtime = name
    }
    deny[msg] {
    runtime["python2.7"]
    msg = "Python 2.7 cannot be used as the runtime for functions"
    }
    deny[msg] {
    not has_field(input.provider.tags, "author")
    msg = "Should set provider tags for author"
    }
    Serverless framework
    snyk.io

    View Slide

  56. version: "3.4"
    services:
    web:
    build: .
    ports:
    - "5000:5000"
    redis:
    image: "redis:latest"
    Docker Compose
    snyk.io

    View Slide

  57. package main
    version {
    to_number(input.version)
    }
    deny[msg] {
    endswith(input.services[_].image, ":latest")
    msg = "No images tagged latest"
    }
    deny[msg] {
    version < 3.5
    msg = "Must be using at least version 3.5 of the Compose file format"
    }
    Docker Compose
    snyk.io

    View Slide

  58. Conclusions
    snyk.io
    If all you remember is...

    View Slide

  59. @jdolitsky @stevelasker @sometorin
    Thanks open source community!

    View Slide

  60. snyk.io
    - Open Policy Agent is incredibly flexible
    - Check out conftest at github.com/instrumenta/conftest
    - Expect lots more integrations in the future
    - Managing configuration as code needs better tools
    Come talk to Snyk at booth #S41
    Summary

    View Slide

  61. Questions?
    snyk.io
    And thanks for listening

    View Slide