Slide 1

Slide 1 text

Testing configuration with Open Policy Agent Gareth Rushgrove

Slide 2

Slide 2 text

Gareth Rushgrove Director, Product Management, Snyk Devops Weekly curator Open Policy Agent, Conftest Maintainer @garethr

Slide 3

Slide 3 text

Agenda Why policy? 01 Policy and developer workflow 02 The importance of sharing 03 Conclusions 04

Slide 4

Slide 4 text

Why policy? Introducing Open Policy Agent

Slide 5

Slide 5 text

policy a set of ideas or a plan of what to do in particular situations that has been agreed to officially by a group of people, a business organization, a government, or a political party. Cambridge Dictionary noun [ C ] UK /ˈpɒl.ə.si/ US /ˈpɑː.lə.si/

Slide 6

Slide 6 text

policy a set of ideas or a plan of what to do in particular situations that has been agreed to officially by a group of people, a business organization, a government, or a political party. Cambridge Dictionary noun [ C ] UK /ˈpɒl.ə.si/ US /ˈpɑː.lə.si/ All Go projects should have been updated to use Go 1.13

Slide 7

Slide 7 text

policy a set of ideas or a plan of what to do in particular situations that has been agreed to officially by a group of people, a business organization, a government, or a political party. Cambridge Dictionary noun [ C ] UK /ˈpɒl.ə.si/ US /ˈpɑː.lə.si/ All Go projects should have been updated to use Go 1.13 All EC2 instances should have tags showing which team owns them

Slide 8

Slide 8 text

policy a set of ideas or a plan of what to do in particular situations that has been agreed to officially by a group of people, a business organization, a government, or a political party. Cambridge Dictionary noun [ C ] UK /ˈpɒl.ə.si/ US /ˈpɑː.lə.si/ All Go projects should have been updated to use Go 1.13 All EC2 instances should have tags showing which team owns them Dockerfiles should not use FROM with images tagged latest

Slide 9

Slide 9 text

Open Policy Agent github.com/open-policy-agent/opa Service OPA .rego Query (any JSON value) Decision (any JSON value) Data (JSON) Policy (Rego) Request, Event, etc. Declarative Express policy in a high-level, declarative language that promotes safe, performant, fine-grained controls. Use a language purpose-built for policy in a world where JSON is pervasive. Context-aware Leverage external information to write the policies you really care about. Write logic that adapts to the world around it and attach that logic to the systems that need it.

Slide 10

Slide 10 text

Conftest github.com/open-policy-agent/conftest

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

Given a Kubernetes YAML file apiVersion: apps/v1 kind: Deployment metadata: name: hello-kubernetes spec: replicas: 3 selector: matchLabels: app: hello-kubernetes template: metadata: labels: app: hello-kubernetes spec: containers: - name: hello-kubernetes YAML

Slide 13

Slide 13 text

Write your policies deny[msg] { input.kind = "Deployment" not input.spec.template.spec.securityContext.runAsNonRoot = true msg = "Containers must not run as root" } We should deny any input for which Deployment is the value for kind and When runAsNonRoot is not true REGO

Slide 14

Slide 14 text

Run tests with conftest $ conftest test deployment.yaml FAIL - Containers must not run as root 1 test, 0 passed, 0 warnings, 1 failure, 0 exceptions

Slide 15

Slide 15 text

Lots of output options $ conftest test service.yaml -o table +---------+--------------+--------------------------------+ | RESULT | FILE | MESSAGE | +---------+--------------+--------------------------------+ | success | service.yaml | data.main.deny | | success | service.yaml | data.main.violation | | success | service.yaml | | | success | service.yaml | | | warning | service.yaml | Found service hello-kubernetes | | | | but services are not allowed | +---------+--------------+--------------------------------+

Slide 16

Slide 16 text

Demo

Slide 17

Slide 17 text

Write unit tests for Rego in Rego test_blank_input { no_violations with input as {} } test_correctly_encrypted_azure_disk { no_violations with input as {"resource": { "azurerm_managed_disk": { "sample": { "encryp } test_unencrypted_azure_disk { deny["Azure disk `sample` is not encrypted"] with input as {"resource": { "azurerm_managed_di } REGO

Slide 18

Slide 18 text

Documentation www.openpolicyagent.org/docs

Slide 19

Slide 19 text

The Rego Playground play.openpolicyagent.org

Slide 20

Slide 20 text

Policy and developer workflow Local testing, CI and more

Slide 21

Slide 21 text

Any configuration format Conftest currently supports YAML, JSON, INI, TOML, HOCON, HCL, CUE, Dockerfile, EDN, VCL, XML and Jsonnet. Lots of infrastructure use cases Pulumi, Terraform, Azure Resource Manager, AWS CloudFormation, Varnish, Dockerfile, Envoy, CircleCI configuration, Tekton, Kubernetes and lots more. Test any configuration file or structured output Anything that outputs, or takes as input, structured data in a standard format.

Slide 22

Slide 22 text

Conftest integrations Easy to run in common CI/CD systems

Slide 23

Slide 23 text

Modern Infrastructure as Code Create and deploy cloud software that use containers, serverless functions, hosted services, and infrastructure, on any cloud. Describe infrastructure in Typescript, Python, Go or .NET. let aws = require("@pulumi/aws"); let sg = new aws.ec2.SecurityGroup("web-sg", { ingress: [{ protocol: "tcp", fromPort: 80, toPort: 80, cidrBlocks: ["0.0.0.0/0"]}], }); for (let i = 0; i < 3; i++) { new aws.ec2.Instance(`web-${i}`, { ami: "ami-7172b611", instanceType: "t2.micro", securityGroups: [ sg.name ], userData: `#!/bin/bash echo "Hello, World!" > index.html nohup python -m SimpleHTTPServer 80 &`, }); } TYPESCRIPT

Slide 24

Slide 24 text

Pulumi CrossGuard supports Open Policy Agent

Slide 25

Slide 25 text

Write policies for any Pulumi provider package kubernetes name = input.metadata.name labels { input.metadata.labels["app.kubernetes.io/name"] input.metadata.labels["app.kubernetes.io/instance"] input.metadata.labels["app.kubernetes.io/version"] input.metadata.labels["app.kubernetes.io/component"] input.metadata.labels["app.kubernetes.io/part-of"] input.metadata.labels["app.kubernetes.io/managed-by"] } deny[msg] { input.kind = "Deployment" not labels msg = sprintf("%s must include Kubernetes recommended labels", [name]) } REGO

Slide 26

Slide 26 text

Gate pulumi up on passing policy $ pulumi up --policy-pack policy Previewing update (dev) View Live: https://app.pulumi.com/garethr/pulumi-oketeto-conftest-demo/dev/previews/b100f975-5715-4ced-acd4-a2dd 961e6617 Type Name Plan Info pulumi:pulumi:Stack pulumi-oketeto-conftest-demo-dev 1 error ~ └─ pulumi:providers:kubernetes kubernetes update [diff: -kubeconfig] Diagnostics: pulumi:pulumi:Stack (pulumi-oketeto-conftest-demo-dev): error: preview failed Policy Violations: [mandatory] kubernetes v0.0.1 deny (nginx: kubernetes:apps/v1:Deployment) nginx-fxpagomr must include Kubernetes recommended labels

Slide 27

Slide 27 text

Output to YAML with Pulumi (beta) from pulumi_kubernetes import Provider # Instantiate a Kubernetes Provider and specify the render directory. render_provider = Provider("k8s-yaml-rendered", render_yaml_to_directory="yaml") PYTHON import * as k8s from "@pulumi/kubernetes"; import * as kx from "@pulumi/kubernetesx"; // Instantiate a Kubernetes Provider and specify the render directory. const provider = new k8s.Provider("render-yaml", { renderYamlToDirectory: "rendered", }); TYPESCRIPT

Slide 28

Slide 28 text

Demo

Slide 29

Slide 29 text

Lots of tools output JSON Any tool that can output to a one of the structured formats supported by Conftest can be tested with Open Policy Agent. Provides a rules engine for anything from kubectl to aws to snyk. Useful for checking output as well as input $ snyk container test your/image --json ... { "title": "CVE-2019-9619", "packageName": "systemd", "language": "linux", "packageManager": "debian:10", "description": "...", "identifiers": { "CVE": [ "CVE-2019-9619" ], }, "severity": "low", "references": [ { "title": "Debian Security Tracker", "url": "https://security-tracker.debian.o

Slide 30

Slide 30 text

Prohibiting high severity vulnerabilities package main deny[msg] { issue = input.vulnerabilities[index] issue.severity = "high" msg = sprintf("High severity issue found. package: %v issue: %v", [issue.name, issue.title]) } Rego for the JSON output from snyk Here we’re saying we want to prohibit any images with known high-severity vulnerabilities. REGO

Slide 31

Slide 31 text

Watching out for crypto issues package main deny[msg] { issue = input.vulnerabilities[index] issue.identifiers["CWE"][_] = "CWE-327" issue.severity = "high" msg = sprintf("High severity cryptography issue (CWE-327). package: %v", [issue.name]) } warn[msg] { issue = input.vulnerabilities[index] issue.identifiers["CWE"][_] = "CWE-327" issue.severity != "high" msg = sprintf("Crypto issue (CWE-327). package: %v severity: %v", [issue.name, issue.severity]) } REGO

Slide 32

Slide 32 text

Run locally and as part of image pipelines $ snyk container test ghcr.io/garethr/snykt/base --file=base/Dockerfile --json | conftest test - WARN - Cryptography issue (CWE-327). package: gnupg severity: low WARN - Cryptography issue (CWE-327). package: openssl/libssl1.0.0 severity: medium WARN - Cryptography issue (CWE-327). package: openssl/libssl1.0.0 severity: low WARN - Cryptography issue (CWE-327). package: openssl severity: medium WARN - High severity issue found. package: apt/libapt-pkg4.12 issue: Arbitrary Code Injection FAIL - High severity issue found. package: apt issue: Arbitrary Code Injection FAIL - High severity issue found. package: bzip2/libbz2-1.0 issue: Out-of-bounds Write FAIL - High severity issue found. package: glibc/libc-bin issue: Out-of-bounds Read Stdin Use conftest and Open Policy Agent with any tool that outputs to JSON.

Slide 33

Slide 33 text

Sharing policy Scaling with reuse

Slide 34

Slide 34 text

Sharing via Git, HTTP, S3 and more $ conftest pull bucket.s3.amazonaws.com/foo # Note here we’re only getting the contents of a specific subdirectory, policy $ conftest pull github.com/garethr/pulumi-okteto-conftest-demo.git//policy Pull policies from remote locations Support for local files, Git, Mercurial, HTTP, Amazon S3, Google GCP

Slide 35

Slide 35 text

Sharing via OCI registries $ conftest push ghcr.io/garethr/pulumi-okteto-conftest-demo/policy ... $ conftest pull oci://ghcr.io/garethr/pulumi-okteto-conftest-demo/policy:latest Powered by the OCI Artifacts specification Not supported by all registries yet, but ACR, ECR, GitHub Container Registry, Harbor and more support this capability now.

Slide 36

Slide 36 text

Sharing via Pulumi $ pulumi policy publish # Note here we’re only getting the contents of a specific subdirectory, policy $ pulumi policy ls NAME VERSIONS kubernetes 1 $ pulumi policy enable garethr/kubernetes 1 Pulumi’s commercial service has sharing built-in Automatic enforcement on pulumi up without having to specify the policy pack locally.

Slide 37

Slide 37 text

Conclusions If all you remember is...

Slide 38

Slide 38 text

Configuration needs tests too Infrastructure as code benefits from software development practices just like other software development.

Slide 39

Slide 39 text

Standardize tools for common problems OPA is useful for individual use cases, but the cost of adoption really pays off when using to solve several problems

Slide 40

Slide 40 text

Policy needs to shift left Catching problems only at the end of the process doesn’t make for a nice developer experience.

Slide 41

Slide 41 text

Thanks for listening Sign up for Snyk for free at snyk.io/signup