Shifting Policy Enforcement Left using GitOps Sandeep Parikh @crcsmnky Google Cloud

Hi, I’m Sandeep Our team helps users adopt tools and processes so that they can deliver software faster. Find me @crcsmnky on Twitter and Github.

Policies

Rules that tell us how we can configure a resource Policies

Policy Management The practice of developing, deploying, and applying policies

Open Policy Agent Open Policy Agent (OPA) is a general-purpose policy engine with uses ranging from authorization and admission control to data filtering. Decouple policy decisions from services to achieve unified control across the entire stack. Unified Express policies in a high-level declarative language that promotes safe, fine-grained logic. Declarative Leverage arbitrary external data in policies to ensure that important requirements are enforced. Context Aware

Gatekeeper OPA Gatekeeper provides first-class integration between OPA and Kubernetes via a custom controller. Gatekeeper turns OPA policies into Kubernetes objects, so they can be customized and deployed using standard workflows. Gatekeeper Kubernetes kubectl AdmissionReview (request) AdmissionReview (response)

Policy objects Policies are written in Rego and packaged as parameterized ConstraintTemplate objects. apiVersion: templates.gatekeeper.sh/v1beta1 kind: ConstraintTemplate metadata: name: k8sblocknodeport spec: crd: spec: names: kind: K8sBlockNodePort targets: - target: admission.k8s.gatekeeper.sh rego: | package k8sblocknodeport violation[{"msg": msg}] { input.review.kind.kind == "Service" input.review.object.spec.type == "NodePort" msg := "Cannot create service of type NodePort" }

apiVersion: templates.gatekeeper.sh/v1beta1 kind: ConstraintTemplate metadata: name: k8sblocknodeport spec: crd: spec: names: kind: K8sBlockNodePort targets: - target: admission.k8s.gatekeeper.sh rego: | package k8sblocknodeport violation[{"msg": msg}] { input.review.kind.kind == "Service" input.review.object.spec.type == "NodePort" msg := "Cannot create service of type NodePort" } Policy objects Policies are written in Rego and packaged as parameterized ConstraintTemplate objects. The ConstraintTemplate extends Gatekeeper by adding a new policy that can be invoked via a new CR.

Policy objects apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sBlockNodePort metadata: name: block-node-port spec: enforcementAction: deny match: kinds: - apiGroups: [""] kinds: ["Service"] --- apiVersion: v1 kind: Service metadata: name: my-service-disallowed spec: type: NodePort ports: - port: 80 targetPort: 80 nodePort: 30007 Constraints are instantiations of a ConstraintTemplate CR and can be optionally scoped to specific objects and/or namespaces.

apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sBlockNodePort metadata: name: block-node-port spec: enforcementAction: deny match: kinds: - apiGroups: [""] kinds: ["Service"] --- apiVersion: v1 kind: Service metadata: name: my-service-disallowed spec: type: NodePort ports: - port: 80 targetPort: 80 nodePort: 30007 Policy objects Constraints are instantiations of a ConstraintTemplate CR and can be optionally scoped to specific objects and/or namespaces. When violated, Constraints can either deny admission or allow entry, and audit the violation in the status field.

Policy Enforcement

Enforcement operations Gatekeeper reviews the request then denies admission or issues warnings, based on violations. But this only happens when resources are deployed. Gatekeeper Kubernetes API kubectl AdmissionReview (request) AdmissionReview (response)

If resources violate any policies they will be rejected. But with GitOps, the controller will continually fail* to sync resources with clusters. Runtime enforcement Kubernetes Repo Gatekeeper GitOps * pending baked-in backoff, depends on your controller, YMMV, etc.

Shift Left

Validate changes against Gatekeeper policies Commits are pushed PRs are submitted Push Deploy ↺ Enforcement Push Review & Enforcement Deploy Merge

Validation tools googlecontainertools.github.io/kpt Kpt is an OSS tool for building declarative workflows on top of resource configuration. conftest.dev Conftest is a utility to help you write tests against structured configuration data. $ conftest test deployment.yaml --policy ./policy $ docker run -i gcr.io/kpt-functions/gatekeeper-validate

Github Actions → Conftest example ● From the creators of Conftest ● Parameterized with ○ Files to validate ○ Policy dir ○ Namespace under test ○ Output format on: push name: Validate jobs: conftest: runs-on: ubuntu-latest steps: - uses: actions/checkout@master - name: test uses: instrumenta/conftest-action@master with: files: deployment.yaml

Github Actions → Kpt example ● Create a workflow using Kpt functions ● Export the workflow to a CI tool: ○ Github Actions ○ GitLab CI ○ Jenkins ○ Cloud Build ○ CircleCI ○ Tekton $ kpt fn export example-package \ --workflow github-actions \ --output main.yaml $ cat main.yaml name: kpt on: push: branches: - master jobs: Kpt: runs-on: ubuntu-latest steps: - name: Run all kpt functions uses: docker://gcr.io/kpt-dev/kpt:latest with: args: fn run example-package

Infrastructure repos and clusters Infra Repo Kubernetes GitOps Gatekeeper Kubernetes GitOps Gatekeeper Kubernetes GitOps Gatekeeper Validation workflow

Application repos and clusters App Repo Kubernetes GitOps Gatekeeper Kubernetes GitOps Gatekeeper Kubernetes GitOps Gatekeeper Validation workflow Infra Repo

Authoring Policies

Getting started The Rego Playground provides a solid editor to get started with OPA and share policies. Try it out at play.openpolicyagent.org https://www.openpolicyagent.org/docs/latest/editor-and-ide-support/

OPA + Editor OPA has integrations for several editors and IDEs → VS Code, ST, IntelliJ, Emacs, VIM. Integrations differ depending on the tools but many offer syntax highlighting, query eval, policy coverage, and more. https://www.openpolicyagent.org/docs/latest/editor-and-ide-support/

Dev loop Don’t forget to test against Gatekeeper itself! Especially to understand failure scenarios. minikube, microk8s, etc.

Example Policies

OSS examples ● open-policy-agent/gatekeeper ○ Getting started examples ○ Includes required labels, allowed repos, container limits, unique service selector ● open-policy-agent/gatekeeper-library ○ Community-owned library of policies ○ General examples (see above) plus others (https-only, disallowed tags, unique ingress) ○ Pod Security Policies implemented as Constraints and ConstraintTemplates

OSS examples ● crcsmnky/gatekeeper-istio ○ Gatekeeper policies for Istio resources ○ Require mTLS activation, disallow all inbound sources, port-naming conventions ● GoogleCloudPlatform/acm-policy-controller-library ○ Anthos Service Mesh (Istio) policies, for use with Anthos Config Management (GitOps) Policy Controller (Gatekeeper) ○ Multiple authz controls, peer authentication, mTLS traffic policies

Bigger Picture

Policy enforcement complexity ● Scoping enforcement to the correct ○ Resources ○ Namespaces ○ Labels, etc. ● Understanding “fail open” vs “fail closed” ● Synchronizing resources to Gatekeeper for policy inputs ● RBAC for administering Constraints and ConstraintTemplates

Shifting security to the left Policy enforcement Signed images Vulnerability scanning Signature validation Audit controls Encrypted images

No content

Thanks! Questions, comments, concerns? Find me @crcsmnky on Twitter and Github