Shift Policy Enforcement Left using GitOps

Open Policy Agent’s Gatekeeper provides a policy-based approach to managing admission control and identifying policy violations, stopping bad things from happening to production deployments. But those checks only happen at deploy time - what if we could move that process back into the GitOps workflow? In this session we’ll review the basics of implementing policy controls using OPA Gatekeeper, and go in-depth on bringing those policy checks back into the branch-commit-merge process. We’ll demonstrate the different approaches to integrating OPA Gatekeeper with GitOps, bringing policy enforcement to the left and identifying policy violations much earlier.

Transcript

1. Shifting Policy Enforcement Left using GitOps Sandeep Parikh @crcsmnky Google Cloud
2. Hi, I’m Sandeep Our team helps users adopt tools and processes so that they can deliver software faster. Find me @crcsmnky on Twitter and Github.
3. Policies
4. Rules that tell us how we can configure a resource Policies
5. Policy Management The practice of developing, deploying, and applying policies
6. Open Policy Agent Open Policy Agent (OPA) is a general-purpose policy engine with uses ranging from authorization and admission control to data filtering. Decouple policy decisions from services to achieve unified control across the entire stack. Unified Express policies in a high-level declarative language that promotes safe, fine-grained logic. Declarative Leverage arbitrary external data in policies to ensure that important requirements are enforced. Context Aware
7. Gatekeeper OPA Gatekeeper provides first-class integration between OPA and Kubernetes via a custom controller. Gatekeeper turns OPA policies into Kubernetes objects, so they can be customized and deployed using standard workflows. Gatekeeper Kubernetes kubectl AdmissionReview (request) AdmissionReview (response)
8. Policy objects Policies are written in Rego and packaged as parameterized ConstraintTemplate objects. apiVersion: templates.gatekeeper.sh/v1beta1 kind: ConstraintTemplate metadata: name: k8sblocknodeport spec: crd: spec: names: kind: K8sBlockNodePort targets: - target: admission.k8s.gatekeeper.sh rego: | package k8sblocknodeport violation[{"msg": msg}] { input.review.kind.kind == "Service" input.review.object.spec.type == "NodePort" msg := "Cannot create service of type NodePort" }
9. apiVersion: templates.gatekeeper.sh/v1beta1 kind: ConstraintTemplate metadata: name: k8sblocknodeport spec: crd: spec: names: kind: K8sBlockNodePort targets: - target: admission.k8s.gatekeeper.sh rego: | package k8sblocknodeport violation[{"msg": msg}] { input.review.kind.kind == "Service" input.review.object.spec.type == "NodePort" msg := "Cannot create service of type NodePort" } Policy objects Policies are written in Rego and packaged as parameterized ConstraintTemplate objects. The ConstraintTemplate extends Gatekeeper by adding a new policy that can be invoked via a new CR.
10. Policy objects apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sBlockNodePort metadata: name: block-node-port spec: enforcementAction: deny match: kinds: - apiGroups: [""] kinds: ["Service"] --- apiVersion: v1 kind: Service metadata: name: my-service-disallowed spec: type: NodePort ports: - port: 80 targetPort: 80 nodePort: 30007 Constraints are instantiations of a ConstraintTemplate CR and can be optionally scoped to specific objects and/or namespaces.
11. apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sBlockNodePort metadata: name: block-node-port spec: enforcementAction: deny match: kinds: - apiGroups: [""] kinds: ["Service"] --- apiVersion: v1 kind: Service metadata: name: my-service-disallowed spec: type: NodePort ports: - port: 80 targetPort: 80 nodePort: 30007 Policy objects Constraints are instantiations of a ConstraintTemplate CR and can be optionally scoped to specific objects and/or namespaces. When violated, Constraints can either deny admission or allow entry, and audit the violation in the status field.
12. Policy Enforcement
13. Enforcement operations Gatekeeper reviews the request then denies admission or issues warnings, based on violations. But this only happens when resources are deployed. Gatekeeper Kubernetes API kubectl AdmissionReview (request) AdmissionReview (response)
14. If resources violate any policies they will be rejected. But with GitOps, the controller will continually fail* to sync resources with clusters. Runtime enforcement Kubernetes Repo Gatekeeper GitOps * pending baked-in backoff, depends on your controller, YMMV, etc.
15. Shift Left
16. Validate changes against Gatekeeper policies Commits are pushed PRs are submitted Push Deploy ↺ Enforcement Push Review & Enforcement Deploy Merge
17. Validation tools googlecontainertools.github.io/kpt Kpt is an OSS tool for building declarative workflows on top of resource configuration. conftest.dev Conftest is a utility to help you write tests against structured configuration data. $ conftest test deployment.yaml --policy ./policy $ docker run -i gcr.io/kpt-functions/gatekeeper-validate
18. Github Actions → Conftest example • From the creators of Conftest • Parameterized with ◦ Files to validate ◦ Policy dir ◦ Namespace under test ◦ Output format on: push name: Validate jobs: conftest: runs-on: ubuntu-latest steps: - uses: actions/checkout@master - name: test uses: instrumenta/conftest-action@master with: files: deployment.yaml
19. Github Actions → Kpt example • Create a workflow using Kpt functions • Export the workflow to a CI tool: ◦ Github Actions ◦ GitLab CI ◦ Jenkins ◦ Cloud Build ◦ CircleCI ◦ Tekton $ kpt fn export example-package \ --workflow github-actions \ --output main.yaml $ cat main.yaml name: kpt on: push: branches: - master jobs: Kpt: runs-on: ubuntu-latest steps: - name: Run all kpt functions uses: docker://gcr.io/kpt-dev/kpt:latest with: args: fn run example-package
20. Infrastructure repos and clusters Infra Repo Kubernetes GitOps Gatekeeper Kubernetes GitOps Gatekeeper Kubernetes GitOps Gatekeeper Validation workflow
21. Application repos and clusters App Repo Kubernetes GitOps Gatekeeper Kubernetes GitOps Gatekeeper Kubernetes GitOps Gatekeeper Validation workflow Infra Repo
22. Authoring Policies
23. Getting started The Rego Playground provides a solid editor to get started with OPA and share policies. Try it out at play.openpolicyagent.org https://www.openpolicyagent.org/docs/latest/editor-and-ide-support/
24. OPA + Editor OPA has integrations for several editors and IDEs → VS Code, ST, IntelliJ, Emacs, VIM. Integrations differ depending on the tools but many offer syntax highlighting, query eval, policy coverage, and more. https://www.openpolicyagent.org/docs/latest/editor-and-ide-support/
25. Dev loop Don’t forget to test against Gatekeeper itself! Especially to understand failure scenarios. minikube, microk8s, etc.
26. Example Policies
27. OSS examples • open-policy-agent/gatekeeper ◦ Getting started examples ◦ Includes required labels, allowed repos, container limits, unique service selector • open-policy-agent/gatekeeper-library ◦ Community-owned library of policies ◦ General examples (see above) plus others (https-only, disallowed tags, unique ingress) ◦ Pod Security Policies implemented as Constraints and ConstraintTemplates
28. OSS examples • crcsmnky/gatekeeper-istio ◦ Gatekeeper policies for Istio resources ◦ Require mTLS activation, disallow all inbound sources, port-naming conventions • GoogleCloudPlatform/acm-policy-controller-library ◦ Anthos Service Mesh (Istio) policies, for use with Anthos Config Management (GitOps) Policy Controller (Gatekeeper) ◦ Multiple authz controls, peer authentication, mTLS traffic policies
29. Bigger Picture
30. Policy enforcement complexity • Scoping enforcement to the correct ◦ Resources ◦ Namespaces ◦ Labels, etc. • Understanding “fail open” vs “fail closed” • Synchronizing resources to Gatekeeper for policy inputs • RBAC for administering Constraints and ConstraintTemplates
31. Shifting security to the left Policy enforcement Signed images Vulnerability scanning Signature validation Audit controls Encrypted images
33. Thanks! Questions, comments, concerns? Find me @crcsmnky on Twitter and Github