www.containerdays.io #CDS23 Evolution of security strategies in K8s environments

www.containerdays.io #CDS23 Agenda ● Introduction to security strategies in K8s environments ● Pod Security Admission(PSA) vs Open Policy Agent (OPA) ● Combination of different security strategies together ● Access to resources in privileged and non-privileged mode

www.containerdays.io #CDS23 Introduction to security strategies in K8s environments ● Cluster Hardening: Implement best practices for securing the Kubernetes cluster itself, including securing access to the API server, enabling RBAC (Role-Based Access Control), and using network policies to control communication between pods. ● Pod Security Policies (PSP): Enforce security policies that define what a pod can and cannot do, including limiting privilege levels, host access, and running as non-root users.

www.containerdays.io #CDS23 Introduction to security strategies in K8s environments ● Secrets Management: Use Kubernetes Secrets to store sensitive information securely, such as API keys, passwords, or certificates. ● Role-Based Access Control (RBAC): Define fine-grained access controls for users and service accounts to limit the scope of actions they can perform within the cluster.

www.containerdays.io #CDS23 Introduction to security strategies in K8s environments ● Limit Resource Consumption: Set resource quotas to limit the amount of CPU, memory, and other resources that can be consumed by pods, preventing resource exhaustion and potential denial-of-service attacks. ● Pod Security Context: Use pod security context to define security settings at the pod level, such as user and group IDs, SELinux, and file system permissions.

www.containerdays.io #CDS23 Introduction to security strategies in K8s environments ● PodSecurityPolicy has been deprecated from Kubernetes 1.21.

www.containerdays.io #CDS23 Introduction to security strategies in K8s environments ● PodSecurityContext, the Kubernetes tool which allows users to specify security contexts and how the pod will be execute.

www.containerdays.io #CDS23 Introduction to security strategies in K8s environments Security Contexts RBAC (Role-Based Access Control) Resource scope Pods Pods, Nodes, cluster Actions Predefined capabilities RBAC policies Extensibility Via integrations with external frameworks, including SELinux and AppArmor Can’t use external tools to define policies.

www.containerdays.io #CDS23 Security Context spec: securityContext: runAsUser: 1000 fsGroup: 2000 allowPrivilegeEscalation: false

www.containerdays.io #CDS23 Security Context apiVersion: v1 kind: Pod metadata: name: scd-3 spec: containers: - name: scd-3 image: nginx securityContext: capabilities: add: ["NET_ADMIN","SYS_TIME"]

www.containerdays.io #CDS23 KubeAudit https://github.com/Shopify/kubeaudit

www.containerdays.io #CDS23 ● New form of admission control is created with the understanding that Kubernetes users are probably going to seek external authorization. ● It can be deactivated partially or entirely to coexist with external admission controllers like OPA. ● KEP-2579: Pod Security Admission Control ● https://github.com/kubernetes/enhancements/blob/mast er/keps/sig-auth/2579-psp-replacement/README.md Pod Security Admission(PSA)

www.containerdays.io #CDS23 Pod Security Admission(PSA) ● Setting Default Security Constraints ● Fine-Grained Control over Policy Definition ● Sub-Namespace Policy Granularity

www.containerdays.io #CDS23 Pod Security Admission(PSA)

www.containerdays.io #CDS23 Pod Security Admission(PSA) ● Pod Security admission places requirements on a Pod's Security Context and other related fields according to the three levels defined by the Pod Security Standards: privileged, baseline, and restricted. ● spec.containers[*].ports ● spec.volumes[*].hostPath ● spec.securityContext ● spec.containers[*].securityContext

www.containerdays.io #CDS23 Pod Security Admission(PSA) kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 featureGates: PodSecurity: true nodes: - role: control-plane - role: worker

www.containerdays.io #CDS23 Pod Security Admission(PSA) Mode Description enforce Policy violations will cause the pod to be rejected. audit Policy violations will trigger the addition of an audit annotation to the event recorded in the audit log, but are otherwise allowed. warn Policy violations will trigger a user-facing warning, but are otherwise allowed.

www.containerdays.io #CDS23 Pod Security Admission(PSA)

www.containerdays.io #CDS23 Pod Security Admission(PSA) kubectl label --overwrite ns test-ns \ pod-security.kubernetes.io/warn=baseline \ pod-security.kubernetes.io/warn-version=v1.22

www.containerdays.io #CDS23 Pod Security Admission(PSA) ● It is consistent in deploying the security levels on namespaces by labels which helps with testing, troubleshooting and maintaining. ● Ability to perform dry runs using --dry-run=server before applying pod-security on namespace labels ● Provides validations for compliance with policies and will not change the pods to enforce compliance.

www.containerdays.io #CDS23 Pod Security Admission(PSA) $ kubectl label --dry-run=server --overwrite ns --all \ pod-security.kubernetes.io/enforce=baseline Warning: kuard: privileged namespace/default labeled namespace/kube-node-lease labeled namespace/kube-public labeled Warning: kube-proxy-vxjwb: host namespaces, hostPath volumes, privileged Warning: kube-proxy-zxqzz: host namespaces, hostPath volumes, privileged Warning: kube-apiserver-kind-control-plane: host namespaces, hostPath volumes Warning: etcd-kind-control-plane: host namespaces, hostPath volumes Warning: kube-controller-manager-kind-control-plane: host namespaces, hostPath volumes Warning: kindnet-cl5ln: non-default capabilities, host namespaces, hostPath volumes Warning: kube-scheduler-kind-control-plane: host namespaces, hostPath volumes Warning: kindnet-6ptww: non-default capabilities, host namespaces, hostPath volumes namespace/kube-system labeled namespace/local-path-storage labeled

www.containerdays.io #CDS23 Pod Security Admission(PSA) apiVersion: v1 kind: Namespace metadata: name: test-ns labels: pod-security.kubernetes.io/enforce: baseline pod-security.kubernetes.io/audit: restricted pod-security.kubernetes.io/warn: restricted

www.containerdays.io #CDS23 Pod Security Admission(PSA) apiVersion: v1 kind: Pod metadata: name: nginx spec: containers: - image: nginx name: nginx ports: - containerPort: 80

www.containerdays.io #CDS23 Pod Security Admission(PSA) $ kubectl apply -f pod.yaml Warning: would violate "latest" version of "restricted" PodSecurity profile: allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") pod/nginx created $ kubectl get pods NAME READY STATUS RESTARTS AGE nginx 1/1 Running 0 6s

www.containerdays.io #CDS23 Pod Security Admission(PSA) {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"808ca159-914c-43fa- b4c8-dee5cb2fc440","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/default/p ods?fieldManager=kubectl-create","verb":"create","user":{"username":"kubernetes-admin","grou ps":["system:masters","system:authenticated"]},"sourceIPs":["172.18.0.1"],"userAgent":"kubectl/ v1.22.0 (darwin/amd64) kubernetes/c2b5237","objectRef":{"resource":"pods","namespace":"default","name":"nginx","api Version":"v1"},"responseStatus":{"metadata":{},"code":201},"requestReceivedTimestamp":"2023 -08-21T03:30:26.605589Z","stageTimestamp":"2023-08-21T03:30:26.627123Z","annotations":{" authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"","pod-security.kubernetes .io/audit":"allowPrivilegeEscalation != false (container \"nginx\" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container \"nginx\" must set securityContext.capabilities.drop=[\"ALL\"]), runAsNonRoot != true (pod or container \"nginx\" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container \"nginx\" must set securityContext.seccompProfile.type to \"RuntimeDefault\" or \"Localhost\")"}}

www.containerdays.io #CDS23 Pod Security Admission(PSA) apiVersion: apiserver.config.k8s.io/v1 kind: AdmissionConfiguration plugins: - name: PodSecurity configuration: apiVersion: pod-security.admission.config.k8s.io/v1alpha1 kind: PodSecurityConfiguration defaults: enforce: "baseline" enforce-version: "latest" audit: "restricted" audit-version: "latest" warn: "restricted" warn-version: "latest" exemptions: usernames: [] runtimeClassNames: [] namespaces: [kube-system]

www.containerdays.io #CDS23 Policy-as-code (PAC) solutions

www.containerdays.io #CDS23 ● Policy agent for cloud-native authorization ● It provides a means of standardizing policy definition and management throughout the cloud-native technology stack. ● When combined with Kubernetes, OPA has the capability to enforce guardrails upon an entire system, requiring users’ permissions to match policy at all times.

www.containerdays.io #CDS23

www.containerdays.io #CDS23 ● Require specific labels on all resources. ● Require container images from the corporate image registry. ● Require all Pods specify resource requests and limits. ● Prevent conflicting Ingress objects from being created.

www.containerdays.io #CDS23

www.containerdays.io #CDS23

www.containerdays.io #CDS23

www.containerdays.io #CDS23 Pod Security Admission(PSA) vs Open Policy Agent(OPA) Pod Security Admission (PSA) Open Policy Agent (OPA) Simplicity Flexibility Native Integration Customization Performance External Control Limited Attack Surface Compliance

www.containerdays.io #CDS23 Pod Security Admission(PSA) vs Open Policy Agent(OPA) ● Which users can access which resources? ● Which subnets egress traffic is allowed to? ● Which clusters a workload must be deployed to? ● Which registries images can be downloaded from? ● Which capabilities a container can execute with? ● Which times of day the system can be accessed at?

www.containerdays.io #CDS23 Combination of different security strategies ● RBAC (Role-Based Access Control) ● PodSecurity Admission Controllers ● Network Policies ● Secrets Management ● Security Contexts ● Runtime Security

www.containerdays.io #CDS23 Access to resources in privileged and non-privileged mode ● Privileged Mode ● Non-Privileged Mode*

www.containerdays.io #CDS23 Access to resources in privileged and non-privileged mode ● Privileged Mode

www.containerdays.io #CDS23 Access to resources in privileged and non-privileged mode ● Non-Privileged Mode

www.containerdays.io #CDS23 Conclusions ● Security ● Functionality ● Isolation ● Attack Surface

www.containerdays.io #CDS23 ¡Thank you! @jmortegac https://www.linkedin.com/in/jmortega1 https://jmortega.github.io https://josemanuelortegablog.com