Evolution of security strategies in K8s environments

In the latest versions of K8s there has been an evolution regarding the definition of security strategies at the level of access policies to the cluster by users and developers. The security contexts (securityContext) allow you to define the configurations at the level of access control and privileges for a pod or container in a simple way using keywords in the configuration files.
To facilitate the implementation of these security strategies throughout the cluster, new strategies have emerged such as the Pod Security Policy (PSP) where the cluster administrator is in charge of defining these policies at the cluster level with the aim that developers can follow these policies.
Other interesting projects include Open Policy Agent (OPA) as the main cloud-native authorization policy agent for creating policies and managing user permissions for access to applications.
The objective of this talk is to present the evolution that has occurred in security strategies and how we could use them together, as well as analyze their behavior in accessing resources. Among the points to be discussed we can highlight:
*Introduction to security strategies in K8s environments
*Pod Security Admission(PSA) vs Open Policy Agent (OPA)
*Combination of different security strategies together
*Access to resources in privileged and non-privileged mode

Transcript

1. www.containerdays.io
   #CDS23
   Evolution of security strategies
   in K8s environments
   

   View Slide

2. www.containerdays.io
   #CDS23
   Agenda
   ● Introduction to security strategies in K8s
   environments
   ● Pod Security Admission(PSA) vs Open Policy Agent
   (OPA)
   ● Combination of different security strategies together
   ● Access to resources in privileged and non-privileged
   mode
   

   View Slide

3. www.containerdays.io
   #CDS23
   Introduction to security strategies in K8s environments
   ● Cluster Hardening: Implement best practices for securing
   the Kubernetes cluster itself, including securing access to
   the API server, enabling RBAC (Role-Based Access
   Control), and using network policies to control
   communication between pods.
   ● Pod Security Policies (PSP): Enforce security policies that
   define what a pod can and cannot do, including limiting
   privilege levels, host access, and running as non-root users.
   

   View Slide

4. www.containerdays.io
   #CDS23
   Introduction to security strategies in K8s environments
   ● Secrets Management: Use Kubernetes Secrets to
   store sensitive information securely, such as API
   keys, passwords, or certificates.
   ● Role-Based Access Control (RBAC): Define
   fine-grained access controls for users and service
   accounts to limit the scope of actions they can
   perform within the cluster.
   

   View Slide

5. www.containerdays.io
   #CDS23
   Introduction to security strategies in K8s environments
   ● Limit Resource Consumption: Set resource quotas
   to limit the amount of CPU, memory, and other
   resources that can be consumed by pods, preventing
   resource exhaustion and potential denial-of-service
   attacks.
   ● Pod Security Context: Use pod security context to
   define security settings at the pod level, such as user
   and group IDs, SELinux, and file system permissions.
   

   View Slide

6. www.containerdays.io
   #CDS23
   Introduction to security strategies in K8s environments
   ● PodSecurityPolicy has been deprecated from
   Kubernetes 1.21.
   

   View Slide

7. www.containerdays.io
   #CDS23
   Introduction to security strategies in K8s environments
   ● PodSecurityContext, the Kubernetes tool which
   allows users to specify security contexts and how the
   pod will be execute.
   

   View Slide

8. www.containerdays.io
   #CDS23
   Introduction to security strategies in K8s environments
   Security Contexts RBAC (Role-Based
   Access Control)
   Resource scope Pods Pods, Nodes, cluster
   Actions Predefined capabilities RBAC policies
   Extensibility Via integrations with
   external frameworks,
   including SELinux and
   AppArmor
   Can’t use external
   tools to define
   policies.
   

   View Slide

9. www.containerdays.io
   #CDS23
   Security Context
   spec:
   securityContext:
   runAsUser: 1000
   fsGroup: 2000
   allowPrivilegeEscalation: false
   

   View Slide

10. www.containerdays.io
    #CDS23
    Security Context
    apiVersion: v1
    kind: Pod
    metadata:
    name: scd-3
    spec:
    containers:
    - name: scd-3
    image: nginx
    securityContext:
    capabilities:
    add: ["NET_ADMIN","SYS_TIME"]
    

    View Slide

11. www.containerdays.io
    #CDS23
    KubeAudit https://github.com/Shopify/kubeaudit
    

    View Slide

12. www.containerdays.io
    #CDS23
    ● New form of admission control is created with the
    understanding that Kubernetes users are probably going
    to seek external authorization.
    ● It can be deactivated partially or entirely to coexist with
    external admission controllers like OPA.
    ● KEP-2579: Pod Security Admission Control
    ● https://github.com/kubernetes/enhancements/blob/mast
    er/keps/sig-auth/2579-psp-replacement/README.md
    Pod Security Admission(PSA)
    

    View Slide

13. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    ● Setting Default Security Constraints
    ● Fine-Grained Control over Policy Definition
    ● Sub-Namespace Policy Granularity
    

    View Slide

14. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    

    View Slide

15. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    ● Pod Security admission places requirements on a Pod's
    Security Context and other related fields according to the
    three levels defined by the Pod Security Standards:
    privileged, baseline, and restricted.
    ● spec.containers[*].ports
    ● spec.volumes[*].hostPath
    ● spec.securityContext
    ● spec.containers[*].securityContext
    

    View Slide

16. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    kind: Cluster
    apiVersion: kind.x-k8s.io/v1alpha4
    featureGates:
    PodSecurity: true
    nodes:
    - role: control-plane
    - role: worker
    

    View Slide

17. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    Mode Description
    enforce Policy violations will cause the pod to be
    rejected.
    audit Policy violations will trigger the addition of an
    audit annotation to the event recorded in the
    audit log, but are otherwise allowed.
    warn Policy violations will trigger a user-facing
    warning, but are otherwise allowed.
    

    View Slide

18. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    

    View Slide

19. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    kubectl label --overwrite ns test-ns \
    pod-security.kubernetes.io/warn=baseline \
    pod-security.kubernetes.io/warn-version=v1.22
    

    View Slide

20. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    ● It is consistent in deploying the security levels on
    namespaces by labels which helps with testing,
    troubleshooting and maintaining.
    ● Ability to perform dry runs using --dry-run=server
    before applying pod-security on namespace labels
    ● Provides validations for compliance with policies and
    will not change the pods to enforce compliance.
    

    View Slide

21. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    $ kubectl label --dry-run=server --overwrite ns --all \
    pod-security.kubernetes.io/enforce=baseline
    Warning: kuard: privileged
    namespace/default labeled
    namespace/kube-node-lease labeled
    namespace/kube-public labeled
    Warning: kube-proxy-vxjwb: host namespaces, hostPath volumes, privileged
    Warning: kube-proxy-zxqzz: host namespaces, hostPath volumes, privileged
    Warning: kube-apiserver-kind-control-plane: host namespaces, hostPath volumes
    Warning: etcd-kind-control-plane: host namespaces, hostPath volumes
    Warning: kube-controller-manager-kind-control-plane: host namespaces, hostPath volumes
    Warning: kindnet-cl5ln: non-default capabilities, host namespaces, hostPath volumes
    Warning: kube-scheduler-kind-control-plane: host namespaces, hostPath volumes
    Warning: kindnet-6ptww: non-default capabilities, host namespaces, hostPath volumes
    namespace/kube-system labeled
    namespace/local-path-storage labeled
    

    View Slide

22. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    apiVersion: v1
    kind: Namespace
    metadata:
    name: test-ns
    labels:
    pod-security.kubernetes.io/enforce: baseline
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted
    

    View Slide

23. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    apiVersion: v1
    kind: Pod
    metadata:
    name: nginx
    spec:
    containers:
    - image: nginx
    name: nginx
    ports:
    - containerPort: 80
    

    View Slide

24. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    $ kubectl apply -f pod.yaml
    Warning: would violate "latest" version of "restricted" PodSecurity profile:
    allowPrivilegeEscalation != false (container "nginx" must set
    securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container
    "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true
    (pod or container "nginx" must set securityContext.runAsNonRoot=true),
    seccompProfile (pod or container "nginx" must set
    securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
    pod/nginx created
    $ kubectl get pods
    NAME READY STATUS RESTARTS AGE
    nginx 1/1 Running 0 6s
    

    View Slide

25. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"808ca159-914c-43fa-
    b4c8-dee5cb2fc440","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/default/p
    ods?fieldManager=kubectl-create","verb":"create","user":{"username":"kubernetes-admin","grou
    ps":["system:masters","system:authenticated"]},"sourceIPs":["172.18.0.1"],"userAgent":"kubectl/
    v1.22.0 (darwin/amd64)
    kubernetes/c2b5237","objectRef":{"resource":"pods","namespace":"default","name":"nginx","api
    Version":"v1"},"responseStatus":{"metadata":{},"code":201},"requestReceivedTimestamp":"2023
    -08-21T03:30:26.605589Z","stageTimestamp":"2023-08-21T03:30:26.627123Z","annotations":{"
    authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"","pod-security.kubernetes
    .io/audit":"allowPrivilegeEscalation != false (container \"nginx\" must set
    securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container \"nginx\"
    must set securityContext.capabilities.drop=[\"ALL\"]), runAsNonRoot != true (pod or container
    \"nginx\" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container
    \"nginx\" must set securityContext.seccompProfile.type to \"RuntimeDefault\" or
    \"Localhost\")"}}
    

    View Slide

26. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    apiVersion: apiserver.config.k8s.io/v1
    kind: AdmissionConfiguration
    plugins:
    - name: PodSecurity
    configuration:
    apiVersion: pod-security.admission.config.k8s.io/v1alpha1
    kind: PodSecurityConfiguration
    defaults:
    enforce: "baseline"
    enforce-version: "latest"
    audit: "restricted"
    audit-version: "latest"
    warn: "restricted"
    warn-version: "latest"
    exemptions:
    usernames: []
    runtimeClassNames: []
    namespaces: [kube-system]
    

    View Slide

27. www.containerdays.io
    #CDS23
    Policy-as-code (PAC) solutions
    

    View Slide

28. www.containerdays.io
    #CDS23
    ● Policy agent for cloud-native authorization
    ● It provides a means of standardizing policy definition
    and management throughout the cloud-native
    technology stack.
    ● When combined with Kubernetes, OPA has the
    capability to enforce guardrails upon an entire
    system, requiring users’ permissions to match policy
    at all times.
    

    View Slide

29. www.containerdays.io
    #CDS23
    

    View Slide

30. www.containerdays.io
    #CDS23
    ● Require specific labels on all resources.
    ● Require container images from the corporate image
    registry.
    ● Require all Pods specify resource requests and limits.
    ● Prevent conflicting Ingress objects from being created.
    

    View Slide

31. www.containerdays.io
    #CDS23
    

    View Slide

32. www.containerdays.io
    #CDS23
    

    View Slide

33. www.containerdays.io
    #CDS23
    

    View Slide

34. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA) vs Open Policy Agent(OPA)
    Pod Security Admission (PSA) Open Policy Agent (OPA)
    Simplicity Flexibility
    Native Integration Customization
    Performance External Control
    Limited Attack Surface Compliance
    

    View Slide

35. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA) vs Open Policy Agent(OPA)
    ● Which users can access which resources?
    ● Which subnets egress traffic is allowed to?
    ● Which clusters a workload must be deployed to?
    ● Which registries images can be downloaded from?
    ● Which capabilities a container can execute with?
    ● Which times of day the system can be accessed at?
    

    View Slide

36. www.containerdays.io
    #CDS23
    Combination of different security strategies
    ● RBAC (Role-Based Access Control)
    ● PodSecurity Admission Controllers
    ● Network Policies
    ● Secrets Management
    ● Security Contexts
    ● Runtime Security
    

    View Slide

37. www.containerdays.io
    #CDS23
    Access to resources in privileged and non-privileged mode
    ● Privileged Mode
    ● Non-Privileged Mode*
    

    View Slide

38. www.containerdays.io
    #CDS23
    Access to resources in privileged and non-privileged mode
    ● Privileged Mode
    

    View Slide

39. www.containerdays.io
    #CDS23
    Access to resources in privileged and non-privileged mode
    ● Non-Privileged Mode
    

    View Slide

40. www.containerdays.io
    #CDS23
    Conclusions
    ● Security
    ● Functionality
    ● Isolation
    ● Attack Surface
    

    View Slide

41. www.containerdays.io
    #CDS23
    ¡Thank you!
    @jmortegac
    https://www.linkedin.com/in/jmortega1
    https://jmortega.github.io
    https://josemanuelortegablog.com
    

    View Slide