$30 off During Our Annual Pro Sale. View Details »

Evolution of security strategies in K8s environments

jmortegac
September 13, 2023

Evolution of security strategies in K8s environments

In the latest versions of K8s there has been an evolution regarding the definition of security strategies at the level of access policies to the cluster by users and developers. The security contexts (securityContext) allow you to define the configurations at the level of access control and privileges for a pod or container in a simple way using keywords in the configuration files.
To facilitate the implementation of these security strategies throughout the cluster, new strategies have emerged such as the Pod Security Policy (PSP) where the cluster administrator is in charge of defining these policies at the cluster level with the aim that developers can follow these policies.
Other interesting projects include Open Policy Agent (OPA) as the main cloud-native authorization policy agent for creating policies and managing user permissions for access to applications.
The objective of this talk is to present the evolution that has occurred in security strategies and how we could use them together, as well as analyze their behavior in accessing resources. Among the points to be discussed we can highlight:
*Introduction to security strategies in K8s environments
*Pod Security Admission(PSA) vs Open Policy Agent (OPA)
*Combination of different security strategies together
*Access to resources in privileged and non-privileged mode

jmortegac

September 13, 2023
Tweet

More Decks by jmortegac

Other Decks in Technology

Transcript

  1. www.containerdays.io
    #CDS23
    Evolution of security strategies
    in K8s environments

    View Slide

  2. www.containerdays.io
    #CDS23
    Agenda
    ● Introduction to security strategies in K8s
    environments
    ● Pod Security Admission(PSA) vs Open Policy Agent
    (OPA)
    ● Combination of different security strategies together
    ● Access to resources in privileged and non-privileged
    mode

    View Slide

  3. www.containerdays.io
    #CDS23
    Introduction to security strategies in K8s environments
    ● Cluster Hardening: Implement best practices for securing
    the Kubernetes cluster itself, including securing access to
    the API server, enabling RBAC (Role-Based Access
    Control), and using network policies to control
    communication between pods.
    ● Pod Security Policies (PSP): Enforce security policies that
    define what a pod can and cannot do, including limiting
    privilege levels, host access, and running as non-root users.

    View Slide

  4. www.containerdays.io
    #CDS23
    Introduction to security strategies in K8s environments
    ● Secrets Management: Use Kubernetes Secrets to
    store sensitive information securely, such as API
    keys, passwords, or certificates.
    ● Role-Based Access Control (RBAC): Define
    fine-grained access controls for users and service
    accounts to limit the scope of actions they can
    perform within the cluster.

    View Slide

  5. www.containerdays.io
    #CDS23
    Introduction to security strategies in K8s environments
    ● Limit Resource Consumption: Set resource quotas
    to limit the amount of CPU, memory, and other
    resources that can be consumed by pods, preventing
    resource exhaustion and potential denial-of-service
    attacks.
    ● Pod Security Context: Use pod security context to
    define security settings at the pod level, such as user
    and group IDs, SELinux, and file system permissions.

    View Slide

  6. www.containerdays.io
    #CDS23
    Introduction to security strategies in K8s environments
    ● PodSecurityPolicy has been deprecated from
    Kubernetes 1.21.

    View Slide

  7. www.containerdays.io
    #CDS23
    Introduction to security strategies in K8s environments
    ● PodSecurityContext, the Kubernetes tool which
    allows users to specify security contexts and how the
    pod will be execute.

    View Slide

  8. www.containerdays.io
    #CDS23
    Introduction to security strategies in K8s environments
    Security Contexts RBAC (Role-Based
    Access Control)
    Resource scope Pods Pods, Nodes, cluster
    Actions Predefined capabilities RBAC policies
    Extensibility Via integrations with
    external frameworks,
    including SELinux and
    AppArmor
    Can’t use external
    tools to define
    policies.

    View Slide

  9. www.containerdays.io
    #CDS23
    Security Context
    spec:
    securityContext:
    runAsUser: 1000
    fsGroup: 2000
    allowPrivilegeEscalation: false

    View Slide

  10. www.containerdays.io
    #CDS23
    Security Context
    apiVersion: v1
    kind: Pod
    metadata:
    name: scd-3
    spec:
    containers:
    - name: scd-3
    image: nginx
    securityContext:
    capabilities:
    add: ["NET_ADMIN","SYS_TIME"]

    View Slide

  11. www.containerdays.io
    #CDS23
    KubeAudit https://github.com/Shopify/kubeaudit

    View Slide

  12. www.containerdays.io
    #CDS23
    ● New form of admission control is created with the
    understanding that Kubernetes users are probably going
    to seek external authorization.
    ● It can be deactivated partially or entirely to coexist with
    external admission controllers like OPA.
    ● KEP-2579: Pod Security Admission Control
    ● https://github.com/kubernetes/enhancements/blob/mast
    er/keps/sig-auth/2579-psp-replacement/README.md
    Pod Security Admission(PSA)

    View Slide

  13. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    ● Setting Default Security Constraints
    ● Fine-Grained Control over Policy Definition
    ● Sub-Namespace Policy Granularity

    View Slide

  14. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)

    View Slide

  15. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    ● Pod Security admission places requirements on a Pod's
    Security Context and other related fields according to the
    three levels defined by the Pod Security Standards:
    privileged, baseline, and restricted.
    ● spec.containers[*].ports
    ● spec.volumes[*].hostPath
    ● spec.securityContext
    ● spec.containers[*].securityContext

    View Slide

  16. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    kind: Cluster
    apiVersion: kind.x-k8s.io/v1alpha4
    featureGates:
    PodSecurity: true
    nodes:
    - role: control-plane
    - role: worker

    View Slide

  17. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    Mode Description
    enforce Policy violations will cause the pod to be
    rejected.
    audit Policy violations will trigger the addition of an
    audit annotation to the event recorded in the
    audit log, but are otherwise allowed.
    warn Policy violations will trigger a user-facing
    warning, but are otherwise allowed.

    View Slide

  18. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)

    View Slide

  19. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    kubectl label --overwrite ns test-ns \
    pod-security.kubernetes.io/warn=baseline \
    pod-security.kubernetes.io/warn-version=v1.22

    View Slide

  20. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    ● It is consistent in deploying the security levels on
    namespaces by labels which helps with testing,
    troubleshooting and maintaining.
    ● Ability to perform dry runs using --dry-run=server
    before applying pod-security on namespace labels
    ● Provides validations for compliance with policies and
    will not change the pods to enforce compliance.

    View Slide

  21. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    $ kubectl label --dry-run=server --overwrite ns --all \
    pod-security.kubernetes.io/enforce=baseline
    Warning: kuard: privileged
    namespace/default labeled
    namespace/kube-node-lease labeled
    namespace/kube-public labeled
    Warning: kube-proxy-vxjwb: host namespaces, hostPath volumes, privileged
    Warning: kube-proxy-zxqzz: host namespaces, hostPath volumes, privileged
    Warning: kube-apiserver-kind-control-plane: host namespaces, hostPath volumes
    Warning: etcd-kind-control-plane: host namespaces, hostPath volumes
    Warning: kube-controller-manager-kind-control-plane: host namespaces, hostPath volumes
    Warning: kindnet-cl5ln: non-default capabilities, host namespaces, hostPath volumes
    Warning: kube-scheduler-kind-control-plane: host namespaces, hostPath volumes
    Warning: kindnet-6ptww: non-default capabilities, host namespaces, hostPath volumes
    namespace/kube-system labeled
    namespace/local-path-storage labeled

    View Slide

  22. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    apiVersion: v1
    kind: Namespace
    metadata:
    name: test-ns
    labels:
    pod-security.kubernetes.io/enforce: baseline
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

    View Slide

  23. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    apiVersion: v1
    kind: Pod
    metadata:
    name: nginx
    spec:
    containers:
    - image: nginx
    name: nginx
    ports:
    - containerPort: 80

    View Slide

  24. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    $ kubectl apply -f pod.yaml
    Warning: would violate "latest" version of "restricted" PodSecurity profile:
    allowPrivilegeEscalation != false (container "nginx" must set
    securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container
    "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true
    (pod or container "nginx" must set securityContext.runAsNonRoot=true),
    seccompProfile (pod or container "nginx" must set
    securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
    pod/nginx created
    $ kubectl get pods
    NAME READY STATUS RESTARTS AGE
    nginx 1/1 Running 0 6s

    View Slide

  25. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"808ca159-914c-43fa-
    b4c8-dee5cb2fc440","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/default/p
    ods?fieldManager=kubectl-create","verb":"create","user":{"username":"kubernetes-admin","grou
    ps":["system:masters","system:authenticated"]},"sourceIPs":["172.18.0.1"],"userAgent":"kubectl/
    v1.22.0 (darwin/amd64)
    kubernetes/c2b5237","objectRef":{"resource":"pods","namespace":"default","name":"nginx","api
    Version":"v1"},"responseStatus":{"metadata":{},"code":201},"requestReceivedTimestamp":"2023
    -08-21T03:30:26.605589Z","stageTimestamp":"2023-08-21T03:30:26.627123Z","annotations":{"
    authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"","pod-security.kubernetes
    .io/audit":"allowPrivilegeEscalation != false (container \"nginx\" must set
    securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container \"nginx\"
    must set securityContext.capabilities.drop=[\"ALL\"]), runAsNonRoot != true (pod or container
    \"nginx\" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container
    \"nginx\" must set securityContext.seccompProfile.type to \"RuntimeDefault\" or
    \"Localhost\")"}}

    View Slide

  26. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA)
    apiVersion: apiserver.config.k8s.io/v1
    kind: AdmissionConfiguration
    plugins:
    - name: PodSecurity
    configuration:
    apiVersion: pod-security.admission.config.k8s.io/v1alpha1
    kind: PodSecurityConfiguration
    defaults:
    enforce: "baseline"
    enforce-version: "latest"
    audit: "restricted"
    audit-version: "latest"
    warn: "restricted"
    warn-version: "latest"
    exemptions:
    usernames: []
    runtimeClassNames: []
    namespaces: [kube-system]

    View Slide

  27. www.containerdays.io
    #CDS23
    Policy-as-code (PAC) solutions

    View Slide

  28. www.containerdays.io
    #CDS23
    ● Policy agent for cloud-native authorization
    ● It provides a means of standardizing policy definition
    and management throughout the cloud-native
    technology stack.
    ● When combined with Kubernetes, OPA has the
    capability to enforce guardrails upon an entire
    system, requiring users’ permissions to match policy
    at all times.

    View Slide

  29. www.containerdays.io
    #CDS23

    View Slide

  30. www.containerdays.io
    #CDS23
    ● Require specific labels on all resources.
    ● Require container images from the corporate image
    registry.
    ● Require all Pods specify resource requests and limits.
    ● Prevent conflicting Ingress objects from being created.

    View Slide

  31. www.containerdays.io
    #CDS23

    View Slide

  32. www.containerdays.io
    #CDS23

    View Slide

  33. www.containerdays.io
    #CDS23

    View Slide

  34. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA) vs Open Policy Agent(OPA)
    Pod Security Admission (PSA) Open Policy Agent (OPA)
    Simplicity Flexibility
    Native Integration Customization
    Performance External Control
    Limited Attack Surface Compliance

    View Slide

  35. www.containerdays.io
    #CDS23
    Pod Security Admission(PSA) vs Open Policy Agent(OPA)
    ● Which users can access which resources?
    ● Which subnets egress traffic is allowed to?
    ● Which clusters a workload must be deployed to?
    ● Which registries images can be downloaded from?
    ● Which capabilities a container can execute with?
    ● Which times of day the system can be accessed at?

    View Slide

  36. www.containerdays.io
    #CDS23
    Combination of different security strategies
    ● RBAC (Role-Based Access Control)
    ● PodSecurity Admission Controllers
    ● Network Policies
    ● Secrets Management
    ● Security Contexts
    ● Runtime Security

    View Slide

  37. www.containerdays.io
    #CDS23
    Access to resources in privileged and non-privileged mode
    ● Privileged Mode
    ● Non-Privileged Mode*

    View Slide

  38. www.containerdays.io
    #CDS23
    Access to resources in privileged and non-privileged mode
    ● Privileged Mode

    View Slide

  39. www.containerdays.io
    #CDS23
    Access to resources in privileged and non-privileged mode
    ● Non-Privileged Mode

    View Slide

  40. www.containerdays.io
    #CDS23
    Conclusions
    ● Security
    ● Functionality
    ● Isolation
    ● Attack Surface

    View Slide

  41. www.containerdays.io
    #CDS23
    ¡Thank you!
    @jmortegac
    https://www.linkedin.com/in/jmortega1
    https://jmortega.github.io
    https://josemanuelortegablog.com

    View Slide