Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kubernetes Security Detection Engineering – Mapping Back To MITRE ATT&CK Matrix | HITB 2023 AMS

Kubernetes Security Detection Engineering – Mapping Back To MITRE ATT&CK Matrix | HITB 2023 AMS

Kubernetes has become a de facto way of running containerized workloads from startups to enterprises and governments, however like most modern technology, it’s not mature, especially in regards to security. Given its nature of being immutable and things happening in a matter of seconds, it’s super hard to perform security detection and incident response.

In this talk we will be focusing on the MITRE ATT&CK matrix for Kubernetes with showcasing what things can go wrong in different phases of the running container workloads, then we map back to what we should observe, collect, analyze, monitor, alert, and respond. We will showcase all the possible mappings of the matrix to the detection engineering. We will also cover some interesting real-world examples of hacks, known vulnerabilities, and misconfiguration. We will also showcase how we simulate these attacks in a controlled environment using the Kubernetes Goat project.

Madhu Akula

April 21, 2023
Tweet

More Decks by Madhu Akula

Other Decks in Technology

Transcript

  1. Kubernetes Security Detection
    Engineering - Mapping Back to
    MITRE ATT&CK Matrix
    HITB 2023 AMS
    Madhu Akula

    View full-size slide

  2. About Me - Madhu Akula
    👉 Pragmatic Security Leader, working on Cloud Native Infra, Security, and Startups
    👉 Creator of Kubernetes Goat, Hacker Container, tools.tldr.run, many other OSS projects.
    👉 Speaks & Trains at Black Hat (USA, EU, Asia), DEF CON, GitHub, USENIX, OWASP, All Day
    DevOps, SANS, DevSecCon, CNCF, c0c0n, Nullcon, null, many others around the globe.
    👉 Author of Security Automation with Ansible2, OWASP KSTG, whitepapers, etc.
    👉 Technical reviewer (multiple books) & Review board member of multiple conferences,
    organizations, communities, advisory, etc.
    👉 Found security vulnerabilities in 200+ organizations and products including Google,
    Microsoft, AT&T, Adobe, eBay, WordPress, Ntop, Cloudflare, Yahoo, LocalBitcoins, etc.
    👉 Certified Kubernetes(CKA/CKS), Offensive Security Certified Professional, etc.
    👉 Never ending learner!
    @madhuakula
    https://madhuakula.com

    View full-size slide

  3. What is Docker?
    https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/#going-back-in-time
    @madhuakula

    View full-size slide

  4. What is Docker?
    ● Docker is an open source platform for building, deploying, and managing
    containerized applications
    ● Docker became the de facto standard to build and share containerized apps -
    from desktop, to the cloud, even edge devices
    ● Docker enables developers to easily pack, ship, and run any application as a
    lightweight, portable, self-sufficient container, which can run virtually
    anywhere
    https://docs.docker.com/get-started/overview/
    @madhuakula

    View full-size slide

  5. What is Kubernetes?
    Kubernetes is a portable, extensible, open-source platform for
    managing containerized workloads and services, that facilitates
    both declarative configuration and automation. It has a large,
    rapidly growing ecosystem. Kubernetes services, support, and tools
    are widely available.
    https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/
    @madhuakula

    View full-size slide

  6. What is Kubernetes?
    @madhuakula

    View full-size slide

  7. The illustrated children's guide to Kubernetes
    https://www.youtube.com/watch?v=3I9PkvZ80BQ
    @madhuakula

    View full-size slide

  8. Why Kubernetes Security?
    @madhuakula
    Lack of
    knowledge in
    security teams

    View full-size slide

  9. Rapidly growing
    Cloud Native
    Landscape
    ecosystem
    Why Kubernetes Security?
    @madhuakula

    View full-size slide

  10. Technology Gap
    Adoption &
    Maturity
    Why Kubernetes Security?
    @madhuakula

    View full-size slide

  11. Why Kubernetes Security?
    @madhuakula

    View full-size slide

  12. MITRE ATT&CK for Kubernetes
    https://attack.mitre.orghttps//microsoft.github.io/Threat-Matrix-for-Kubernetes/
    @madhuakula

    View full-size slide

  13. Kubernetes Attack Path / Kill Chain
    @madhuakula

    View full-size slide

  14. Practical MITRE ATT&CK for Kubernetes - Attack Path
    https://youtu.be/7nc78ZrvP4Y
    @madhuakula

    View full-size slide

  15. Let’s map back Attack Path
    to
    Detection Engineering
    @madhuakula

    View full-size slide

  16. Defense In Depth - Layered Approach
    Some of the very high level abstraction layers, each layer contains many ways how
    we can secure and defend against attackers.
    ● Application Security
    ● Supply Chain Security
    ● Infrastructure Security
    ● Runtime Security
    ● Continuous Security
    @madhuakula

    View full-size slide

  17. Why Layered Approach?
    https://github.com/cncf/financial-user-group/blob/master/projects/k8s-threat-model/AttackTrees/AttackerOnTheNetwork.md
    Attackers have many ways! Defenders have many layers!
    @madhuakula

    View full-size slide

  18. @madhuakula
    🔥 Let’s focus on detecting
    Container Escape /
    Privilege Escalation

    View full-size slide

  19. There are many things we need to think as you seen it’s a layered approach!
    👉 Here are some of the things you can think of looking for detection from various
    perspectives of an attacker
    Detection - Container Escape / Privilege Escalation
    @madhuakula
    🛡 Standard Linux Server logging (SSH, System, Services, etc. /var/log/*)
    🛡 Container/Pod logs, and Node level system logging
    🛡 Components (Kubelet, Runtime, Middlewares, Entrypoints, etc.)
    🛡 Admission Controller/Audit Logging from Kubernetes
    🛡 Runtime Security logging (Things like Falco, Tetragon, etc. - SYSCALLs, PS, FIM, Net, Flow, etc.
    )
    🛡 [Near] Real-time Proactive Monitoring, Detection / Prevention

    View full-size slide

  20. Demo Time 🤞
    @madhuakula

    View full-size slide

  21. What just happened!
    @madhuakula https://madhuakula.com/kubernetes-goat/docs/scenarios/scenario-21/ebpf-runtime-security-monitoring-and-detection-i
    n-kubernetes-cluster-using-cilium-tetragon/welcome

    View full-size slide

  22. ✅ List of logs for Kubernetes Security Detection
    👉 API Server logs: Kubernetes API server, including audit logs, requests and responses, and
    authentication logs.
    👉 Kubernetes Audit Logs: Audit Kubernetes can provide insight into activities related to API server
    requests and resource changes.
    👉 Controller Manager logs: Kubernetes Controller Manager, including events, component status,
    and leader election.
    👉 etcd logs: Kubernetes etcd datastore, including request and response logs, snapshot creation, and
    cluster state changes.
    👉 Kubelet logs: Kubernetes Kubelet, including container logs, node status, and pod events.
    👉 kube-proxy logs: Kubernetes kube-proxy, including service proxying, health checks, and NAT
    operations.
    @madhuakula

    View full-size slide

  23. ✅ List of logs for Kubernetes Security Detection
    👉 Network policy logs: Kubernetes Network Policies, including rule matches and denied
    connections.
    👉 Pod logs: Kubernetes pods, including container logs, application logs, and any error or warning
    messages.
    👉 RBAC logs: Kubernetes Role-Based Access Control, including user authentication and
    authorization events.
    👉 Scheduler logs: Kubernetes scheduler, including scheduling decisions, pod status updates, and
    failed scheduling attempts.
    👉 Service Mesh Logs: Service mesh components like Istio or Linkerd can provide insight into
    network activity within the mesh, including service-to-service communication and security policies.
    @madhuakula

    View full-size slide

  24. ✅ List of logs for Kubernetes Security Detection
    👉 Container Runtime Logs: Container runtimes like Docker or containerd can provide insight into
    container activities such as process execution, file system operations, and network communication.
    👉 Container Network Interface (CNI) Plugin Logs: CNI plugins can provide visibility into network
    activities, including network policies, network connectivity, and network segmentation.
    👉 Host System Logs: The host system can provide insight into activities related to system-level
    events, including process creation, file system operations, and network communication.
    👉 Application Logs: Applications running inside containers can provide insight into application-level
    activities, including user actions and application-level vulnerabilities.
    👉 Many other context related Telemetry (Open Telemetry), Tracing data is a gold mine of
    information for the detection engineering to make the most out of the systems
    @madhuakula

    View full-size slide

  25. How can we learn and
    practice to defend against
    MITRE ATT&CK ?
    @madhuakula

    View full-size slide

  26. Kubernetes Goat is an interactive
    Kubernetes security learning playground.
    Intentionally vulnerable by design scenarios
    to showcase the common
    misconfigurations, real-world
    vulnerabilities, and security issues in
    Kubernetes clusters, containers, and cloud
    native environments.
    What is Kubernetes Goat 🐐
    @madhuakula

    View full-size slide

  27. ⚡ Get Started with Kubernetes Goat 🐐
    https://madhuakula.com/kubernetes-goat
    @madhuakula

    View full-size slide

  28. 🔥 Kubernetes Goat Audience
    💥 Attackers & Red Teams 🛡 Defenders & Blue Teams
    🧰 Products & Vendors
    🔐 Developers & DevOps Teams
    💡 Interested in Kubernetes Security
    @madhuakula

    View full-size slide

  29. 🚀 Scenarios in Kubernetes Goat
    1. Sensitive keys in codebases
    2. DIND (docker-in-docker) exploitation
    3. SSRF in the Kubernetes (K8S) world
    4. Container escape to the host system
    5. Docker CIS benchmarks analysis
    6. Kubernetes CIS benchmarks analysis
    7. Attacking private registry
    8. NodePort exposed services
    9. Helm v2 tiller to PwN the cluster - [Deprecated]
    10. Analyzing crypto miner container
    11. Kubernetes namespaces bypass
    12. Gaining environment information
    13. DoS the Memory/CPU resources
    14. Hacker container preview
    15. Hidden in layers
    16. RBAC least privileges misconfiguration
    17. KubeAudit - Audit Kubernetes clusters
    18. Falco - Runtime security monitoring & detection
    19. Popeye - A Kubernetes cluster sanitizer
    20. Secure network boundaries using NSP
    21. Cilium Tetragon - eBPF-based Security Observability
    and Runtime Enforcement
    22. Securing Kubernetes Clusters using Kyverno Policy
    Engine
    More scenarios releasing soon… ❤
    @madhuakula

    View full-size slide

  30. 🧰 How can I setup Kubernetes Goat
    ☸ Vanilla Kubernetes Cluster
    ☁ AWS Kubernetes (EKS)
    ☁ GCP Kubernetes (GKE)
    ☁ Azure Kubernetes (AKS)
    ☸ Kubernetes IN Docker (KiND)
    ☸ Lightweight Kubernetes (K3S)
    ☸ Digital Ocean, Vagrant, Many others…
    @madhuakula

    View full-size slide

  31. ⎈ Setting up in your Kubernetes Cluster
    ● Make sure you have Kubernetes cluster with cluster-admin privileges. Also kubectl and helm
    installed in your system before running the following commands to setup the Kubernetes
    Goat
    ● Now you can access the Kubernetes Goat by navigating to http://127.0.0.1:1234
    @madhuakula

    View full-size slide

  32. ⚡ Get Started with Kubernetes Goat 🐐
    @madhuakula

    View full-size slide

  33. ⚡ Get Started with Kubernetes Goat 🐐
    @madhuakula

    View full-size slide

  34. 🔟 OWASP Kubernetes Top 10
    https://owasp.org/www-project-kubernetes-top-ten/
    @madhuakula

    View full-size slide

  35. 🛡 MITRE ATT&CK for Kubernetes Goat
    https://madhuakula.com/kubernetes-goat/docs/mitre/mitre-attack
    @madhuakula

    View full-size slide

  36. 🥳 Adoption of Kubernetes Goat
    https://youtu.be/62_Cj6yseno?t=352
    @madhuakula

    View full-size slide

  37. Spread the ❤ #KubernetesGoat
    🙌 Give it a try
    🚀 Contribute ideas & suggestions
    🤝 Work with the project & improve
    🙏 Share your valuable feedback
    🌟 Star in GitHub
    🎉 Spread word #KubernetesGoat
    @madhuakula

    View full-size slide

  38. Thank you!
    https://madhuakula.com
    @madhuakula

    View full-size slide