Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ship of Fools: Shoring Up Kubernetes Security

Ship of Fools: Shoring Up Kubernetes Security

Hackers gonna hack. They have their own motivations, and they don’t care about your constraints. As attackers, they want to find vulnerabilities and exploit them. As a defender, your mission is to stop them. Mistakes can be easy to make, but with the right configuration and attention to security best practices many attacks can be prevented.

This talk will give you practical advice about securing your Kubernetes clusters, from an attacker’s perspective. We’ll walk through the attack process from discovery to post-exploitation, and you’ll walk away with tools and techniques that can be used for prevention along the way. Learn how to keep your infrastructure safer by making a hacker’s job harder.

Ian Coldwater

July 13, 2018
Tweet

More Decks by Ian Coldwater

Other Decks in Technology

Transcript

  1. SHIP OF FOOLS
    Shoring Up Kubernetes Security
    @IanColdwater

    View full-size slide

  2. WHO AM I?
    My name is Ian Coldwater.
    I do DevSecOps at Jamf Software, where I
    focus on container security and hardening
    cloud infrastructure.
    I’m also an ethical hacker.
    @IanColdwater

    View full-size slide

  3. AS AN ATTACKER, I WANT TO PWN YOU.
    @IanColdwater

    View full-size slide

  4. WHAT'S A CONTAINER?
    @IanColdwater

    View full-size slide

  5. WHAT IS KUBERNETES?
    • Kubernetes is an open-source system for automating deployment, scaling, and
    management of containerized applications. It groups containers that make up an
    application into logical units for easy management and discovery.
    @IanColdwater

    View full-size slide

  6. @IanColdwater

    View full-size slide

  7. @IanColdwater

    View full-size slide

  8. EVEN HACKERS AREN'T IMMUNE TO THIS
    @IanColdwater https://hackernoon.com/capturing-all-the-flags-in-bsidessf-ctf-by-pwning-our-infrastructure-3570b99b4dd0

    View full-size slide

  9. SOME CONSIDERATIONS
    • Kubernetes has a very active community of contributors who are pushing changes very
    quickly. Security features have vastly improved in the last several releases.
    • Older Kubernetes releases are still commonly found in production, and left a lot wide
    open by default.
    • Attack and defense can vary with individual configurations.
    @IanColdwater

    View full-size slide

  10. GOOD NEWS AND BAD NEWS
    • It is possible to secure your Kubernetes cluster.
    • It's probably not going to come that way by default.
    • Security doesn't end there.
    @IanColdwater

    View full-size slide

  11. @IanColdwater

    View full-size slide

  12. WHAT IS YOUR THREAT MODEL?
    • What are you trying to protect?
    • Who are you trying to protect it from?
    • What capabilities do your adversaries have?
    • What capabilities do you have to defend against them?
    @IanColdwater

    View full-size slide

  13. KUBERNETES THREAT MODEL
    • External attacker
    • Application or container compromise
    • Compromised user or credentials
    @IanColdwater

    View full-size slide

  14. AN ATTACKER'S WORKFLOW
    • Discovery
    • Enumeration
    • Getting In
    • Post-Exploitation
    • Exfiltration
    • Privilege Escalation
    • Lateral Movement
    • Persistence?
    • Rinse and Repeat
    @IanColdwater

    View full-size slide

  15. EXTERNALLY VISIBLE PORTS
    • 2379/tcp open | etcd
    • 4194/tcp open | cAdvisor
    • 443/tcp open | API Server (sometimes this is port 6443 or 8443)
    • 8080/tcp open | Insecure API Server
    • 10250/tcp open | kubelet
    • 10255/tcp open | kubelet (read only)
    • ???/open | various network plugins
    @IanColdwater

    View full-size slide

  16. @IanColdwater

    View full-size slide

  17. PREVENTING EXTERNAL ATTACKERS
    • Don't leave your ports open if you don't have to
    • Make sure that all management ports that are visible externally require authentication
    • Limit SSH access to Kubernetes nodes
    @IanColdwater

    View full-size slide

  18. DEFENDERS THINK IN LISTS.
    ATTACKERS THINK IN GRAPHS.
    What’s in your graph?
    19
    @IanColdwater

    View full-size slide

  19. USER OR CREDENTIAL COMPROMISE
    @IanColdwater

    View full-size slide

  20. PREVENTING USER COMPROMISE
    • Don’t fall for social engineering!
    • Keep credentials encrypted and limit access to them.
    @IanColdwater

    View full-size slide

  21. CONTAINER COMPROMISE
    @IanColdwater

    View full-size slide

  22. KUBERNETES CONTROL PLANE
    @IanColdwater

    View full-size slide

  23. CONTAINER BREAKOUTS
    shit just got real
    @IanColdwater

    View full-size slide

  24. PREVENTING CONTAINER COMPROMISE
    • Write your own applications as securely as possible.
    • Treat other people’s code with caution!
    • Run static code analysis on your applications and containers to check for vulnerabilities.
    You can use open source tools for this such as Clair by CoreOS.
    • If you find vulnerabilities, patch or mitigate them.
    @IanColdwater

    View full-size slide

  25. DEFENSE IN DEPTH
    • Reduce your attack surface
    • Limit your blast radius
    @IanColdwater

    View full-size slide

  26. PRINCIPLE OF LEAST PRIVILEGE
    @IanColdwater

    View full-size slide

  27. CONTROLS
    • Network Policies
    • Admission Controllers
    • Role-Based Access Control
    • Pod Security Policies
    • Resource Quotas
    • Logging and Monitoring
    @IanColdwater

    View full-size slide

  28. GENERAL RECOMMENDATIONS
    • If you can upgrade, upgrade. If you can’t upgrade, mitigate.
    • Secure defaults are very important!
    • Be careful with your secrets
    • Log and monitor…outside your cluster.
    @IanColdwater

    View full-size slide

  29. TL;DR
    • A lot of this is standard security advice.
    • Practice good cyber hygiene, and get the basics right!
    @IanColdwater

    View full-size slide

  30. YOU GOT THIS!
    I believe in you!
    @IanColdwater

    View full-size slide

  31. RESOURCES
    • securing your cluster - goo.gl/gE9sj6
    • hacking and hardening kubernetes by
    example - goo.gl/QJhsDb
    • a hacker's guide to kubernetes and the cloud
    - goo.gl/CZgG6V
    • preventing attacks at scale - goo.gl/Y7EeU7
    • shipping in pirate-infested waters - 

    goo.gl/TRwXCu
    • CIS benchmarks - goo.gl/v2ZWXR
    • lessons from the cryptojacking attack at tesla -
    goo.gl/u4ucvo
    • analysis of a kubernetes hack - goo.gl/VKteVq
    • attackers think in graphs - goo.gl/imExzC
    • github.com/kelseyhightower/nocode - the
    best way to write secure and reliable
    applications!
    @IanColdwater

    View full-size slide