Advanced Persistence Threats: The Future of Kubernetes Attacks

Presented with Brad Geesaman at KubeCon EU/Virtual 2020.

What would happen if your cluster was successfully compromised by an attacker who understands Kubernetes at a deep level? How could they attempt to avoid detection, cover their tracks, achieve full cluster access, obtain persistence, steal credentials, and launch additional attacks in your environment? As Kubernetes grows in popularity, the sophistication of attackers will improve, and security by obscurity will no longer be sufficient. Cluster operators need to be aware of what a skilled and knowledgeable attacker can be capable of.

Let’s explore the dark corners of clusters and shine a light on how features such as privileged containers and validating webhooks can be used to maliciously mutate pods, exfiltrate data, deploy “shadow” control planes, and more. The audience will learn how to detect these advanced approaches and how to prevent these attacks using practical, proven methods.

Transcript

1. ADVANCED PERSISTENCE THREATS
   The Future of Kubernetes Attacks
   @bradgeesaman
   @IanColdwater
   

   View Slide

2. • Ian Coldwater is a Lead Platform Security
   Engineer at Salesforce, who specializes in
   hacking and hardening Kubernetes,
   containers and cloud infrastructure.
   • Brad Geesaman is the co-founder of Darkbit,
   who helps clients improve the security of
   their clusters in cloud-native environments.
   @bradgeesaman
   @IanColdwater
   

   View Slide

3. EARLY K8S ARCHITECTURE
   @bradgeesaman
   @IanColdwater
   

   View Slide

4. View Slide

5. K8S COMES AT YOU FAST
   @bradgeesaman
   @IanColdwater
   

   View Slide

6. LOOKING FORWARD
   @bradgeesaman
   @IanColdwater
   

   View Slide

7. GOALS
   What might an attacker want to do?
   @bradgeesaman
   @IanColdwater goose.game
   

   View Slide

8. DEMO
   Tapping into the API Server Data Flow
   @bradgeesaman
   @IanColdwater
   

   View Slide

9. VALIDATING WEBHOOKS
   @bradgeesaman
   @IanColdwater
   

   View Slide

10. VALIDATING WEBHONKS
    @bradgeesaman
    @IanColdwater
    

    View Slide

11. View Slide

12. View Slide

13. DEMO
    Shadow API Server
    @bradgeesaman
    @IanColdwater
    • launch an in-cluster “shadow” 

    API server that silently bypasses 

    main API servers
    • no security policy
    • no logs
    • no crime!
    

    View Slide

14. SHADOW API SERVER
    @bradgeesaman
    @IanColdwater
    

    View Slide

15. DEMO - C2BERNETES
    Use Kubernetes as a C2 infrastructure across multiple clusters
    @bradgeesaman
    @IanColdwater
    

    View Slide

16. WHAT IS K3S?
    • A lightweight Kubernetes distribution
    designed for resource-constrained
    environments
    • Runs as a single <40MB binary
    • Has a simplified communication channel:
    only requires a single TLS connection
    outbound from nodes to the control plane
    • This is very likely to be available and blend in
    with other valid traffic :)
    @bradgeesaman
    @IanColdwater
    

    View Slide

17. KUBERNETES VS K3S
    @bradgeesaman
    @IanColdwater
    

    View Slide

18. View Slide

19. ALL CLOUDS ARE BROKEN
    @bradgeesaman
    @IanColdwater
    

    View Slide

20. C2: CLUSTER OF CLUSTERS
    @bradgeesaman
    @IanColdwater
    

    View Slide

21. WHAT'S COMING
    @bradgeesaman
    @IanColdwater
    

    View Slide

22. CHECK YOUR --PRIVILEGE
    New as of Kubernetes 1.19.0
    @bradgeesaman
    @IanColdwater
    kubectl run --privileged
    

    View Slide

23. DYNAMIC CONFIGURATION
    @bradgeesaman
    @IanColdwater
    • Dynamic Audit Sink configuration
    --feature-gates=DynamicAuditing=true
    • Dynamic Kubelet configuration
    --feature-gates=DynamicKubeletConfig=true
    

    View Slide

24. Greetz to https://github.com/kayrus/kubelet-exploit
    @bradgeesaman
    @IanColdwater
    

    View Slide

25. DEMO
    Bringing It Back
    @bradgeesaman
    @IanColdwater
    

    View Slide

26. COMING FULL CIRCLE
    @bradgeesaman
    @IanColdwater
    

    View Slide

27. View Slide

28. View Slide