Defenders Guide to Cloud Native Infrastructure Security - Github Satellite 2020

While DevOps teams have moved toward cloud, containers, Kubernetes, serverless, and cloud-native infrastructure, security teams are still catching up. In this talk, Madhu will discuss how to get started with setting up real-world cloud-native infrastructure using containers, serverless, and service mesh with automated deployments. What's more, each phase will contain built-in security checks with open source tools and cloud services.

Madhu will perform security checks at multiple layers—like Infrastructure Security, Supply Chain Security and Run Time Security—with real-world scenarios. At the end of the talk he'll verify the security of the cloud-native infrastructure by performing an automated security scan with the help of CIS Benchmarks. Following this talk, you'll feel comfortable applying these practical security skills to your daily operations, no matter your infrastructure.

Transcript

1. Defenders Guide to
   Cloud Native Infrastructure
   Security
   Madhu Akula @ Xebia
   @madhuakula
   #GitHubSatellite
   

   View Slide

2. @madhuakula
   About Me
   ● Cloud Native Security Specialist @ Xebia
   ● Security (Cloud, Containers, Kubernetes &
   Automation)
   ● Speaker & Trainer @ BlackHat, DEF CON, USENIX
   LISA, OWASP, All Day DevOps, null, etc.
   ● Co-Author of Security Automation with Ansible 2
   ● Never Ending Learner!
   ● https://madhuakula.com
   #GitHubSatellite
   

   View Slide

3. ● What & Why Cloud Native Infrastructure?
   ● Code to Production workflow
   ● Why security defence?
   ● Architecture & Attack surface
   ● Layers of security defence (defence in depth)
   ● Key takeaways
   ● References & Resources
   ● Next steps to learn more and more…
   What You Will Learn?
   @madhuakula
   #GitHubSatellite
   

   View Slide

4. Cloud Native is used to describe containerised application to dynamically
   schedule, orchestrate and manage through continuous delivery workflows.
   Which allows to optimize resource utilization, and microservices-oriented to
   increase the overall agility and maintainability and support the life cycle of
   applications.
   - Cloud Native Computing Foundation
   What is Cloud Native?
   @madhuakula
   #GitHubSatellite
   

   View Slide

5. What is Cloud Native?
   @madhuakula
   https://landscape.cncf.io
   #GitHubSatellite
   

   View Slide

6. https://github.com/cncf/toc/blob/master/DEFINITION.md
   Cloud native technologies empower organizations to build and run
   scalable applications in modern, dynamic environments such as
   public, private, and hybrid clouds. Containers, service meshes,
   microservices, immutable infrastructure, and declarative APIs
   exemplify this approach.
   These techniques enable loosely coupled systems that are resilient,
   manageable, and observable. Combined with robust automation,
   they allow engineers to make high-impact changes frequently and
   predictably with minimal toil.
   Why Cloud Native?
   @madhuakula
   #GitHubSatellite
   

   View Slide

7. https://medium.com/@pkerrison/pizza-as-a-service-2-0-5085cd4c365e @madhuakula
   #GitHubSatellite
   

   View Slide

8. Operat
   e
   Develop
   Design
   Deploy
   Test
   Code to Production Workflow
   @madhuakula
   #GitHubSatellite
   

   View Slide

9. Cloud Native Microservices Demo Application
   Online Boutique is a cloud-native demo application with 10 microservices showcasing
   Kubernetes, Istio, gRPC and OpenCensus.
   https://github.com/GoogleCloudPlatform/microservices-demo/ @madhuakula
   #GitHubSatellite
   

   View Slide

10. Why Security Defence?
    @madhuakula
    #GitHubSatellite
    

    View Slide

11. https://blog.madhuakula.com/some-tips-to-review-docker-hub-hack-of-190k-accounts-addcd602aade
    Why Security Defence?
    @madhuakula
    https://kromtech.com/blog/security-center/cryptojacking-invades-cloud-how-modern-
    containerization-trend-is-exploited-by-attackers
    https://www.youtube.com/watch?v=4CTK2aUXTH
    o
    https://github.com/Frichetten/CVE-2019-5736-PoC
    https://github.com/eoftedal/writings/blob/master/published/CVE-2019-9901-
    path-traversal.md
    https://engineering.bitnami.com/articles/helm-security.html
    #GitHubSatellite
    

    View Slide

12. Many other vulnerabilities and real-
    world impacts...
    @madhuakula
    #GitHubSatellite
    

    View Slide

13. Architecture & Attack Surface
    @madhuakula
    #GitHubSatellite
    

    View Slide

14. Cloud Native Attack Surface
    ● Application Code
    ● Container Image
    ● Orchestration Platform
    ● Runtime
    ● Microservices & Communication
    ● API Gateway & Proxies
    ● Network & Load Balancers
    ● AuthN & AuthZ
    ● Storage
    ● Management
    ● Many other...
    @madhuakula
    #GitHubSatellite
    

    View Slide

15. Container Attack Surface
    ● Namespaces
    ● Control Groups
    ● Daemon
    ● Configuration
    ● Capabilities
    ● Content Trust
    ● Container Registry
    ● Volumes
    ● Networks
    ● Many other...
    @madhuakula
    #GitHubSatellite
    

    View Slide

16. https://www.microsoft.com/security/blog/2020/04/02/attack-matrix-kubernetes/
    Kubernetes Attack Surface
    @madhuakula
    #GitHubSatellite
    

    View Slide

17. Layers of Security Defense
    (defense in depth)
    @madhuakula
    #GitHubSatellite
    

    View Slide

18. Code Quality Analysis
    https://www.sonarqube.org/ @madhuakula
    #GitHubSatellite
    

    View Slide

19. Security Linters
    https://find-sec-bugs.github.io/ @madhuakula
    #GitHubSatellite
    

    View Slide

20. Dependency Security Analysis
    https://help.github.com/en/github/managing-security-vulnerabilities/viewing-and-updating-vulnerable-
    dependencies-in-your-repository @madhuakula
    #GitHubSatellite
    

    View Slide

21. Static Code Security Analysis
    https://brakemanscanner.org/ @madhuakula
    #GitHubSatellite
    

    View Slide

22. Dynamic Security Analysis
    https://help.github.com/en/github/managing-security-vulnerabilities/viewing-and-updating-vulnerable-
    dependencies-in-your-repository @madhuakula
    #GitHubSatellite
    

    View Slide

23. Semantic Code Analysis
    https://github.com/features/security @madhuakula
    #GitHubSatellite
    

    View Slide

24. Container Image Linter
    https://github.com/goodwithtech/dockle @madhuakula
    #GitHubSatellite
    

    View Slide

25. Sensitive Information Analysis
    https://github.com/dxa4481/truffleHog @madhuakula
    #GitHubSatellite
    

    View Slide

26. Vulnerability Analysis for Containers
    https://github.com/aquasecurity/trivy @madhuakula
    #GitHubSatellite
    

    View Slide

27. Risk Analysis for K8S Manifests
    https://kubesec.io @madhuakula
    #GitHubSatellite
    

    View Slide

28. Exploring Docker Image Layers
    https://github.com/wagoodman/dive @madhuakula
    #GitHubSatellite
    

    View Slide

29. Container Image Integrity Analysis
    @madhuakula
    #GitHubSatellite
    

    View Slide

30. Centralised Logging & Monitoring
    @madhuakula
    #GitHubSatellite
    

    View Slide

31. Network Security Policies
    https://github.com/ahmetb/kubernetes-network-policy-recipes
    Provides isolation between Kubernetes resources (pods, namespaces, svc, etc.) using
    labels and selectors across the cluster.
    @madhuakula
    #GitHubSatellite
    

    View Slide

32. Security Profiles
    https://github.com/genuinetools/bane @madhuakula
    #GitHubSatellite
    

    View Slide

33. Metadata Concealment
    https://github.com/features/security
    ● Most of the cloud providers has fix for this in some way
    ● GKE: Workload Identity, Metadata Concealment for Nodes
    https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity
    ● AWS: IMDSv2 for SSRF
    https://aws.amazon.com/blogs/security/defense-in-depth-open-firewalls-
    reverse-proxies-ssrf-vulnerabilities-ec2-instance-metadata-service/
    @madhuakula
    

    View Slide

34. RBAC with least privilege access possible
    https://kubernetes.io/docs/reference/access-authn-authz/rbac/
    Role-based access control (RBAC) is a method of regulating access to computer or
    network resources based on the roles of individual users within your organization.
    Useful utilities to check out is
    ● https://github.com/liggitt/audit2rbac
    ● https://github.com/FairwindsOps/rbac-manager
    ● https://github.com/jtblin/kube2iam
    @madhuakula
    #GitHubSatellite
    

    View Slide

35. Secrets Injection into K8S Pod
    https://www.hashicorp.com/blog/injecting-vault-secrets-into-kubernetes-pods-via-a-sidecar/ @madhuakula
    #GitHubSatellite
    

    View Slide

36. TLS with Cert-Manager
    Automate certificate management
    in cloud native environments. cert-
    manager builds on top of
    Kubernetes, introducing certificate
    authorities and certificates as first-
    class resource types in the
    Kubernetes API. This makes it
    possible to provide 'certificates as a
    service' to developers working
    within your Kubernetes cluster.
    @madhuakula
    #GitHubSatellite
    

    View Slide

37. Pod Security Policies
    A Pod Security Policy is a cluster-
    level resource that controls
    security sensitive aspects of the
    pod specification. The
    PodSecurityPolicy objects define a
    set of conditions that a pod must
    run with in order to be accepted
    into the system, as well as
    defaults for the related fields.
    Good utility to check out is
    https://github.com/sysdiglabs/ku
    be-psp-advisor
    https://kubernetes.io/docs/concepts/policy/pod-security-policy
    @madhuakula
    #GitHubSatellite
    

    View Slide

38. Open Policy Agent - Policy Engine
    Policy-based control for cloud
    native environments Flexible, fine-
    grained control for administrators
    across the stack
    https://www.openpolicyagent.org
    @madhuakula
    #GitHubSatellite
    

    View Slide

39. Container Runtime Sandbox
    ● gVisor is a user-space kernel,
    written in Go, that implements a
    substantial portion of the Linux
    system call interface. It provides
    an additional layer of isolation
    between running applications and
    the host operating system.
    ● Firecracker is an open source
    virtualization technology that is
    purpose-built for creating and
    managing secure, multi-tenant
    container and function-based
    services.
    ● Many other...
    @madhuakula
    #GitHubSatellite
    

    View Slide

40. Sysdig Falco - Runtime Security Detection
    https://www.youtube.com/watch?v=zd0ksjZI5Vk
    https://falco.org
    @madhuakula
    #GitHubSatellite
    

    View Slide

41. Audit Your Kubernetes Clusters
    https://github.com/Shopify/kubeaudit @madhuakula
    #GitHubSatellite
    

    View Slide

42. Docker CIS Benchmarks
    https://github.com/docker/docker-bench-security
    A script that checks for dozens of
    common best-practices around
    deploying Docker containers in
    production
    ● Host configuration
    ● Docker daemon configuration
    and files
    ● Docker container images
    ● Docker runtime
    ● Docker security operations
    ● Docker swarm configuration
    @madhuakula
    #GitHubSatellite
    

    View Slide

43. Kubernetes CIS Benchmarks
    https://github.com/aquasecurity/kube-bench
    ● Master Node Security
    Configuration
    ○ API Server
    ○ Scheduler
    ○ Controller Manager
    ○ Configuration Files
    ○ Etcd
    ○ General Security Primitives
    ○ PodSecurityPolicices
    ● Worker Node Security
    Configuration
    ○ Kubelet
    ○ Configuration Files
    @madhuakula
    #GitHubSatellite
    

    View Slide

44. Security Best Practices
    ● Application Code
    ○ Code Linters
    ○ Dependency Scanning
    ○ Code Analysis (static, dynamic, variant and manual analysis)
    ● Infrastructure Code
    ○ Dockerfile (cis benchmarks, security best practices)
    ○ Kubernetes manifests/Helm charts (cis benchmarks, least privilege)
    ○ Host images, Host infrastructure (terraform, cloud infra security configs)
    ○ Container Registry, Config Management
    ● Sensitive information checks (secrets, api keys, etc.)
    ● Version Control System (Config, PRs, MRs, etc.)
    ● Manual Review/Approval/Verification
    @madhuakula
    #GitHubSatellite
    

    View Slide

45. Security Best Practices
    ● Secure Defaults
    ● Least privilege principle
    ● Network Security Policies
    ● RBAC reviews
    ● Service Mesh
    ● Open Security Policy Agent (Multiple levels applying policy engine checks)
    ● Proactive Logging & Monitoring for detection
    ● Falco - Syscall monitoring & Threat detection engine
    ● RASP - Runtime application security protection
    ● Logging & Monitoring with Centralized Monitoring
    ● Proactive Security Monitoring & Detection
    ● Many other...
    @madhuakula
    #GitHubSatellite
    

    View Slide

46. The (12) Twelve Factor App
    In the modern era, the twelve-factor app is a methodology for building modern,
    scalable, maintainable software-as-a-service apps.
    @madhuakula
    https://12factor.net
    #GitHubSatellite
    

    View Slide

47. Key Takeaways
    @madhuakula
    #GitHubSatellite
    

    View Slide

48. ● Security is everyone’s responsibility (Dev, Ops and Security, etc.)
    ● Threat model your architecture and identify risks/threats
    ● Follow and apply secure defaults
    ● Know what you have (Inventory of assets)
    ● Adopt zero trust model and trust nothing (Zoning, Containment & Segmentation)
    ● Apply security at each layer (Defense in depth strategy)
    ● Follow least privilege principle
    ● AuthN & AuthZ
    ● Encryption at REST & TRANSIT
    ● Proactive monitoring & Active defense
    ● Continuously analyse and apply feedback loops
    ● Crawl, Walk, Run
    What are Your Key Takeaways?
    @madhuakula
    #GitHubSatellite
    

    View Slide

49. References & Resources
    @madhuakula
    #GitHubSatellite
    

    View Slide

50. ● Docker Security Docs
    ● Kubernetes Security Docs
    ● Attack matrix for Kubernetes
    ● Breaking & Pwning Docker Containers & Kubernetes Clusters
    ● Advanced Persistence Threats: The Future of Kubernetes Attacks
    ● 11 Ways (Not) to Get Hacked
    ● Attacking & Auditing Docker Containers using Open Source @ DEFCON 26
    ● Attacking and Auditing Docker Containers and Kubernetes Clusters @ DEFCON 27
    ● contained.af
    ● CIS Benchmarks Docker
    ● Understanding and Hardening Linux Containers
    ● Abusing Privileged and Unprivileged Linux Containers
    ● Container Security Notes
    ● Linux Container Security
    ● Docker Runtime Privileges and Capabilities
    ● Apparmor Security Profiles on Docker
    ● Seccomp Security Profiles on Docker
    ● Docker Labs Capabilities
    ● Practical SELinux and Containers
    ● Containers and Operating systems morning paper gist
    ● Kubernetes Webinar series
    References & Resources
    @madhuakula
    #GitHubSatellite
    

    View Slide

51. Next steps to learn more and more...
    @madhuakula
    #GitHubSatellite
    

    View Slide

52. Recommended Reads & More Learning
    ● Google SRE - 3 books
    ● Cloud Native Infrastructure Book
    ● Cloud Native Transformation Book
    ● Kubernetes-Security.info
    ● DevOps Security Checklist
    ● Kubernetes Attack Audit Reports
    ● CNCF Landscape
    ● Known CVE’s and Vulnerability Research
    ● K8S Slack Channels/Working Groups
    ● Katacoda Playgrounds & Play with Docker & Play with Kubernetes
    ● Many other...
    @madhuakula
    #GitHubSatellite
    

    View Slide

53. Madhu Akula
    https://madhuakula.com
    @madhuakula
    Thank You!
    

    View Slide