Runtime security protection for web applications on K8s

Transcript

1. Kubernetes Summit 2023 Runtime security protection for web applications on K8s Rich Lee Architect @ Cathay Holdings 1
2. 李琦 Rich rich04230@gmail.com RICH0423 @rich04230 Architect at Cathay Financial Holding - DDT Center • Cloud-Native Application Integration • Cloud Architecture • Event-driven Architecture
3. 3 Agenda • Cloud Native Threat • Runtime Security – Container Sandbox - gVisor – Real-time detection - Falco – Demo
4. 4 OWASP K8s TOP 10 OWASP K8s TOP 10
5. 5 OWASP K8s TOP 10 OWASP K8s TOP 10 K01: Insecure Workload Configurations K02: Supply Chain Vulnerabilities K03: Overly Permissive RBAC Configurations K04: Lack of Centralized Policy Enforcement K05: Inadequate Logging and Monitoring K06: Broken Authentication Mechanisms K07: Missing Network Segmentation Controls K08: Secrets Management Failures K09: Misconfigured Cluster Components K10: Outdated and Vulnerable Kubernetes Components Top 10 Kubernetes Risks - 2022
6. 6 OWASP K8s TOP 10
7. 7 Vulnerabilities and Threats Kernel Container Pod Container Pod Host OS
8. 8 Vulnerabilities and Threats Kernel Container Pod Container Pod Host OS kubectl apply -f dep.yml {… privileged: true…} K8s: Create Privileged Pod
9. 9 Vulnerabilities and Threats Kernel Container Pod Container Pod Host OS kubectl apply -f dep.yml {… privileged: true…} K8s: Create Privileged Pod Application: Log4Shell Download request Java Process Class File LDAP
10. 10 Vulnerabilities and Threats Kernel Container Pod Container Pod Host OS kubectl apply -f dep.yml {… privileged: true…} K8s: Create Privileged Pod login status Cloud: Login without MFA Application: Log4Shell Download request Java Process Class File LDAP
11. 11 Cloud Native Security Edge Services WAF LB Ingress API Gateway CDN LB WAF Ingress API GW CNI Service Mesh Sandbox Microservices Network Runtime Security Container Runtime End-to-end protection
12. 12 Cloud Native Security Edge Services WAF LB Ingress API Gateway CDN LB WAF Ingress API GW CNI Service Mesh Sandbox Microservices Network Runtime Security Container Runtime End-to-end protection • gVisor • Falco
13. 13 Runtime Security Protection Flow Prevent Detect Response • Vulnerabilities Scanning • Container Isolation • Hardening • Threat detection • Real-time incident detection • Alert and audit • Remediation • Recovery
14. Container Sandbox
15. 15 Container Runtime Isolation • Protects attackers from escaping the runtime environment • Reduces execution of trusted, privileged code • Don't want to expose the system to risk of any single bug CNCF – Container Runtime
16. 16 Container Runtime Isolation • gVisor is an application kernel for containers written in Go. • Reduces the host attack surface: – Calls to the host OS(syscall) are controlled by the Sentry – No syscalls are "passed through".
17. 17 Container Runtime Isolation • gVisor is an application kernel for containers written in Go. • Reduces the host attack surface: – Calls to the host OS(syscall) are controlled by the Sentry – No syscalls are "passed through".
18. 19 K8s RuntimeClass API • A RuntimeClass is a Kubernetes API resource that defines the configuration of the container runtime. – It allows users to select one of a supported list of container runtimes in the cluster.
19. 20 K8s RuntimeClass API • A RuntimeClass is a Kubernetes API resource that defines the configuration of the container runtime. – It allows users to select one of a supported list of container runtimes in the cluster. Control Plane kubelet runc pod pod pod Native Runtime
20. 21 K8s RuntimeClass API • A RuntimeClass is a Kubernetes API resource that defines the configuration of the container runtime. – It allows users to select one of a supported list of container runtimes in the cluster. Control Plane kubelet runc pod pod pod kubelet pod pod pod Native Runtime Sandboxed Runtime
21. 22 K8s RuntimeClass API • A RuntimeClass is a Kubernetes API resource that defines the configuration of the container runtime. – It allows users to select one of a supported list of container runtimes in the cluster. Control Plane kubelet runc pod pod pod kubelet kubelet runc pod pod pod pod pod pod pod Native Runtime Sandboxed Runtime
22. 23 Security Protection Flow Prevent Detect Response • Vulnerabilities Scanning • Container Isolation • Hardening • Threat detection • Real-time incident detection • Alert and audit • Remediation • Recovery
23. 24 Falco Introduction Falco, the cloud-native runtime security incubating project, is the de facto Kubernetes threat detection engine
24. 25 Falco Rules - rule: Terminal shell in container desc: A shell has been spawned in a container. condition: > spawned_process and container and shell_procs output: > A shell was spawned in a container (user=%user.name user_loginuid=%user.loginuid %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline container_id=%container.id)
25. 26 Falco Rules - rule: Terminal shell in container desc: A shell has been spawned in a container. condition: > spawned_process and container and shell_procs output: > A shell was spawned in a container (user=%user.name user_loginuid=%user.loginuid %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline container_id=%container.id) - list: shell_binaries items: [ash, bash, csh, ksh, sh, tcsh, zsh, dash] - macro: shell_procs condition: proc.name in (shell_binaries) - macro: container condition: (container.id != host) - macro: spawned_process condition: > evt.type in (execve, execveat) and evt.dir=<
26. 27 Falco Rules - rule: Terminal shell in container desc: A shell has been spawned in a container. condition: > spawned_process and container and shell_procs output: > A shell was spawned in a container (user=%user.name user_loginuid=%user.loginuid %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline container_id=%container.id) - list: shell_binaries items: [ash, bash, csh, ksh, sh, tcsh, zsh, dash] - macro: shell_procs condition: proc.name in (shell_binaries) - macro: container condition: (container.id != host) - macro: spawned_process condition: > evt.type in (execve, execveat) and evt.dir=<
27. 28 Falco Rules • Falco ships with more than 70 default rules: – Privilege escalation – R/W to sensitive directories – Executing shell – Execute SSH binaries – Mutating binaries – Creating symlinks – more… (falco_rules.yaml)
28. 29 Falco Architecture Falco sources Falco rule engine Syscall gRPC Falco rules FILE SHELL STDOUT HTTPS Input Process Output
29. https://tinyurl.com/rich-k8s-summit-2023
30. 31 K8s Audit Logs • Kubernetes auditing provides a security-relevant, chronological set of records documenting the sequence of actions in a cluster. • The defined audit levels are: – None – Metadata – Request – RequestResponse # audit-policy.yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: # Log pod changes at RequestResponse level - level: RequestResponse resources: - group: "" resources: ["pods"] # Log "pods/log", "pods/status" at Metadata level - level: Metadata resources: - group: "" resources: ["pods/log", "pods/status"]
31. - rule: Create Privileged Pod desc: > Detect an attempt to start a pod with a privileged container condition: kevt and pod and kcreate and ka.req.pod.containers.privileged intersects (true) exceptions: - name: image_repos fields: ka.req.pod.containers.image.repository comps: in values: [falco_privileged_images] output: Pod started with privileged container (user=%ka.user.name pod=%ka.resp.name ns=%ka.target.namespace images=%ka.req.pod.containers.image) priority: WARNING source: k8s_audit tags: [k8s] 32 K8s Audit Rule Example K8s audit rules
32. Falco sources Falco rule engine Syscall K8s Audit Logs K8s audit rules HTTPS Sideakick UI audit logs and metadata Falco plugins (k8saudit) Demo
33. 35 Falco Alerts
34. 36 Falco Sidekick https://github.com/falcosecurity/falcosidekick
35. 38 Falco Sidekick & Plugins Syscall K8s Audit Logs gRPC Falco Sidekick FILE STDOUT SHELL HTTPS
36. 39 Falco Sidekick & Plugins Syscall K8s Audit Logs gRPC Falco Sidekick FILE STDOUT SHELL HTTPS … extend Falco outputs
37. 40 Falco Sidekick & Plugins Syscall K8s Audit Logs gRPC Falco Sidekick FILE STDOUT SHELL HTTPS … extend Falco outputs extend Falco functionality Plugins Cloud Activity Logs
38. 42 Security Framework Support • CIS Benchmarks • MITRE ATT&CK Framework • Compliance – PCI DSS, NIST
39. 43 Security Framework Support • CIS Benchmarks • MITRE ATT&CK Framework • Compliance – NIST, PCI DSS - rule: Debugfs Launched in Privileged Container desc: > Detect file system debugger debugfs launched inside a privileged container which might lead to container escape. condition: > spawned_process and container and container.privileged=true and proc.name=debugfs priority: WARNING tags: [maturity_stable, container, cis, process, mitre_privilege_escalation, T1611] MITRE T1611 – Escape to Host
40. 44 Security Framework Support • CIS Benchmarks • MITRE ATT&CK Framework • Compliance – PCI DSS, NIST - rule: Unexpected K8s NodePort Connection desc: Detect attempts to use K8s NodePorts from a container condition: (inbound_outbound) and fd.sport >= 30000 and fd.sport <= 32767 and container and not nodeport_containers output: Unexpected K8s NodePort Connection (command=%proc.cmdline pid=%proc.pid connection=%fd.name container_id=%container.id image=%container.image.repository) priority: NOTICE tags: [network, k8s, container, mitre_persistence, T1205.001, NIST_800-53_AC-6] NIST AC-6 – Least Privilege
41. Thank You. 45