Securing your Kubernetes applications

While there have been many improvements around securing containers, there is still a large gap in monitoring the behavior of containers in production. That is the reason we created Falco, the open source behavioral activity monitor for containerized environments.

Falco can detect and alert on anomalous behavior at the application, file system and network level.

In this session we get a deep dive into Falco and explain the following points:

* How does behavioral security differ from existing security solutions like image scanning, seccomp, SELinux or AppArmor?

* How Falco does its magic?

* What Falco can detect? Creating your own rules and customize the existing ones for your Kubernetes applications.

* How to deploy Falco in your Kubernetes cluster?

* Reacting to security incidents, what we can do to stop the attackers in real-time?

*Post-mortem analysis and forensics on containers with Sysdig Inspect. Even when containers does not exist anymore!

Transcript

1. Néstor Salceda, Integrations Engineer LibreCon Bilbao, Nov 22th 2018 Securing your Kubernetes applications
2. @nestorsalceda • Open Source enthusiast • Security and Monitoring passionate • I work at Sysdig • Daddy of twins • Kubernetes member: Maintainer of Sysdig and Falco Helm charts • Top 3 Contributor to Falco • Judo, Aikido and other Gendai Budo martial arts lover and practicioner
3. Anomaly Detection in run-time: Falco Active Security: Kubernetes Response Engine Forensics: Sysdig Inspect Current challenges of Container Security Agenda Layers of Container Security
4. • Container Security Challenges
5. Breaches may extend for days or weeks before detected Attacks are changing to abuse activities rather than data exfiltration (crypto mining) Ephemeral nature of containers means that in the event of a security breach you may never know Many security paradigms are still reactive Main Challenges
6. • Layers of Container Security
7. Which layers? Runtime Build Infrastructure
8. Networking: Filtering, Istio, Calico ... Cluster Security: RBAC, Audit Events, Security Policies, Affinity, Network Policies ... Container Runtime: SELinux, AppArmor, CIS Benchmarks, InSpec ... Host Security: SecComp, SELinux, AppArmor, Resource Constraints ... Infrastructure
9. Vulnerability Management: • Image Scanning: Sysdig Secure, Anchore, Clair • Upstream OS • Application Vulnerabilities Image / Software Origin: • Signed Images / Layers • Artifact Signing • Trusted Registries Build
10. Secure Secrets: How secrets are stored or used? Anomaly Detection: Someone altered my runtime environment? Forensics: What happened if compromised? Service / Container Admission: What is allowed to run? Runtime
11. Processes are “scoped” as to what’s expected Container images are immutable, runtime environments often aren’t How do you detect abnormal behavior? See containers like isolated processes Anomaly Detection
12. Containers are highly volatile: Imagine Grisom doing CSI stuff without the corpse What did happen inside the container? When a security incident has already happened Forensics
13. What is Falco?
14. • Detects suspicious activity defined by a set of rules • Uses Sysdig’s flexible and powerful filtering expressions Behavioral Activity Monitor • Uses Sysdig’s container and orchestrator support Full Support of Containers Orchestration Flexible Notification Methods Open Source Software • Files • STDOUT • Syslog • Execute other programs • And more ... • CNCF Sandbox Project • Welcome contributions • Transparency & Governance
15. falco_probe Kernel Module Kernel User Syscalls Sysdig Libraries Events Alerting Falco Rules Suspicious Events File Syslog Stdout Filter Expression Shell
16. Filter expressions A shell is run in a container container.id != host and proc.name = bash Overwrite system binaries fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin) and write Container namespace change evt.type = setns and not proc.name in (docker, sysdig) Non-device files written in /dev (evt.type = create or evt.arg.flags contains O_CREAT) and proc.name != blkid and fd.directory = /dev and fd.name != /dev/null Process tries to access camera evt.type = open and fd.name = /dev/video0 and not proc.name in (skype, webex)
17. Rules - macro: bin_dir condition: fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin) - list: shell_binaries items: [bash, csh, ksh, sh, tcsh, zsh, dash] - rule: write_binary_dir desc: an attempt to write to any file below a set of binary directories condition: bin_dir and evt.dir = < and open_write and not package_mgmt_procs output: "File below a known binary directory opened for writing (user=%user.name command=%proc.cmdline file=%fd.name)" priority: WARNING
18. More rules implemented in draios/falco-extras repository: • Traefik • Redis • Nginx • PostgreSQL • ... Falco ships with a nice default ruleset for best practices: • Writing files in bin or etc • Reading sensitive files • Terminal spawning in a container • ... Batteries included
19. Requests made by anonymous user Attach to cluster-admin Role Service Account Created in Kube Namespace Create / Modify ConfigMaps which exposes secrets K8s Audit Events Support
20. Try it out! $ helm install --name sysdig-falco-1 --set fakeEventGenerator.enabled=true stable/falco
21. Active Security
23. See it in action!
24. Start a capture Network isolate Demisto/Phantom integration Delete a pod Playbooks Available Forbid that a node schedules more pods Slack notification
25. Forensics
26. Correlate events to reconstruct the attack Blameless Post-Mortem incident report Capture system calls using Sysdig Forensics
27. The ephemeral nature of containers changes the rules Security offers us an opportunity to be proactive Containers add more infrastructure, layers and risks. But we have seen same security threats before: DDoS, Injections ... Just a quick summary
28. Do you want work with me? Monitoring / Security Open Source Remote
29. Blog https://www.sysdig.com/blog/tag/falco Sysdig Secure https://www.sysdig.com/product/secure Website https://www.sysdig.com/opensource/falco https://falco.org Join the community Public Slack https://slack.sysdig.com https://slack.sysdig.com/messages/falco
30. Docker Hub https://hub.docker.com/r/falcosecurity/falco GitHub https://github.com/falcosecurity/falco Learn more Wiki https://github.com/falcosecurity/falco/wiki Sysdig Docker Usage Report 2018 https://sysdig.com/blog/2018-docker-usage-report
31. Eskerrik Asko Questions? nestor.salceda@sysdig.com @nestorsalceda