While there have been many improvements around securing containers, there is still a large gap in monitoring the behavior of containers in production. That is the reason we created Falco, the open source behavioral activity monitor for containerized environments.
Falco can detect and alert on anomalous behavior at the application, file system and network level.
In this session we get a deep dive into Falco and explain the following points:
* How does behavioral security differ from existing security solutions like image scanning, seccomp, SELinux or AppArmor?
* How Falco does its magic?
* What Falco can detect? Creating your own rules and customize the existing ones for your Kubernetes applications.
* How to deploy Falco in your Kubernetes cluster?
* Reacting to security incidents, what we can do to stop the attackers in real-time?
*Post-mortem analysis and forensics on containers with Sysdig Inspect. Even when containers does not exist anymore!
Néstor Salceda, Integrations Engineer
LibreCon Bilbao, Nov 22th 2018
• Open Source enthusiast
• Security and Monitoring passionate
• I work at Sysdig
• Daddy of twins
• Kubernetes member: Maintainer of Sysdig and
Falco Helm charts
• Top 3 Contributor to Falco
• Judo, Aikido and other Gendai Budo martial arts
lover and practicioner
Anomaly Detection in run-time: Falco
Active Security: Kubernetes Response Engine
Forensics: Sysdig Inspect
Current challenges of Container Security
Layers of Container Security
Breaches may extend for days or weeks before
Attacks are changing to abuse activities rather than data
exfiltration (crypto mining)
Ephemeral nature of containers means that in the event
of a security breach you may never know
Many security paradigms are still reactive
Networking: Filtering, Istio, Calico ...
Cluster Security: RBAC, Audit Events, Security Policies,
Affinity, Network Policies ...
Container Runtime: SELinux, AppArmor, CIS
Benchmarks, InSpec ...
Host Security: SecComp, SELinux, AppArmor, Resource
● Image Scanning: Sysdig Secure, Anchore, Clair
● Upstream OS
● Application Vulnerabilities
Image / Software Origin:
● Signed Images / Layers
● Artifact Signing
● Trusted Registries
Secure Secrets: How secrets are stored or used?
Anomaly Detection: Someone altered my runtime
Forensics: What happened if compromised?
Service / Container Admission: What is allowed to run?
Processes are “scoped” as to what’s expected
Container images are immutable, runtime
environments often aren’t
How do you detect abnormal behavior?
See containers like isolated processes
Containers are highly volatile: Imagine Grisom doing
CSI stuff without the corpse
What did happen inside the container?
When a security incident has already happened
What is Falco?
• Detects suspicious activity
defined by a set of rules
• Uses Sysdig’s flexible and
• Uses Sysdig’s container
and orchestrator support
Full Support of
• Execute other programs
• And more ...
• CNCF Sandbox Project
• Welcome contributions
• Transparency &
A shell is run in a container container.id != host and proc.name = bash
Overwrite system binaries
fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin)
Container namespace change
evt.type = setns and not proc.name in
Non-device files written in /dev
(evt.type = create or evt.arg.flags contains O_CREAT) and proc.name
!= blkid and fd.directory = /dev and fd.name != /dev/null
Process tries to access camera
evt.type = open and fd.name = /dev/video0 and not proc.name
in (skype, webex)
- macro: bin_dir
condition: fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin)
- list: shell_binaries
items: [bash, csh, ksh, sh, tcsh, zsh, dash]
- rule: write_binary_dir
desc: an attempt to write to any file below a set of binary directories
condition: bin_dir and evt.dir = < and open_write and not package_mgmt_procs
output: "File below a known binary directory opened for writing
(user=%user.name command=%proc.cmdline file=%fd.name)"
More rules implemented in draios/falco-extras repository:
Falco ships with a nice default ruleset for best practices:
● Writing files in bin or etc
● Reading sensitive files
● Terminal spawning in a container
Requests made by anonymous user
Attach to cluster-admin Role
Service Account Created in Kube Namespace
Create / Modify ConfigMaps which exposes secrets
K8s Audit Events Support
Try it out!
$ helm install --name sysdig-falco-1
See it in action!
Start a capture
Delete a pod
Forbid that a node schedules more pods
Correlate events to reconstruct the attack
Blameless Post-Mortem incident report
Capture system calls using Sysdig
The ephemeral nature of containers changes the rules
Security offers us an opportunity to be proactive
Containers add more infrastructure, layers and risks. But
we have seen same security threats before: DDoS,
Just a quick summary
Do you want work with me?
Monitoring / Security Open Source Remote
Join the community
Sysdig Docker Usage Report 2018