Securing your Kubernetes applications

While there have been many improvements around securing containers, there is still a large gap in monitoring the behavior of containers in production. That is the reason we created Falco, the open source behavioral activity monitor for containerized environments.

Falco can detect and alert on anomalous behavior at the application, file system and network level.

In this session we get a deep dive into Falco and explain the following points:

* How does behavioral security differ from existing security solutions like image scanning, seccomp, SELinux or AppArmor?

* How Falco does its magic?

* What Falco can detect? Creating your own rules and customize the existing ones for your Kubernetes applications.

* How to deploy Falco in your Kubernetes cluster?

* Reacting to security incidents, what we can do to stop the attackers in real-time?

*Post-mortem analysis and forensics on containers with Sysdig Inspect. Even when containers does not exist anymore!

Transcript

1. Néstor Salceda, Integrations Engineer
   LibreCon Bilbao, Nov 22th 2018
   Securing your
   Kubernetes
   applications
   

   View Slide

2. @nestorsalceda
   • Open Source enthusiast
   • Security and Monitoring passionate
   • I work at Sysdig
   • Daddy of twins
   • Kubernetes member: Maintainer of Sysdig and
   Falco Helm charts
   • Top 3 Contributor to Falco
   • Judo, Aikido and other Gendai Budo martial arts
   lover and practicioner
   

   View Slide

3. Anomaly Detection in run-time: Falco
   Active Security: Kubernetes Response Engine
   Forensics: Sysdig Inspect
   Current challenges of Container Security
   Agenda
   Layers of Container Security
   

   View Slide

4. •
   Container
   Security
   Challenges
   

   View Slide

5. Breaches may extend for days or weeks before
   detected
   Attacks are changing to abuse activities rather than data
   exfiltration (crypto mining)
   Ephemeral nature of containers means that in the event
   of a security breach you may never know
   Many security paradigms are still reactive
   Main Challenges
   

   View Slide

6. •
   Layers of
   Container
   Security
   

   View Slide

7. Which layers?
   Runtime
   Build
   Infrastructure
   

   View Slide

8. Networking: Filtering, Istio, Calico ...
   Cluster Security: RBAC, Audit Events, Security Policies,
   Affinity, Network Policies ...
   Container Runtime: SELinux, AppArmor, CIS
   Benchmarks, InSpec ...
   Host Security: SecComp, SELinux, AppArmor, Resource
   Constraints ...
   Infrastructure
   

   View Slide

9. Vulnerability Management:
   ● Image Scanning: Sysdig Secure, Anchore, Clair
   ● Upstream OS
   ● Application Vulnerabilities
   Image / Software Origin:
   ● Signed Images / Layers
   ● Artifact Signing
   ● Trusted Registries
   Build
   

   View Slide

10. Secure Secrets: How secrets are stored or used?
    Anomaly Detection: Someone altered my runtime
    environment?
    Forensics: What happened if compromised?
    Service / Container Admission: What is allowed to run?
    Runtime
    

    View Slide

11. Processes are “scoped” as to what’s expected
    Container images are immutable, runtime
    environments often aren’t
    How do you detect abnormal behavior?
    See containers like isolated processes
    Anomaly Detection
    

    View Slide

12. Containers are highly volatile: Imagine Grisom doing
    CSI stuff without the corpse
    What did happen inside the container?
    When a security incident has already happened
    Forensics
    

    View Slide

13. What is Falco?
    

    View Slide

14. • Detects suspicious activity
    defined by a set of rules
    • Uses Sysdig’s flexible and
    powerful filtering
    expressions
    Behavioral
    Activity
    Monitor
    • Uses Sysdig’s container
    and orchestrator support
    Full Support of
    Containers
    Orchestration
    Flexible
    Notification
    Methods
    Open
    Source
    Software
    • Files
    • STDOUT
    • Syslog
    • Execute other programs
    • And more ...
    • CNCF Sandbox Project
    • Welcome contributions
    • Transparency &
    Governance
    

    View Slide

15. falco_probe
    Kernel
    Module
    Kernel
    User
    Syscalls
    Sysdig Libraries
    Events
    Alerting
    Falco Rules
    Suspicious
    Events
    File
    Syslog
    Stdout
    Filter Expression
    Shell
    

    View Slide

16. Filter expressions
    A shell is run in a container container.id != host and proc.name = bash
    Overwrite system binaries
    fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin)
    and write
    Container namespace change
    evt.type = setns and not proc.name in
    (docker, sysdig)
    Non-device files written in /dev
    (evt.type = create or evt.arg.flags contains O_CREAT) and proc.name
    != blkid and fd.directory = /dev and fd.name != /dev/null
    Process tries to access camera
    evt.type = open and fd.name = /dev/video0 and not proc.name
    in (skype, webex)
    

    View Slide

17. Rules
    - macro: bin_dir
    condition: fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin)
    - list: shell_binaries
    items: [bash, csh, ksh, sh, tcsh, zsh, dash]
    - rule: write_binary_dir
    desc: an attempt to write to any file below a set of binary directories
    condition: bin_dir and evt.dir = < and open_write and not package_mgmt_procs
    output: "File below a known binary directory opened for writing
    (user=%user.name command=%proc.cmdline file=%fd.name)"
    priority: WARNING
    

    View Slide

18. More rules implemented in draios/falco-extras repository:
    ● Traefik
    ● Redis
    ● Nginx
    ● PostgreSQL
    ● ...
    Falco ships with a nice default ruleset for best practices:
    ● Writing files in bin or etc
    ● Reading sensitive files
    ● Terminal spawning in a container
    ● ...
    Batteries included
    

    View Slide

19. Requests made by anonymous user
    Attach to cluster-admin Role
    Service Account Created in Kube Namespace
    Create / Modify ConfigMaps which exposes secrets
    K8s Audit Events Support
    

    View Slide

20. Try it out!
    $ helm install --name sysdig-falco-1
    --set fakeEventGenerator.enabled=true
    stable/falco
    

    View Slide

21. Active
    Security
    

    View Slide

22. View Slide

23. See it in action!
    

    View Slide

24. Start a capture
    Network isolate
    Demisto/Phantom integration
    Delete a pod
    Playbooks Available
    Forbid that a node schedules more pods
    Slack notification
    

    View Slide

25. Forensics
    

    View Slide

26. Correlate events to reconstruct the attack
    Blameless Post-Mortem incident report
    Capture system calls using Sysdig
    Forensics
    

    View Slide

27. The ephemeral nature of containers changes the rules
    Security offers us an opportunity to be proactive
    Containers add more infrastructure, layers and risks. But
    we have seen same security threats before: DDoS,
    Injections ...
    Just a quick summary
    

    View Slide

28. Do you want work with me?
    Monitoring / Security Open Source Remote
    

    View Slide

29. Blog
    https://www.sysdig.com/blog/tag/falco
    Sysdig Secure
    https://www.sysdig.com/product/secure
    Website
    https://www.sysdig.com/opensource/falco
    https://falco.org
    Join the community
    Public Slack
    https://slack.sysdig.com
    https://slack.sysdig.com/messages/falco
    

    View Slide

30. Docker Hub
    https://hub.docker.com/r/falcosecurity/falco
    GitHub
    https://github.com/falcosecurity/falco
    Learn more
    Wiki
    https://github.com/falcosecurity/falco/wiki
    Sysdig Docker Usage Report 2018
    https://sysdig.com/blog/2018-docker-usage-report
    

    View Slide

31. Eskerrik Asko
    Questions?
    nestor.salceda@sysdig.com
    @nestorsalceda
    

    View Slide