Remediate Kubernetes Security Threats in Real-Time with Falco Talon Marat Salakhutdinov Senior Customer Solutions Engineer

Marat Salakhutdinov LinkedIn: https://www.linkedin.com/in/salakhutdinov/ Email: marat@sysdig.com Senior Customer Solutions Engineer

Sysdig Inc. Proprietary Information Agenda 1 Introductions 2 Runtime Security: Why? 3 What is Falco? 4 How it started? 5 Falco Talon 6 Demo 7 Q&A 4

Runtime Security: Why?

Sysdig 2023 Global Cloud Threat Report ● Cloud Automation Weaponized ● 10 Minutes to Pain - every second counts ● A 90% Safe Supply Chain Isn’t Safe Enough ● Attackers are Hiding Among the Clouds ● 65% of Cloud Attacks Target Telcos and FinTech 6

The 5/5/5 Benchmark for Cloud Detection and Response ● 5 Seconds to Detect Threats Collect detection signals from the cloud service provider and cloud security tools within 5 seconds to ensure visibility into ephemeral assets. ● 5 Minutes to Correlate and Triage Automate triage by gathering full context for all correlated signals within 5 minutes of receiving the first relevant alert. ● 5 Minutes to Initiate Response Use the flexibility of the cloud to initiate tactical response actions within 5 minutes of a high-fidelity detection. 7

What is Falco?

What is Falco? Falco is an open source runtime security solution for threat detection across Kubernetes, containers, hosts and the cloud. 6.8K 60M+ pulls CNCF Graduated Project

The Security Camera for Modern Apps CNCF GRADUATED PROJECT created by Sysdig

What is Falco? ○ Runtime security engine ○ Observability for endpoints and cloud infrastructure ○ Built on eBPF ○ Integrated with Kubernetes CNCF GRADUATED PROJECT

Falco rule - rule: Terminal shell in container desc: A shell was used as the entrypoint/exec point into a container with an attached terminal. condition: > spawned_process and container and shell_procs and proc.tty != 0 and container_entrypoint and not user_expected_terminal_shell_in_container_conditions output: > A shell was spawned in a container with an attached terminal (user=%user.name user_loginuid=%user.loginuid %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline terminal=%proc.tty container_id=%container.id image=%container.image.repository) priority: NOTICE tags: [container, shell, mitre_execution] alerts

Falco alert

Beyond system calls and containers Plugins are dynamic shared libraries which allow Falco to collect and extract fields from streams of events

Falcosidekick push push push push pull push push push push only if priority > critical

CNCF GRADUATED PROJECT Users and Builders

How it started?

The Dawn of Falco From Inception to Cloud Native Excellence 2016 First commit 2018 eBPF probe 2019 Falcosidekick 2020 2021 Plugins 2022 falcoctl 2023 eBPF CO-RE 2024 Enhanced governance CNCF incubation CNCF sandbox 2017 eBPF eBPF CO-RE Rules Maturity Framework Kernel Version Testing I learned about Falco during preparation for CKS exam 😁 🎉Graduation of Falco🎉

What’s new in Falco? ● Detection Improvements: symlink resolution ● Rule language improvements: override option ● Testing and stability improvements: increased e2e tests coverage, new kernel and linux distributions ● Performance improvements: Falco joined CNCF’s green reviews working group ● Rules maturity framework: stable, incubating, sandbox, deprecated ● Plugins: ○ new plugins: Anomaly Detection, K8S Cluster Metadata, Hashicorp Vault ○ improvements in plugin API: C++ SDK, Go SDK, more to come ● Falcosidekick - more outputs: Dynatrace, Sumologic, Qucikwit, etc ● falcoctl - now used for downloading drivers ● Falco playground: try and test falco rules at play.falco.org

The Falco Roadmap - Core Enhancements ● High-Level enanchements ○ Standardizing feature adoption and deprecation policies ○ Streamlined configuration and CLI standardization ○ Addressing legacy language inconsistencies and introducing new constructs ○ Introducing advanced metrics ○ Making the modern eBPF the default driver ● Distribution ○ Streamlined DEB/RPM packages, following Linux distro best practices ○ Switching to a distroless container image by default ○ Complete supply chain security initiative (ie. ensuring signatures for all artifacts and SBOM)

The Falco Roadmap - Future Directions ● Falco in 3rd-party distributions ○ artifacthub.io integration ○ Curated presence in cloud marketplaces (ie. Azure, AWS, GCP, …) ○ Linux distros inclusion ● Advanced Integrations ○ Event enrichment with cloud provider metadata ○ Comprehensive K8s AuditLog support across cloud providers ○ More K8s enhancements ● Innovations ○ On-host anomaly detection ○ Plugin framework expansion with extended API access ○ Plugins SDK for Rust!

Resources ● Get started at Falco.org ● Check out the Falco project in Github ● Get involved in the Falco community ● Meet the maintainers on the Falco Slack ● Follow @falco_org on ● Join a Falco workshop ● Mailing list: cncf-falco-dev@lists.cncf.io

Falco Talon

What is Falco Talon? Falco Talon is a Response Engine for managing threats in your Kubernetes. It enhances the solutions proposed by the Falco community with a no-code tailor made solution. With easy rules, you can react to events from Falco in milliseconds.

Architecture ┌──────────┐ ┌───────────────┐ ┌─────────────┐ │ Falco ├─► Falcosidekick ├─► Falco Talon │ └──────────┘ └───────────────┘ └─────────────┘ or ┌──────────┐ ┌─────────────┐ │ Falco ├─► Falco Talon │ └──────────┘ └─────────────┘

Glossary ● event: an event detected by Falco and sent to its outputs ● rule: defines criterias for linking the events with the actions to apply ● action: each rule can sequentially run actions, each action refers to an actionner ● actionner: defines what the action will do ● notifier: defines what outputs to notify with the result of the action

Falco Talon rule - action: Terminate Pod actionner: kubernetes:terminate parameters: ignore_daemonsets: true ignore_statefulsets: true grace_period_seconds: 0 - rule: Terminate the pod match: rules: - Terminal shell in container - Netcat Remote Code Execution in Container output_fields: - k8s.ns.name!=kube-system, k8s.ns.name!=falco actions: - action: Terminate Pod

Actionners Actionners are the built-it actions to react to the events. ● kubernetes:terminate ● kubernetes:labelize ● kubernetes:networkpolicy ● kubernetes:exec ● kubernetes:script ● kubernetes:log ● kubernetes:delete ● calico:networkpolicy ● aws:lambda

Notifiers Notifiers are the built-it outputs to forward the result of the Falco Talon actions. ● K8s Events ● Slack ● Loki ● Elasticsearch ● SMTP ● Webhook

Installation Install Falco Talon with Helm git clone https://github.com/Falco-Talon/falco-talon.git cd falco-talon/deployment/helm/ helm install falco-talon . -n falco --create-namespace Configure Falcosidekick helm install falco falcosecurity/falco --namespace falco \ --create-namespace \ --set tty=true \ --set falcosidekick.enabled=true \ --set falcosidekick.config.webhook.address=http://falco-talon:2803

Resources ● Github repo: https://github.com/falco-talon/falco-talon ● Docs: https://docs.falco-talon.org ● Falco Talon overview video from CNCF Live: https://www.youtube.com/watch?v=1ewRLb4cack ● Falco Talon examples by Nigel Douglas: https://github.com/nigel-falco/falco-talon-testing

Demo time

Demo Environment Details K8S cluster running on an EC2 node (with IMDSv1). • Vulnerable Spring Boot Application • Falco as a daemon set on k8s cluster • Falco Sidekick • Falco Talon • Falco Sidekick UI • Falco Cloudtrail plugin • Falco AWS Cloudtrail terraform module An attacker host to execute the infiltration and exploit of the attack. • Rootkit installed. • Other tools to escalate privileges and lateral movement.

SCARLETEEL attack 34

Demo 35

Try it yourself! 36

Q&A

Thank you!