Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
eBPF for Security Observability
Search
Liz Rice
June 24, 2022
Technology
0
1.3k
eBPF for Security Observability
As seen at DevOpsDays Amsterdam and KCD Berlin
Liz Rice
June 24, 2022
Tweet
Share
More Decks by Liz Rice
See All by Liz Rice
Unleashing the kernel with eBPF
lizrice
0
140
eBPF's Abilities and Limitations: The Truth
lizrice
0
260
Simplifying multi-cloud and multi-cluster Kubernetes deployments with Cilium
lizrice
0
150
When is a Secure Connection not encrypted? And other stories
lizrice
1
66
Keeping it simple: Cilium Mesh - networking for multi-cloud Kubernetes and beyond
lizrice
1
570
How Many Proxies Do You Need
lizrice
1
130
Beginner's Guide to eBPF Programming for Networking
lizrice
1
2.2k
Contributing to Open Source - what's in it for my business?
lizrice
0
45
Cloud Native eBPF Superpowers
lizrice
0
240
Other Decks in Technology
See All in Technology
AWS re:Invent 2024登壇資料(GBL206-JA: Unleashing the power of generative AI on AWS for your business)
minorun365
PRO
7
250
Kubernetesを知る
logica0419
18
5.3k
プロダクトの爆速開発を支える、 「作らない・削る・尖らせる」技術
applism118
10
8.7k
セキュリティ系アップデート全体像と AWS Organizations 新ポリシー「宣言型ポリシー」を紹介 / reGrowth 2024 Security
masahirokawahara
0
170
品質管理チームのEMとして大事にしていること / QA EM
nihonbuson
0
840
【AWS re:Invent 2024】Amazon Bedrock アップデート総まとめ
minorun365
PRO
7
610
大規模サーバ移行を成功に導くための事前調査フェーズの工夫事例
fukuchiiinu
2
110
多様なロール経験が導いたエンジニアキャリアのナビゲーション
coconala_engineer
1
160
Kubernetesトラフィックルーティング徹底解説/Kubernetes-traffic-deep-dive
oracle4engineer
PRO
3
350
宇宙最速のランチRecap LT会(AWS re:Invent 2024)
watany
1
380
GDGoC開発体験談 - Gemini生成AI活用ハッカソン / GASとFirebaseで挑むパン屋のフードロス解決 -
hotekagi
1
760
Kaggleふりかえり会〜LLM 20 Questions & ISIC 2024
recruitengineers
PRO
2
190
Featured
See All Featured
Become a Pro
speakerdeck
PRO
25
5k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
232
17k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
6
480
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
4
410
The Pragmatic Product Professional
lauravandoore
32
6.3k
Into the Great Unknown - MozCon
thekraken
33
1.5k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
27
2.1k
Docker and Python
trallard
41
3.1k
A Philosophy of Restraint
colly
203
16k
No one is an island. Learnings from fostering a developers community.
thoeni
19
3k
Gamification - CAS2011
davidbonilla
80
5.1k
Transcript
eBPF for Security Observability Liz Rice | @lizrice Chief Open
Source Officer, Isovalent
@lizrice
@lizrice What is ? extended Berkeley Packet Filter
@lizrice What is ? Makes the kernel programmable
@lizrice userspace kernel app event system calls eBPF program Run
custom code in the kernel
@lizrice SEC("kprobe/sys_execve") int hello(void *ctx) { bpf_trace_printk("Hello World!"); return 0;
} $ sudo ./hello bash-20241 [004] d... 84210.752785: 0: Hello World! bash-20242 [004] d... 84216.321993: 0: Hello World! bash-20243 [004] d... 84225.858880: 0: Hello World! Info about process that called execve syscall + userspace code to load eBPF program eBPF Hello World
Dynamic changes to kernel behaviour
Dynamic tracing tools
@lizrice userspace kernel Tracing tool event eBPF program Use eBPF
to collect event metrics eBPF Map metrics load Gather & show metrics
@lizrice eBPF tracing tools from iovisor/bcc
@lizrice eBPF tracing - opensnoop ~/bcc/libbpf-tools$ sudo ./opensnoop PID COMM
FD ERR PATH 5040 node 21 0 /proc/5132/cmdline 5040 node 21 0 /proc/6460/cmdline 5040 node 21 0 /proc/6460/cmdline 6461 opensnoop 18 0 /etc/localtime 5040 node 21 0 /proc/5132/cmdline 5040 node 21 0 /proc/6460/cmdline 5060 node 23 0 /home/liz/.vscode-server/data/User/workspaceStorage/48b53 5040 node 21 0 /proc/5132/cmdline 5040 node 21 0 /proc/6460/cmdline 5040 node 21 0 /proc/5132/cmdline 5040 node 21 0 /proc/6460/cmdline …
eBPF and Kubernetes
@lizrice userspace kernel pod container pod container container One kernel
per host
@lizrice userspace kernel networking access files create containers One kernel
per host pod container pod container container
@lizrice userspace kernel app app pods networking access files create
containers Kernel aware of everything on the host
@lizrice userspace app kernel app pods networking access files create
containers eBPF programs can be aware of everything
@lizrice $ kubectl gadget trace open NODE NAMESPACE POD CONTAINER
PID COMM FD ERR PATH kind-2-control-plane default xwing spaceship 361876 vi 3 0 /etc/passwd eBPF tracing on Kubernetes - Inspektor Gadget Kubernetes info
@lizrice eBPF observability tools -
@lizrice eBPF observability tools - Cilium Hubble
eBPF observability
eBPF security observability
@lizrice Security observability
@lizrice Security observability
@lizrice What activity do we care about for security? eBPF
programs
@lizrice Syscall checks within the kernel
@lizrice TOCTTOU vulnerabilities with syscalls For more details • Leo
Di Donato & KP Singh at CN eBPF Day 2021 • Rex Guo & Junyuan Zeng at DEFCON 29 on Phantom attacks Attacker changes params after inspection
@lizrice Need to make the check at the right place
@lizrice Linux Security Modules • Stable interface • Safe places
to make checks
@lizrice BPF LSM • Stable interface • Safe places to
make checks + eBPF benefits • Dynamic • Protect pre-existing processes
@lizrice $ sudo ./chmoddemo & [1] 7631 $ sudo cat
/sys/kernel/debug/tracing/trace_pipe chmod-7776 [001] d... 38197.342160: bpf_trace_printk: lsm path_chmod liz BPF LSM hook has kernel info populated SEC("lsm/path_chmod") int BPF_PROG(path_chmod, const struct path *path, umode_t mode) { bpf_printk("lsm path_chmod %s\n", path->dentry->d_iname); return 0; } Filename known to kernel
@lizrice BPF LSM • Stable interface • Safe places to
make checks + eBPF benefits • Dynamic • Protect pre-existing processes But needs kernel 5.7+ & Kubernetes context?
How stable is the Linux kernel?
@lizrice Cilium Tetragon • Safe places to make checks +
eBPF benefits • Dynamic • Protect pre-existing processes Uses kernel knowledge to hook into sufficiently stable functions Adds Kubernetes context
@lizrice Photo credit: Bibafu A Tetragonisca angustula bee guarding the
nest-entrance
@lizrice apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: "etc-files" spec: kprobes:
- call: "fd_install" … matchArgs: - index: 1 operator: "Prefix" values: - "/etc/" … Cilium Tetragon tracing policy + Policy “follows” file descriptor through read, write & close events
@lizrice $ kubectl logs ds/tetragon -c export-stdout -f | tetragon
observe 🚀 process default/xwing /usr/bin/vi /etc/passwd 📬 open default/xwing /usr/bin/vi /etc/passwd 📪 close default/xwing /usr/bin/vi 📬 open default/xwing /usr/bin/vi /etc/passwd 📝 write default/xwing /usr/bin/vi /etc/passwd 1275 bytes 📪 close default/xwing /usr/bin/vi 💥 exit default/xwing /usr/bin/vi /etc/passwd 0 Cilium Tetragon observe Policy events Kubernetes info
@lizrice Combined network and runtime visibility
eBPF preventative runtime security
@lizrice Network policy → eBPF programs drop packets
@lizrice Preventative actions from user space
@lizrice Preventative actions from kernel
@lizrice $ kubectl logs ds/tetragon -c export-stdout -f | tetragon
observe 🚀 process default/xwing /usr/bin/vi /etc/passwd 📬 open default/xwing /usr/bin/vi /etc/passwd 📪 close default/xwing /usr/bin/vi 📬 open default/xwing /usr/bin/vi /etc/passwd 📝 write default/xwing /usr/bin/vi /etc/passwd 1269 bytes 💥 exit default/xwing /usr/bin/vi /etc/passwd SIGKILL Cilium Tetragon observe Killed before write
eBPF security observability • Dynamic instrumentation - zero app modifications
• Contextual information, Kubernetes identity-aware • Option for runtime enforcement from the kernel
Thank you! cilium/tetragon @ciliumproject cilium.io | ebpf.io @lizrice