Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
eBPF in Microservices Observability at eBPF Day
Search
JBD
October 11, 2021
Programming
1
2.1k
eBPF in Microservices Observability at eBPF Day
JBD
October 11, 2021
Tweet
Share
More Decks by JBD
See All by JBD
eBPF in Microservices Observability
rakyll
1
1.7k
OpenTelemetry at AWS
rakyll
1
1.9k
Debugging Code Generation in Go
rakyll
5
1.6k
Are you ready for production?
rakyll
8
2.8k
Servers are doomed to fail
rakyll
3
1.5k
Serverless Containers
rakyll
1
260
Critical Path Analysis
rakyll
0
630
Monitoring and Debugging Containers
rakyll
2
1.1k
CPDD
rakyll
0
4.2k
Other Decks in Programming
See All in Programming
大規模アプリのDIフレームワーク刷新戦略 ~過去最大規模の並行開発を止めずにアプリ全体に導入するまで~
mot_techtalk
0
380
CSC509 Lecture 06
javiergs
PRO
0
240
WebエンジニアがSwiftをブラウザで動かすプレイグラウンドを作ってみた
ohmori_yusuke
0
170
SpecKitでどこまでできる? コストはどれくらい?
leveragestech
0
520
私はどうやって技術力を上げたのか
yusukebe
43
17k
XP, Testing and ninja testing ZOZ5
m_seki
2
290
タスクの特性や不確実性に応じた最適な作業スタイルの選択(ペアプロ・モブプロ・ソロプロ)と実践 / Optimal Work Style Selection: Pair, Mob, or Solo Programming.
honyanya
3
140
Railsだからできる 例外業務に禍根を残さない 設定設計パターン
ei_ei_eiichi
0
250
(Extension DC 2025) Actor境界を越える技術
teamhimeh
1
220
私達はmodernize packageに夢を見るか feat. go/analysis, go/ast / Go Conference 2025
kaorumuta
2
490
いま中途半端なSwift 6対応をするより、Default ActorやApproachable Concurrencyを有効にしてからでいいんじゃない?
yimajo
2
340
CSC509 Lecture 04
javiergs
PRO
0
290
Featured
See All Featured
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.2k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
2.6k
Building an army of robots
kneath
306
46k
Reflections from 52 weeks, 52 projects
jeffersonlam
352
21k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
9
580
Documentation Writing (for coders)
carmenintech
75
5k
The Art of Programming - Codeland 2020
erikaheidi
56
14k
Visualization
eitanlees
148
16k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
54
3k
RailsConf 2023
tenderlove
30
1.2k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Transcript
@rakyll eBPF in Microservices Observability Jaana Dogan Principal Engineer, AWS
[email protected]
@rakyll About me • Not a Linux developer. • Working
on monitoring, observability and performance. • Multi-tenancy and microservices focus.
@rakyll
@rakyll How does eBPF work? process JIT compiler Verifier Sockets
TCP/IP BPF Maps code (accessible from the user space)
@rakyll Where can eBPF hook into? - Kernel and user
functions - System calls - Network events - Kernel tracepoints
@rakyll Challenges in microservices
@rakyll Challenges in microservices We don’t just monitor VMs or
processes. We monitor critical paths.
@rakyll What’s next? service service database storage service
@rakyll What’s next? service service database storage service
@rakyll Challenges in microservices Context matters. Downstream stack don’t have
context.
@rakyll What’s next? process Linux kernel process process M:N Problem
@rakyll What’s next? process Linux kernel process process RPCs M:N
Problem
@rakyll What’s next? process Linux kernel process process RPCs container
container M:N Problem
@rakyll What’s next? process Linux kernel process process RPCs container
container Kubernetes pod, ECS task M:N Problem
@rakyll Challenges in microservices First, we debug the path of
the request. We debug functions or syscalls secondarily.
@rakyll Challenges in microservices Too much data. Need runtime controls
to modify the collection.
@rakyll Challenges in microservices Instrumentation is a two-year roadmap. Data
is not consistent.
@rakyll Recap Out of the box instrumentation is critical. Networking
observability is essential. Extensibility in runtime is needed. Decoration and enrichment is needed.
@rakyll How does eBPF help?
@rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols
(MySQL, Postgres, ...)
@rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols
(MySQL, Postgres, ...)
@rakyll Service Maps
@rakyll Distributed Traces Automatically create trace span if a trace
header is present. Your job is to generate and propagate the header. GET /users HTTP/1.1 Host: users.service Accept-Encoding: gzip, deflate Connection: Keep-Alive Traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
@rakyll Continuous Profiling
@rakyll Extensibility
@rakyll Decorating with Context eBPF agent process JIT compiler Verifier
Sockets TCP/IP BPF Maps API Server
@rakyll Decorating with Context
@rakyll Several projects... - Cillium/Hubble - Pixie - Flowmill -
Prodfiler - Parca
@rakyll What’s next? - High level language? - More platforms
supporting eBPF? - Reusable eBPF event processors? - Signed programs?
@rakyll Thank you Jaana Dogan
[email protected]
@rakyll After party? Pixie KubeCon Happy Hour hosted by New
Relic Tomorrow, 8:30 PM PDT RSVP: pixiehh.eventbrite.com
@rakyll Runtime Extensibility eBPF agent process JIT compiler Verifier Sockets
TCP/IP BPF Maps code