Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
eBPF in Microservices Observability
JBD
August 18, 2021
Programming
1
1.3k
eBPF in Microservices Observability
JBD
August 18, 2021
Tweet
Share
More Decks by JBD
See All by JBD
eBPF in Microservices Observability at eBPF Day
rakyll
1
1.6k
OpenTelemetry at AWS
rakyll
1
1.6k
Debugging Code Generation in Go
rakyll
5
1.3k
Are you ready for production?
rakyll
8
2k
Servers are doomed to fail
rakyll
3
1.2k
Serverless Containers
rakyll
1
160
Critical Path Analysis
rakyll
0
230
Monitoring and Debugging Containers
rakyll
2
970
CPDD
rakyll
0
3.8k
Other Decks in Programming
See All in Programming
もしも、 上司に鬼退治を命じられたら~プロジェクト計画編~
higuuu
0
290
Reactでアプリケーションを構築する多様化
sakito
4
3.4k
The future of trust stores in Python
sethmlarson
0
180
競プロへの誘 -いざな-
u76ner
0
380
[RailsConf 2022] The pitfalls of realtime-ification
palkan
0
240
Explore Java 17 and beyond
josepaumard
3
660
モデリングの費用対効果
masuda220
PRO
3
910
ソフトウェアテストなんて他人事! だと思っていた私が始めた小さな取り組み
izumii19
0
240
職場にPythonistaを増やす方法
soogie
0
320
Your Test Suite is Making Too Many Database Calls!
joelq
0
120
アプリのログをチーム外で活用してもらうためにやったこと
shotakashihara
0
190
How useEvent would change our applications
koba04
1
1.7k
Featured
See All Featured
The Illustrated Children's Guide to Kubernetes
chrisshort
14
35k
Embracing the Ebb and Flow
colly
73
3.3k
Done Done
chrislema
174
14k
Art Directing for the Web. Five minutes with CSS Template Areas
malarkey
196
9.4k
Scaling GitHub
holman
451
140k
Bootstrapping a Software Product
garrettdimon
295
110k
Building Flexible Design Systems
yeseniaperezcruz
310
33k
Visualization
eitanlees
124
11k
Three Pipe Problems
jasonvnalue
89
8.6k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
4
450
Fireside Chat
paigeccino
11
1.3k
Building Applications with DynamoDB
mza
83
4.6k
Transcript
@rakyll eBPF in Microservices Observability Jaana Dogan Principal Engineer, AWS
jbd@amazon.com
@rakyll About me • Not a Linux developer. • Working
on monitoring, observability and performance. • Multi-tenancy and microservices focus.
@rakyll
@rakyll How does eBPF work? process JIT compiler Verifier Sockets
TCP/IP BPF Maps code (accessible from the user space)
@rakyll Where can eBPF hook into? - Kernel and user
functions - System calls - Network events - Kernel tracepoints
@rakyll Challenges in microservices
@rakyll Challenges in microservices We don’t just monitor VMs or
processes. We monitor critical paths.
@rakyll What’s next? service service database storage service
@rakyll What’s next? service service database storage service
@rakyll Challenges in microservices Context matters. Downstream stack don’t have
context.
@rakyll What’s next? process Linux kernel process process M:N Problem
@rakyll What’s next? process Linux kernel process process RPCs M:N
Problem
@rakyll What’s next? process Linux kernel process process RPCs container
container M:N Problem
@rakyll What’s next? process Linux kernel process process RPCs container
container Kubernetes pod, ECS task M:N Problem
@rakyll Challenges in microservices We initially debug RPCs. We debug
functions or syscalls secondarily.
@rakyll Challenges in microservices Too much data. Need runtime controls
to modify the collection.
@rakyll Challenges in microservices Instrumentation is a two-year roadmap. Data
is not consistent.
@rakyll Networking observability is core. Out of the box instrumentation
is essential. Extensibility in runtime is critical. Decoration and enrichment is needed.
@rakyll How does eBPF help?
@rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols
(MySQL, Postgres, ...)
@rakyll Service Maps
@rakyll Distributed Traces Automatically create request span if a trace
header is present. GET /users HTTP/1.1 Host: users.service Accept-Encoding: gzip, deflate Connection: Keep-Alive Traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
@rakyll Fleet-wide Profiling context kernel
@rakyll Decorating with Context eBPF agent process JIT compiler Verifier
Sockets TCP/IP BPF Maps API Server
@rakyll Runtime Extensibility eBPF agent process JIT compiler Verifier Sockets
TCP/IP BPF Maps code
@rakyll Examples - Cillium/Hubble - Pixie - Flowmill
@rakyll What’s next? - High level language to write probes.
- Make eBPF agents widely available. - More platforms supporting eBPF. - Reusable eBPF event processing.
@rakyll Thank you Jaana Dogan jbd@amazon.com