Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
eBPF in Microservices Observability
Search
JBD
August 18, 2021
Programming
1
1.6k
eBPF in Microservices Observability
JBD
August 18, 2021
Tweet
Share
More Decks by JBD
See All by JBD
eBPF in Microservices Observability at eBPF Day
rakyll
1
2k
OpenTelemetry at AWS
rakyll
1
1.8k
Debugging Code Generation in Go
rakyll
5
1.5k
Are you ready for production?
rakyll
8
2.6k
Servers are doomed to fail
rakyll
3
1.5k
Serverless Containers
rakyll
1
240
Critical Path Analysis
rakyll
0
500
Monitoring and Debugging Containers
rakyll
2
1.1k
CPDD
rakyll
0
4.1k
Other Decks in Programming
See All in Programming
Amazon Neptuneで始める初めてのグラフDB ー グラフDBを使う意味を考える ー
satoshi256kbyte
2
260
What we keep in mind when migrating from Serverless Framework to AWS CDK and AWS SAM
kasacchiful
1
140
rails_girls_is_my_gate_to_join_the_ruby_commuinty
maimux2x
0
200
全部見せます! クラシルリワードのSwiftTesting移行プロジェクト
uetyo
0
200
あなたのアプリ、ログはでてますか?あるいはログをだしてますか? (Funabashi.dev用 軽量版)
uzulla
2
110
私のEbitengineの第一歩
qt_luigi
0
440
労務ドメインを快適に開発する方法 / How to Comfortably Develop in the Labor Domain
yuki21
1
250
Some more adventure of Happy Eyeballs
coe401_
2
180
僕が思い描くTypeScriptの未来を勝手に先取りする
yukukotani
9
2.4k
Regular Expressions, REXML, Automata Learning
makenowjust
0
220
月間4.5億回再生を超える大規模サービス TVer iOSアプリのリアーキテクチャ戦略 - iOSDC2024
techtver
PRO
1
830
エンジニア1年目で複雑なコードの改善に取り組んだ話
mtnmr
3
2k
Featured
See All Featured
Bash Introduction
62gerente
608
210k
How to Think Like a Performance Engineer
csswizardry
16
950
Visualization
eitanlees
142
15k
Raft: Consensus for Rubyists
vanstee
135
6.5k
Making Projects Easy
brettharned
113
5.8k
The Brand Is Dead. Long Live the Brand.
mthomps
53
38k
The Straight Up "How To Draw Better" Workshop
denniskardys
230
130k
Building Flexible Design Systems
yeseniaperezcruz
325
37k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
28
1.6k
Bootstrapping a Software Product
garrettdimon
PRO
304
110k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
248
20k
Fashionably flexible responsive web design (full day workshop)
malarkey
401
65k
Transcript
@rakyll eBPF in Microservices Observability Jaana Dogan Principal Engineer, AWS
[email protected]
@rakyll About me • Not a Linux developer. • Working
on monitoring, observability and performance. • Multi-tenancy and microservices focus.
@rakyll
@rakyll How does eBPF work? process JIT compiler Verifier Sockets
TCP/IP BPF Maps code (accessible from the user space)
@rakyll Where can eBPF hook into? - Kernel and user
functions - System calls - Network events - Kernel tracepoints
@rakyll Challenges in microservices
@rakyll Challenges in microservices We don’t just monitor VMs or
processes. We monitor critical paths.
@rakyll What’s next? service service database storage service
@rakyll What’s next? service service database storage service
@rakyll Challenges in microservices Context matters. Downstream stack don’t have
context.
@rakyll What’s next? process Linux kernel process process M:N Problem
@rakyll What’s next? process Linux kernel process process RPCs M:N
Problem
@rakyll What’s next? process Linux kernel process process RPCs container
container M:N Problem
@rakyll What’s next? process Linux kernel process process RPCs container
container Kubernetes pod, ECS task M:N Problem
@rakyll Challenges in microservices We initially debug RPCs. We debug
functions or syscalls secondarily.
@rakyll Challenges in microservices Too much data. Need runtime controls
to modify the collection.
@rakyll Challenges in microservices Instrumentation is a two-year roadmap. Data
is not consistent.
@rakyll Networking observability is core. Out of the box instrumentation
is essential. Extensibility in runtime is critical. Decoration and enrichment is needed.
@rakyll How does eBPF help?
@rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols
(MySQL, Postgres, ...)
@rakyll Service Maps
@rakyll Distributed Traces Automatically create request span if a trace
header is present. GET /users HTTP/1.1 Host: users.service Accept-Encoding: gzip, deflate Connection: Keep-Alive Traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
@rakyll Fleet-wide Profiling context kernel
@rakyll Decorating with Context eBPF agent process JIT compiler Verifier
Sockets TCP/IP BPF Maps API Server
@rakyll Runtime Extensibility eBPF agent process JIT compiler Verifier Sockets
TCP/IP BPF Maps code
@rakyll Examples - Cillium/Hubble - Pixie - Flowmill
@rakyll What’s next? - High level language to write probes.
- Make eBPF agents widely available. - More platforms supporting eBPF. - Reusable eBPF event processing.
@rakyll Thank you Jaana Dogan
[email protected]