Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
eBPF in Microservices Observability at eBPF Day
Search
JBD
October 11, 2021
Programming
1
2.1k
eBPF in Microservices Observability at eBPF Day
JBD
October 11, 2021
Tweet
Share
More Decks by JBD
See All by JBD
eBPF in Microservices Observability
rakyll
1
1.7k
OpenTelemetry at AWS
rakyll
1
1.9k
Debugging Code Generation in Go
rakyll
5
1.6k
Are you ready for production?
rakyll
8
2.8k
Servers are doomed to fail
rakyll
3
1.5k
Serverless Containers
rakyll
1
260
Critical Path Analysis
rakyll
0
620
Monitoring and Debugging Containers
rakyll
2
1.1k
CPDD
rakyll
0
4.2k
Other Decks in Programming
See All in Programming
Infer入門
riru
4
1.6k
MCPで実現するAIエージェント駆動のNext.jsアプリデバッグ手法
nyatinte
7
970
個人軟體時代
ethanhuang13
0
220
LLMOpsのパフォーマンスを支える技術と現場で実践した改善
po3rin
8
990
Portapad紹介プレゼンテーション
gotoumakakeru
1
130
あのころの iPod を どうにか再生させたい
orumin
2
2.5k
The state patternの実践 個人開発で培ったpractice集
miyanokomiya
0
150
フロントエンドのmonorepo化と責務分離のリアーキテクト
kajitack
2
150
rage against annotate_predecessor
junk0612
0
140
AIエージェント開発、DevOps and LLMOps
ymd65536
1
360
旅行プランAIエージェント開発の裏側
ippo012
1
490
Improving my own Ruby thereafter
sisshiki1969
1
130
Featured
See All Featured
Testing 201, or: Great Expectations
jmmastey
45
7.6k
GitHub's CSS Performance
jonrohan
1032
460k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.4k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Gamification - CAS2011
davidbonilla
81
5.4k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
139
34k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.8k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Practical Orchestrator
shlominoach
190
11k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.6k
Building an army of robots
kneath
306
46k
Transcript
@rakyll eBPF in Microservices Observability Jaana Dogan Principal Engineer, AWS
[email protected]
@rakyll About me • Not a Linux developer. • Working
on monitoring, observability and performance. • Multi-tenancy and microservices focus.
@rakyll
@rakyll How does eBPF work? process JIT compiler Verifier Sockets
TCP/IP BPF Maps code (accessible from the user space)
@rakyll Where can eBPF hook into? - Kernel and user
functions - System calls - Network events - Kernel tracepoints
@rakyll Challenges in microservices
@rakyll Challenges in microservices We don’t just monitor VMs or
processes. We monitor critical paths.
@rakyll What’s next? service service database storage service
@rakyll What’s next? service service database storage service
@rakyll Challenges in microservices Context matters. Downstream stack don’t have
context.
@rakyll What’s next? process Linux kernel process process M:N Problem
@rakyll What’s next? process Linux kernel process process RPCs M:N
Problem
@rakyll What’s next? process Linux kernel process process RPCs container
container M:N Problem
@rakyll What’s next? process Linux kernel process process RPCs container
container Kubernetes pod, ECS task M:N Problem
@rakyll Challenges in microservices First, we debug the path of
the request. We debug functions or syscalls secondarily.
@rakyll Challenges in microservices Too much data. Need runtime controls
to modify the collection.
@rakyll Challenges in microservices Instrumentation is a two-year roadmap. Data
is not consistent.
@rakyll Recap Out of the box instrumentation is critical. Networking
observability is essential. Extensibility in runtime is needed. Decoration and enrichment is needed.
@rakyll How does eBPF help?
@rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols
(MySQL, Postgres, ...)
@rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols
(MySQL, Postgres, ...)
@rakyll Service Maps
@rakyll Distributed Traces Automatically create trace span if a trace
header is present. Your job is to generate and propagate the header. GET /users HTTP/1.1 Host: users.service Accept-Encoding: gzip, deflate Connection: Keep-Alive Traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
@rakyll Continuous Profiling
@rakyll Extensibility
@rakyll Decorating with Context eBPF agent process JIT compiler Verifier
Sockets TCP/IP BPF Maps API Server
@rakyll Decorating with Context
@rakyll Several projects... - Cillium/Hubble - Pixie - Flowmill -
Prodfiler - Parca
@rakyll What’s next? - High level language? - More platforms
supporting eBPF? - Reusable eBPF event processors? - Signed programs?
@rakyll Thank you Jaana Dogan
[email protected]
@rakyll After party? Pixie KubeCon Happy Hour hosted by New
Relic Tomorrow, 8:30 PM PDT RSVP: pixiehh.eventbrite.com
@rakyll Runtime Extensibility eBPF agent process JIT compiler Verifier Sockets
TCP/IP BPF Maps code