Upgrade to Pro — share decks privately, control downloads, hide ads and more …

eBPF in Microservices Observability

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for JBD JBD
August 18, 2021
Programming
1
1.7k

eBPF in Microservices Observability

Avatar for JBD

JBD

August 18, 2021
Tweet

More Decks by JBD

See All by JBD
eBPF in Microservices Observability at eBPF Day
Avatar for JBD rakyll
1
2.2k
OpenTelemetry at AWS
Avatar for JBD rakyll
1
1.9k
Debugging Code Generation in Go
Avatar for JBD rakyll
5
1.6k
Are you ready for production?
Avatar for JBD rakyll
8
2.9k
Servers are doomed to fail
Avatar for JBD rakyll
3
1.6k
Serverless Containers
Avatar for JBD rakyll
1
270
Critical Path Analysis
Avatar for JBD rakyll
0
680
Monitoring and Debugging Containers
Avatar for JBD rakyll
2
1.1k
CPDD
Avatar for JBD rakyll
0
4.3k

Other Decks in Programming

See All in Programming
16年目のピクシブ百科事典を支える最新の技術基盤 / The Modern Tech Stack Powering Pixiv Encyclopedia in its 16th Year
Avatar for ahu ahuglajbclajep
5
1k
Rust 製のコードエディタ “Zed” を使ってみた
Avatar for NearMeの技術発表資料です nearme_tech
PRO
0
210
CSC307 Lecture 01
Avatar for Javier Gonzalez-Sanchez javiergs
PRO
0
690
並行開発のためのコードレビュー
Avatar for Miyuki miyukiw
0
1.2k
なぜSQLはAIぽく見えるのか/why does SQL look AI like
Avatar for florets1 florets1
0
480
Fluid Templating in TYPO3 14
Avatar for Simon Praetorius s2b
0
130
組織で育むオブザーバビリティ
Avatar for ryotaro kobayashi ryota_hnk
0
180
プロダクトオーナーから見たSOC2 _SOC2ゆるミートアップ#2
Avatar for Kenta Suzuki kekekenta
0
220
QAフローを最適化し、品質水準を満たしながらリリースまでの期間を最短化する #RSGT2026
Avatar for shibayu36 shibayu36
2
4.4k
Oxlint JS plugins
Avatar for kazupon kazupon
1
1k
Apache Iceberg V3 and migration to V3
Avatar for Tomohiro Tanaka tomtanaka
0
170
HTTPプロトコル正しく理解していますか？ 〜かわいい猫と共に学ぼう。ฅ^•ω•^ฅ ﾆｬ〜
Avatar for ♛Heitor Hirose hekuchan
2
690

Featured

See All Featured
XXLCSS - How to scale CSS and keep your sanity
Avatar for Zaharenia Atzitzikaki sugarenia
249
1.3M
Navigating Algorithm Shifts & AI Overviews - #SMXNext
Avatar for Aleyda Solis aleyda
0
1.1k
Chasing Engaging Ingredients in Design
Avatar for Sebastian Deterding codingconduct
0
110
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
Avatar for Rebecca Miller-Webster rmw
35
3.4k
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
240
Abbi's Birthday
Avatar for Violet Bock coloredviolet
1
4.8k
Avoiding the “Bad Training, Faster” Trap in the Age of AI
Avatar for Mike Taylor tmiket
0
79
How to build an LLM SEO readiness audit: a practical framework
Avatar for Nick Samuel nmsamuel
1
650
SEOcharity - Dark patterns in SEO and UX: How to avoid them and build a more ethical web
Avatar for Sara Fernández Carmona sarafernandez
0
120
Between Models and Reality
Avatar for mayunak mayunak
1
190
How To Speak Unicorn (iThemes Webinar)
Avatar for Michelle Schulp Hunt marktimemedia
1
380
Writing Fast Ruby
Avatar for Erik Berlin sferik
630
62k

Transcript

  1. @rakyll eBPF in Microservices Observability Jaana Dogan Principal Engineer, AWS

    [email protected]

  2. @rakyll About me • Not a Linux developer. • Working

    on monitoring, observability and performance. • Multi-tenancy and microservices focus.

  3. @rakyll

  4. @rakyll How does eBPF work? process JIT compiler Verifier Sockets

    TCP/IP BPF Maps code (accessible from the user space)

  5. @rakyll Where can eBPF hook into? - Kernel and user

    functions - System calls - Network events - Kernel tracepoints

  6. @rakyll Challenges in microservices

  7. @rakyll Challenges in microservices We don’t just monitor VMs or

    processes. We monitor critical paths.

  8. @rakyll What’s next? service service database storage service

  9. @rakyll What’s next? service service database storage service

  10. @rakyll Challenges in microservices Context matters. Downstream stack don’t have

    context.

  11. @rakyll What’s next? process Linux kernel process process M:N Problem

  12. @rakyll What’s next? process Linux kernel process process RPCs M:N

    Problem

  13. @rakyll What’s next? process Linux kernel process process RPCs container

    container M:N Problem

  14. @rakyll What’s next? process Linux kernel process process RPCs container

    container Kubernetes pod, ECS task M:N Problem

  15. @rakyll Challenges in microservices We initially debug RPCs. We debug

    functions or syscalls secondarily.

  16. @rakyll Challenges in microservices Too much data. Need runtime controls

    to modify the collection.

  17. @rakyll Challenges in microservices Instrumentation is a two-year roadmap. Data

    is not consistent.

  18. @rakyll Networking observability is core. Out of the box instrumentation

    is essential. Extensibility in runtime is critical. Decoration and enrichment is needed.

  19. @rakyll How does eBPF help?

  20. @rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols

    (MySQL, Postgres, ...)

  21. @rakyll Service Maps

  22. @rakyll Distributed Traces Automatically create request span if a trace

    header is present. GET /users HTTP/1.1 Host: users.service Accept-Encoding: gzip, deflate Connection: Keep-Alive Traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

  23. @rakyll Fleet-wide Profiling context kernel

  24. @rakyll Decorating with Context eBPF agent process JIT compiler Verifier

    Sockets TCP/IP BPF Maps API Server

  25. @rakyll Runtime Extensibility eBPF agent process JIT compiler Verifier Sockets

    TCP/IP BPF Maps code

  26. @rakyll Examples - Cillium/Hubble - Pixie - Flowmill

  27. @rakyll What’s next? - High level language to write probes.

    - Make eBPF agents widely available. - More platforms supporting eBPF. - Reusable eBPF event processing.

  28. @rakyll Thank you Jaana Dogan [email protected]