Upgrade to Pro — share decks privately, control downloads, hide ads and more …

eBPF in Microservices Observability at eBPF Day

E7526ec3e801f8ba99f6746498a154a6?s=47 JBD
October 11, 2021
Programming
1
1.3k

eBPF in Microservices Observability at eBPF Day

E7526ec3e801f8ba99f6746498a154a6?s=128

JBD

October 11, 2021
Tweet

More Decks by JBD

See All by JBD
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
1
1k
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
1
1.5k
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
5
1.2k
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
8
1.8k
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
3
1.2k
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
1
140
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
0
170
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
2
910
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
0
3.8k

Other Decks in Programming

See All in Programming
E86b46be0f4e61db6b28becc5493bbab?s=48 januswel
0
1.1k
D3be793aaf76ed1915295001712ce0f1?s=48 jrfk
0
310
30e63a1ad21d3944b288fc3d33d1e920?s=48 amrrs
0
130
B3b2139e4f2c0eca4efe2379fcebc1c5?s=48 afilina
PRO
1
470
B793da271a45d149b52f507bca9f137c?s=48 77web
2
630
0187ef0e45a8d745eafc223cf2ebb90c?s=48 tosh7
0
240
89d1ae72e8683e44c744f4cab8d99f39?s=48 brainpadpr
0
510
A3fa7c92c26f1170762202d574c19e20?s=48 soh335
1
310
4ce0d60e368cef89450f1d9dfdcb493f?s=48 n8ebel
1
190
Cb504213a5310e01faabbc0b3fca3d39?s=48 napple29
0
160
23e7f90fa36af44f13000c1229240984?s=48 uri
2
200
81190ba1ece00c93c5d9736c2940895c?s=48 tasugi
0
110

Featured

See All Featured
027d1ebf66cc039a0bd3b55eeadbe75d?s=48 sachag
270
17k
053e75a5b48b44d6dd0612795dfb326d?s=48 notwaldorf
54
3.3k
5f2da528927a2ec9ba4fec2069cbc958?s=48 kneath
298
39k
3d6ace9554821d552146413bcdf874f6?s=48 trishagee
37
4.9k
245cee81a9c424266e5e401d844ea881?s=48 lara
27
3.4k
C0390e8b11079f8761c0cd3274ed61cb?s=48 productmarketing
11
1.1k
0c6fd6a08d0f898159cda7cafcacf07f?s=48 afnizarnur
180
14k
7fa6101cd43067653cf4ac6c7ca54305?s=48 akmur
255
20k
C86443373fbfe92996aa882d0d7a59f8?s=48 imathis
484
150k
3ab1249be442027903e1180025340b3f?s=48 reverentgeek
171
7.7k
433acaea1012b25d97ae66da9b998dad?s=48 malarkey
122
16k
5b6a2bd86ef643786a381a212e0ef2f1?s=48 geoffreycrofte
42
1.6k

Transcript

  1. @rakyll eBPF in Microservices Observability Jaana Dogan Principal Engineer, AWS

    jbd@amazon.com

  2. @rakyll About me • Not a Linux developer. • Working

    on monitoring, observability and performance. • Multi-tenancy and microservices focus.

  3. @rakyll

  4. @rakyll How does eBPF work? process JIT compiler Verifier Sockets

    TCP/IP BPF Maps code (accessible from the user space)

  5. @rakyll Where can eBPF hook into? - Kernel and user

    functions - System calls - Network events - Kernel tracepoints

  6. @rakyll Challenges in microservices

  7. @rakyll Challenges in microservices We don’t just monitor VMs or

    processes. We monitor critical paths.

  8. @rakyll What’s next? service service database storage service

  9. @rakyll What’s next? service service database storage service

  10. @rakyll Challenges in microservices Context matters. Downstream stack don’t have

    context.

  11. @rakyll What’s next? process Linux kernel process process M:N Problem

  12. @rakyll What’s next? process Linux kernel process process RPCs M:N

    Problem

  13. @rakyll What’s next? process Linux kernel process process RPCs container

    container M:N Problem

  14. @rakyll What’s next? process Linux kernel process process RPCs container

    container Kubernetes pod, ECS task M:N Problem

  15. @rakyll Challenges in microservices First, we debug the path of

    the request. We debug functions or syscalls secondarily.

  16. @rakyll Challenges in microservices Too much data. Need runtime controls

    to modify the collection.

  17. @rakyll Challenges in microservices Instrumentation is a two-year roadmap. Data

    is not consistent.

  18. @rakyll Recap Out of the box instrumentation is critical. Networking

    observability is essential. Extensibility in runtime is needed. Decoration and enrichment is needed.

  19. @rakyll How does eBPF help?

  20. @rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols

    (MySQL, Postgres, ...)

  21. @rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols

    (MySQL, Postgres, ...)

  22. @rakyll Service Maps

  23. @rakyll Distributed Traces Automatically create trace span if a trace

    header is present. Your job is to generate and propagate the header. GET /users HTTP/1.1 Host: users.service Accept-Encoding: gzip, deflate Connection: Keep-Alive Traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

  24. @rakyll Continuous Profiling

  25. @rakyll Extensibility

  26. @rakyll Decorating with Context eBPF agent process JIT compiler Verifier

    Sockets TCP/IP BPF Maps API Server

  27. @rakyll Decorating with Context

  28. @rakyll Several projects... - Cillium/Hubble - Pixie - Flowmill -

    Prodfiler - Parca

  29. @rakyll What’s next? - High level language? - More platforms

    supporting eBPF? - Reusable eBPF event processors? - Signed programs?

  30. @rakyll Thank you Jaana Dogan jbd@amazon.com

  31. @rakyll After party? Pixie KubeCon Happy Hour hosted by New

    Relic Tomorrow, 8:30 PM PDT RSVP: pixiehh.eventbrite.com

  32. @rakyll Runtime Extensibility eBPF agent process JIT compiler Verifier Sockets

    TCP/IP BPF Maps code