@rakyll
eBPF in Microservices
Observability
Jaana Dogan
Principal Engineer, AWS
[email protected]
Slide 2
Slide 2 text
@rakyll
About me
● Not a Linux developer.
● Working on monitoring, observability and
performance.
● Multi-tenancy and microservices focus.
Slide 3
Slide 3 text
@rakyll
Slide 4
Slide 4 text
@rakyll
How does eBPF work?
process
JIT compiler
Verifier Sockets
TCP/IP
BPF
Maps
code
(accessible from the
user space)
Slide 5
Slide 5 text
@rakyll
Where can eBPF hook into?
- Kernel and user functions
- System calls
- Network events
- Kernel tracepoints
Slide 6
Slide 6 text
@rakyll
Challenges in
microservices
Slide 7
Slide 7 text
@rakyll
Challenges in microservices
We don’t just monitor VMs or processes.
We monitor critical paths.
Slide 8
Slide 8 text
@rakyll
What’s next?
service
service
database storage
service
Slide 9
Slide 9 text
@rakyll
What’s next?
service
service
database storage
service
Slide 10
Slide 10 text
@rakyll
Challenges in microservices
Context matters.
Downstream stack don’t have context.
Slide 11
Slide 11 text
@rakyll
What’s next?
process
Linux kernel
process process
M:N Problem
Slide 12
Slide 12 text
@rakyll
What’s next?
process
Linux kernel
process process
RPCs
M:N Problem
Slide 13
Slide 13 text
@rakyll
What’s next?
process
Linux kernel
process process
RPCs
container container
M:N Problem
Slide 14
Slide 14 text
@rakyll
What’s next?
process
Linux kernel
process process
RPCs
container container
Kubernetes pod,
ECS task
M:N Problem
Slide 15
Slide 15 text
@rakyll
Challenges in microservices
We initially debug RPCs.
We debug functions or syscalls secondarily.
Slide 16
Slide 16 text
@rakyll
Challenges in microservices
Too much data.
Need runtime controls to modify the collection.
Slide 17
Slide 17 text
@rakyll
Challenges in microservices
Instrumentation is a two-year roadmap.
Data is not consistent.
Slide 18
Slide 18 text
@rakyll
Networking observability is core.
Out of the box instrumentation is essential.
Extensibility in runtime is critical.
Decoration and enrichment is needed.