DISTRIBUTED SYSTEM TRACING
@dustyburwell
Monitoring Performance in Distributed Systems
Slide 2
Slide 2 text
No content
Slide 3
Slide 3 text
Dusty Burwell
@dustyburwell
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
No content
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
Traditional Options
Slide 11
Slide 11 text
CPU Tracing
Statistical or Detailed
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
No content
Slide 15
Slide 15 text
Memory Dumping
Slide 16
Slide 16 text
Structured Log Aggregation
Slide 17
Slide 17 text
Structured Log Aggregation
a.k.a. read the f’n error
Slide 18
Slide 18 text
Microservices Tracing
Slide 19
Slide 19 text
Dapper
http://bit.ly/cm-dapper
Slide 20
Slide 20 text
Vocabulary
Slide 21
Slide 21 text
Trace
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
Trace ID
Slide 24
Slide 24 text
Span
Slide 25
Slide 25 text
Trace ID
Slide 26
Slide 26 text
Trace ID
Slide 27
Slide 27 text
Trace ID
Span ID
Slide 28
Slide 28 text
Trace ID
Span ID
Slide 29
Slide 29 text
Trace ID
Span ID
Parent Span ID
Slide 30
Slide 30 text
Trace ID
Span ID
Parent Span ID
Client Start - timestamp
Server Start - timestamp
Server End - timestamp
Client End - timestamp
Slide 31
Slide 31 text
Annotation
Slide 32
Slide 32 text
Trace ID
Span ID
Parent Span ID
Client Start - timestamp
Server Start - timestamp
Server End - timestamp
Client End - timestamp
Slide 33
Slide 33 text
Trace ID
Span ID
Parent Span ID
Client Start - timestamp
Server Start - timestamp
Server End - timestamp
Client End - timestamp
Slide 34
Slide 34 text
Trace ID
Span ID
Parent Span ID
Client Start - timestamp
Server Start - timestamp
Server End - timestamp
Client End - timestamp
Service Name
Endpoint
Is Error?
Error code
Etc.
Slide 35
Slide 35 text
No content
Slide 36
Slide 36 text
Parts
Slide 37
Slide 37 text
Instrumentation
Slide 38
Slide 38 text
Transport libraries (HTTP, etc)
Middleware or similar
Headers passed from instrumented
libraries and received by
middleware pass along Trace ID
and Parent Span ID