Propagating context and tracing across your distributed process boundaries using OpenTelemetry

Slide 1

Slide 1 text

Tracing across your distributed process boundaries using 5th Oct 2021 Dasith Wijesiriwardena @dasiths OpenTelemetry Melb .NET Meetup 15-16 Sep 2021

Slide 2

Slide 2 text

I am here because I have a love hate relationship with distributed systems. HELLO! I AM Dasith @dasiths dasith.me

Slide 3

Slide 3 text

Distributed tracing using OpenTelemetry Introduction to distributed tracing Agenda Demo using OpenTelemetry .NET SDK Introduction to OpenTelemetry @dasiths

Slide 4

Slide 4 text

“ Peter Drucker @dasiths You can’t manage what you can’t measure..

Slide 5

Slide 5 text

@dasiths

Slide 6

Slide 6 text

Introduction to Distributed Tracing @dasiths

Slide 7

Slide 7 text

Distributed Tracing @dasiths Distributed tracing, also called distributed request tracing, is a method used to profile and monitor applications, especially those built using a microservices architecture. Distributed tracing helps pinpoint where failures occur and what causes poor performance. https://opentracing.io/docs/overview/what-is-tracing/

Slide 8

Slide 8 text

@dasiths Example Distributed Tracing API Gateway Middleware Backend Database APP

Slide 9

Slide 9 text

@dasiths Example Distributed Tracing Middleware Backend Database Correlation_ID = ABC123 Request API Gateway APP

Slide 10

Slide 10 text

@dasiths Example Distributed Tracing Middleware Backend Database Correlation_ID = ABC123 Request API Gateway APP

Slide 11

Slide 11 text

@dasiths Example Distributed Tracing Middleware Backend Database Correlation_ID = ABC123 Request API Gateway APP

Slide 12

Slide 12 text

@dasiths Example Distributed Tracing Middleware Backend Database Correlation_ID = ABC123 Request API Gateway APP

Slide 13

Slide 13 text

@dasiths Example Request Distributed Tracing Middleware Backend Database Correlation_ID = ABC123 API Gateway APP

Slide 14

Slide 14 text

@dasiths Example Distributed Tracing Middleware Backend Database Correlation_ID = ABC123 Request API Gateway APP fields @timestamp, @message | filter @message like /some-unique-thing-for-the-request/ | sort @timestamp desc | limit 20

Slide 15

Slide 15 text

Slide 16

Slide 16 text

@dasiths Example Request Distributed Tracing Middleware Backend Database https://link.springer.com/chapter/10.1007/978-3-030-61140-8_9 Correlation_ID = ABC123 API Gateway APP

Slide 17

Slide 17 text

@dasiths Example Request Distributed Tracing Middleware Backend Database https://link.springer.com/chapter/10.1007/978-3-030-61140-8_9 Correlation_ID = ABC123 API Gateway APP

Slide 18

Slide 18 text

@dasiths Example Request Distributed Tracing Backend Database Middleware API Gateway APP

Slide 19

Slide 19 text

@dasiths Example Request Distributed Tracing Backend Database Middleware API Gateway APP

Slide 20

Slide 20 text

@dasiths Example Request Distributed Tracing Backend Database Middleware API Gateway APP

Slide 21

Slide 21 text

Looking at the Modern Landscape @dasiths

Slide 22

Slide 22 text

• Runs natively on the cloud • Distributed (i.e. Microservices) • Managed by many teams •Broad set of languages and frameworks • Built on top of open source technologies Modern Software.. @dasiths

Slide 23

Slide 23 text

Enter OpenTelemetry @dasiths

Slide 24

Slide 24 text

@dasiths An observability framework for cloud-native software. OpenTelemetry is a collection of tools, APIs, and SDKs. You can use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) for analysis in order to understand your software's performance and behavior. https://opentelemetry.io/

Slide 25

Slide 25 text

Slide 26

Slide 26 text

OpenTelemetry Architecture @dasiths Signals Tracing Metrics Baggage Context Propagation

Slide 27

Slide 27 text

OpenTelemetry Architecture @dasiths Signals Tracing Metrics Baggage Context Propagation Frameworks (.NET, Spring, etc) Applications (Your code) Clients (HTTP, DB, etc ) Tracing

Slide 28

Slide 28 text

OpenTelemetry Architecture @dasiths Signals Tracing Metrics Baggage Context Propagation Cross Cutting Conventions Self Contained API SDK Contrib

Slide 29

Slide 29 text

OpenTelemetry Components @dasiths • Proto •Specification • Instrumentation Libraries •Collector

Slide 30

Slide 30 text

@dasiths Collectors and Exporters SDK SDK SDK SDK SDK Directly exporting traces using the SDK to the monitoring backend Apps instrumented using the OpenTelemetry SDK Monitoring backends with their own data protocols

Slide 31

Slide 31 text

@dasiths Collectors and Exporters Using a Collector (agent) as a proxy to forward telemetry data Apps using the OpenTelemetry Collector (Agent) Monitoring backends with their own data protocols Standalone OpenTelemetry Collector

Slide 32

Slide 32 text

@dasiths Closer Look At The Collector https://opentelemetry.io/docs/

Slide 33

Slide 33 text

@dasiths Reference Architecture https://opentelemetry.io/docs/

Slide 34

Slide 34 text

@dasiths Example: Dapr

Slide 35

Slide 35 text

All About The SPAN @dasiths

Slide 36

Slide 36 text

Defined By All About The Span @dasiths Tracing Span https://medium.com/nikeengineering/hit-the-ground-running-with-distributed-tracing-core-concepts-ff5ad47c7058 A B H C D E F G Signals Tracing Metrics Baggage Context Propagation

Slide 37

Slide 37 text

All About The Span @dasiths https://medium.com/nikeengineering/hit-the-ground-running-with-distributed-tracing-core-concepts-ff5ad47c7058 Temporal Relationship Casual Relationship

Slide 38

Slide 38 text

Context Propagation @dasiths Span (A) Time APP TraceId: 1 ParentSpan: Null

Slide 39

Slide 39 text

@dasiths Context Propagation Span (A) Span (B) Time Context Propagation TraceId: 1 ParentSpan: Null TraceId: 1 ParentSpan: A APP

Slide 40

Slide 40 text

@dasiths Context Propagation Context Propagation Span (C) Span (D) Context Propagation Span (A) Span (B) Time @ Context Propagation TraceId: 1 ParentSpan: Null TraceId: 1 ParentSpan: A TraceId: 1 ParentSpan: B TraceId: 1 ParentSpan: B APP

Slide 41

Slide 41 text

@dasiths Context Propagation Context Propagation Span (C) Span (D) • TraceId • SpanId • TraceFlags • TraceState Context Propagation Span (A) Span (B) Span Context Time @ Context Propagation TraceId: 1 ParentSpan: Null TraceId: 1 ParentSpan: A TraceId: 1 ParentSpan: B TraceId: 1 ParentSpan: B APP

Slide 42

Slide 42 text

@dasiths Context Propagation Context Propagation Span (C) Span (D) • TraceId • SpanId • TraceFlags • TraceState Context Propagation Span (A) Span (B) Span Context Time Process 2 Process 1 @ Context Propagation TraceId: 1 ParentSpan: Null TraceId: 1 ParentSpan: A TraceId: 1 ParentSpan: B TraceId: 1 ParentSpan: B APP

Slide 43

Slide 43 text

Pr @dasiths Context Propagation Context Propagation Span (D) • TraceId • SpanId • TraceFlags • TraceState Context Propagation Span Context Span (C) Span (A) Span (B) Over-the-wire vs in-process Time Process 1 Process 2 @ Context Propagation TraceId: 1 ParentSpan: Null TraceId: 1 ParentSpan: A TraceId: 1 ParentSpan: B TraceId: 1 ParentSpan: B APP

Slide 44

Slide 44 text

Pr @dasiths Context Propagation Context Propagation Span (D) • TraceId • SpanId • TraceFlags • TraceState Context Propagation Span Context Span (C) Span (A) Span (B) Inject Context To Outgoing Request Over-the-wire vs in-process Time Process 2 Process 1 @ Context Propagation TraceId: 1 ParentSpan: Null TraceId: 1 ParentSpan: A TraceId: 1 ParentSpan: B TraceId: 1 ParentSpan: B Extract Context From Incoming Request APP

Slide 45

Slide 45 text

Pr @dasiths Context Propagation Context Propagation Span (D) Context Propagation Baggage Baggage Baggage Span (C) Span (A) Span (B) Baggage Time Process 1 Process 2 @ Context Propagation Inject Context To Outgoing Request Extract Context From Incoming Request TraceId: 1 ParentSpan: Null TraceId: 1 ParentSpan: A TraceId: 1 ParentSpan: B TraceId: 1 ParentSpan: B APP

Slide 46

Slide 46 text

Pr @dasiths Context Propagation Context Propagation Span (D) Context Propagation Baggage Baggage Baggage Span (C) Span (A) Span (B) Time Process 1 Process 2 @ Inject Context To Outgoing Request Extract Context From Incoming Request KeyValue Pairs Baggage Data Baggage Context Propagation TraceId: 1 ParentSpan: Null TraceId: 1 ParentSpan: A TraceId: 1 ParentSpan: B TraceId: 1 ParentSpan: B APP

Slide 47

Slide 47 text

DEMO @dasiths https://github.com/dasiths/OpenTelemetryDistributedTracingSample

Slide 48

Slide 48 text

• Span (Activity) + Baggage ▪ Trace Context, Parent->Child, Tags, Events • Context Propagation ▪ Injecting & extracting, built-in vs custom instrumentation ▪ HTTP W3C trace context vs custom serialisation • Exporting ▪ Forwarding to & observing the traces in Zipkin Demo Highlights @dasiths

Slide 49

Slide 49 text

Pr @dasiths Propagation

Slide 50

Slide 50 text

Exporters (Traces + Metrics) @dasiths And lots more…

Slide 51

Slide 51 text

Start using OpenTelemetry today! Distributed tracing is essential for performance and reliability of modern cloud native software. It’s no longer a “nice to have”. @dasiths

Slide 52

Slide 52 text

Any questions? THANKS! @dasiths dasith.me https://www.nationalgeographic.com/travel/destinations/asia/sri-lanka/

Slide 53

Slide 53 text

Presentation template designed by powerpointify.com Special thanks to all people who made and shared these awesome resources for free: CREDITS Photographs by unsplash.com Free Fonts used: https://www.fontsquirrel.com/fonts/oswald @dasiths