Observability with Distributed Tracing

Why Distributed Tracing? Microservices are great for turning method calls in to distributed computing problems A user request has to propagate through load balancers, api gateways, microservices which perform more requests, asyncronous messages, etc. Data-driven decisions win over decisions based on feelings, or the opinion of the most senior employee in the room

Open Tracing Offer a consistent, uni ded and tracer-agnostic instrumentation API Abstracts away the differences among numerous tracer implementations OT is not only for tracing

Contextualized Logs Correlate logs for seamlessly visualization in the UI Hint: Use request.id as a tag and trace request e2e

Metrics Jaeger exposes prometheus metrics of the outgoing calls Latency, RPS, HTTP Request Status

The Challenges of Tracing Every component of a request needs to be modi ed to propagate tracing More challenging at places with polyglot architecture Sampling Strategy (Constant, Probabilistic, Rate Limiting, Remote, etc.) Engineers needs to instrument in the code (White Box)

OpenCensus Provides libraries for metrics collection and tracing Supported languages include Go, Java, C++, Ruby, Erlang, Python, and PHP Supported backends include Datadog, Instana, Jaeger, SignalFX, Stackdriver, and Zipkin Originates from Google

ocsql example

Service Meshes Integration of tracing funcionality is almost effortless Data Planes implement tracing and stats collection at the proxy level Applications that are part of the mesh needs to forward headers to the next hop in the mesh

Conclusion As Brian Knox who manages the Observability Team at Digital Ocean said: The goal of an Observability team is not to collect logs, metrics, or traces. It is to build a culture of engineering based on facts and feedback, and then spread that culture within the broader organization. “ “

Thank you