Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Unifying Your Observability Pipeline

Unifying Your Observability Pipeline

“If a microservice falls down in the middle of a server farm, does my pager make a sound?”

If your service is automatically monitored, then the answer is “yes!”. But now that you’ve been paged and roused from your slumber… what happens next? Do you stumble to your computer, bleary-eyed, trying to find the elusive problem by cross-referencing dashboards and server logs across eleven different browser tabs? Or do you have better tools that you can use to integrate your monitoring data?

Unless you’re an engineer at a massive company like Google, the answer is probably “no” - most companies don’t have the resources to build all their own monitoring tools in-house. When using third-party and open-source tools for monitoring, there will always be gaps in between.

Fortunately, there’s a way teams can get the best of both worlds: high-resolution visibility into your systems, but without having to write your entire monitoring stack yourselves. We built a custom-built, open-source distributed tracing and monitoring pipeline that allows us to inspect each step of an HTTP request and diagnose the root causes of errors, leveraging the same open-source and third-party monitoring platforms you’re already used to. And with a monitoring pipeline that unifies metrics, logs, and traces, you can live the observability dream: the right data, in the right form, right when you need it.

Aditya Mukerjee

July 13, 2018

More Decks by Aditya Mukerjee

Other Decks in Programming


  1. What tools can we use? Metrics/dashboards? Logs? Request traces? No

    context! Hard to aggregate! Require planning! @chimeracoder
  2. What’s the difference? •If you squint, it’s hard to tell

    them apart •A log is a metric with “longer” information •A trace is a metric that allows “inner joins” @chimeracoder
  3. Trying out Veneur •Free and open source! http://github.com/stripe/veneur •Six-week release

    cycle • Drop-in support for statsd, Graphite, Datadog, SignalFx, Prometheus, and more •Native Kubernetes support •Public images on Docker Hub @chimeracoder