How to instrument Go code in 2020 - Björn Rabenstein - Grafana Labs

6e3ea86995d93d35c0fadf2694bca773?s=47 GoDays
January 23, 2020

How to instrument Go code in 2020 - Björn Rabenstein - Grafana Labs

If you are an astute follower of Peter Bourgon's [Go best practices](https://peter.bourgon.org/go-best-practices-2016/#logging-and-instrumentation), you have known it since 2016: “You should be instrumenting every significant component of your codebase.” But what does that even mean? Back then, Peter was mostly talking about metrics (and flatteringly recommended [Prometheus](https://prometheus.io) for it). However, logging can also be seen as instrumentation. (Peter considered it related but separate in his post from 2016.) And then there is instrumentation for distributed tracing. All three of them, metrics, logs, and traces, are nowadays often referred to as the [three pillars ob observability](https://www.oreilly.com/library/view/distributed-systems-observability/9781492033431/ch04.html). Wouldn't it be nice if you could do all three of them within the same framework? [OpenTelemetry](https://opentelemetry.io) is a project promising exactly that. There is a wide field of techniques between simple standard Go packages like `expvar` and `log` and the newest and most ambitious projects like OpenTelemetry. This talk doesn't intend to give you hands-on instructions for using any of them. Instead, it will provide a (necessarily incomplete) overview of established and new approaches and some guidelines how to navigate the many choices.

6e3ea86995d93d35c0fadf2694bca773?s=128

GoDays

January 23, 2020
Tweet

Transcript

  1. 2.

    | 2

  2. 6.

    | 6

  3. 7.

    | 7

  4. 8.

    | 8

  5. 9.

    | 9

  6. 10.
  7. 11.

    | 11 What Go “built in” “Modern” needs Example Debugging

    Profiling/Tracing runtime/pprof runtime/trace Logging log Metrics expvar
  8. 12.

    | 12 What Go “built in” “Modern” needs Example Debugging

    Almost useless… You’ll find one if you really need it. Profiling/Tracing runtime/pprof runtime/trace Distributed tracing opentracing/ opentracing-go Logging log Structured logging go-kit/log Metrics expvar Labeled metrics prometheus/ client_golang
  9. 13.

    | 13 What Go “built in” “Modern” needs Example Debugging

    Almost useless… You’ll find one if you really need it. Profiling/Tracing runtime/pprof runtime/trace Distributed tracing opentracing/ opentracing-go Logging log Structured logging go-kit/log Metrics expvar Labeled metrics prometheus/ client_golang
  10. 15.
  11. 19.

    A rich and universal data model | 19 Metrics are

    labeled. Logs are structured. OpenTracing.
  12. 20.

    For ALL data collection models | 20 Push vs pull.

    On prem vs. cloud. Distributed vs. replicated. Location of sampling, aggregation, buffering. Exposition formats. Transport protocols.
  13. 24.
  14. 27.

    | 27 Relax! Logs and metrics are the easy part

    anyway. And not everything is a trace. Step 2:
  15. 28.