Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Future of BEAM observability

Future of BEAM observability

Łukasz Niemier

June 18, 2019
Tweet

More Decks by Łukasz Niemier

Other Decks in Programming

Transcript

  1. Future of BEAM
    observability

    View full-size slide

  2. • Kobil Systems GmbH

    • EEF Observability WG

    • OpenCenus/OpenTelemetry

    View full-size slide

  3. Three pillars of observability
    • Logs

    • Metrics

    • Traces

    View full-size slide

  4. Logs
    • What have happened?

    • Sent on each action (ex. request)

    • Detailed look into service

    • Always stored in full form

    • Structured data

    View full-size slide

  5. Metrics
    • Aggregated numeric values

    • Cheap on the wire

    • „Overall preview” of the platform performance

    • Sometimes sampled

    View full-size slide

  6. Traces
    • Cross-service data

    • Very heavy on the wire

    • Multiple sources

    • Often sampled

    View full-size slide

  7. Lager
    • For a long time THE Erlang logging solution

    • Heavily rely on Erlang’s parse transformators

    • Supports all 7 syslog levels

    • Very configureable

    • Built in support for log rotation

    View full-size slide

  8. Elixir’s Logger
    • Built in into Elixir

    • gen_event based

    • Textual logs only (with addition of metadata)

    • Metadata stored as keyword list

    • Heavily rely on macros

    • OOtB supports only logging to stdout

    View full-size slide

  9. logger
    • Built in into OTP’s kernel application

    • Tries to do as much work as possible within logging
    process (instead of handler)

    • Replaces old error_logger module

    • Supports textual and structured logging (+ metadata)

    View full-size slide

  10. Metrics
    • Telemetry
    • Various aggregators:

    • OpenCensus

    • Exometer/Elixometer

    • Folsom

    • Prometheus

    • Various implementations of statsd/Influx/graphite/etc.

    • Propertiary implementations (ex. NewRelic)

    View full-size slide

  11. Unified API
    • Telemetry (migrated to Erlang)

    • Telemetry.Metrics (to be migrated)
    • Telemetry.Pooler

    View full-size slide

  12. Tracing
    • Black box tracing and debugging in BEAM is top of the
    class

    • White box tracing is still lacking

    View full-size slide

  13. „Standards”
    • OpenZipkin (Zipkin)

    • OpenTracing (Jaeger)

    • Numerous propertiary APM protocols (DataDog,
    NewRelic, etc.)

    View full-size slide

  14. BEAM projects
    • OpenCensus (to be moved to OpenTelemetry)

    • Otter (no updates in 2 years)

    • Spandex (team joined OpenCensus team)

    • Propertiary libraries: NewRelic and AppSignal

    View full-size slide

  15. Erlang Ecosystem
    Foundation Observability
    Working Group

    View full-size slide

  16. Members
    • Tristan Sloughter (OpenCensus)

    • Vince Foley (New Relic)

    • Łukasz Niemier (OpenCensus)

    • Ilya Khaprov (prometheus.erl, prometheus.ex, OpenCensus)

    • Bryan Naegele

    • Greg Mefford (Spandex)

    • Arkadiusz Gil (Telemetry)

    • Zach Daniel (Spandex)

    • Mark Allen (Lager)

    • Andrew Thompson (Lager)

    • Jose Valim

    View full-size slide

  17. Goals
    • Improve white box monitoring (when you can see the
    code)

    • Improve black box monitoring (when you do not have
    access to code)

    • Provide common interfaces for publishing metrics and
    traces from OTP and libraries without vendor lock-in

    • Cooperation with OAM providers (NewRelic, DataDog,
    etc.) to provide better integrations with BEAM

    View full-size slide

  18. Current projects
    • Migrate OpenCensus to OpenTelemetry

    • Merge Spandex with OpenTelemetry

    • Release stable version of Telemetry and Telemetry.Metrics

    • Integrate Elixir’s Logger with Erlang’s logger

    View full-size slide