OpenTelemetry - ein offener Standard für den Durchblick in Anwendungslandschaften (code.talks 2022)

Um in verteilten Anwendungslandschaften den Überblick zu behalten, gibt es schon seit einiger Zeit unterschiedliche Ansätze um Daten wie Metriken, Traces und Logs zentralisiert zu analysieren. Die Uneinheitlichkeit machte es bislang aufwändig wirklich alle Komponenten einfach zu berücksichtigen. Im Jahr 2019 wurde unter dem Dach der Cloud Native Computing Foundation (CNCF) OpenTelemetry als herstellerneutrales, quelloffenes Ökosystem mit dem Ziel geschaffen die Erfassung und Übertragung von Telemetriedaten zu standardisieren.

In diesem Lightning Talk werden die Spezifikation und deren Bestandteile kurz vorgestellt. Es wird gezeigt, wie sie in Anwendungen integriert werden kann. Zuletzt wird auf die Interaktion mit Cloud-Infrastruktur eingegangen, da OpenTelemetry sich insbesondere auf cloud-native Anwendungen fokussiert.

Transcript

1. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
   © 2022, Amazon Web Services, Inc. or its affiliates.
   OpenTelemetry - ein offener Standard für den
   Durchblick in Anwendungslandschaften
   Dennis Kieselhorst @ Amazon Web Services
   Cloud & infrastructure
   

   View Slide

2. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
   © 2022, Amazon Web Services, Inc. or its affiliates.
   © 2022, Amazon Web Services, Inc. or its affiliates.
   OpenTelemetry - ein offener Standard
   für den Durchblick in
   Anwendungslandschaften
   Dennis Kieselhorst
   Sr. Solutions Architect
   Amazon Web Services
   

   View Slide

3. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
   © 2022, Amazon Web Services, Inc. or its affiliates.
   

   View Slide

4. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
   © 2022, Amazon Web Services, Inc. or its affiliates.
   What is observability?
   DETECT
   INVESTIGATE
   REMEDIATE
   

   View Slide

5. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
   © 2022, Amazon Web Services, Inc. or its affiliates.
   Foundation for observability
   METRICS
   TRACES
   LOGS
   

   View Slide

6. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
   © 2022, Amazon Web Services, Inc. or its affiliates.
   Foundation for observability
   METRICS
   TRACES
   LOGS
   ✓Discrete events
   ✓Usually timestamped
   ✓Can be structured
   

   View Slide

7. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
   © 2022, Amazon Web Services, Inc. or its affiliates.
   Foundation for observability
   METRICS
   TRACES
   LOGS
   ✓ Regularly sampled data
   points (timestamped)
   ✓ Numeric value
   ✓ Dimension/Labels
   grafana.com/grafana/dashboards/1244
   

   View Slide

8. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
   © 2022, Amazon Web Services, Inc. or its affiliates.
   Foundation for observability
   METRICS
   TRACES
   LOGS
   jaegertracing.io/docs/1.18/
   Request path across
   a number of services
   

   View Slide

9. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
   © 2022, Amazon Web Services, Inc. or its affiliates.
   CNCF Cloud Native Landscape
   http://l.cncf.io
   

   View Slide

10. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    CNCF Cloud Native Landscape
    http://l.cncf.io
    

    View Slide

11. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    CNCF end-user technology radar
    „1. The most
    commonly adopted
    tools are open source.
    2. There’s no
    consolidation in the
    observability space.
    3. Prometheus and
    Grafana are
    frequently used
    together.”
    https://radar.cncf.io/2020-09-observability
    https://www.cncf.io/blog/2020/09/11/cncf-end-user-technology-radar-observability-september-2020/
    

    View Slide

12. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    What is OpenTelemetry
    OpenTelemetry is a collection of tools, APIs, and SDKs.
    You can use it to instrument, generate, collect, and export
    telemetry data (metrics, logs, and traces) for analysis in order to
    understand your software's performance and behavior.
    A N O B S E R V A B I L I T Y F R A M E W O R K F O R C L O U D - N A T I V E S O F T W A R E
    Broad Language Support
    Java | C# | Go | JavaScript | PHP | Python |
    Rust | C++ | Erlang/Elixir | .NET | Swift
    Integrates With Popular
    Frameworks and Libraries
    MySQL | Redis | Django | Kafka | Jetty | Akka |
    RabbitMQ | Spring | Flask | net/http | gorilla/mux |
    WSGI | JDBC | PostgreSQL
    https://opentelemetry.io
    

    View Slide

13. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    OpenTelemetry Reference architecture
    https://opentelemetry.io/docs/
    

    View Slide

14. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    OpenTelemetry status
    METRICS
    TRACES
    LOGS
    API: stable
    SDK: mixed
    Protocol: stable
    https://opentelemetry.io/status/
    API: draft
    SDK: draft
    Protocol: stable
    API: stable
    SDK: stable
    Protocol: stable
    

    View Slide

15. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Tracing concepts
    Frontend API Gateway
    Trace
    Amazon
    SQS queue
    Amazon
    DynamoDB
    Table
    Function
    Function
    AWS X-Ray Segments/ OpenTelemetry Spans
    Sub-segments/ Spans
    Request
    Response
    Amazon
    Rekognition
    

    View Slide

16. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Tracing concepts in a nutshell
    • Span represents an operation within a transaction. Typically encapsulates:
    operation name, a start and finish timestamp, attributes (a list of key-value
    pairs), events, a parent span identifier, links to other spans, and a
    SpanContext.
    • Trace defined implicitly by its spans. A trace can be thought of as a directed
    acyclic graph (DAG) of spans where the edges between spans are defined as
    parent/child relationships.
    • SpanContext represents all the information that identifies Span in the Trace
    and MUST be propagated to child Spans and across process boundaries. A
    SpanContext contains the tracing identifiers and the options that are
    propagated from parent to child Spans.
    • Tracer is responsible for tracking the currently active span.
    https://github.com/open-telemetry/opentelemetry-
    specification/blob/main/specification/overview.md#tracing-signal
    

    View Slide

17. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Demo
    19
    Blog: https://a.co/5sIvwvL
    Workshop: https://observability.workshop.aws
    

    View Slide

18. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    

    View Slide

19. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Getting started: OpenTelemetry with Java applications
    Manual instrumentation
    opentelemetry-api and
    opentelemetry-sdk libraries
    available to e.g. create spans
    programmatically
    https://opentelemetry.io/docs/java/manual_instrumentation/
    Auto instrumentation
    Java agent attached to the JVM
    https://github.com/open-telemetry/opentelemetry-java-
    instrumentation#getting-started
    https://github.com/open-telemetry/opentelemetry-java-
    instrumentation/blob/main/docs/supported-libraries.md
    https://opentelemetry.io/docs/java/
    

    View Slide

20. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Collector
    Deployment
    • Agent: A Collector instance running with the application or on the same host as the
    application (e.g. binary, sidecar, or daemonset).
    • Gateway: One or more Collector instances running as a standalone service (e.g.
    container or deployment) typically per cluster, datacenter or region.
    Components
    • Receivers: How to get data into the Collector; these can be push or pull based
    • Processors: What to do with received data
    • Exporters: Where to send received data; these can be push or pull based
    • Pipelines: Enable the components
    V E N D O R - A G N O S T I C W A Y T O R E C E I V E , P R O C E S S A N D E X P O R T T E L E M E T R Y D A T A
    https://opentelemetry.io/docs/concepts/data-collection/
    

    View Slide

21. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Collector setup in the demo
    

    View Slide

22. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    OpenTelemetry Collector
    https://opentelemetry.io/docs/collector/
    

    View Slide

23. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Collector configuration
    https://github.com/aws-observability/aws-otel-
    lambda/blob/main/adot/collector/config.yaml
    https://github.com/open-
    telemetry/opentelemetry-
    collector/blob/main/examples/de
    mo/otel-collector-config.yaml
    https://opentelemetry.io/docs/collector/configuration/
    

    View Slide

24. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    AWS Distro for OpenTelemetry
    A S E C U R E , O P E N S O U R C E D I S T R I B U T I O N S U P P O R T E D B Y A W S
    33
    • Upstream-first distro of the popular CNCF project
    • Certified by AWS for security and predictability
    • Backed by AWS support
    • One-click deploy and configure from Amazon ECS and
    AWS Lambda consoles
    • Exporters for AWS monitoring solutions including –
    Amazon Managed Service for Prometheus, Amazon
    CloudWatch, AWS X-Ray, Amazon OpenSearch Service,
    and 3rd party Partner solutions
    

    View Slide

25. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Q&A
    37
    

    View Slide

26. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Resources
    • Observability Workshop: https://observability.workshop.aws
    • Developer Portal: https://aws-otel.github.io
    • Overview page: http://aws.amazon.com/otel
    • Roadmap: https://github.com/orgs/aws-observability/projects/4
    • Feature requests, bugs: https://github.com/aws-
    observability/aws-otel-community/issues
    • OpenTelemetry spec https://github.com/open-
    telemetry/opentelemetry-specification/tree/main/specification
    • Contact us on Gitter: https://gitter.im/aws-otel-
    distro/community
    

    View Slide

27. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Thank you!
    © 2022, Amazon Web Services, Inc. or its affiliates.
    Dennis Kieselhorst
    kieselhorst
    

    View Slide