OpenTelemetry - ein offener Standard für den Durchblick in Anwendungslandschaften (code.talks 2022)

Um in verteilten Anwendungslandschaften den Überblick zu behalten, gibt es schon seit einiger Zeit unterschiedliche Ansätze um Daten wie Metriken, Traces und Logs zentralisiert zu analysieren. Die Uneinheitlichkeit machte es bislang aufwändig wirklich alle Komponenten einfach zu berücksichtigen. Im Jahr 2019 wurde unter dem Dach der Cloud Native Computing Foundation (CNCF) OpenTelemetry als herstellerneutrales, quelloffenes Ökosystem mit dem Ziel geschaffen die Erfassung und Übertragung von Telemetriedaten zu standardisieren.

In diesem Lightning Talk werden die Spezifikation und deren Bestandteile kurz vorgestellt. Es wird gezeigt, wie sie in Anwendungen integriert werden kann. Zuletzt wird auf die Interaktion mit Cloud-Infrastruktur eingegangen, da OpenTelemetry sich insbesondere auf cloud-native Anwendungen fokussiert.

Transcript

1. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. OpenTelemetry - ein offener Standard für den Durchblick in Anwendungslandschaften Dennis Kieselhorst @ Amazon Web Services Cloud & infrastructure
2. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. OpenTelemetry - ein offener Standard für den Durchblick in Anwendungslandschaften Dennis Kieselhorst Sr. Solutions Architect Amazon Web Services
3. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates.
4. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. What is observability? DETECT INVESTIGATE REMEDIATE
5. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Foundation for observability METRICS TRACES LOGS
6. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Foundation for observability METRICS TRACES LOGS ✓Discrete events ✓Usually timestamped ✓Can be structured
7. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Foundation for observability METRICS TRACES LOGS ✓ Regularly sampled data points (timestamped) ✓ Numeric value ✓ Dimension/Labels grafana.com/grafana/dashboards/1244
8. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Foundation for observability METRICS TRACES LOGS jaegertracing.io/docs/1.18/ Request path across a number of services
9. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. CNCF Cloud Native Landscape http://l.cncf.io
10. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. CNCF Cloud Native Landscape http://l.cncf.io
11. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. CNCF end-user technology radar „1. The most commonly adopted tools are open source. 2. There’s no consolidation in the observability space. 3. Prometheus and Grafana are frequently used together.” https://radar.cncf.io/2020-09-observability https://www.cncf.io/blog/2020/09/11/cncf-end-user-technology-radar-observability-september-2020/
12. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. What is OpenTelemetry OpenTelemetry is a collection of tools, APIs, and SDKs. You can use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) for analysis in order to understand your software's performance and behavior. A N O B S E R V A B I L I T Y F R A M E W O R K F O R C L O U D - N A T I V E S O F T W A R E Broad Language Support Java | C# | Go | JavaScript | PHP | Python | Rust | C++ | Erlang/Elixir | .NET | Swift Integrates With Popular Frameworks and Libraries MySQL | Redis | Django | Kafka | Jetty | Akka | RabbitMQ | Spring | Flask | net/http | gorilla/mux | WSGI | JDBC | PostgreSQL https://opentelemetry.io
13. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. OpenTelemetry Reference architecture https://opentelemetry.io/docs/
14. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. OpenTelemetry status METRICS TRACES LOGS API: stable SDK: mixed Protocol: stable https://opentelemetry.io/status/ API: draft SDK: draft Protocol: stable API: stable SDK: stable Protocol: stable
15. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Tracing concepts Frontend API Gateway Trace Amazon SQS queue Amazon DynamoDB Table Function Function AWS X-Ray Segments/ OpenTelemetry Spans Sub-segments/ Spans Request Response Amazon Rekognition
16. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Tracing concepts in a nutshell • Span represents an operation within a transaction. Typically encapsulates: operation name, a start and finish timestamp, attributes (a list of key-value pairs), events, a parent span identifier, links to other spans, and a SpanContext. • Trace defined implicitly by its spans. A trace can be thought of as a directed acyclic graph (DAG) of spans where the edges between spans are defined as parent/child relationships. • SpanContext represents all the information that identifies Span in the Trace and MUST be propagated to child Spans and across process boundaries. A SpanContext contains the tracing identifiers and the options that are propagated from parent to child Spans. • Tracer is responsible for tracking the currently active span. https://github.com/open-telemetry/opentelemetry- specification/blob/main/specification/overview.md#tracing-signal
17. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. Demo 19 Blog: https://a.co/5sIvwvL Workshop: https://observability.workshop.aws
18. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates.
19. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Getting started: OpenTelemetry with Java applications Manual instrumentation opentelemetry-api and opentelemetry-sdk libraries available to e.g. create spans programmatically https://opentelemetry.io/docs/java/manual_instrumentation/ Auto instrumentation Java agent attached to the JVM https://github.com/open-telemetry/opentelemetry-java- instrumentation#getting-started https://github.com/open-telemetry/opentelemetry-java- instrumentation/blob/main/docs/supported-libraries.md https://opentelemetry.io/docs/java/
20. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Collector Deployment • Agent: A Collector instance running with the application or on the same host as the application (e.g. binary, sidecar, or daemonset). • Gateway: One or more Collector instances running as a standalone service (e.g. container or deployment) typically per cluster, datacenter or region. Components • Receivers: How to get data into the Collector; these can be push or pull based • Processors: What to do with received data • Exporters: Where to send received data; these can be push or pull based • Pipelines: Enable the components V E N D O R - A G N O S T I C W A Y T O R E C E I V E , P R O C E S S A N D E X P O R T T E L E M E T R Y D A T A https://opentelemetry.io/docs/concepts/data-collection/
21. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Collector setup in the demo
22. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. OpenTelemetry Collector https://opentelemetry.io/docs/collector/
23. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Collector configuration https://github.com/aws-observability/aws-otel- lambda/blob/main/adot/collector/config.yaml https://github.com/open- telemetry/opentelemetry- collector/blob/main/examples/de mo/otel-collector-config.yaml https://opentelemetry.io/docs/collector/configuration/
24. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. AWS Distro for OpenTelemetry A S E C U R E , O P E N S O U R C E D I S T R I B U T I O N S U P P O R T E D B Y A W S 33 • Upstream-first distro of the popular CNCF project • Certified by AWS for security and predictability • Backed by AWS support • One-click deploy and configure from Amazon ECS and AWS Lambda consoles • Exporters for AWS monitoring solutions including – Amazon Managed Service for Prometheus, Amazon CloudWatch, AWS X-Ray, Amazon OpenSearch Service, and 3rd party Partner solutions
25. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Q&A 37
26. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Resources • Observability Workshop: https://observability.workshop.aws • Developer Portal: https://aws-otel.github.io • Overview page: http://aws.amazon.com/otel • Roadmap: https://github.com/orgs/aws-observability/projects/4 • Feature requests, bugs: https://github.com/aws- observability/aws-otel-community/issues • OpenTelemetry spec https://github.com/open- telemetry/opentelemetry-specification/tree/main/specification • Contact us on Gitter: https://gitter.im/aws-otel- distro/community
27. CODE.TALKS 2022 - OPENTELEMETRY - EIN OFFENER STANDARD FÜR DEN DURCHBLICK IN ANWENDUNGSLANDSCHAFTEN © 2022, Amazon Web Services, Inc. or its affiliates. Thank you! © 2022, Amazon Web Services, Inc. or its affiliates. Dennis Kieselhorst kieselhorst