Building custom OpenTelemetry Collector distributions with ocb

You probably know that the OpenTelemetry Collector comes in two flavors: the core and the contrib distributions. What you might not know is that you can mix and match components to create your distribution. Come to this lightning session and learn how to assemble a custom distribution with the components you care about using ocb, the OpenTelemetry Collector Builder.

Transcript

1. @jpkrohling Juraci Paixão Kröhling Software engineer @jpkrohling Building custom OpenTelemetry Collector distributions with ocb
2. @jpkrohling @jpkrohling Juraci Paixão Kröhling Software engineer OpenTelemetry Collector • Quick intro • OTel Collector concepts • OTel Collector distributions OpenTelemetry Collector Builder • Intro • Example • Demo Presenter Agenda
3. @jpkrohling OpenTelemetry Collector
4. @jpkrohling @jpkrohling Vendor-agnostic way to receive, process and export telemetry data. Source: https://opentelemetry.io/docs/collector/
5. @jpkrohling @jpkrohling OpenTelemetry Collector Concepts Obtains the telemetry data point, either actively (pull model) or passively (push model) Receiver Processor Exporter Has the ability to look into and potentially change the in-flight telemetry data points Gets the telemetry data points out of the collector, either passively (pull model) or actively (push model)
6. @jpkrohling @jpkrohling OpenTelemetry Collector Concepts Components that interacts with the collector but not directly part of a pipeline. Like health-check endpoints, or auth/z. Extension Connectors A component that is the exporter of one pipeline and receiver in another, perhaps transforming one signal into another (spans to logs, spans to metrics, …)
7. @jpkrohling OpenTelemetry Collector conceptual architecture
8. @jpkrohling @jpkrohling OpenTelemetry Concept Distribution A collection of components assembled in a package, making it easier to accomplish a task.
9. @jpkrohling @jpkrohling OpenTelemetry Collector Concepts Distribution A collection of components (receivers, processors, …) assembled in a binary.
10. @jpkrohling Our official distributions Core Selected components, geared towards interoperability with other open-source projects. Contrib Everything under the sun, including vendor-specific components.
11. @jpkrohling @jpkrohling What is included in the “core” distribution Receivers • otlp • hostmetrics • jaeger • kafka • opencensus • prometheus • zipkin Processors • batch • memorylimiter • attributes • resource • span • probabilisticsampler • filter Exporters • logging • otlp • otlphttp • file • jaeger • kafka • opencensus • prometheus • prometheusremotewrite • zipkin
12. @jpkrohling @jpkrohling What is included in the “contrib” distribution Receivers • otlp • activedirectoryds • aerospike • apache • awscloudwatch • awscontainerinsight • awsecscontainermetrics • awsfirehose • awsxray • azureeventhub • bigip • carbon • cloudfoundry • collectd • couchdb • + 60 others Processors • batch • memorylimiter • attributes • cumulativetodelta • datadog • deltatorate • filter • groupbyattrs • groupbytrace • k8sattributes • metricsgeneration • metricstransform • probabilisticsampler • redaction • resourcedetection • + 7 others Exporters • logging • otlp • otlphttp • alibabacloudlogservice • awscloudwatchlogs • awsemf • awskinesis • awsxray • azuredataexplorer • azuremonitor • carbon • clickhouse • coralogix • datadog • dynatrace • + 29 others
13. @jpkrohling What if I just need one or two extra components to core?
14. @jpkrohling Custom distributions!
15. @jpkrohling OpenTelemetry Collector Builder (ocb)
16. @jpkrohling @jpkrohling The OpenTelemetry Collector Builder (ocb) is a command-line interface tool that generates sources and binaries for OpenTelemetry Collector distributions. me
17. @jpkrohling Example manifest
18. @jpkrohling Key takeaways
19. @jpkrohling @jpkrohling Key takeaways • The Collector is available officially in two distributions: core and contrib • Core has essential components, suitable for a wide variety of applications • Contrib has everything that is part of the OTel Collector project • Custom distributions can be used for a more streamlined (smaller, safer) solution • The OpenTelemetry Collector Builder can be used to generate custom distributions, potentially with your own custom components
20. @jpkrohling Q&A
21. @jpkrohling Grafana Labs’ and CNCF’s Slack GitHub: jpkrohling @jpkrohling Let’s connect! I’m here: Thanks for participating!