Ask an OpenShift Admin episode 93: OpenShift Logging and Observability

These slides were used with episode 93 of Ask an OpenShift Admin: https://www.youtube.com/watch?v=QZ4Hv85lEJ0

Transcript

1. V0000000 Ask an Openshift Admin February 15, 2023 Openshift Logging and Observability 1
2. V0000000 Disclaimer The content set forth herein is Red Hat confidential information and does not constitute in any way a binding or legal agreement or impose any legal obligation or duty on Red Hat. 2 This information is provided for discussion purposes only and is subject to change for any or no reason.
3. V0000000 Deliver ONE unified, consistent, and simplified Observability experience across any footprint: the public cloud, on-prem, and edge Our Observability Strategy 3
4. V0000000 4 Store: Metrics with Prometheus/Thanos Logs with Loki Traces with Jaeger/Elasticsearch Observability "Turn your data into answers!" Data Visualization Data Analytics Data Delivery Data Storage Visualize: Out of the box experience & full support in OpenShift Web Console Collect: Metrics with Prometheus Logs with Vector Traces with OpenTelemetry Product Manager: Jamie Parker, Roger Floren & Vanessa Martini Data Collection Deliver: Aggregate & Normalize data Transport it with Observability Operator Analyze: Query metrics Search metrics targets Filter logs by severity 1 2 3 5 4 OpenShift Observability Third Party Integration
5. V0000000 Observability Metrics Logs Traces Monitoring OpenShift 4.12 Logging v5.6 Q42022 Logging Distributed Tracing • Support for forwarding logs to Splunk • Support for forwarding logs to Google Cloud Platform • Add the openshift cluster ID to log records so that clusters can be uniquely identified in aggregated logs • GA release of Vector as an alternate collector to Fluentd • Exposed stream-based retention capabilities in the Loki Stack custom resource for OpenShift Application owners and OpenShift Administrators • Log Exploration UI also available in OpenShift Developer Console • Improved UI experience in OpenShift Console: > Custom time range > Predefined filters to easily search and filter logs (namespace, pod, container) • OpenTelemetry as Traces Collector (now in Tech Preview) • Multi-cluster support in OpenTelemetry • Kubernetes Attribute Processor • Option to specify Topology Spread Constraints for Prometheus, Alertmanager, and Thanos Ruler. • Option to improve consistency of prometheus-adapter CPU and RAM time series. • TP: Allow admin users to create new alerting rules based on platform metrics • Version updates to monitoring stack components and dependencies • Runbooks URLs enabled in the Alerting UI of OpenShift Console • Improved UI experience in OpenShift Console: > Easier selection of records in Metrics UI 5
6. V0000000 Upstream Projects Contribution 6 Grafana Loki for log aggregation https://grafana.com/oss/loki/ Vector for log collection https://vector.dev/ Prometheus for metrics https://prometheus.io/ Thanos for metrics https://thanos.io/ Jaeger for distributed tracing https://www.jaegertracing.io/ OpenTelemetry for standardizing data https://opentelemetry.io/ Grafana Tempo for traces https://grafana.com/oss/tempo/ Kafka for event streaming https://kafka.apache.org/intro
7. V0000000 OpenShift Logging 7
8. V0000000 Initial Openshift Logging Stack 8 Elasticsearch • Log Store • Log Search Fluentd • Log Collection • Log Transportation Kibana • Log Visualization
9. V0000000 Current Openshift Logging Stack 9 Vector as collector Loki as log store ▸ Loki indexes log labels instead of log lines for better performance ▸ Log visualization natively inside the OpenShift Console ▸ Vector has excellent Log Per Second performance Major updates and features ▸ Pod labels for k8s are preserved ▸ Support Cloudwatch output for Vector ▸ CloudWatch log forwarding add-on supports STS installations ▸ Loki allows multiple tenants to use a single Loki instance.
10. V0000000 Openshift Logging - Vector Collection 10 AWS Cloudwatch GCP Stackdriver Loki Splunk
11. V0000000 Openshift Logging - Loki Storage 11 • LogQL query language • Efficient memory use via log chunking • Multi tenant storage with data isolation • Cluster ID added for easier log aggregation • When used as the Openshift log store, enables log analytics within the Openshift console UI
12. V0000000 New entry: Aggregated Logs view in Developer Console PMs: Roger Floren, Jamie Parker & Vanessa Martini Improved UX: Filter by content (namespace, pod, container) AND Search by content AND Filter by severity OpenShift Observability 12
13. V0000000 The Future 13
14. V0000000 14 Upcoming on the Roadmap Vector • Http Forwarding • Syslog Forwarding • Multicluster Collection Loki • Log based alerting • Zone aware replication Console UI • Even more log viewing and aggregation enhancements • Correlation
15. V0000000 15 OpenShift 4.12 / Logging 5.6 Blog Logging 5.5 Blog Notable Blogs OpenShift Monitoring for 4.12 Blog Network Observability Support in OpenShift 4.12 Blog Red Hat OpenShift Observability Brings Flexible Insights to Management of Clusters Blog In-depth OpenShift Monitoring for 4.12 Blog What is OpenTelemetry? Blog
16. V0000000 linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHat 16 Red Hat is the world’s leading provider of enterprise open source software solutions. Award-winning support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. Thank you