Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ask an OpenShift Admin episode 93: OpenShift Logging and Observability

Ask an OpenShift Admin episode 93: OpenShift Logging and Observability

These slides were used with episode 93 of Ask an OpenShift Admin: https://www.youtube.com/watch?v=QZ4Hv85lEJ0

Red Hat Livestreaming

February 22, 2023
Tweet

More Decks by Red Hat Livestreaming

Other Decks in Technology

Transcript

  1. V0000000
    Ask an Openshift Admin
    February 15, 2023
    Openshift Logging and
    Observability
    1

    View Slide

  2. V0000000
    Disclaimer
    The content set forth herein is Red Hat confidential
    information and does not constitute in any way a binding or
    legal agreement or impose any legal obligation or duty on
    Red Hat.
    2
    This information is provided for discussion purposes only
    and is subject to change for any or no reason.

    View Slide

  3. V0000000
    Deliver ONE unified, consistent, and simplified
    Observability experience across any footprint:
    the public cloud, on-prem, and edge
    Our Observability Strategy
    3

    View Slide

  4. V0000000
    4
    Store:
    Metrics with Prometheus/Thanos
    Logs with Loki
    Traces with Jaeger/Elasticsearch
    Observability
    "Turn your data
    into answers!"
    Data
    Visualization
    Data
    Analytics
    Data Delivery
    Data Storage
    Visualize:
    Out of the box experience
    & full support in
    OpenShift Web Console
    Collect:
    Metrics with Prometheus
    Logs with Vector
    Traces with OpenTelemetry
    Product Manager: Jamie Parker, Roger Floren & Vanessa Martini
    Data Collection
    Deliver:
    Aggregate & Normalize data
    Transport it with Observability
    Operator
    Analyze:
    Query metrics
    Search metrics targets
    Filter logs by severity
    1
    2
    3
    5
    4
    OpenShift Observability
    Third Party Integration

    View Slide

  5. V0000000
    Observability
    Metrics Logs Traces
    Monitoring
    OpenShift 4.12 Logging v5.6 Q42022
    Logging
    Distributed Tracing
    ● Support for forwarding logs to Splunk
    ● Support for forwarding logs to Google Cloud
    Platform
    ● Add the openshift cluster ID to log records so
    that clusters can be uniquely identified in
    aggregated logs
    ● GA release of Vector as an alternate collector
    to Fluentd
    ● Exposed stream-based retention capabilities in
    the Loki Stack custom resource for OpenShift
    Application owners and OpenShift
    Administrators
    ● Log Exploration UI also available in OpenShift
    Developer Console
    ● Improved UI experience in OpenShift Console:
    > Custom time range
    > Predefined filters to easily search and
    filter logs (namespace, pod, container)
    ● OpenTelemetry as Traces Collector (now in
    Tech Preview)
    ● Multi-cluster support in OpenTelemetry
    ● Kubernetes Attribute Processor
    ● Option to specify Topology Spread Constraints
    for Prometheus, Alertmanager, and Thanos
    Ruler.
    ● Option to improve consistency of
    prometheus-adapter CPU and RAM time series.
    ● TP: Allow admin users to create new alerting
    rules based on platform metrics
    ● Version updates to monitoring stack
    components and dependencies
    ● Runbooks URLs enabled in the Alerting UI of
    OpenShift Console
    ● Improved UI experience in OpenShift Console:
    > Easier selection of records in Metrics UI
    5

    View Slide

  6. V0000000
    Upstream Projects Contribution
    6
    Grafana Loki for log aggregation
    https://grafana.com/oss/loki/
    Vector for log collection
    https://vector.dev/
    Prometheus for metrics
    https://prometheus.io/
    Thanos for metrics
    https://thanos.io/
    Jaeger for distributed tracing
    https://www.jaegertracing.io/
    OpenTelemetry for standardizing data
    https://opentelemetry.io/
    Grafana Tempo for traces
    https://grafana.com/oss/tempo/
    Kafka for event streaming
    https://kafka.apache.org/intro

    View Slide

  7. V0000000
    OpenShift Logging
    7

    View Slide

  8. V0000000
    Initial Openshift Logging Stack
    8
    Elasticsearch
    ● Log Store
    ● Log Search
    Fluentd
    ● Log Collection
    ● Log Transportation
    Kibana
    ● Log Visualization

    View Slide

  9. V0000000
    Current Openshift Logging Stack
    9
    Vector as collector Loki as log store
    ▸ Loki indexes log labels instead of log lines for
    better performance
    ▸ Log visualization natively inside the OpenShift
    Console
    ▸ Vector has excellent Log Per Second performance
    Major updates and features
    ▸ Pod labels for k8s are preserved
    ▸ Support Cloudwatch output for Vector
    ▸ CloudWatch log forwarding add-on supports STS
    installations
    ▸ Loki allows multiple tenants to use a single Loki
    instance.

    View Slide

  10. V0000000
    Openshift Logging - Vector Collection
    10
    AWS Cloudwatch
    GCP Stackdriver
    Loki
    Splunk

    View Slide

  11. V0000000
    Openshift Logging - Loki Storage
    11
    ● LogQL query language
    ● Efficient memory use via log chunking
    ● Multi tenant storage with data isolation
    ● Cluster ID added for easier log
    aggregation
    ● When used as the Openshift log store,
    enables log analytics within the Openshift
    console UI

    View Slide

  12. V0000000
    New entry:
    Aggregated Logs
    view in Developer
    Console
    PMs: Roger Floren, Jamie Parker & Vanessa Martini
    Improved UX:
    Filter by content
    (namespace, pod,
    container) AND
    Search by content
    AND Filter by
    severity
    OpenShift Observability
    12

    View Slide

  13. V0000000
    The Future
    13

    View Slide

  14. V0000000
    14
    Upcoming on the Roadmap
    Vector
    ● Http Forwarding
    ● Syslog Forwarding
    ● Multicluster Collection
    Loki
    ● Log based alerting
    ● Zone aware replication
    Console UI
    ● Even more log viewing and aggregation
    enhancements
    ● Correlation

    View Slide

  15. V0000000
    15
    OpenShift 4.12 / Logging 5.6
    Blog
    Logging 5.5
    Blog
    Notable Blogs
    OpenShift Monitoring for 4.12
    Blog
    Network Observability Support in OpenShift 4.12
    Blog
    Red Hat OpenShift Observability Brings Flexible
    Insights to Management of Clusters
    Blog
    In-depth OpenShift Monitoring for 4.12
    Blog
    What is OpenTelemetry?
    Blog

    View Slide

  16. V0000000
    linkedin.com/company/red-hat
    youtube.com/user/RedHatVideos
    facebook.com/redhatinc
    twitter.com/RedHat
    16
    Red Hat is the world’s leading provider of enterprise
    open source software solutions. Award-winning
    support, training, and consulting services make
    Red Hat a trusted adviser to the Fortune 500.
    Thank you

    View Slide