Monitoring in Kubernetes with Prometheus and Grafana

© Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 1
© Copyright 2020 Rancher Labs. All Rights Reserved. 1 Monitoring in Kubernetes with Prometheus and Grafana BASTIAN HOFMANN Field Engineer - DACH

Rancher Technical Overview Rancher Technical Overview © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 5 Rancher’s recipe for production quality Kubernetes at scale Shared Tooling & Services Security & Authentication Simplified Cluster Operations & Infrastructure Management Policy management Pod & network security policies CIS benchmark monitoring RBAC policies Configuration enforcement Visibility & diagnostics Centralized audit Monitoring & alerting Kubernetes version management Node pool management Cluster provisioning Amazon EKS Azure AKS Google GKE Cloud Datacenter Dev Branch Edge Secure Application Deployment Routing Autoscaling Metrics Load Balancing Canary Git Deployments Beta © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 1 Rancher enables production quality Kubernetes operations everywhere Amazon EKS Azure AKS Google GKE Containerized App 1 Containerized App 2 Containerized App 3 Cloud Datacenter Dev Branch Edge Central Management Shared Tooling & Services Security & Authentication Simplified Cluster Operations & Infrastructure Management Policy management Pod & network security policies CIS benchmark monitoring RBAC policies Configuration enforcement Visibility & diagnostics Centralized audit Monitoring & alerting Kubernetes version management Node pool management Cluster provisioning Infrastructure agnostic

Agenda • How to set up Prometheus and Grafana • How to get and visualize metrics from Kubernetes • How to get and visualize metrics from your own applications • How to add alerts • How to collect logs from your applications • How Service Meshes help with traffic observability

4 Demos, Demos, Demos

5 First, we need a Kubernetes Cluster

6 Demo

7 Extending Kubernetes

Standardization of compute, network and service discovery

Extensible API

Easy integration of additional tools

Custom Resource Definitions • Extend the Kubernetes API with additional Resource Definitions • Certificate • MySQLCluster • Prometheus • … • Deploy a controller into the cluster that listens on creation, change and deletion of these resources and perform the necessary actions

13 Prometheus-operator

Manages and Configures Prometheus, Alertmanager (and Grafana)

CustomResourceDefinitions • Workload Management • Prometheus • Alertmanager • Prometheus Configuration • ServiceMonitor • PodMonitor • Rule

17 Demo

18 Monitoring external resources

Blackbox-monitor

20 Demo

21 Central log management

Several solutions • Commercial • OpenSource • ElasticSearch – FluentD/FluentBit/Logstash – Kibana • Loki – Promtail – Grafana

23 Demo

24 Service Meshes

Service Meshes • Observability • Traffic Control • Traffic Security • Connectivity • On top of the Kubernetes Pod network

Service Meshes • Istio • Linkerd • Maesh • Kuma • …

28 Service Mesh Features

Traffic Security • Automatic mutual TLS encryption of traffic • Automatic certificate management • Optional certificate-based authentication of traffic

Traffic Control • Advanced traffic management between services in the mesh • Blue/Green Deployments • Canary Deployments • A/B Testing • Fault injection • Circuit breakers

Multi-cluster Service Mesh • Connect separate Kubernetes cluster together • Securely handle traffic between clusters

Observability • Monitor all TCP connections, HTTP and GRCP requests • Bytes in/out • Amount of requests • Success rate • Response time • Visualize service communication

Tracing • Trace a single incoming request through all services • Enhance with application spans through Open Tracing

38 Demo

39 Thank you Bastian Hofmann Field Engineer, DACH * [email protected]

Monitoring in Kubernetes with Prometheus and Grafana

Monitoring in Kubernetes with Prometheus and Grafana

More Decks by Bastian Hofmann

Other Decks in Programming

Featured

Transcript