Kubernetes Monitoring Introduction

Transcript

1. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 1/31 Kubernetes Monitoring Introduction 1 / 31
2. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 2/31 Kyohei Mizumoto(@kyohmizu) C# Software Engineer Interests Docker/Kubernetes Go Security whoami 2 / 31
3. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 3/31 Required Kubernetesの概要程度の知識 Targets Kubernetes Monitoring未経験者 3 / 31
4. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 4/31 Contents 1. Monitoring Overview 2. Datadog 3. Prometheus 4. Demo 4 / 31
5. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 5/31 Monitoring Overview 5 / 31
6. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 6/31 Monitoring Problem Detection ダッシュボード、アラート Problem Resolution 根本原因の特定、トラブルシュート Continuous Improvement キャパシティ、コスト最適化 6 / 31
7. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 7/31 Observability A measure of how well internal states of a system can be inferred from knowledge of its external outputs 7 / 31
8. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 8/31 Landscape 8 / 31
9. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 9/31 Tools Datadog Prometheus Managed Service Azure Monitor for containers Stackdriver(for GKE) 9 / 31
10. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 10/31 Tools Datadog Prometheus Managed Service Azure Monitor for containers Stackdriver(for GKE) 10 / 31
11. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 11/31 Datadog 11 / 31
12. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 12/31 SaaS型の監視＆解析ツール 課⾦制 様々なサービスのメトリクスをシー ムレスに集約 Datadog 12 / 31
13. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 13/31 Dashboard 13 / 31
14. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 14/31 Metrics ホストのCPU使⽤率、ディスク使⽤率 ノード上のコンテナのCPU使⽤率 Deployment のPod数、要求起動数 Jobの成功数、失敗数 etc... 14 / 31
15. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 15/31 Prometheus 15 / 31
16. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 16/31 CNCFがホストするOSSの監視ツー ル 時系列でメトリクスを収集 Reliability Prometheus 16 / 31
17. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 17/31 Architecture pull metrics HDD / SSD Pushgateway Short-lived jobs Jobs / Exporters Storage Retrieval PromQL Prometheus Server Node Service Discovery find targets Prometheus Server Alertmanager push alerts Web UI Grafana API clients PagerDuty Email DNS Kubernetes Consul ... Custom integration notify ... 17 / 31
18. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 18/31 Architecture Prometheus Server メトリクスの収集(Pull型)、保存 Alert Manager Exporter 要求に応じてメトリクスを送信 Push Gateway メトリクスをPushしておく 18 / 31
19. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 19/31 Get Started 19 / 31
20. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 20/31 Use Helm Helmを使⽤してマニフェストを作成 公式サイト︓https://helm.sh/ # Chartのダウンロード $ helm fetch stable/prometheus --version 8.8.0 # Chartを元にマニフェストを作成 $ helm template --name sample-prometheus \ prometheus-8.8.0.tgz \ > sample-prometheus.yaml 20 / 31
21. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 21/31 Values LBとして外部に公開する場合は、valuesファイルを⽤意 しておく alertmanager: service: type: LoadBalancer pushgateway: service: type: LoadBalancer server: service: type: LoadBalancer 21 / 31
22. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 22/31 Deploy サンプルをデプロイ https://github.com/Kyohei-M/slide-k8s-monitoring/blob/master/sample- prometheus.yaml # Prometheusを起動 $ kubectl apply -f sample-prometheus.yaml 22 / 31
23. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 23/31 Demo 23 / 31
24. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 24/31 Prometheus Server 24 / 31
25. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 25/31 Prometheus Server 25 / 31
26. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 26/31 PromQL Prometheus Query Language # Example # apiserverのhttpリクエスト合計 http_requests_total{job="apiserver"} # 直近5分のhttpリクエストの増加率 rate(http_requests_total[5m])[30m:1m] # 直近1時間での空きメモリ量の差 delta(node_memory_MemFree_bytes[1h]) 26 / 31
27. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 27/31 Grafana OSSのデータ可視化ツール 27 / 31
28. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 28/31 Grafana helmでインストール $ helm install stable/grafana 28 / 31
29. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 29/31 Books 29 / 31
30. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 30/31 Links Monitoring and Observability https://thenewstack.io/monitoring-and-observability-whats-the- difference-and-why-does-it-matter/ Datadog https://www.datadoghq.com/ Prometheus https://prometheus.io/ Grafana https://grafana.com/ 30 / 31
31. 2019/3/28 Kubernetes Monitoring Introduction 127.0.0.1:5500/index.html#2 31/31 Thank you! 31 / 31