Monitoring Kubernetes with Prometheus

Prometheus Tokyo Meetup #1

Transcript

1. Monitoring k8s
   Prometheus Tokyo Meetup #1
   with Prometheus
   

   View Slide

2. ABOUT ME
   INTRODUCTION
   AYA IGARASHI
   @Ladicle
   NTT Communications
   Software Engineer
   

   View Slide

3. AGENDA
   INTRODUCTION
   features
   02
   01
   03
   04
   Prometheus & Kubernetes Overview
   Why use Prometheus?
   Introduce good points
   Summarize
   k8s service discovery
   easy installation and powerful monitoring
   pros and cons
   

   View Slide

4. Prometheus?
   What is
   

   View Slide

5. What is Prometheus?
   OVERVIEW
   ● OSS monitoring & alerting tool
   ● Built at SoundCloud (inspired by Borgmon)
   ● Joined the
   ● Written in Go
   ● Pull model
   Characteristics
   Not Fit
   ● 100% accuracy, such as billing
   

   View Slide

6. Architecture
   OVERVIEW
   

   View Slide

7. Kubernetes?
   What is
   

   View Slide

8. What is Kubernetes(k8s)?
   OVERVIEW
   ● OSS management tool of container
   ● Built at Google (inspired by Borg)
   ● Joined the CNCF
   ● Written in Go
   ● Latest version is 1.6 (Support RBAC)
   Characteristics
   Container Runtime
   ● Docker
   ● rkt
   ● frakit etc….
   

   View Slide

9. Prometheus?
   Why use
   

   View Slide

10. Why use Prometheus?
    Deciding factors
    ○ Containers are discovered via service discovery
    ○ Flexible query and dashboarding support
    ○ Smart alerting system such as grouping
    OVERVIEW
    

    View Slide

11. Architecture
    OVERVIEW
    ○ NodeExporter
    ■ VM(baremetal)
    ○ BlackboxExporter
    ■ HTTP/TCP/DNS
    ○ Kube-state-metrix
    ■ Kubernetes cluster
    

    View Slide

12. k8s Service Discovery
    OVERVIEW
    ● Scrape node, pod, endpoint and service
    ○ Node: NodeExporter
    ○ Pod / Endpoint: CustomExporter
    ○ Service: BlackboxExporter
    What can it do?
    ● Provided official prometheus chart
    ○ https://github.com/kubernetes/charts/tree/master/stable/prometheus
    How to use it?
    

    View Slide

13. How to use it?
    OVERVIEW
    ● Prometheus Server configuration
    Scrape pods and endpoints
    annotations:
    prometheus.io/scrape: 'true'
    prometheus.io/port: '2379'
    prometheus.io/metrics: /metrics
    

    View Slide

14. How to use it?
    OVERVIEW
    Scrape service with BlackboxExporter
    ● Probe endpoints over HTTP, HTTPS, DNS, TCP and ICMP.
    ○ https://github.com/prometheus/blackbox_exporter
    

    View Slide

15. Dashboard
    OVERVIEW
    ● Provided official grafana chart
    ○ https://github.com/kubernetes/charts/tree/master/stable/grafana
    How to install it?
    ● Published k8s dashboards
    ○ https://grafana.com/dashboards?search=kubernetes
    Beautiful dashboards
    

    View Slide

16. Summarize
    OVERVIEW
    ● 日本で使っているところが少ない
    ● BlackboxExporterがchartsに入っていない
    ● K8s 1.6のRBACにchartsが対応していない
    Cons.
    Pros.
    ● β版だが普通に動く(今後のupdateに注意)
    ● インストール周りが整備済み
    ● PromQLが強力
    ● アラートをまとめるのが楽
    

    View Slide

17. Thanks For Watching!
    See you next time
    

    View Slide