[workshop] Getting started with prometheus

In February 2020 we (@Vorobiev_Cloud, @MstrsObserver) were giving Prometheus workshop at InstallFest 2020. You can find the repository with the practical part here: https://github.com/pipetail/installfest20

Transcript

1. Getting started with Prometheus Anton Vorobiev @Vorobiev_Cloud Štěpán Vraný @MstrsObserver
2. I’m Anton. - Helping companies with Cloud and automation
3. and I’m Štěpán. - Big fan of DevOps culture - Cloud Engineer (Whatever it means) - Obsessed with automation - Helping with the adoption of Public Cloud, Containers and other fun stuff
4. Shall we start?
5. Prometheus. What the hell is that?
6. What is prometheus - Built at SoundCloud at 2012 - Graduated CNCF project (in 2016, second project after Kubernetes) - Powerful tool for metrics and alerting - works well in machine-centric or dynamic service-oriented architectures systems
7. What? How?
8. What does Prometheus do - Gathers data - A lot of data - From everywhere - Promotes instrumentation - It draws nice graphs - ... and generates PagerDuty alerts
10. Data gathering - HTTP protocol - Plain text - Pull model
11. Metrics format
12. Metrics format (detailed view) traefik_entrypoint_request_duration_seconds_count{ code="404", entrypoint="traefik", method="GET", protocol="http" } 44
13. Data gathering - Hundreds of thousands metrics per second per server - Very efficient storage - Tunable retention - Very reasonable defaults
14. Data sources Instrumented applications / infrastructure components - developers of the given piece of software instrumented app directly with Prometheus Exporters - for application / components where we don’t have access to the code
15. Data sources Push gateway - For data that cannot be pulled - CronJobs and other short-lived services
16. Instrumented applications - Plenty of official / third party libraries - Go, Java, Python, Ruby, Elixir, pretty much everything - Apps can be also instrumented with OpenTelemetry! https://github.com/open-telemetry
17. Instrumented applications
18. Already instrumented applications - Some application work with Prometheus out of the box - Docker, Kubernetes, Etcd, Traefik, ...
19. Exporters - See the full list here https://prometheus.io/docs/instrumenting/exporters/ - Mix of official and third-party libraries - Databases, storage systems, … - Cloud providers
20. Data processing - Very powerful PromQL https://prometheus.io/docs/prometheus/latest/querying/basics/ - Interactive console
21. Data processing - Graphs for the results
22. Alerting - Deduplication of alerts - Highly Available setup possible - A lot of targets (email, PagerDuty, ...) - Templated alerts - Label-based alert routing
23. Alert example (Slack)
24. Service discovery - Static - Kubernetes, Consul, AWS, GCP, Azure, … - File-based service discovery
25. Hands-on session https:/ /cutt.ly/Prometheus2020
26. Visit next Prometheus session tomorrow! 10:30–11:25, Track II, delivered by @verosk
27. Thank you! Anton Vorobiev @Vorobiev_Cloud Štěpán Vraný @MstrsObserver