Prometheus

This presentation gives an overview of the Prometheus project. It explains Prometheus in terms of it's visualisation, time series processing capabilities and architecture. It also examines it's query language PromQL.

Links for further information and connecting

http://www.amazon.com/Michael-Frampton/e/B00NIQDOOM/

https://nz.linkedin.com/pub/mike-frampton/20/630/385

https://open-source-systems.blogspot.com/

Transcript

1. What Is Apache Prometheus ? • A system for event monitoring and alerting • Open source / Apache 2.0 license • Records real-time metrics in a time series database • Released by Cloud Native Computing Foundation • Storage in memory / on disk / various integrations • Has multiple visualisation options • Written in Go
2. What Is Apache Prometheus ? • Provides PromQL query language • Has a highly dimensional data model • Has many third party integration options • Many client libraries available • Alerting via PromQL and alertManager • Has wide community support
3. Prometheus Client Libraries • Official client libraries include – Go, Java, Scala, Python, Ruby • Unofficial client libraries include – Bash, C, C++, Common Lisp, Dart – Elixir, Erlang, Haskell, Lua for Nginx – Lua for Tarantool, .NET / C#, Node.js – Perl, PHP, R, Rust • Also use text base exposition formats to – Expose metrics to Prometheus
4. Prometheus Storage • By default Prometheus stores to local file system • But this storage is not clustered or replicated • Does not scale and is not durable, alternatives include • AppOptics: (W) • Azure Data Explorer: (R/W) • Azure Event Hubs: (W) • Chronix: (W) • Cortex: (R/W) • CrateDB: (R/W) • Elasticsearch: (W) • Gnocchi: (W) • Google Cloud Spanner: (R/W) • Graphite: (W) • InfluxDB: (R/W) • IRONdb: (R/W) • Kafka: (W) • M3DB: (R/W) • OpenTSDB: (W) • PostgreSQL/TimescaleDB: (R/W) • QuasarDB: (R/W) • SignalFx: (W) • Splunk: (R/W) • TiKV: (R/W) • Thanos: (R/W) • VictoriaMetrics: (W) • Wavefront: (W)
5. Architecture • Prometheus server scrapes and stores time series data • Client libraries for instrumenting application code • Push gateway for supporting short-lived jobs • Special-purpose exporters for services like – HAProxy, StatsD, Graphite, etc. • Alertmanager to handle alerts • Various support tools
6. Architecture
7. Prometheus Usage • Works well for – Recording any purely numeric time series data – Machine-centric monitoring – Monitoring dynamic service-oriented architectures • Does not works for – Situations where you need 100% accuracy – Such as for per-request billing
8. Prometheus Users
9. Visualisation • Uses Grafana data source for visualisation • Can create Grafana graphs and dashboards • Can import pre-built dashboards from Grafana.com • Can have dashboards update in real time • See next slide for an example
10. Visualisation
11. Prometheus PromQL • Select and aggregate time series data in real time • Display data as graph, table or send to HTTP API • PromQL expressions evaluate to – Instant vector • A set of time series containing a single sample for each time series, all sharing the same timestamp – Range vector • A set of time series containing a range of data points over time for each time series – Scalar, String • Instant vector is the only type that can be directly graphed
12. Prometheus PromQL Examples • Return all time series with the metric http_requests_total – http_requests_total • Return all time series with the metric – http_requests_total and – the given job and handler labels: – http_requests_total{job="apiserver", handler="/api/comments"} • Return a whole range of time – (in this case 5 minutes) for the same vector – making it a range vector: – http_requests_total{job="apiserver", handler="/api/comments"}[5m]
13. Available Books • See “Big Data Made Easy” – Apress Jan 2015 • See “Mastering Apache Spark” – Packt Oct 2015 • See “Complete Guide to Open Source Big Data Stack – “Apress Jan 2018” • Find the author on Amazon – www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ • Connect on LinkedIn – www.linkedin.com/in/mike-frampton-38563020
14. Connect • Feel free to connect on LinkedIn – www.linkedin.com/in/mike-frampton-38563020 • See my open source blog at – open-source-systems.blogspot.com/ • I am always interested in – New technology – Opportunities – Technology based issues – Big data integration