Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lightning Talk Prometheus
Search
Mattias Gees
March 01, 2018
Programming
0
62
Lightning Talk Prometheus
Lightning talk I gave internally at Skyscrapers about Prometheus.
Mattias Gees
March 01, 2018
Tweet
Share
More Decks by Mattias Gees
See All by Mattias Gees
A Cloud Native Journey At Scale (Belgium Kubernetes Meetup)
mattiasgees
0
32
Tarmak, why do we need another Kubernetes provisioner?
mattiasgees
1
260
Pod autoscaling with custom-metrics
mattiasgees
0
73
Terraform workspaces
mattiasgees
0
100
Introduction to Ansible
mattiasgees
0
110
Other Decks in Programming
See All in Programming
コードの90%をAIが書く世界で何が待っているのか / What awaits us in a world where 90% of the code is written by AI
rkaga
46
31k
Deep Dive into ~/.claude/projects
hiragram
8
1.5k
KotlinConf 2025 現地で感じたServer-Side Kotlin
n_takehata
1
230
CursorはMCPを使った方が良いぞ
taigakono
1
180
git worktree × Claude Code × MCP ~生成AI時代の並列開発フロー~
hisuzuya
1
480
Cursor AI Agentと伴走する アプリケーションの高速リプレイス
daisuketakeda
1
130
[初登壇@jAZUG]アプリ開発者が気になるGoogleCloud/Azure+wasm/wasi
asaringo
0
130
イベントストーミング図からコードへの変換手順 / Procedure for Converting Event Storming Diagrams to Code
nrslib
1
400
Composerが「依存解決」のためにどんな工夫をしているか #phpcon
o0h
PRO
1
230
データの民主化を支える、透明性のあるデータ利活用への挑戦 2025-06-25 Database Engineering Meetup#7
y_ken
0
320
Rubyでやりたい駆動開発 / Ruby driven development
chobishiba
1
410
Result型で“失敗”を型にするPHPコードの書き方
kajitack
4
380
Featured
See All Featured
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
10
930
Making Projects Easy
brettharned
116
6.3k
GraphQLの誤解/rethinking-graphql
sonatard
71
11k
Building Applications with DynamoDB
mza
95
6.5k
GraphQLとの向き合い方2022年版
quramy
48
14k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
Adopting Sorbet at Scale
ufuk
77
9.4k
The Language of Interfaces
destraynor
158
25k
The Pragmatic Product Professional
lauravandoore
35
6.7k
Producing Creativity
orderedlist
PRO
346
40k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Transcript
Prometheus Mattias Gees
Components • Prometheus • Alertmanager • Grafana • (Pushgateway)
Architecture
Time series database!
Data Model Notation <metric name>{<label name>=<label value>, ...} Example api_http_requests_total{method="POST",
handler="/messages"}
Metric Types • Counter • Gauge • Histogram • Summary
Metrics gathering /metrics
/metrics # HELP go_gc_duration_seconds A summary of the GC invocation
durations. # TYPE go_gc_duration_seconds summary go_gc_duration_seconds{quantile="0"} 6.4062e-05 go_gc_duration_seconds{quantile="0.25"} 0.000108055 go_gc_duration_seconds{quantile="0.5"} 0.000136816 go_gc_duration_seconds{quantile="0.75"} 0.000248504 go_gc_duration_seconds{quantile="1"} 0.006248475 go_gc_duration_seconds_sum 0.175922918 go_gc_duration_seconds_count 779
Push & Pull • Client library • Exporters • Software
directly • Push based
Client library • Implemented in application • Standard metrics (some)
• Custom metrics
Client library Official • Go • Java or Scala •
Python • Ruby
Client library Unofficial • Bash • Node.JS • PHP •
...
Exporters • Converts application metrics to prometheus formatted Metrics •
3rd party applications • Databases, middleware, messaging systems, APIs, logging, ...
Exporters Examples • MySQL • Mongo • Elasticsearch • Ubiquiti
UniFi exporter • RabbitMQ exporter • Apache exporter
Exporters Examples • PHP-FPM exporter • Nginx exporter • AWS
ECS exporter • AWS SQS exporter • Fluentd exporter • Grok exporter
Software directly • Etcd • Kubernetes • Neo4j • ...
Push • Short lived processes • crons • batch •
Pushgateway
Config
Service discovery • Kubernetes • EC2 • DNS • File
• ...
Example - job_name: infrastructure/k8s-monitor-exporter-kube-controller-manager/0 scrape_interval: 15s scrape_timeout: 10s metrics_path: /metrics
scheme: http kubernetes_sd_configs: - api_server: null role: endpoints namespaces: names: - kube-system bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token tls_config: ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt insecure_skip_verify: true relabel_configs: - source_labels: [__meta_kubernetes_service_label_app] separator: ; regex: exporter-kube-controller-manager replacement: $1 action: keep
query
query
Alerts alert: K8SApiserverDown expr: absent(up{job="kubernetes"} == 1) for: 20m labels:
severity: critical annotations: description: No API servers are reachable or all have disappeared from service discovery
Alertmanager
Alertmanager • Global config • Route • Receivers • Templates
Alertmanager Route route: receiver: opsgenie group_by: - job routes: -
receiver: slack match: severity: warning - receiver: opsgenieproxy match: alertname: DeadMansSwitch group_wait: 1s group_interval: 1s repeat_interval: 1s group_wait: 30s group_interval: 5m repeat_interval: 12h
Alertmanager Receivers receivers: - name: opsgenie opsgenie_configs: - send_resolved: true
api_key: <secret> api_host: https://api.opsgenie.com/ message: '{{ template "opsgenie.default.message" . }}' description: '{{ template "opsgenie.default.description" . }}' source: '{{ template "opsgenie.default.source" . }}' tags: sla_test,kubernetes,client_skyscrapers - name: opsgenieproxy webhook_configs: - send_resolved: false url: http://k8s-monitor-opsgenie-heartbeat-proxy/proxy - name: slack slack_configs: - send_resolved: true api_url: <secret> username: skyscrapers-test color: '{{ if eq .Status "firing" }}danger{{ else }}good{{ end }}' title: '{{ template "slack.default.title" . }}' title_link: '{{ template "slack.default.titlelink" . }}' pretext: '{{ template "slack.default.pretext" . }}' text: '{{ template "slack.default.text" . }}' fallback: '{{ template "slack.default.fallback" . }}' icon_emoji: '{{ template "slack.default.iconemoji" . }}' icon_url: '{{ template "slack.default.iconurl" . }}'
Grafana
Grafana
Others • Federation • Storage • Recording rules • Kubernetes
Questions?
More information https://prometheus.io/ Docs are good ;)