Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
InfluxDB + Prometheus
Search
Paul Dix
August 18, 2017
Programming
2
3.2k
InfluxDB + Prometheus
Slides from my Promcon2017 talk on integrating InfluxDB and Prometheus.
Paul Dix
August 18, 2017
Tweet
Share
More Decks by Paul Dix
See All by Paul Dix
InfluxDB IOx Project Update - 2021-02-10
pauldix
0
230
InfluxDB IOx data lifecycle and object store persistence
pauldix
1
630
InfluxDB 2.0 and Flux
pauldix
1
730
Flux and InfluxDB 2.0
pauldix
1
1.3k
Querying Prometheus with Flux
pauldix
1
920
Flux (#fluxlang): a new (time series) data scripting language
pauldix
7
5.2k
At Scale, Everything is Hard
pauldix
2
720
IFQL and the future of InfluxData
pauldix
2
1.4k
Time series & monitoring with InfluxDB and the TICK stack
pauldix
0
470
Other Decks in Programming
See All in Programming
学習を成果に繋げるための個人開発の考え方 〜 「学習のための個人開発」のすすめ / personal project for leaning
panda_program
1
110
kiroでゲームを作ってみた
iriikeita
0
180
なぜ今、Terraformの本を書いたのか? - 著者陣に聞く!『Terraformではじめる実践IaC』登壇資料
fufuhu
4
640
技術的負債で信頼性が限界だったWordPress運用をShifterで完全復活させた話
rvirus0817
1
2k
React 使いじゃなくても知っておきたい教養としての React
oukayuka
18
5.8k
20250808_AIAgent勉強会_ClaudeCodeデータ分析の実運用〜競馬を題材に回収率100%の先を目指すメソッドとは〜
kkakeru
0
200
TROCCO×dbtで実現する人にもAIにもやさしいデータ基盤
nealle
0
300
バイブコーディングの正体——AIエージェントはソフトウェア開発を変えるか?
stakaya
5
1k
UbieのAIパートナーを支えるコンテキストエンジニアリング実践
syucream
2
660
あまり知られていない MCP 仕様たち / MCP specifications that aren’t widely known
ktr_0731
0
290
A Gopher's Guide to Vibe Coding
danicat
0
170
tool ディレクティブを導入してみた感想
sgash708
1
150
Featured
See All Featured
Designing for humans not robots
tammielis
253
25k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
131
19k
RailsConf 2023
tenderlove
30
1.2k
Scaling GitHub
holman
462
140k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
Docker and Python
trallard
45
3.5k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
8
470
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Testing 201, or: Great Expectations
jmmastey
45
7.6k
Unsuck your backbone
ammeep
671
58k
Bash Introduction
62gerente
614
210k
Balancing Empowerment & Direction
lara
2
580
Transcript
InfluxDB + Prometheus Paul Dix CTO @ InfluxData @pauldix paul@influxdb.com
Open Source Time Series Database
• OSS - MIT License • Written in Go •
SQL-ish query language • Time Series Merge Tree storage engine & inverted index • Commercial Enterprise - HA + Scale Out Clustering
Data Model Measurement cpu,host=serverA,num=1,region=west idle=1.667,system=2342.2 1492214400000000000 Tags Fields nanosecond epoch
float64, int64, bool, string uint64 in next release
Example Query select percentile(90, value) from cpu where time >
now() - 12h and “region” = ‘west’ group by time(10m), host
Process: • OSS - MIT License • Written in Go
• Process, monitor, alert, act/execute • TICK script • Streaming & Batch • Store data back into InfluxDB • User Defined Functions • Service Discovery & Pull
Collect: • OSS - MIT License • Written in Go
• Agent deployed across infrastructure • Input plugins - system, docker, postgres, mysql, cassandra, elastic, hadoop, redis, nginx, apache, etc. • Output plugins - InfluxDB, Graphite, Kafka, etc.
Why is a competitor at Promcon?
What is Prometheus?
Prometheus Server
Alert Manager
Exposition Format
Client Libraries • Supported • Go • Java • Python
• Ruby • Community • C++ • C# • Node.js • PHP • etc…
PromQL
API /api/v1/series /api/v1/label/<name>/values
Remote Read/Write API
Prometheus is more than just the server
Influx + Prometheus today
We’re embracing pull & push
Prometheus Scrape Targets Kapacitor
https://github.com/prometheus/prometheus/issues/2911 https://github.com/influxdata/prometheus_metric_normalizer
Remote Read/Write
Prometheus Config # Remote write configuration (for Graphite, OpenTSDB, or
InfluxDB). remote_write: - url: "http://localhost:9201/write" # Remote read configuration (for InfluxDB only at the moment). remote_read: - url: "http://localhost:9201/read"
metric -> measurement labels -> tags value -> only field
(float64)
Remote Read Protobuf over HTTP
Reads message Query { int64 start_timestamp_ms = 1; int64 end_timestamp_ms
= 2; repeated LabelMatcher matchers = 3; } enum MatchType { EQUAL = 0; NOT_EQUAL = 1; REGEX_MATCH = 2; REGEX_NO_MATCH = 3; } message LabelMatcher { MatchType type = 1; string name = 2; string value = 3; }
Remote Read JSON over HTTP
Unnecessary Overhead
Can we push down processing?
Influx + Prometheus future
First, pull remote gateway into InfluxDB!
Influx Data Model tags + value + time
Like Prometheus message Sample { double value = 1; int64
timestamp_ms = 2; } message LabelPair { string name = 1; string value = 2; } message TimeSeries { repeated LabelPair labels = 1; // Sorted by time, oldest sample first. repeated Sample samples = 2; }
metric is __name__ label
But we’ll continue to support int64, bool, string, and uint64
IFQL - (Influx Functional Query Language) select(database:"foo",where:{'metric'="requests"}) .range(start:-4h) .window(period:10m) .sum()
.interpolate(start:-4h,every:10m,value:{mean($)})
Decouple QL from Processing from Storage
None
DAG { "operations": [ { "id": "select", "kind": "select", "spec":
{ "database": "mydb" } }, { "id": "range", "kind": "range", "spec": { "start": "-4h", "stop": "now" } }, { "id": "sum", "kind": "sum" } ], "edges": [ { "parent": "select", "child": "range" }, { "parent": "range", "child": "sum" } ] } }
Pushdown Predicates
Opportunities to improve storage/remote?
Bulk Write
Long term storage and query for Prometheus SaaS & On-Premise
Thank you @pauldix paul@influxdb.com