Slide 1

Slide 1 text

Querying Prometheus with Flux (#fluxlang) Paul Dix @pauldix paul@influxdata.com

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

• Data-scripting language • Functional • MIT Licensed • Language & Runtime/Engine

Slide 4

Slide 4 text

Prometheus users: so what?

Slide 5

Slide 5 text

High availability?

Slide 6

Slide 6 text

Sharded Data?

Slide 7

Slide 7 text

Federation?

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

subqueries

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

subqueries recording rules

Slide 15

Slide 15 text

Ad hoc exporation

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

Focus is Strength

Slide 18

Slide 18 text

Saying No is an Asset

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

Liberate the silo!

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

Language Elements

Slide 23

Slide 23 text

// get all data from the telegraf db from(db:"telegraf") // filter that by the last hour |> range(start:-1h) // filter further by series with a specific measurement and field |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system")

Slide 24

Slide 24 text

// get all data from the telegraf db from(db:"telegraf") // filter that by the last hour |> range(start:-1h) // filter further by series with a specific measurement and field |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system") Comments

Slide 25

Slide 25 text

// get all data from the telegraf db from(db:"telegraf") // filter that by the last hour |> range(start:-1h) // filter further by series with a specific measurement and field |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system") Functions

Slide 26

Slide 26 text

// get all data from the telegraf db from(db:"telegraf") // filter that by the last hour |> range(start:-1h) // filter further by series with a specific measurement and field |> filter(fn: r => r._measurement == "cpu" and r._field == "usage_system") Pipe forward operator

Slide 27

Slide 27 text

// get all data from the telegraf db from(db:"telegraf") // filter that by the last hour |> range(start:-1h) // filter further by series with a specific measurement and field |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system") Named Arguments

Slide 28

Slide 28 text

// get all data from the telegraf db from(db:"telegraf") // filter that by the last hour |> range(start:-1h) // filter further by series with a specific measurement and field |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system") String Literal

Slide 29

Slide 29 text

// get all data from the telegraf db from(db:"telegraf") // filter that by the last hour |> range(start:-1h) // filter further by series with a specific measurement and field |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system") Duration Literal (relative time)

Slide 30

Slide 30 text

// get all data from the telegraf db from(db:"telegraf") // filter that by the last hour |> range(start:”2018-08-09T14:00:00Z“) // filter further by series with a specific measurement and field |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system") Time Literal

Slide 31

Slide 31 text

// get all data from the telegraf db from(db:"telegraf") // filter that by the last hour |> range(start:-1h) // filter further by series with a specific measurement and field |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system") Anonymous Function

Slide 32

Slide 32 text

Operators + == != ( ) - < !~ [ ] * > =~ { } / <= = , : % >= <- . |>

Slide 33

Slide 33 text

Types • int • uint • float64 • string • duration • time • regex • array • object • function • namespace • table • table stream

Slide 34

Slide 34 text

Ways to run Flux - (interpreter, fluxd api server, InfluxDB 1.7 & 2.0)

Slide 35

Slide 35 text

Flux builder in Chronograf

Slide 36

Slide 36 text

Flux builder in Grafana

Slide 37

Slide 37 text

Flux is about:

Slide 38

Slide 38 text

Time series in Prometheus

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

// get data from Prometheus on http://localhost:9090 fromProm(query:`node_cpu_seconds_total{cpu=“0”,mode=“idle”}`) // filter that by the last minute |> range(start:-1m)

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

Multiple time series in Prometheus

Slide 50

Slide 50 text

fromProm(query: `node_cpu_seconds_total{cpu=“0”,mode=~”idle|user”}`) |> range(start:-1m) |> keep(columns: [“name”, “cpu”, “host”, “mode”, “_value”, “_time”])

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

Tables are the base unit

Slide 57

Slide 57 text

Not tied to a specific data model/schema

Slide 58

Slide 58 text

Filter function

Slide 59

Slide 59 text

fromProm() |> range(start:-1m) |> filter(fn: (r) => r.__name__ == “node_cpu_seconds_total” and r.mode == “idle” and r.cpu == “0”) |> keep(columns: [“name”, “cpu”, “host”, “mode”, “_value”, “_time”])

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

No content

Slide 62

Slide 62 text

No content

Slide 63

Slide 63 text

No content

Slide 64

Slide 64 text

fromProm() |> range(start:-1m) |> filter(fn: (r) => r.__name__ == “node_cpu_seconds_total” and r.mode in [“idle”, “user”] and r.cpu == “0”) |> keep(columns: [“name”, “cpu”, “host”, “mode”, “_value”, “_time”])

Slide 65

Slide 65 text

No content

Slide 66

Slide 66 text

No content

Slide 67

Slide 67 text

No content

Slide 68

Slide 68 text

Aggregate functions

Slide 69

Slide 69 text

fromProm() |> range(start:-30s) |> filter(fn: (r) => r.__name__ == “node_cpu_seconds_total” and r.mode == “idle” and r.cpu =~ /0|1/) |> count() |> keep(columns: [“name”, “cpu”, “host”, “mode”, “_value”, “_time”])

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

No content

Slide 72

Slide 72 text

No content

Slide 73

Slide 73 text

No content

Slide 74

Slide 74 text

No content

Slide 75

Slide 75 text

No content

Slide 76

Slide 76 text

_start and _stop are about windows of data

Slide 77

Slide 77 text

fromProm(query: `node_cpu_seconds_total{cpu=“0”,mode=“idle”}` |> range(start: -1m)

Slide 78

Slide 78 text

No content

Slide 79

Slide 79 text

fromProm(query: `node_cpu_seconds_total{cpu=“0”,mode=“idle”}` |> range(start: -1m) |> window(every: 20s)

Slide 80

Slide 80 text

No content

Slide 81

Slide 81 text

fromProm(query: `node_cpu_seconds_total{cpu=“0”,mode=“idle”}` |> range(start: -1m) |> window(every: 20s)j |> min()

Slide 82

Slide 82 text

No content

Slide 83

Slide 83 text

fromProm(query: `node_cpu_seconds_total{cpu=“0”,mode=“idle”}` |> range(start: -1m) |> window(every: 20s)j |> min() |> window(every:inf)

Slide 84

Slide 84 text

No content

Slide 85

Slide 85 text

Window converts N tables to M tables based on time boundaries

Slide 86

Slide 86 text

Group converts N tables to M tables based on values

Slide 87

Slide 87 text

fromProm(query: `node_cpu_seconds_total{cpu=~“0|1”,mode=“idle”}`) |> range(start: -1m)

Slide 88

Slide 88 text

No content

Slide 89

Slide 89 text

fromProm(query: `node_cpu_seconds_total{cpu=~“0|1”,mode=“idle”}`) |> range(start: -1m) |> group(columns: [“__name__”, “mode”])

Slide 90

Slide 90 text

No content

Slide 91

Slide 91 text

No content

Slide 92

Slide 92 text

No content

Slide 93

Slide 93 text

Nested range vectors fromProm(host:”http://localhost:9090") |> filter(fn: (r) => r.__name__ == "node_disk_written_bytes_total") |> range(start:-1h) // transform into non-negative derivative values |> derivative() // break those out into tables for each 10 minute block of time |> window(every:10m) // get the max rate of change in each 10 minute window |> max() // and put everything back into a single table |> window(every:inf) // and now let’s convert to KB |> map(fn: (r) => r._value / 1024.0)

Slide 94

Slide 94 text

Multiple Servers dc1 = fromProm(host:”http://prom.dc1.local:9090") |> filter(fn: (r) => r.__name__ == “node_network_receive_bytes_total”) |> range(start:-1h) |> insertGroupKey(key: “dc”, value: “1”) dc2 = fromProm(host:”http://prom.dc2.local:9090") |> filter(fn: (r) => r.__name__ == “node_network_receive_bytes_total”) |> range(start:-1h) |> insertGroupKey(key: “dc”, value: “2”) dc1 |> union(streams: [dc2]) |> limit(n: 2) |> derivative() |> group(columns: [“dc”]) |> sum()

Slide 95

Slide 95 text

Work with data from many sources • from() // influx • fromProm() • fromMySQL() • fromCSV() • fromS3() • …

Slide 96

Slide 96 text

Defining Functions fromProm(query: `node_cpu_seconds_total{cpu=“0”,mode=“idle”}` |> range(start: -1m) |> window(every: 20s)j |> min() |> window(every:inf)

Slide 97

Slide 97 text

Defining Functions windowAgg = (every, fn, <-stream) => { return stream |> window(every: every) |> fn() |> window(every:inf) } fromProm(query: `node_cpu_seconds_total{cpu=“0”,mode=“idle”}` |> range(start: -1m) |> windowAgg(every:20s, fn: min)

Slide 98

Slide 98 text

Packages & Namespaces package “flux-helpers” windowAgg = (every, fn, <-stream) => { return stream |> window(every: every) |> fn() |> window(every:inf) } // in a new script import helpers “github.com/pauldix/flux-helpers" fromProm(query: `node_cpu_seconds_total{cpu=“0”,mode=“idle”}` |> range(start: -1m) |> helpers.windowAgg(every:20s, fn: min)

Slide 99

Slide 99 text

Project Status • Everything in this talk is prototype (as of 2018-08-09) • Proposed Final Language Spec • Release flux, fluxd, InfluxDB 1.7, InfluxDB 2.0 alpha • Iterate with community to finalize spec • Optimizations! • https://github.com/influxdata/flux

Slide 100

Slide 100 text

Future work

Slide 101

Slide 101 text

More complex Flux compilations to PromQL?

Slide 102

Slide 102 text

PromQL parser for Flux engine?

Slide 103

Slide 103 text

Add Flux into Prometheus?

Slide 104

Slide 104 text

Arrow API for Prometheus

Slide 105

Slide 105 text

Apache Arrow

Slide 106

Slide 106 text

Stream from Prometheus

Slide 107

Slide 107 text

Pushdown matcher and range

Slide 108

Slide 108 text

Later pushdown more?

Slide 109

Slide 109 text

Standardized Remote Read API?

Slide 110

Slide 110 text

Arrow is becoming the lingua franca in data science and big data

Slide 111

Slide 111 text

fromProm(query: `{__name__=~/node_.*/}`) |> range(start:-1h) |> toCSV(file: “node-data.csv”) |> toFeather(file: “node-data.feather”)

Slide 112

Slide 112 text

Much more work to be done…

Slide 113

Slide 113 text

Prometheus + Flux = Possibilities

Slide 114

Slide 114 text

Thank you Paul Dix @pauldix paul@influxdata.com