(Presented at London Prometheus Meetup 20/09/2019)
PromQL, the Prometheus Query Language, is a concise, powerful and increasingly popular language for querying time series data. But PromQL queries can take a long time when they have to consider >100k series and months of data. Even with Prometheus’ compression, a 90-day query over 200k series can touch ~100GB of data.
In this talk, we will present a series of techniques employed by Cortex (a CNCF project for clustered Prometheus) for accelerating PromQL queries - namely query results caching, time slice parallelisation, aggregation sharding, and automatic recoding rule substitutions.
But there’s more: we will show how you can use this technology to get these improvements with Thanos and Prometheus.