$30 off During Our Annual Pro Sale. View Details »

Grafana Loki: Like Prometheus, but for logs.

Grafana
January 31, 2019

Grafana Loki: Like Prometheus, but for logs.

(Presented at CNCF Paris Meetup on 31/01/2019)

Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate, as it does not index the contents of the logs, but rather a set of labels for each log stream.

In this talk we will discuss the motivation behind Loki, its design and architecture, and what the future holds. Its early days - the project was only launched at KubeCon Seattle in Dec 2018 - so this is a great opportunity to influence the direction the project goes!

Grafana

January 31, 2019
Tweet

More Decks by Grafana

Other Decks in Technology

Transcript

  1. Grafana Loki: Like Prometheus, but for logs.
    Tom Wilkie, Feb 2019

    View Slide

  2. Tom Wilkie VP Product, Grafana Labs
    Previously: Kausal, Weaveworks, Google, Acunu, Xensource
    Prometheus & Cortex maintainer, mixins authors etc
    Twitter: @tom_wilkie Email: [email protected]

    View Slide

  3. View Slide

  4. Loki is a horizontally-scalable, highly-available, multi-
    tenant log aggregation system inspired by Prometheus.
    03/18 Project started

    12/18 Launched at KubeCon

    12/18 #1 on HN for ~12hrs!

    01/19 ~5k GitHub stars

    https://github.com/grafana/loki
    goo.gl/5DEVH6

    View Slide

  5. #0 Simple and cost effective to operate

    #1 Integrate with existing observability tools

    #2 Cloud Native through-and-through

    View Slide

  6. #0 Simple to scale

    View Slide

  7. Existing log aggregation systems do full text indexing and support complex queries
    DEwMGIwZ => {
    time: “2018-01-31 15:41:04”,
    job: “frontend”,
    env: “dev”,
    line: “POST /api/prom/push...”
    }
    (“time", “2018-01-31 15:41:04”) -> “DEwMGIwZ”
    (“job”, “frontend”) -> “DEwMGIwZ”
    (“env”, “dev”) -> “DEwMGIwZ”
    (“line”, “POST”) -> “DEwMGIwZ”
    (“line”, “/api/prom/push”) -> “DEwMGIwZ”
    (“line”, “HTTP/1.1”) -> “DEwMGIwZ”
    (“line”, “502”) -> “DEwMGIwZ”

    View Slide

  8. (“time", “2018-01-31 15:41:04”) -> “DEwMGIwZ”
    (“job”, “frontend”) -> “DEwMGIwZ”
    (“env”, “dev”) -> “DEwMGIwZ”
    (“line”, “POST”) -> “DEwMGIwZ”
    (“line”, “/api/prom/push”) -> “DEwMGIwZ”
    (“line”, “HTTP/1.1”) -> “DEwMGIwZ”
    (“line”, “502”) -> “DEwMGIwZ”
    NodeN

    Node1
    Node0
    Existing log aggregation systems do full text indexing and support complex queries

    View Slide

  9. {job=“frontend”, env=“dev”} => {
    time: “2018-01-31 15:41:04”,
    line: “POST /api/prom/push HTTP/1.1 502 0"
    }
    Loki doesn’t index the text of the logs, instead grouping entries
    into “streams” and indexing those with labels.

    View Slide

  10. #1 Integrated with
    existing tools

    View Slide

  11. 1. Alert 2. Dashboard 3. Adhoc Query
    4. Log Aggregation
    5. Distributed Tracing
    Fix!

    View Slide

  12. Prometheus’ data model is very simple:

    → [ (t0, v0), (t1, v1), ... ]
    Timestamps are millisecond int64, values are float64

    Identifiers are bags of (label, value) pairs:

    {job=“foo”, instance=“bar”, ... }
    https://www.slideshare.net/Docker/monitoring-the-prometheus-way-julius-voltz-prometheus

    View Slide

  13. Apps
    Apps
    Apps
    apps
    k8s
    #0 Prometheus talks to k8s to discover list of targets
    #1 Target information is “relabelled” to build labels
    #2 Metrics are pulled from apps
    #3 Target labels added to series labels

    View Slide

  14. Loki’s data model is very similar:

    → [ (t0, v0), (t1, v1), ... ]
    Timestamps are nanosecond floats, values are byte arrays.

    Identifiers are the same - label sets.
    https://www.slideshare.net/Docker/monitoring-the-prometheus-way-julius-voltz-prometheus

    View Slide

  15. prom
    tail
    Apps
    Apps
    Apps
    apps
    k8s

    View Slide

  16. View Slide

  17. 1. Alert 2. Dashboard 3. Adhoc Query
    4. Log Aggregation
    5. Distributed Tracing
    Fix!

    View Slide

  18. #2 Cloud Native

    View Slide

  19. containerised Kubernetes Native
    (optionally)

    Microservices
    Cloud Storage

    View Slide

  20. (optionally) microservices
    promtail
    prom
    tail
    Apps Apps Apps
    Apps
    Apps
    Apps
    Apps
    Apps
    Apps
    prom
    tail
    prom
    tail

    View Slide

  21. Demo(?)

    View Slide

  22. #0 Simple and cost effective to operate

    #1 Integrate with existing observability tools

    #2 Cloud Native through-and-through

    View Slide

  23. Thanks! Questions?

    View Slide

  24. +
    Grafana Cloud is a hosted and fully managed SaaS metrics
    platform that helps Ops and Dev teams using Grafana
    to understand the behavior of their applications and
    infrastructure
    Grafana Cloud allows users to provision and manage
    the best open source observability tools - Grafana and
    Prometheus - all through a simple UI and single API.
    What is Grafana Cloud?
    Store, visualize and alert without the headache of scaling or managing
    your own monitoring stack.
    Your complete, fully managed, hosted metrics platform.
    Grafana Cloud:

    View Slide