KubeCon EU 2020 Virtual - Introduction to Thanos

Transcript

1. @ThanosMetrics
   GiedriusS / stag1e metalmatze
   

   View Slide

2. View Slide

3. View Slide

4. View Slide

5. View Slide

6. View Slide

7. View Slide

8. View Slide

9. View Slide

10. View Slide

11. View Slide

12. @ThanosMetrics
    ○
    ○
    ○
    ○
    

    View Slide

13. @ThanosMetrics
    ○
    ○
    

    View Slide

14. @ThanosMetrics
    

    View Slide

15. @ThanosMetrics
    ●
    ●
    ●
    ● 1400
    ●
    ●
    ●
    ●
    ●
    

    View Slide

16. @ThanosMetrics
    

    View Slide

17. @ThanosMetrics
    

    View Slide

18. @ThanosMetrics
    
    

    View Slide

19. @ThanosMetrics
    
    

    View Slide

20. @ThanosMetrics
    

    View Slide

21. @ThanosMetrics
    

    View Slide

22. @ThanosMetrics
    /metrics
    # TYPE counter
    app_request_total 1337
    # TYPE gauge
    app_request_in_flight 3
    # TYPE histogram
    app_request_duration_bucket {le="0.005"} 500
    app_request_duration_bucket {le="0.01"} 213
    

    View Slide

23. @ThanosMetrics
    Scrape Engine
    Rule + Alert Engine Query Engine
    Compactor
    Local Storage
    SVC 1
    SVC 2
    SVC 3
    /metrics
    Alertmanager Grafana
    

    View Slide

24. @ThanosMetrics
    SVC 2
    SVC 3
    SVC 4
    NODE 1 NODE 2
    SVC 1
    

    View Slide

25. @ThanosMetrics
    SVC 2
    SVC 3
    SVC 4
    NODE 1 NODE 2
    SVC 1
    

    View Slide

26. @ThanosMetrics
    SVC 2
    SVC 3
    SVC 4
    NODE 1
    SVC 1
    NODE 2
    

    View Slide

27. @ThanosMetrics
    NODE 1
    Local Storage
    

    View Slide

28. @ThanosMetrics
    NODE 1
    Local Storage
    

    View Slide

29. @ThanosMetrics
    Monitoring Cluster
    Cluster 1 Cluster 2 Cluster 3
    /federate /federate /federate
    

    View Slide

30. @ThanosMetrics
    Monitoring Cluster
    Cluster 1 Cluster 2 Cluster 3
    /federate /federate /federate
    

    View Slide

31. @ThanosMetrics
    Monitoring Cluster
    Cluster 1 Cluster 2 Cluster 3
    /federate /federate /federate
    

    View Slide

32. @ThanosMetrics
    Monitoring Cluster
    /federate /federate /federate /federate
    ...
    Cluster 1 Cluster 2 Cluster 3 Cluster N
    

    View Slide

33. @ThanosMetrics
    

    View Slide

34. @ThanosMetrics
    Downsampling & Compaction
    Unlimited Retention
    Prometheus Compatible Global Query View
    

    View Slide

35. @ThanosMetrics
    Monitoring Cluster
    /federate /federate /federate
    Cluster 1 Cluster 2 Cluster 3
    

    View Slide

36. @ThanosMetrics
    Monitoring Cluster
    /query /query /query
    Cluster 1 Cluster 2 Cluster 3
    ?
    

    View Slide

37. @ThanosMetrics
    Monitoring Cluster
    /query /query /query /query
    ...
    Cluster 1 Cluster 2 Cluster 3 Cluster N
    ?
    

    View Slide

38. @ThanosMetrics
    Monitoring Cluster
    Cluster 1
    ...
    ?
    ?
    Cluster 2
    ?
    Cluster N
    ?
    /query /query /query
    

    View Slide

39. @ThanosMetrics
    Monitoring Cluster
    Cluster 1
    gRPC
    ...
    ?
    ?
    Cluster 2
    gRPC
    ?
    Cluster N
    gRPC
    ?
    

    View Slide

40. @ThanosMetrics
    Monitoring Cluster
    Cluster 1
    gRPC
    ...
    Cluster 2
    gRPC
    Cluster N
    gRPC
    Query
    

    View Slide

41. @ThanosMetrics
    Thanos Deep Dive:
    Inside a Distributed
    Monitoring System
    KubeCon San Diego talk
    

    View Slide

42. @ThanosMetrics
    Monitoring Cluster
    Cluster 1
    gRPC
    ...
    Cluster 2
    gRPC
    Cluster N
    gRPC
    Query
    Query
    

    View Slide

43. @ThanosMetrics
    Monitoring Cluster
    Cluster 1
    gRPC
    ...
    Cluster 2 Cluster N
    gRPC gRPC
    gRPC gRPC
    gRPC
    Query
    

    View Slide

44. @ThanosMetrics
    Monitoring Cluster
    Cluster 1
    ...
    Cluster 2 Cluster N
    StoreAPI StoreAPI StoreAPI
    Query
    Query Query Query
    

    View Slide

45. @ThanosMetrics
    Monitoring Cluster
    Cluster 1
    ...
    Cluster 2 Cluster N
    Query
    Query Query Query
    

    View Slide

46. @ThanosMetrics
    Monitoring Cluster
    Cluster 1
    ...
    Cluster 2 Cluster N
    Object
    Storage
    Query
    TSDB
    blocks
    TSDB
    blocks
    Query Query Query
    

    View Slide

47. @ThanosMetrics
    Monitoring Cluster
    Cluster 1
    ...
    Cluster 2 Cluster N
    Object
    Storage
    Query Store
    Query Query Query
    

    View Slide

48. @ThanosMetrics
    Monitoring Cluster
    Cluster 1
    ...
    Query
    Cluster 2 Cluster N
    Object
    Storage
    Compact
    Store
    Query Query Query
    

    View Slide

49. @ThanosMetrics
    Monitoring Cluster
    Cluster 1
    ...
    Query
    Cluster 2 Cluster N
    Object
    Storage
    Compact
    Store
    Query Query Query
    Object
    Storage
    Replicate
    

    View Slide

50. @ThanosMetrics
    Query Sidecar Store Compact Replicate
    

    View Slide

51. @ThanosMetrics
    Cluster 1 Cluster 2 Cluster 3
    Monitoring Cluster
    Query
    ?
    

    View Slide

52. @ThanosMetrics
    Cluster 1 Cluster 2 Cluster 3
    Monitoring Cluster
    Query
    ?
    

    View Slide

53. @ThanosMetrics
    Monitoring Cluster
    ...
    Cluster 1 Cluster 2 Cluster 3 Cluster N
    Query
    ?
    

    View Slide

54. @ThanosMetrics
    ?
    ?
    ?
    ?
    ?
    ?
    

    View Slide

55. @ThanosMetrics
    ?
    ?
    ?
    ?
    ?
    ?
    

    View Slide

56. @ThanosMetrics
    ?
    ?
    ?
    ?
    ?
    ?
    Monitoring Cluster
    ...
    Cluster 1 Cluster 2 Cluster 3 Cluster N
    Query
    

    View Slide

57. @ThanosMetrics
    ?
    ?
    ?
    ?
    ?
    ?
    ...
    Cluster 1 Cluster 2 Cluster 3 Cluster N
    Monitoring Cluster
    Object
    Storage
    Query Store
    

    View Slide

58. @ThanosMetrics
    ?
    ?
    ?
    ?
    ?
    ...
    Cluster 1 Cluster 2 Cluster 3 Cluster N
    Monitoring Cluster
    Object
    Storage
    Query Store
    Receive
    Ingesting Millions of
    Metrics with Thanos
    Receive - Lucas
    Today (13:00)
    

    View Slide

59. @ThanosMetrics
    ?
    ?
    ?
    ?
    ?
    ...
    Cluster 1 Cluster 2 Cluster 3 Cluster N
    Object
    Storage
    Query
    Store
    Receive
    Rule
    

    View Slide

60. @ThanosMetrics
    Query Receive Store Rule
    

    View Slide

61. @ThanosMetrics
    Scrape Engine
    Rule + Alert Engine Query Engine
    Compactor
    Local Storage
    Alertmanager Grafana
    SVC 1
    SVC 2
    SVC 3
    /metrics
    

    View Slide

62. @ThanosMetrics
    Scrape Engine
    Rule + Alert Engine
    Compactor
    Local Storage
    Alertmanager
    SVC 1
    SVC 2
    SVC 3
    /metrics
    Grafana
    Query
    

    View Slide

63. @ThanosMetrics
    Sidecar/Receive
    Compactor
    Object Storage
    Grafana
    Query
    SVC 1
    SVC 2
    SVC 3
    /metrics
    Rule + Alert Engine
    Alertmanager
    

    View Slide

64. @ThanosMetrics
    Object Storage
    Compact/Store
    SVC 1
    SVC 2
    SVC 3
    /metrics
    Sidecar/Receive
    Grafana
    Query
    Rule + Alert Engine
    Alertmanager
    

    View Slide

65. @ThanosMetrics
    Object Storage
    Compact/Store
    Rule
    Alertmanager
    SVC 1
    SVC 2
    SVC 3
    /metrics
    Sidecar/Receive
    Grafana
    Query
    

    View Slide

66. @ThanosMetrics
    https://www.katacoda.com/thanos
    

    View Slide

67. @ThanosMetrics
    ●
    ○
    ●
    ○
    ○
    ●
    ●
    ○
    

    View Slide

68. @ThanosMetrics
    ● Started participating in various initiatives
    such as the Community Bridge
    mentorship program late last year
    ● Increases diversity in open source
    communities and gets more open source
    code written and released for the benefit
    of all.
    ● Lots of projects going on at the moment
    and some awesome stuff has been
    implemented already such as delayed
    deletions by Khyati Soneji
    

    View Slide

69. @ThanosMetrics
    

    View Slide

70. @ThanosMetrics
    

    View Slide