ClickHouse Cloud をオブザーバビリティ基盤の候補に選んだ理由

9/30に実施された clickhouse イベントに登壇した資料です。

Transcript

1. © Demae - can Co., Ltd. ClickHouse Cloud をををををををを ををををををををををををを 岡岡 岡 2025.09.30 ををををををを
2. © Demae - can Co., Ltd. をををを をををを • をを を (Okada Sho) • を LINE - > ををを SRE をををを ( をを )
3. © Demae - can Co., Ltd. CONFIDENTIAL PoC ををををを PoC ををを 01 02 03 をををを Agenda
4. © Demae - can Co., Ltd. PoC ををををを 4
5. © Demae - can Co., Ltd. PoC ををををを ををををを AWS ををををををを をををを をををを agent Private Cloud agent ををををををををを ( をを /をををを /ををををを ) ををををををををををををををををををを Slack Grafana (self - hosted) ををを agent
6. © Demae - can Co., Ltd. PoC ををををを ををををををををををををををををを • ををををををををを x をををををを • をををを : 250+ • をををををを : 50+ TB/day • ををををををををををををををををををををををを
7. © Demae - can Co., Ltd. PoC ををををを をををををををををををををををを
8. © Demae - can Co., Ltd. PoC ををををを ををををををををををををををを • bitdrift “Why is observability so expensive?” https://blog.bitdrift.io/post/observability - cost - crisis • Honeycomb “The Cost Crisis in Observability Tooling” https://www.honeycomb.io/blog/cost - crisis - observability - tooling • ...and more
9. © Demae - can Co., Ltd. PoC ををををを をををををををを 1 : bitdrift • https://bitdrift.io/ • ををををををををををををををををををを • をををををををををををををををををををををををををを • をををををををををををををををををををををををををををををををををを
10. © Demae - can Co., Ltd. PoC ををををを をををををををを 2 : Honeycomb • https://www.honeycomb.io/ • ををををををををを • をををををを /ををををを /ををををををををををををををををを • 1ををををををををををををををををを • をををををををををををををををををを • ををををををををををををををををををををををををををををををを
11. © Demae - can Co., Ltd. PoC ををををを をををををををををををを • Honeycomb をををををををを / ををををををををををををををををををををを • をををををををををををををををををををををををををを • ををををををををををををををををををををををを • ををををををををを
12. © Demae - can Co., Ltd. PoC ををををを をををををををををををををををををををををををををを
13. © Demae - can Co., Ltd. PoC ををををを ClickHouse をををを • Temporal を ClickHouse ををををををををををををををををををををををををを “Building Chronicle: How Temporal supercharged their observability with ClickHouse ” https://clickhouse.com/blog/building - chronicle - how - temporal - supercharged - their - observability - with - clickhouse • ををを ClickHouse ををを Datadog - > ClickHouse ををををををををををををををを “How we Built a 19 PiB Logging Platform with ClickHouse and Saved Millions” https://clickhouse.com/blog/building - a - logging - platform - with - clickhouse - and - saving - millions - over - datadog
14. © Demae - can Co., Ltd. PoC ををををを ををををを ClickHouse をををを • をををををををををををををを • をを S3 • Grafana を ClickHouse をををををを • ををををををををを Grafana ををををををを • ClickHouse をををををををををををををををを Grafana ををををををををををををををををををををををををををををを • ををををを (ClickHouse Cloud) • ををををををををををををををを • をををを • Tesla/OpenAI/Anthropic ををををををををををををををを ClickHouse ををを • をををををををを • Qiita Organization をををを “ClickHouse Japan” をををををををををを - https://qiita.com/organizations/clickhouse • ををををををををを
15. © Demae - can Co., Ltd. PoC ををををを ClickHouse ををを • をををををををををををををををををををを • をををををををををををををををををををををを • ををををををををををををを PoC ををを
16. © Demae - can Co., Ltd. PoC 16
17. © Demae - can Co., Ltd. PoC をを Kubernetes Cluster OTel Collector ClickHouse Cloud ををををををををををを をを ClickHouse Exporter • をををを (Dev) ををををををををををををををを ClickHouse Exporter を ClickHouse Cloud をををを • をををををををををを Grafana (self - hosted) をををを /をををを OTLP Exporter Grafana (self - hosted) Kubernetes Cluster OTel Collector ClickHouse Exporter OTLP Exporter
18. © Demae - can Co., Ltd. PoC をををををを • ClickHouse Exporter をををををををををををををををを • ををを /ををををを /ををををををををををををを otel_logs , otel_metrics _*, otel_traces を • をををををををを ”PARTITION BY toDate (Timestamp) ”を ”ORDER BY ( ServiceName , ..)” をををををををを
19. © Demae - can Co., Ltd. PoC をををををを : をを • PoC をを JSON をををを Map(String, String) ををををををを • ClickHouse Exporter をををををををををを JSON をををををををを • “[exporter/ clickhouseexporter ] Support JSON type for logs and traces #40547” https://github.com/open - telemetry/opentelemetry - collector - contrib/pull/40547 • をををををををををををををををををををををををををををを JSON をををををををををを
20. © Demae - can Co., Ltd. PoC をををををを • ををををををを ( ををををををを ) • ををを • をををををを /をををを
21. © Demae - can Co., Ltd. PoC をを : ををををををを (をを ) • ををををををををををををををををををををををををををを ClickHouse Cloud を Grafana をををを • ををををををををををををををを • をををを • をををををををををををををををををををを”をを”を”ををををを“ををををををを • をををををををををををををを ”PARTITION BY toDate (Timestamp) ”を ”ORDER BY ( ServiceName , ..) ” ををををををを をををををををををををを
22. © Demae - can Co., Ltd. PoC をを : ををををををを (をををを ) • をををををををををををををををををををををををををををををををををををををを • ををををををを CPU を Memory をををををををををををををををををををををををををををををををを flush をををををををををををを
23. © Demae - can Co., Ltd. PoC をを : Gateway ( OTel Collector) をををををををををををを Kubernetes Cluster OTel Collector ClickHouse Cloud OTLP Exporter Kubernetes Cluster OTel Collector OTLP Exporter Gateway (OTel Collector) OTLP Receiver Batch Processor ClickHouse Exporter
24. © Demae - can Co., Ltd. PoC をを : ををを • ををををを Storage x Compute x Egress • Storage • をををををををを 1/10 • ををををををを 1/10 • ををををををををを 1/100 ををををを • Compute • ClickHouse Cloud をををををををををををを • ををををををををををををををををををををををををををををををををををををを • Egress • Grafana をを ClickHouse Cloud をををををををををををををををををををををを • をををををををををををををををををををををををををををををを 岡岡岡岡岡岡岡岡岡岡岡岡岡岡岡岡岡
25. © Demae - can Co., Ltd. PoC をを : をををををを /をををを • をををををを • RBAC を SSO ををををををををををを • SOC2 Type2 を ISO 27001 • ををををををををををを • をををを • SLA ををををを 24/365 ををををを
26. © Demae - can Co., Ltd. PoC をを • をををををををををををををををををををををををををを • ををををををををををををををををををををををを • をををををを /をををををををををを • PoC ををををををををををををををを
27. © Demae - can Co., Ltd. PoC をををを :ClickHouse HyperDX demo site: https://play.hyperdx.io/ HyperDX (ををををを ) Private Preview • HyperDX を ClickHouse をををををををををををををををををををををををを • ををを /ををををを /ををををををををををををををををををををををををを • Private Preview ををををををををを ClickHouse Cloud ををををををををををを
28. © Demae - can Co., Ltd. PoC HyperDX (ををををを ) ををををを • ををををを • ををををををををををををををを Grafana ををををををををををををををををををを /ををををををををを • ClickHouse Cloud ををををを ClickHouse をををををををををををををををを • ををを • ををををををををををををををををををををををををををををををををををををををををををを • をををを • ををを • ををををををををををををををををををををををををををををををををををををををを
29. © Demae - can Co., Ltd. • をををををををををををををををををををををををををををををををををををををを • ををををを ClickHouse ををををををを • をををををを • ををををををををををををををををををををををを • をををを • UI をを ( HyperDX ) ををををををををを 岡岡岡岡岡岡岡岡岡岡岡岡岡岡岡岡岡岡岡岡岡岡 ClickHouse 岡岡岡岡岡岡岡岡岡岡 ををを
30. © Demae - can Co., Ltd. THANK YOU. ををををををををををををををををををを