Prometheusで監視・モニタリング #hackt

Transcript

1. PrometheusͰ ؂ࢹɾϞχλϦϯά Hacker Tackle 2016 2016/09/10 @matsumana
2. ࣗݾ঺հ • ໊લɿ দ࡚ ֶ • Twitterɿ @matsumana • ৬ۀɿ ιϑτ΢ΣΞΤϯδχΞ
 ʢϓϩάϥϚɺWebαʔϏεӡ༻ʣ
3. ΞδΣϯμ • Prometheusͷ঺հ • ଞͷπʔϧͱͷൺֱ • promgenͷ঺հ • exporterʹ͍ͭͯ
4. Ξϯέʔτ
5. ݱࡏԿΛ࢖ͬͯ·͔͢ʁ • Nagios • Ganglia • Munin • Cacti • Zabbix • AWS CloudWatch • Datadog • Mackerel • New Relic • ͦΕҎ֎
6. PrometheusͷࣄΛ ஌ͬͯ·͔ͨ͠ʁ
7. Prometheusͱ͸ • SoundCloud͕ࣾ։ൃ͠ɺOSSͱͯ͠ެ։͍ͯ͠Δ؂ࢹɾϞχλϦϯάπʔϧ • ࠷ॳͷcommit͸2012/11/24 • 2016/07/18ʹ1.0.0ϦϦʔε
 (ࠓ೔࣌఺Ͱͷ࠷৽͸1.1.0) • ࠷ۙ஫໨౓্͕ঢத • Prometheus Casual Talks @౦ژ → ࢀՃऀ100໊ • Prometheus Core Developer Björn Rabenstein Open Q&A @ϩϯυϯ
 ˠ ࢀՃऀ 27໊ • PromCon 2016 @ϕϧϦϯ → ࢀՃऀ 80໊
8. Prometheusͷػೳ • time series database (࣌ܥྻσʔλϕʔε) • ϝτϦΫεऩू & อଘ • อଘͨ͠ϝτϦΫεʹର͢ΔΫΤϦ • ͖͍͠஋Λઃఆͯ͠ΞϥʔτΛඈ͹͢ • μογϡϘʔυ
9. Prometheusʹແ͍ػೳ • ੜϩά/Πϕϯτσʔλͷऩू
 (Elasticsearchͷ༷ͳ) • ϦΫΤετͷτϨʔγϯά
 (Zipkinͷ༷ͳ) • ػցֶश΍σʔλϚΠχϯάʹΑΔݕग़ • ϝτϦΫεͷ௕ظอଘετϨʔδ • ࣗಈతͳਫฏεέʔϦϯά • Ϣʔβೝূ
10. ͦͷଞͷಛ௃ • golangͰॻ͔Ε͍ͯΔ • pullܕ
 ؂ࢹର৅ϊʔυʹexportorΛΠϯετʔϧͯ͠Prometheus͔ΒϙʔϦϯά • Grafanaͱ࿈ܞՄೳ • PrometheusࣗମʹPromDashͱ͍͏μογϡϘʔυπʔϧ͕͋Δ͕ɺ Grafanaͱͷ࿈ܞ͕ਪ঑ • CNCF(Cloud Native Computing Foundation)ͷϓϩδΣΫτʹͳͬͨ • Kubernetesͱ౷߹Մೳ
 ※ͱ͸ݴͬͯ΋Docker༻Ͱ͸ͳ͍ͷͰɺ΋ͪΖΜΦϯϓϨͰ΋࢖͑Δ
11. ΞʔΩςΫνϟ
12. pros • Grafanaͱ࿈ܞग़དྷΔ • ΫΤϦ͕ॻ͚Δ • αʔόεϖοΫ΍ϝτϦΫε਺ʹΑΔͱࢥ͏͕
 1୆ͷαʔόͰ୔ࢁͷϊʔυΛ؂ࢹͰ͖ΔΒ͍͠ • 1୆ͷαʔόͰ3,000ϊʔυ͘Β͍؂ࢹग़དྷΔΒ͍͠ • AWSͷ৔߹ɺ50ʙ100ϊʔυ͘Β͍ͳΒαʔόεϖοΫ͸ t2.medium(CPU: 2Core, Mem: 4GB)Ͱे෼Β͍͠ • exportor͕๛෋ɻ΋͠ཉ͍͠΋ͷ͕ແ͚Ε͹؆୯ʹ࡞ΕΔ
13. cons • ໊લ͕ΞϨͳͷͰάάΓʹ͍͘
 ʢөըɺΪϦγϟਆ࿩ͷਆʣ • ΫΤϦͷॻ͖ํ͕Α͘Θ͔Βͳ͍ • ؂ࢹର৅ϊʔυͷ௥Ճ͸ઃఆϑΝΠϧʹॻ͍ͯ
 ϓϩηεΛ࠶ىಈʢGUI͕ͳ͍ʣ • TimeZone͕GMT (GrafanaΛ࢖͑͹໰୊ͳ͍)
14. ଞͷπʔϧͱͷൺֱ • ྫ͑͹Zabbixͱൺֱ͢ΔͱɺͲΜͳײ͡ʁ • γϯϓϧ ʘ(^o^)ʗ • ίϯϙʔωϯτ͕ૄ݁߹ͳͷ͸ɺ
 1ͭͷαʔϏεͰ਺ϊʔυ؂ࢹ͢Δ͚ͩͳΒZabbixͷ༷ͳ
 ΦʔϧΠϯϫϯͷํ͕͍͍͔΋ (´ɾωɾʆ) • WebUIͰઃఆग़དྷͳ͍ (´ɾωɾʆ) • ϝϯςφϯε͕࣌ؒઃఆग़དྷͳ͍ (´ɾωɾʆ) • Ξϥʔτૹ৴ઌϝʔϧΞυϨεͷઃఆ͕ࡉ͔͘ग़དྷͳ͍ (´ɾωɾʆ)
15. Prometheus
 ྑ͍Μ͚ͩͲ ͦͷ··࢖͏ͷ͸ ݁ߏͭΒ͍
16. ͦ͜Ͱ
17. promgen • GitHubͰެ։த • https://github.com/line/promgen • WebUIͰઃఆ͕Մೳ
18. promgen Service
19. promgen Project
20. ΞʔΩςΫνϟ (࠶ܝ)
21. Prometheus w/ promgen
22. exporter
23. Α͘࢖͍ͦ͏ͳexporter • Default port allocations
 Wiki͸ࣗ༝ʹฤूՄೳɻศརͳexporterΛ࡞ͬͨΒొ࿥͠Α͏ʂ • node_exporter • blackbox_exporter • nginx_exporter • apache_exporter • mysqld_exporter • redis_exporter • memcached_exporter • jmx_exporter • zookeeper_exporter • elasticsearch_exporter
24. exporterͷ࡞Γํ • ΫϥΠΞϯτϥΠϒϥϦΛ࢖͏ͱָ • client_java • client_golang • client_python • client_ruby
25. exporterͷಈ࡞Πϝʔδ
26. exporterΛ؂ࢹର৅ΞϓϦʹ૊ΈࠐΉࣄ΋Մೳ
27. γϯϓϧͳ exporterͷιʔεΛྫʹ ࡞ΓํΛઆ໌͠·͢
28. σϞ • promgen • Prometheus • Grafana
29. ϦϯΫू • ΦϑΟγϟϧ • prometheus.io • Github Organization • YouTubeνϟϯωϧ ʢPromConͷಈը͋Γʣ ← ͓͢͢Ί • Prometheus Casual Talks #1Λ։࠵͠·ͨ͠ • Prometheus London meetupͰLT͖ͯ͠·ͨ͠ • PromCon 2016ʹߦͬͯൃද͖ͯ͠·ͨ͠ • promgen - prometheus managemnet tool / simpleclient_java hacks @ Prometheus casual