sensuとその周辺のはなし

Transcript

1. sensuͱͦͷपลͷ͸ͳ͠ Monitoring Casual #6 2014.6.12 ϋονגࣜձࣾ Nobutoshi Ogata
2. ࣗݾ঺հ • ඌܗெढ़ a.k.a @nobu666 • ɹɹɹˠ ϋονגࣜձࣾ • talentio ͱ͍͏αʔϏεΛ࡞͍ͬͯ·͢ • ਓࡐ࠾༻ͱٻ৬׆ಈͷͨΊͷαʔϏεͰ͢ • ΠϯϑϥΤϯδχΞɺͱ͍ͭͭL4͔Β্
3. talentioͷαʔόߏ੒֓ཁ %JSFDU
   $POOFDU ౿Έ୆ DB memchached DC AWS ELB WWW batch sensu
4. ೖ͙ࣾͯ͢͠ͷঢ়ଶ • ϝʔϧ΋ඈ͹ͳ͍ঢ়ଶͷZabbix • ઃఆΛࢼΈΔ͕UIͷμϝ͞ʹ৺͕ંΕΔ • all-in-oneͳͷ͸͍͍Μ͚ͩͲͳʔ • ͳΔ΂͘গͳ͍ઃఆͰࢮ׆؂ࢹೖΕ͍ͨ • ͔֦ͭுੑ͕͋Δͱͳ͓Α͍
6. sensuͷ͍͍ͱ͜Ζ • clientͷొ࿥Λ͠ͳ͍Ͱ͍͍ • client.jsonʹద౰ʹattributesΛ௥Ճͯ͠ɺ sensu-serverͰҰׅ؅ཧͰ͖Δ • nagios pluginΛύΫΕΔ
7. sensuͷͭΒΈ (1) • No keep-alive… ͕ফ͑ͯ͘Εͳ͍ • Ͳͳ͔ͨղܾํ๏Λ͝ଘ஌Ͱͨ͠Βڭ͑ͯ Լ͍͞…
8. sensuͷͭΒΈ (2) • Πϥͬͱ͢Δ • ͷͰ͜͏͠·ͨ͠… $ cat /etc/init.d/sensu #!/bin/bash /etc/init.d/sensu-server $1 /etc/init.d/sensu-api $1
9. sensuͰ΍ͬͯΔ͜ͱ • Health check • process(http/mysql/batch) • response(http) • replication status(mysql/solr) • disk usage, load ave. • Notification • mail • slack
10. ଟগखΛೖΕͯΔ • backup༻ʹ͍͔ͭ͘ͷmysqlΛmysqld_multiͰू໿ ͯ͠Δ • check-mysql-alive.jsonʹ͍͍ͪͪ෇͚଍͢ͷ͕Ί ΜͲ͍ͷͰɺmysqld_multi reportͱmy.cnf͔Β؂ ࢹର৅ΛউखʹܾΊΔΑ͏ʹ • slack௨஌࣌ ```hogehoge``` ͱ͔ͨͬͨ͠ͷͰࢦఆͷ จࣈྻͰғΊΔΑ͏ʹpull requestத
11. sensuͰ΍ͬͯͳ͍͜ͱ • metricsͷऩू • why? • graphiteΊΜͲ͍ʢ୯ʹΑ͘Θ͔ͬͯͳ͍ʣ • es+kibana͸େ͛͞ͳؾ͕͢Δ • ͍·͸NewRelic࢖ͬͯΔ • agentͷinstallͱpluginͷinstall͕ͪΐͬͱΊΜͲ͍ • ΊΜͲ͍ͬͭʔ͔ͨ·ʹsyntax errorͱ͔Ͱͯಈ͔ͳ͍ • mackerel΋ͪΐͬͱ͓ࢼ͠த
12. ͜Ε͔Β΍Γ͍ͨ͜ͱ • ansible࿈ܞ • reloadͩͷrestartͩͷͷλΠϛϯάͰalert͕ඈΜͰ͠·͏ͷͰ໧Β͍ͤͨ • sensu-cliͰͰ͖Δͷ͔ͳ • twilio + slack࿈ܞ • ਂ໷ͩͱϝʔϧ͚ͩͰ͸ؾ͔ͮͳ͍ͷͰి࿩໐Β͍ͨ͠ • ి࿩ʹग़ͯ͠·͏ͱ͔͔ۚΔͷͰɺslack͔Βి࿩ͱΊΔ • hubot + slack࿈ܞ • sensu-adminͷը໘͔ΒͰ͖Δ͜ͱ͸slack͔Β΍ΕΔΑ͏ʹ͍ͨ͠
13. ͝੩ௌ ͋Γ͕ͱ͏͍͟͝·ͨ͠