sensuとその周辺のはなし

Transcript

1. 1.

   sensuͱͦͷपลͷ͸ͳ͠ Monitoring Casual #6 2014.6.12 ϋονגࣜձࣾ Nobutoshi Ogata

2. 2.

   ࣗݾ঺հ • ඌܗெढ़ a.k.a @nobu666 • ɹɹɹˠ ϋονגࣜձࣾ • talentio

   ͱ͍͏αʔϏεΛ࡞͍ͬͯ·͢ • ਓࡐ࠾༻ͱٻ৬׆ಈͷͨΊͷαʔϏεͰ͢ • ΠϯϑϥΤϯδχΞɺͱ͍ͭͭL4͔Β্
3. 3.

   talentioͷαʔόߏ੒֓ཁ %JSFDU
   $POOFDU ౿Έ୆ DB memchached DC AWS ELB WWW batch

   sensu
4. 4.

   ೖ͙ࣾͯ͢͠ͷঢ়ଶ • ϝʔϧ΋ඈ͹ͳ͍ঢ়ଶͷZabbix • ઃఆΛࢼΈΔ͕UIͷμϝ͞ʹ৺͕ંΕΔ • all-in-oneͳͷ͸͍͍Μ͚ͩͲͳʔ • ͳΔ΂͘গͳ͍ઃఆͰࢮ׆؂ࢹೖΕ͍ͨ •

   ͔֦ͭுੑ͕͋Δͱͳ͓Α͍
5. 5.

   None
6. 6.

   sensuͷ͍͍ͱ͜Ζ • clientͷొ࿥Λ͠ͳ͍Ͱ͍͍ • client.jsonʹద౰ʹattributesΛ௥Ճͯ͠ɺ sensu-serverͰҰׅ؅ཧͰ͖Δ • nagios pluginΛύΫΕΔ

7. 7.

   sensuͷͭΒΈ (1) • No keep-alive… ͕ফ͑ͯ͘Εͳ͍ • Ͳͳ͔ͨղܾํ๏Λ͝ଘ஌Ͱͨ͠Βڭ͑ͯ Լ͍͞…

8. 8.

   sensuͷͭΒΈ (2) • Πϥͬͱ͢Δ • ͷͰ͜͏͠·ͨ͠… $ cat /etc/init.d/sensu #!/bin/bash

   /etc/init.d/sensu-server $1 /etc/init.d/sensu-api $1
9. 9.

   sensuͰ΍ͬͯΔ͜ͱ • Health check • process(http/mysql/batch) • response(http) • replication

   status(mysql/solr) • disk usage, load ave. • Notification • mail • slack
10. 10.

    ଟগखΛೖΕͯΔ • backup༻ʹ͍͔ͭ͘ͷmysqlΛmysqld_multiͰू໿ ͯ͠Δ • check-mysql-alive.jsonʹ͍͍ͪͪ෇͚଍͢ͷ͕Ί ΜͲ͍ͷͰɺmysqld_multi reportͱmy.cnf͔Β؂ ࢹର৅ΛউखʹܾΊΔΑ͏ʹ •

    slack௨஌࣌ ```hogehoge``` ͱ͔ͨͬͨ͠ͷͰࢦఆͷ จࣈྻͰғΊΔΑ͏ʹpull requestத
11. 11.

    sensuͰ΍ͬͯͳ͍͜ͱ • metricsͷऩू • why? • graphiteΊΜͲ͍ʢ୯ʹΑ͘Θ͔ͬͯͳ͍ʣ • es+kibana͸େ͛͞ͳؾ͕͢Δ •

    ͍·͸NewRelic࢖ͬͯΔ • agentͷinstallͱpluginͷinstall͕ͪΐͬͱΊΜͲ͍ • ΊΜͲ͍ͬͭʔ͔ͨ·ʹsyntax errorͱ͔Ͱͯಈ͔ͳ͍ • mackerel΋ͪΐͬͱ͓ࢼ͠த
12. 12.

    ͜Ε͔Β΍Γ͍ͨ͜ͱ • ansible࿈ܞ • reloadͩͷrestartͩͷͷλΠϛϯάͰalert͕ඈΜͰ͠·͏ͷͰ໧Β͍ͤͨ • sensu-cliͰͰ͖Δͷ͔ͳ • twilio +

    slack࿈ܞ • ਂ໷ͩͱϝʔϧ͚ͩͰ͸ؾ͔ͮͳ͍ͷͰి࿩໐Β͍ͨ͠ • ి࿩ʹग़ͯ͠·͏ͱ͔͔ۚΔͷͰɺslack͔Βి࿩ͱΊΔ • hubot + slack࿈ܞ • sensu-adminͷը໘͔ΒͰ͖Δ͜ͱ͸slack͔Β΍ΕΔΑ͏ʹ͍ͨ͠
13. 13.

    ͝੩ௌ ͋Γ͕ͱ͏͍͟͝·ͨ͠