Monitorel - Web API for Server Resource Graph

RRDtool in the hell

Transcript

1. Monitorel Web API for server resource graph Kyoto.pm 5 Tech Talk
2. Hatena
   JEZ@VVLJ Twitter
   !Z@VVL CPAN
   
   
   :66,* ͸ͯͳΞϧόΠλʔ !γεςϜϓϥοτϑΥʔϜ෦ ೥ʹ͓͍ͯ৽͘͠ 1FSMΛษڧ͠Α͏ͱ͢Δ ෺޷͖ͳϫΧϞϊ
3. Perl CPAN WebService::Qiita RRDTool::Rawish Plack::Middleware::GitStatus Net::Signalet vim-plugin unite-perl-module.vim perl-local-lib-path.vim
4. ἤΒΕΔKyoto.pm
5. ࠓ೔ͷ͓࿩ αʔόϦιʔεՄࢹԽ ͭͬͨ͘΍ͭ ( Monitorel ) αʔόϦιʔεΛάϥϑԽ͢ΔͨΊͷAPI
6. αʔόϦιʔε ҎԼͷΑ͏ʹ࣌ؒมԽ͢Δ΋ͷΛɹɹɹɹɹ αʔόϦιʔεͱ͢Δ OS͔ΒͱΕΔ΍ͭ(load averageͱ͔CPU usage) ϛυϧ΢ΣΞ͔ΒͱΕΔ΍ͭ( MySQLͩͱ Connections, Qcache_hits, Slow queries) 1ݸ1ݸͷ஋ΛϝτϦΫεͱ͢Δ ࣌ؒมԽ͠ͳ͍΋ͷΛϗετ৘ใͱ͢Δ IPΞυϨε, MACΞυϨε, ϚγϯεϖοΫͳͲ
7. αʔόϦιʔεՄࢹԽ Munin Cacti Zabbix Ganglia collectd etc...
8. Muninʢྫʣ
9. ࣾ಺޲͚ʹΧελϚΠζͨ͠Γ͢Δ ίʔυʹखΛೖΕΔͷ͕೉͍͠ɹ ʢ◦◦ݴޠͱ͔͞ΘΕͳ͍ʣɹɹɹɹɹ ʢόʔδϣϯΞοϓʹ௥ै͢Δͷ͕ ೉͍͠ʣ
10. ϑϧεΫϥον? http:/ /anond.hatelabo.jp/20120313004820
11. αʔόϦιʔε؅ཧπʔϧߏ੒ཁૉ Server Server αʔόϦιʔε Λ؅ཧ͢ΔԿ͔ Web UI ϗετ৘ใ؅ཧ ͷͨΊͷDB Server Server ϗετͷIPΞυϨε, MACΞυϨε, ϚγϯεϖοΫ, ϥοΫ൪߸ Graph image ϝτϦΫε
12. UIΛ࢖͍΍͍ͨ͘͢͠ ϗετͷσʔλͷ࣋ͪํΛม͍͑ͨ ͚ͩͳͷʹ ϝτϦΫεͷऔಘɾอଘɾάϥϑԽͱ͔ ͷγεςϜ·Ͱ࡞Δͷ͸ͪΐͬͱ...
13. αʔόϦιʔε؅ཧ͚ͩ΍ͬͯ͘ΕΔɹ ൚༻తͳπʔϧ ͕͋Ε͹͍͍ͷͰ͸
14. Monitorel Proof of Concept
15. Architecture Server Server Server ɾ ɾ ɾ Monitorel Web UI HTTP request Graph image SNMP, HTTP, Telnet... DB (Hosts)
16. Architecture Server Server Server ɾ ɾ ɾ Monitorel API Job Queue Web UI DB (Hosts) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL request enqueue job update metrics Monitorel
17. ࢥ૝ αʔόϦιʔεऔಘɾอଘɾάϥϑԽ͚ͩ΍Δ Web UIͱMonitorel͸HTTPͰ͠Ό΂Δ Web UIͷ࣮૷ݴޠ͸ͳΜͰ΋Α͍ Web UIʹ͸ؔ༩͠ͳ͍ ϗετ৘ใͷ࣋ͪํʹ͸ؔ༩͠ͳ͍ ϗετ৘ใΛ؅ཧ͢ΔDB͸Ϣʔβ͕ͭ͘Δ
18. Architecture Server Server Server ɾ ɾ ɾ Monitorel API Job Queue Web UI DB (Servers) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL enqueue job update metrics Interface
19. Monitorel 
    *OUFSGBDF Web UI - Monitorel API HTTPͰૄ݁߹ʹ͢Δ Applicationͷ࣮૷ݴޠ͸RubyͰ΋ͳΜͰ΋Α͍ cron enqueue - Job Queue DB(Hosts)Ͱ؅ཧ͞Ε͍ͯΔϗετ਺෼ͷɹ δϣϒΛQueueʹ౤͛Δ ఆظతʹcronͰ࣮ߦ HTTPͰδϣϒΛinsertͰ͖ΔΑ͏ʹ͍ͨ͠
20. Architecture Server Server Server ɾ ɾ ɾ Monitorel Web API Job Queue Web UI DB (Hosts) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL enqueue job update metrics
21. Web API άϥϑͷϦΫΤετΛड͚෇͚ͯάϥϑը૾Λฦ͢ ͚ͩ ϦΫΤετύϥϝʔλʢͲͷϗετͷͲͷϝτϦ Ϋε͔, ظؒͳͲʣΛύʔε͢Δ Round Robin Databaseʹ໰͍߹Θͤͯάϥϑը૾ Խ
22. Architecture Server Server Server ɾ ɾ ɾ Web API Job Queue Web UI DB (Servers) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL enqueue job update metrics
23. Job Queue ͳͥJob Queue ͔ ඞͣ͠΋Job QueueͰ͋Δඞཁ͸ͳ͍ αʔόʹର͢ΔωοτϫʔΫI/O଴͕ͪൃੜ͢Δ ͨΊɼඇಉظͰ΍Γ͍ͨ Job QueueΛ࢖ͬͨେྔͷλεΫ؅ཧͷϊ΢ϋ΢ ͕͋Δ ϚϧνϓϩηεͰৗற͢ΔσΟεύονϟͱҰॹʹ ࢖͏ Parallel::PreforkͰ֤ϓϩηε͕ͦΕͧΕJobΛσΟ εύον SFG
    IUUQCMPHQPSHFOUSZ
24. Agent JobͷҰ෦ ϦϞʔταʔό͔ΒϝτϦΫεΛͱͬͯ͘Δ΍ͭ ରԠϛυϧ΢ΣΞɼϓϩτίϧ Apache, Nginx, MySQL, Munin, Latency, Plack, Perlbal, Redis, SNMP, TheSchwartz ϓϥΨϒϧ Monitorel::Worker::Agent::Something SNMP΍HTTP, TelnetͰϝτϦΫεऔಘ
25. Architecture Server Server Server ɾ ɾ ɾ Web API Job Queue Web UI DB (Servers) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL enqueue job update metrics
26. RRDtool
27. RRDTool RRD (Round Robin Database) ࣌ܥྻσʔλΛ֨ೲ͢ΔͨΊͷσʔλϑΥʔϚοτ อଘظؒΛݻఆͯ͠ݹ͍஋͸ࣺ͍ͯͯ͘ RRDtool RRDͷ࣮૷ʢσϑΝΫτελϯμʔυʣ RRDͷ࡞੒ɾߋ৽ɾࢀরػೳͱάϥϑԽػೳΛ΋ͭ I/O͕΍ͨΒ଎͍(Β͍͠) େ఍ͷαʔό؅ཧπʔϧͰ࢖ΘΕ͍ͯΔ
28. CPAN Modules Web API Amon2 RRDTool Wrapper RRDTool::Rawish Job Queue TheSchwartz or Qudo, Parallel::Prefork Agent IO::Socket::INET, Net::SNMP, Net::Telnet
29. Demo
30. TODO RRDtool͔Βͷ୤٫ʢRedisʣ ը૾Ͱ͸ͳ͘ϝτϦΫεΛJSONͰฦͯ͠JS Ͱάϥϑඳը AgentʹJVM, Solr, MongoDB, LVS,..., ͳͲ௥Ճ
31. Monitorel (POC) IUUQTHJUIVCDPNZVVLJ.POJUPSFM