Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Monitorel - Web API for Server Resource Graph
Search
Yuuki Tsubouchi (yuuk1)
July 13, 2013
Programming
1
3.9k
Monitorel - Web API for Server Resource Graph
RRDtool in the hell
Yuuki Tsubouchi (yuuk1)
July 13, 2013
Tweet
Share
More Decks by Yuuki Tsubouchi (yuuk1)
See All by Yuuki Tsubouchi (yuuk1)
eBPFを用いたAIネットワーク監視システム論文の実装 / eBPF Japan Meetup #4
yuukit
3
850
クラウドのテレメトリーシステム研究動向2025年
yuukit
3
960
博士論文公聴会: Scaling Telemetry Workloads in Cloud Applications: Techniques for Instrumentation, Storage, and Mining / PhD Defence
yuukit
1
180
博士学位論文予備審査 / Scaling Telemetry Workloads in Cloud Applications: Techniques for Instrumentation, Storage, and Mining
yuukit
1
1.9k
MetricSifter:クラウドアプリケーションにおける故障箇所特定の効率化のための多変量時系列データの特徴量削減 / FIT 2024
yuukit
2
260
工学としてのSRE再訪 / Revisiting SRE as Engineering
yuukit
19
14k
Cloudless Computingの論文紹介
yuukit
2
560
#SRE論文紹介 Detection is Better Than Cure: A Cloud Incidents Perspective V. Ganatra et. al., ESEC/FSE’23
yuukit
3
2.1k
エンジニアのためのSRE論文への招待 / Introduction to SRE Papers for Engineers
yuukit
2
11k
Other Decks in Programming
See All in Programming
『自分のデータだけ見せたい!』を叶える──Laravel × Casbin で複雑権限をスッキリ解きほぐす 25 分
akitotsukahara
2
620
RailsGirls IZUMO スポンサーLT
16bitidol
0
170
PostgreSQLのRow Level SecurityをPHPのORMで扱う Eloquent vs Doctrine #phpcon #track2
77web
2
500
Kotlin エンジニアへ送る:Swift 案件に参加させられる日に備えて~似てるけど色々違う Swift の仕様 / from Kotlin to Swift
lovee
1
260
地方に住むエンジニアの残酷な現実とキャリア論
ichimichi
5
1.5k
Systèmes distribués, pour le meilleur et pour le pire - BreizhCamp 2025 - Conférence
slecache
0
120
ふつうの技術スタックでアート作品を作ってみる
akira888
0
460
WebViewの現在地 - SwiftUI時代のWebKit - / The Current State Of WebView
marcy731
0
110
C++20 射影変換
faithandbrave
0
570
来たるべき 8.0 に備えて React 19 新機能と React Router 固有機能の取捨選択とすり合わせを考える
oukayuka
2
910
イベントストーミング図からコードへの変換手順 / Procedure for Converting Event Storming Diagrams to Code
nrslib
2
640
20250628_非エンジニアがバイブコーディングしてみた
ponponmikankan
0
660
Featured
See All Featured
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
5
260
Testing 201, or: Great Expectations
jmmastey
42
7.6k
How to train your dragon (web standard)
notwaldorf
94
6.1k
Building Adaptive Systems
keathley
43
2.6k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
Build The Right Thing And Hit Your Dates
maggiecrowley
36
2.8k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.9k
Writing Fast Ruby
sferik
628
62k
The Straight Up "How To Draw Better" Workshop
denniskardys
234
140k
The Art of Programming - Codeland 2020
erikaheidi
54
13k
RailsConf 2023
tenderlove
30
1.1k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
46
9.6k
Transcript
Monitorel Web API for server resource graph Kyoto.pm 5 Tech
Talk
HatenaJEZ@VVLJ Twitter!Z@VVL CPAN:66,* ͯͳΞϧόΠλʔ !γεςϜϓϥοτϑΥʔϜ෦ ʹ͓͍ͯ৽͘͠ 1FSMΛษڧ͠Α͏ͱ͢Δ ͖ͳϫΧϞϊ
Perl CPAN WebService::Qiita RRDTool::Rawish Plack::Middleware::GitStatus Net::Signalet vim-plugin unite-perl-module.vim perl-local-lib-path.vim
ἤΒΕΔKyoto.pm
ࠓͷ͓ αʔόϦιʔεՄࢹԽ ͭͬͨͭ͘ ( Monitorel ) αʔόϦιʔεΛάϥϑԽ͢ΔͨΊͷAPI
αʔόϦιʔε ҎԼͷΑ͏ʹ࣌ؒมԽ͢ΔͷΛɹɹɹɹɹ αʔόϦιʔεͱ͢Δ OS͔ΒͱΕΔͭ(load averageͱ͔CPU usage) ϛυϧΣΞ͔ΒͱΕΔͭ( MySQLͩͱ Connections, Qcache_hits,
Slow queries) 1ݸ1ݸͷΛϝτϦΫεͱ͢Δ ࣌ؒมԽ͠ͳ͍ͷΛϗετใͱ͢Δ IPΞυϨε, MACΞυϨε, ϚγϯεϖοΫͳͲ
αʔόϦιʔεՄࢹԽ Munin Cacti Zabbix Ganglia collectd etc...
Muninʢྫʣ
͚ࣾʹΧελϚΠζͨ͠Γ͢Δ ίʔυʹखΛೖΕΔͷ͕͍͠ɹ ʢ◦◦ݴޠͱ͔͞ΘΕͳ͍ʣɹɹɹɹɹ ʢόʔδϣϯΞοϓʹै͢Δͷ͕ ͍͠ʣ
ϑϧεΫϥον? http:/ /anond.hatelabo.jp/20120313004820
αʔόϦιʔεཧπʔϧߏཁૉ Server Server αʔόϦιʔε Λཧ͢ΔԿ͔ Web UI ϗετใཧ ͷͨΊͷDB Server
Server ϗετͷIPΞυϨε, MACΞυϨε, ϚγϯεϖοΫ, ϥοΫ൪߸ Graph image ϝτϦΫε
UIΛ͍͍ͨ͘͢͠ ϗετͷσʔλͷ࣋ͪํΛม͍͑ͨ ͚ͩͳͷʹ ϝτϦΫεͷऔಘɾอଘɾάϥϑԽͱ͔ ͷγεςϜ·Ͱ࡞Δͷͪΐͬͱ...
αʔόϦιʔεཧ͚ͩͬͯ͘ΕΔɹ ൚༻తͳπʔϧ ͕͋Ε͍͍ͷͰ
Monitorel Proof of Concept
Architecture Server Server Server ɾ ɾ ɾ Monitorel Web UI
HTTP request Graph image SNMP, HTTP, Telnet... DB (Hosts)
Architecture Server Server Server ɾ ɾ ɾ Monitorel API Job
Queue Web UI DB (Hosts) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL request enqueue job update metrics Monitorel
ࢥ αʔόϦιʔεऔಘɾอଘɾάϥϑԽ͚ͩΔ Web UIͱMonitorelHTTPͰ͠ΌΔ Web UIͷ࣮ݴޠͳΜͰΑ͍ Web UIʹؔ༩͠ͳ͍ ϗετใͷ࣋ͪํʹؔ༩͠ͳ͍ ϗετใΛཧ͢ΔDBϢʔβ͕ͭ͘Δ
Architecture Server Server Server ɾ ɾ ɾ Monitorel API Job
Queue Web UI DB (Servers) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL enqueue job update metrics Interface
Monitorel *OUFSGBDF Web UI - Monitorel API HTTPͰૄ݁߹ʹ͢Δ Applicationͷ࣮ݴޠRubyͰͳΜͰΑ͍ cron
enqueue - Job Queue DB(Hosts)Ͱཧ͞Ε͍ͯΔϗετͷɹ δϣϒΛQueueʹ͛Δ ఆظతʹcronͰ࣮ߦ HTTPͰδϣϒΛinsertͰ͖ΔΑ͏ʹ͍ͨ͠
Architecture Server Server Server ɾ ɾ ɾ Monitorel Web API
Job Queue Web UI DB (Hosts) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL enqueue job update metrics
Web API άϥϑͷϦΫΤετΛड͚͚ͯάϥϑը૾Λฦ͢ ͚ͩ ϦΫΤετύϥϝʔλʢͲͷϗετͷͲͷϝτϦ Ϋε͔, ظؒͳͲʣΛύʔε͢Δ Round Robin Databaseʹ͍߹Θͤͯάϥϑը૾
Խ
Architecture Server Server Server ɾ ɾ ɾ Web API Job
Queue Web UI DB (Servers) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL enqueue job update metrics
Job Queue ͳͥJob Queue ͔ ඞͣ͠Job QueueͰ͋Δඞཁͳ͍ αʔόʹର͢ΔωοτϫʔΫI/O͕ͪൃੜ͢Δ ͨΊɼඇಉظͰΓ͍ͨ Job
QueueΛͬͨେྔͷλεΫཧͷϊϋ ͕͋Δ ϚϧνϓϩηεͰৗற͢ΔσΟεύονϟͱҰॹʹ ͏ Parallel::PreforkͰ֤ϓϩηε͕ͦΕͧΕJobΛσΟ εύον SFGIUUQCMPHQPSHFOUSZ
Agent JobͷҰ෦ ϦϞʔταʔό͔ΒϝτϦΫεΛͱͬͯ͘Δͭ ରԠϛυϧΣΞɼϓϩτίϧ Apache, Nginx, MySQL, Munin, Latency, Plack,
Perlbal, Redis, SNMP, TheSchwartz ϓϥΨϒϧ Monitorel::Worker::Agent::Something SNMPHTTP, TelnetͰϝτϦΫεऔಘ
Architecture Server Server Server ɾ ɾ ɾ Web API Job
Queue Web UI DB (Servers) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL enqueue job update metrics
RRDtool
RRDTool RRD (Round Robin Database) ࣌ܥྻσʔλΛ֨ೲ͢ΔͨΊͷσʔλϑΥʔϚοτ อଘظؒΛݻఆͯ͠ݹ͍ࣺ͍ͯͯ͘ RRDtool RRDͷ࣮ʢσϑΝΫτελϯμʔυʣ RRDͷ࡞ɾߋ৽ɾࢀরػೳͱάϥϑԽػೳΛͭ
I/O͕ͨΒ͍(Β͍͠) େͷαʔόཧπʔϧͰΘΕ͍ͯΔ
CPAN Modules Web API Amon2 RRDTool Wrapper RRDTool::Rawish Job Queue
TheSchwartz or Qudo, Parallel::Prefork Agent IO::Socket::INET, Net::SNMP, Net::Telnet
Demo
TODO RRDtool͔Βͷ٫ʢRedisʣ ը૾Ͱͳ͘ϝτϦΫεΛJSONͰฦͯ͠JS Ͱάϥϑඳը AgentʹJVM, Solr, MongoDB, LVS,..., ͳͲՃ
Monitorel (POC) IUUQTHJUIVCDPNZVVLJ.POJUPSFM