Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Monitorel - Web API for Server Resource Graph
Search
Yuuki Tsubouchi (yuuk1)
July 13, 2013
Programming
1
3.9k
Monitorel - Web API for Server Resource Graph
RRDtool in the hell
Yuuki Tsubouchi (yuuk1)
July 13, 2013
Tweet
Share
More Decks by Yuuki Tsubouchi (yuuk1)
See All by Yuuki Tsubouchi (yuuk1)
eBPFを用いたAIネットワーク監視システム論文の実装 / eBPF Japan Meetup #4
yuukit
3
850
クラウドのテレメトリーシステム研究動向2025年
yuukit
3
960
博士論文公聴会: Scaling Telemetry Workloads in Cloud Applications: Techniques for Instrumentation, Storage, and Mining / PhD Defence
yuukit
1
170
博士学位論文予備審査 / Scaling Telemetry Workloads in Cloud Applications: Techniques for Instrumentation, Storage, and Mining
yuukit
1
1.9k
MetricSifter:クラウドアプリケーションにおける故障箇所特定の効率化のための多変量時系列データの特徴量削減 / FIT 2024
yuukit
2
260
工学としてのSRE再訪 / Revisiting SRE as Engineering
yuukit
19
14k
Cloudless Computingの論文紹介
yuukit
2
560
#SRE論文紹介 Detection is Better Than Cure: A Cloud Incidents Perspective V. Ganatra et. al., ESEC/FSE’23
yuukit
3
2.1k
エンジニアのためのSRE論文への招待 / Introduction to SRE Papers for Engineers
yuukit
2
11k
Other Decks in Programming
See All in Programming
NPOでのDevinの活用
codeforeveryone
0
720
Goで作る、開発・CI環境
sin392
0
190
deno-redisの紹介とJSRパッケージの運用について (toranoana.deno #21)
uki00a
0
180
A2A プロトコルを試してみる
azukiazusa1
2
1.3k
スタートアップの急成長を支えるプラットフォームエンジニアリングと組織戦略
sutochin26
0
340
High-Level Programming Languages in AI Era -Human Thought and Mind-
hayat01sh1da
PRO
0
710
イベントストーミング図からコードへの変換手順 / Procedure for Converting Event Storming Diagrams to Code
nrslib
2
590
生成AIコーディングとの向き合い方、AIと共創するという考え方 / How to deal with generative AI coding and the concept of co-creating with AI
seike460
PRO
1
350
PipeCDのプラグイン化で目指すところ
warashi
1
250
GitHub Copilot and GitHub Codespaces Hands-on
ymd65536
2
140
地方に住むエンジニアの残酷な現実とキャリア論
ichimichi
5
1.5k
Composerが「依存解決」のためにどんな工夫をしているか #phpcon
o0h
PRO
1
250
Featured
See All Featured
Done Done
chrislema
184
16k
A designer walks into a library…
pauljervisheath
207
24k
Into the Great Unknown - MozCon
thekraken
39
1.9k
The Cult of Friendly URLs
andyhume
79
6.5k
What's in a price? How to price your products and services
michaelherold
246
12k
Gamification - CAS2011
davidbonilla
81
5.3k
Building Adaptive Systems
keathley
43
2.6k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
138
34k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.8k
YesSQL, Process and Tooling at Scale
rocio
173
14k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.5k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
34
5.9k
Transcript
Monitorel Web API for server resource graph Kyoto.pm 5 Tech
Talk
HatenaJEZ@VVLJ Twitter!Z@VVL CPAN:66,* ͯͳΞϧόΠλʔ !γεςϜϓϥοτϑΥʔϜ෦ ʹ͓͍ͯ৽͘͠ 1FSMΛษڧ͠Α͏ͱ͢Δ ͖ͳϫΧϞϊ
Perl CPAN WebService::Qiita RRDTool::Rawish Plack::Middleware::GitStatus Net::Signalet vim-plugin unite-perl-module.vim perl-local-lib-path.vim
ἤΒΕΔKyoto.pm
ࠓͷ͓ αʔόϦιʔεՄࢹԽ ͭͬͨͭ͘ ( Monitorel ) αʔόϦιʔεΛάϥϑԽ͢ΔͨΊͷAPI
αʔόϦιʔε ҎԼͷΑ͏ʹ࣌ؒมԽ͢ΔͷΛɹɹɹɹɹ αʔόϦιʔεͱ͢Δ OS͔ΒͱΕΔͭ(load averageͱ͔CPU usage) ϛυϧΣΞ͔ΒͱΕΔͭ( MySQLͩͱ Connections, Qcache_hits,
Slow queries) 1ݸ1ݸͷΛϝτϦΫεͱ͢Δ ࣌ؒมԽ͠ͳ͍ͷΛϗετใͱ͢Δ IPΞυϨε, MACΞυϨε, ϚγϯεϖοΫͳͲ
αʔόϦιʔεՄࢹԽ Munin Cacti Zabbix Ganglia collectd etc...
Muninʢྫʣ
͚ࣾʹΧελϚΠζͨ͠Γ͢Δ ίʔυʹखΛೖΕΔͷ͕͍͠ɹ ʢ◦◦ݴޠͱ͔͞ΘΕͳ͍ʣɹɹɹɹɹ ʢόʔδϣϯΞοϓʹै͢Δͷ͕ ͍͠ʣ
ϑϧεΫϥον? http:/ /anond.hatelabo.jp/20120313004820
αʔόϦιʔεཧπʔϧߏཁૉ Server Server αʔόϦιʔε Λཧ͢ΔԿ͔ Web UI ϗετใཧ ͷͨΊͷDB Server
Server ϗετͷIPΞυϨε, MACΞυϨε, ϚγϯεϖοΫ, ϥοΫ൪߸ Graph image ϝτϦΫε
UIΛ͍͍ͨ͘͢͠ ϗετͷσʔλͷ࣋ͪํΛม͍͑ͨ ͚ͩͳͷʹ ϝτϦΫεͷऔಘɾอଘɾάϥϑԽͱ͔ ͷγεςϜ·Ͱ࡞Δͷͪΐͬͱ...
αʔόϦιʔεཧ͚ͩͬͯ͘ΕΔɹ ൚༻తͳπʔϧ ͕͋Ε͍͍ͷͰ
Monitorel Proof of Concept
Architecture Server Server Server ɾ ɾ ɾ Monitorel Web UI
HTTP request Graph image SNMP, HTTP, Telnet... DB (Hosts)
Architecture Server Server Server ɾ ɾ ɾ Monitorel API Job
Queue Web UI DB (Hosts) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL request enqueue job update metrics Monitorel
ࢥ αʔόϦιʔεऔಘɾอଘɾάϥϑԽ͚ͩΔ Web UIͱMonitorelHTTPͰ͠ΌΔ Web UIͷ࣮ݴޠͳΜͰΑ͍ Web UIʹؔ༩͠ͳ͍ ϗετใͷ࣋ͪํʹؔ༩͠ͳ͍ ϗετใΛཧ͢ΔDBϢʔβ͕ͭ͘Δ
Architecture Server Server Server ɾ ɾ ɾ Monitorel API Job
Queue Web UI DB (Servers) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL enqueue job update metrics Interface
Monitorel *OUFSGBDF Web UI - Monitorel API HTTPͰૄ݁߹ʹ͢Δ Applicationͷ࣮ݴޠRubyͰͳΜͰΑ͍ cron
enqueue - Job Queue DB(Hosts)Ͱཧ͞Ε͍ͯΔϗετͷɹ δϣϒΛQueueʹ͛Δ ఆظతʹcronͰ࣮ߦ HTTPͰδϣϒΛinsertͰ͖ΔΑ͏ʹ͍ͨ͠
Architecture Server Server Server ɾ ɾ ɾ Monitorel Web API
Job Queue Web UI DB (Hosts) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL enqueue job update metrics
Web API άϥϑͷϦΫΤετΛड͚͚ͯάϥϑը૾Λฦ͢ ͚ͩ ϦΫΤετύϥϝʔλʢͲͷϗετͷͲͷϝτϦ Ϋε͔, ظؒͳͲʣΛύʔε͢Δ Round Robin Databaseʹ͍߹Θͤͯάϥϑը૾
Խ
Architecture Server Server Server ɾ ɾ ɾ Web API Job
Queue Web UI DB (Servers) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL enqueue job update metrics
Job Queue ͳͥJob Queue ͔ ඞͣ͠Job QueueͰ͋Δඞཁͳ͍ αʔόʹର͢ΔωοτϫʔΫI/O͕ͪൃੜ͢Δ ͨΊɼඇಉظͰΓ͍ͨ Job
QueueΛͬͨେྔͷλεΫཧͷϊϋ ͕͋Δ ϚϧνϓϩηεͰৗற͢ΔσΟεύονϟͱҰॹʹ ͏ Parallel::PreforkͰ֤ϓϩηε͕ͦΕͧΕJobΛσΟ εύον SFGIUUQCMPHQPSHFOUSZ
Agent JobͷҰ෦ ϦϞʔταʔό͔ΒϝτϦΫεΛͱͬͯ͘Δͭ ରԠϛυϧΣΞɼϓϩτίϧ Apache, Nginx, MySQL, Munin, Latency, Plack,
Perlbal, Redis, SNMP, TheSchwartz ϓϥΨϒϧ Monitorel::Worker::Agent::Something SNMPHTTP, TelnetͰϝτϦΫεऔಘ
Architecture Server Server Server ɾ ɾ ɾ Web API Job
Queue Web UI DB (Servers) HTTP request Graph image Round Robin Database Graph image SNMP, HTTP, Telnet... rrd graph cron enqueue hosts list HTTP or SQL enqueue job update metrics
RRDtool
RRDTool RRD (Round Robin Database) ࣌ܥྻσʔλΛ֨ೲ͢ΔͨΊͷσʔλϑΥʔϚοτ อଘظؒΛݻఆͯ͠ݹ͍ࣺ͍ͯͯ͘ RRDtool RRDͷ࣮ʢσϑΝΫτελϯμʔυʣ RRDͷ࡞ɾߋ৽ɾࢀরػೳͱάϥϑԽػೳΛͭ
I/O͕ͨΒ͍(Β͍͠) େͷαʔόཧπʔϧͰΘΕ͍ͯΔ
CPAN Modules Web API Amon2 RRDTool Wrapper RRDTool::Rawish Job Queue
TheSchwartz or Qudo, Parallel::Prefork Agent IO::Socket::INET, Net::SNMP, Net::Telnet
Demo
TODO RRDtool͔Βͷ٫ʢRedisʣ ը૾Ͱͳ͘ϝτϦΫεΛJSONͰฦͯ͠JS Ͱάϥϑඳը AgentʹJVM, Solr, MongoDB, LVS,..., ͳͲՃ
Monitorel (POC) IUUQTHJUIVCDPNZVVLJ.POJUPSFM