Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Metrics Driven Monitoring with Sensu/Graphite/G...
Search
Jaime Gago
February 09, 2015
Technology
0
1.1k
Metrics Driven Monitoring with Sensu/Graphite/Grafana
Metrics Driven Monitoring with Sensu/Graphite/Grafana
Jaime Gago
February 09, 2015
Tweet
Share
More Decks by Jaime Gago
See All by Jaime Gago
Automating an AWS Complex Infrastructure With Ansible: Lessons learned
jaimegago
1
270
Other Decks in Technology
See All in Technology
LLMを搭載したプロダクトの品質保証の模索と学び
qa
1
1.1k
初めてAWSを使うときのセキュリティ覚書〜初心者支部編〜
cmusudakeisuke
1
280
KotlinConf 2025_イベントレポート
sony
1
140
Snowflake Intelligenceにはこうやって立ち向かう!クラシルが考えるAI Readyなデータ基盤と活用のためのDataOps
gappy50
0
280
AIエージェントで90秒の広告動画を制作!台本・音声・映像・編集をつなぐAWS最新アーキテクチャの実践
nasuvitz
3
360
Evolución del razonamiento matemático de GPT-4.1 a GPT-5 - Data Aventura Summit 2025 & VSCode DevDays
lauchacarro
0
210
Snowflake×dbtを用いたテレシーのデータ基盤のこれまでとこれから
sagara
0
130
株式会社ログラス - 会社説明資料【エンジニア】/ Loglass Engineer
loglass2019
4
65k
「その開発、認知負荷高すぎませんか?」Platform Engineeringで始める開発者体験カイゼン術
sansantech
PRO
2
880
現場で効くClaude Code ─ 最新動向と企業導入
takaakikakei
1
260
テストを軸にした生き残り術
kworkdev
PRO
0
220
Snowflake Intelligence × Document AIで“使いにくいデータ”を“使えるデータ”に
kevinrobot34
1
120
Featured
See All Featured
The World Runs on Bad Software
bkeepers
PRO
70
11k
Typedesign – Prime Four
hannesfritz
42
2.8k
Writing Fast Ruby
sferik
628
62k
How to Ace a Technical Interview
jacobian
279
23k
Intergalactic Javascript Robots from Outer Space
tanoku
272
27k
KATA
mclloyd
32
14k
Imperfection Machines: The Place of Print at Facebook
scottboms
268
13k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Music & Morning Musume
bryan
46
6.8k
Building a Modern Day E-commerce SEO Strategy
aleyda
43
7.6k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Transcript
Metrics Driven Monitoring with Sensu/Graphite/Grafana @JaimeGagoTech
VS 10/7/14, 5:14 PM ,000,000 ,000,000 ,000,000 ,000,000 ,000,000 ,000,000
,000,000 ,000,000 ,000,000 ,000,000 ,000,000 ,000,000 ,000,000 ,000,000 ,000,000 ,000,000 ,000,000 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940
GRAPHITE METRIC or EVENT SOURCE FOO METRIC or EVENT SOURCE
BAR Render URL API Render URL API Sensu Graphite Grafana 30000ft Overview TCP Socket TCP Socket STDOUT ALERT! RESTART SERVICE FOO
http://grafana.org • FOSS • Kibana Fork • Client Only •
Fast Growing community
None
Getting Data From Graphite http://my.graphite.org/render?target=absolute(some.metric*.memory.memory-free)&format=json&from=-20min { { { { Graphite
Host Metric Target Accepts: -Graphite functions -Globs Output Period for query JSON PNG CSV
Getting Metrics to Graphite Plain text protocol 'root.foo.metric 42 1412728523'
STDOUT TCP SOCKET Python snippet message = 'root.foo.metric 42 %d\n' % int(time.time())' carbon_server = 'graphite.ops.bluejeans.com' carbon_port = 2003 sock = socket.socket() sock.connect((carbon_server, carbon_port) sock.sendall(message) sock.close() GRAPHITE Whisper
Getting Events to Graphite Json Blob {"what": "Event - deploy",
"tags": “deploy", "data": "deploy of master branch happened at Fri Jan 3 22:34:41 UTC 2014"}' STDOUT Http post http://graphite/events/ Python Snippet graphite_event_tags = 'sensu-mco-restart %s %s' % (init_script, " ".join(restarted_hosts)) graphite_event_data = ('The Sensu-Mcollective remediation pipeline ' 'send restart to %s on : %s') % (init_script, ",".join(restarted_hosts)) graphite_event_payload = {'what': 'Sensu-MCollective-Restart', 'tags': graphite_event_tags, 'data': graphite_event_data} try: if options.debug: print "Sent graphite event" print 'graphite payload: %s' % (graphite_event_payload) request = urllib2.Request(graphite_events_url, json.dumps(graphite_event_payload)) response = urllib2.urlopen(request) except Exception as e: if options.debug: print 'graphite payload: %s' % (graphite_event_payload) print response print e logger.critical(e) GRAPHITE MySQL
Getting Events from Graphite Http GET GRAPHITE MySQL http://graphite/events/get_data [
{ "data": "deploy happened", "id": 6, "tags": "tag1 ", "what": "grafana demo", "when": 1412974823.0 }, { "data": "deploy happened", "id": 7, "tags": "tag1 ", "what": "grafana demo", "when": 1412976219.0 }, { "data": "Foo happened", "id": 8, "tags": "foo ", "what": "grafana", "when": 1412977300.0 } … ]
Sensu API /aggregates /checks /clients /events /health /info /stashes
Config Dive In
Self Healing Example
Scaling
Self Service Monitoring