Monitoring Casual in Kyoto / Sensu + Graphite

Slide 1

Slide 1 text

Sensu & Graphite Moden Monitoring Tool .POJUPSJOH$BTVBM5BML*O,ZPUP JEZ@VVLJ

Slide 2

Slide 2 text

id:y_uuki

Slide 3

Slide 3 text

2011 / 08 ͸ͯͳΠϯλʔϯ

Slide 4

Slide 4 text

2012 / 09 ΞϧόΠτࢯ

Slide 5

Slide 5 text

2013 / 07

Slide 6

Slide 6 text

2013 / 09

Slide 7

Slide 7 text

2013 / 11

Slide 8

Slide 8 text

2013 / 12

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

In Kyoto

Slide 11

Slide 11 text

Monitoring

Slide 12

Slide 12 text

NFUSJDTDPMMFDUJPO Today NFUSJDTWJTVBMJTBUJPO health check alert notiﬁcation

Slide 13

Slide 13 text

Monitoring Tool

Slide 14

Slide 14 text

Monitoring Tool Requirement ✓ Scalable & Robust ✓ Pluggable ✓ Programmable ✓ Friendly to immutable infrastructure ✓ Simple

Slide 15

Slide 15 text

Case 1

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

? Scalable & Robust - Munin 2.0 ? ✓ Pluggable - munin-node plugin ✗ Programmable - API ͳ͠ ✗ Friendly to immutable infrastructure - ઃఆϑΝΠϧʹ؂ࢹϊʔυͷϦετ͕ඞཁ (serf ͰͳΜͱ͔ͳΔ) ✗ Simple - because of RRDtool

Slide 18

Slide 18 text

Case 2

Slide 19

Slide 19 text

Collectd + Cacti + RRDtool

Slide 20

Slide 20 text

✗ Scalable & Robust - because of RRDtool (cannot replicate) ✓ Pluggable - collectd input / output plugin (like ﬂuentd) ✗ Programmable - Cacti ͸ API ͳ͠ ✗ Friendly to immutable infrastructure - Must conﬁgure node list ✗ Simple - because of RRDtool

Slide 21

Slide 21 text

Case 3

Slide 22

Slide 22 text

Graduate from

Slide 23

Slide 23 text

(SBQIJUF Scalable Realtime Graphing

Slide 24

Slide 24 text

Graphite Data Flow Graphite Webapp Carbon Whisper Collector TCP HTTP metrics Graph JSON Store Get Graphite

Slide 25

Slide 25 text

3&45"1* - Graph rendering - JSON ! )JHI*OEFQFOEFODZ"SDIJUFDUVSF - Webapp Frontend - Storage

Slide 26

Slide 26 text

Replication / Sharding Carbon Relay Whisper metrics Carbon Whisper Carbon Whisper Carbon

Slide 27

Slide 27 text

Many Better Dashboards Graphite Webapp Carbon Whisper Collector TCP HTTP metrics JSON Store Get Graphite Dashboard Graphiti, Gdash, Pencil, Graphene…

Slide 28

Slide 28 text

4FOTV A monitoring framework that aims! to be simple, malleable, and scalable

Slide 29

Slide 29 text

.POJUPSJOH3PVUFS - Checks plugin: ex. Apache is up or down - Handlers plugin: ex. sending alert e-mails ! )JHI*OEFQFOEFODZ"SDIJUFDUVSF - RabbitMQ - Redis - JSON

Slide 30

Slide 30 text

Sensu Data Flow sensu- client sensu- client RabbitMQ sensu server Redis sensu- client run checks checks run handlers store notiﬁcation checks check request

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

More Detail • https://speakerdeck.com/joemiller/introduction- to-sensu • http://www.slideshare.net/jeremy_carroll/ sensu-14485155 • https://speakerdeck.com/joemiller/practical- examples-with-sensu-monitoring-framework

Slide 34

Slide 34 text

(SBQIJUF 4FOTV

Slide 35

Slide 35 text

Sensu & Graphite Data Flow sensu- client sensu- client sensu- client RabbitMQ sensu server Redis Graphite Webapp Carbon Whisper

Slide 36

Slide 36 text

✓ Scalable & Robust - Maybe ✓ Pluggable - sensu plugin & sensu handler ✓ Programmable - Graphite: REST API & Carbon TCP API! - Sensu: Events API & Checks API … etc ✓ Friendly to Immutable Infrastructure - ؂ࢹϊʔυͷઃఆ͍Βͣ ✓ Simple - Sensu: Ruby, Redis, RabbitMQ, JSON! - Graphite: REST API

Slide 37

Slide 37 text

Graphite & RabbitMQ Integration

Slide 38

Slide 38 text

Sensu & Graphite Data Flow sensu- client sensu- client sensu- client RabbitMQ sensu server Redis Graphite Webapp Carbon Whisper

Slide 39

Slide 39 text

Stack Visualisation Storage Collector Collector Agent .VOJO $BDUJ $PMMFDUE (SBQIJUF 4FOTV 33%UPPM

Slide 40

Slide 40 text

Slide 41

Slide 41 text

OpenTSDB Kairos InﬂuxDB

Slide 42

Slide 42 text

Slide 43

Slide 43 text

ﬂuentd?

Slide 44

Slide 44 text

Integration with other tools

Slide 45

Slide 45 text

log health Infrastructure Information metrics Dynamic Static Host, Service, Role, Network, Location …

Slide 46

Slide 46 text

log health Integration Viewer metrics Dynamic Static Integration View Host, Service, Role, Network, Location …

Slide 47

Slide 47 text

log health Host, Service, Role, Network, Location … Integration with API metrics Dynamic Static Integration View JSON

Slide 48

Slide 48 text

Monitoring Tool Requirement ✓ Scalable & Robust ✓ Pluggable ✓ Programmable ✓ Friendly to immutable infrastructure ✓ Simple

Slide 49

Slide 49 text

ҟৗͳ౒ྗ͕ඞཁͳπʔϧʹݻࣥͤͣʹ ےͷྑ͍πʔϧΛબΜͰ͍͖͍ͨ

Slide 50

Slide 50 text

Graduate from

Slide 51

Slide 51 text

Sensu & Graphite Moden Monitoring Tool Monitoring Casual Talk In Kyoto id:y_uuki

Slide 52

Slide 52 text

Thanks