Monitoring Casual in Kyoto / Sensu + Graphite

Monitoring Casual in Kyoto / Sensu + Graphite

Monitoring Casual in Kyoto 11/30

A658ec7f1badf73819dfa501165016c1?s=128

Yuuki Tsubouchi (yuuk1)

November 30, 2013
Tweet

Transcript

  1. Sensu & Graphite Moden Monitoring Tool .POJUPSJOH$BTVBM5BML*O,ZPUP JEZ@VVLJ

  2. id:y_uuki

  3. 2011 / 08 ͸ͯͳΠϯλʔϯ

  4. 2012 / 09 ΞϧόΠτࢯ

  5. 2013 / 07

  6. 2013 / 09

  7. 2013 / 11

  8. 2013 / 12

  9. None
  10. In Kyoto

  11. Monitoring

  12. NFUSJDTDPMMFDUJPO Today NFUSJDTWJTVBMJTBUJPO health check alert notification

  13. Monitoring Tool

  14. Monitoring Tool Requirement ✓ Scalable & Robust ✓ Pluggable ✓

    Programmable ✓ Friendly to immutable infrastructure ✓ Simple 
  15. Case 1

  16. None
  17. ? Scalable & Robust - Munin 2.0 ? ✓ Pluggable

    - munin-node plugin ✗ Programmable - API ͳ͠ ✗ Friendly to immutable infrastructure - ઃఆϑΝΠϧʹ؂ࢹϊʔυͷϦετ͕ඞཁ (serf ͰͳΜͱ͔ͳΔ) ✗ Simple - because of RRDtool
  18. Case 2

  19. Collectd + Cacti + RRDtool

  20. ✗ Scalable & Robust - because of RRDtool (cannot replicate)

    ✓ Pluggable - collectd input / output plugin (like fluentd) ✗ Programmable - Cacti ͸ API ͳ͠ ✗ Friendly to immutable infrastructure - Must configure node list ✗ Simple - because of RRDtool
  21. Case 3

  22. Graduate from

  23. (SBQIJUF Scalable Realtime Graphing

  24. Graphite Data Flow Graphite Webapp Carbon Whisper Collector TCP HTTP

    metrics Graph JSON Store Get Graphite
  25. 3&45"1* - Graph rendering - JSON ! )JHI*OEFQFOEFODZ"SDIJUFDUVSF - Webapp

    Frontend - Storage
  26. Replication / Sharding Carbon Relay Whisper metrics Carbon Whisper Carbon

    Whisper Carbon
  27. Many Better Dashboards Graphite Webapp Carbon Whisper Collector TCP HTTP

    metrics JSON Store Get Graphite Dashboard Graphiti, Gdash, Pencil, Graphene…
  28. 4FOTV A monitoring framework that aims! to be simple, malleable,

    and scalable
  29. .POJUPSJOH3PVUFS - Checks plugin: ex. Apache is up or down

    - Handlers plugin: ex. sending alert e-mails ! )JHI*OEFQFOEFODZ"SDIJUFDUVSF - RabbitMQ - Redis - JSON
  30. Sensu Data Flow sensu- client sensu- client RabbitMQ sensu server

    Redis sensu- client run checks checks run handlers store notification checks check request
  31. None
  32. None
  33. More Detail • https://speakerdeck.com/joemiller/introduction- to-sensu • http://www.slideshare.net/jeremy_carroll/ sensu-14485155 • https://speakerdeck.com/joemiller/practical-

    examples-with-sensu-monitoring-framework
  34. (SBQIJUF 4FOTV

  35. Sensu & Graphite Data Flow sensu- client sensu- client sensu-

    client RabbitMQ sensu server Redis Graphite Webapp Carbon Whisper
  36. ✓ Scalable & Robust - Maybe ✓ Pluggable - sensu

    plugin & sensu handler ✓ Programmable - Graphite: REST API & Carbon TCP API! - Sensu: Events API & Checks API … etc ✓ Friendly to Immutable Infrastructure - ؂ࢹϊʔυͷઃఆ͍Βͣ ✓ Simple - Sensu: Ruby, Redis, RabbitMQ, JSON! - Graphite: REST API
  37. Graphite & RabbitMQ Integration

  38. Sensu & Graphite Data Flow sensu- client sensu- client sensu-

    client RabbitMQ sensu server Redis Graphite Webapp Carbon Whisper
  39. Stack Visualisation Storage Collector Collector Agent .VOJO $BDUJ $PMMFDUE (SBQIJUF

    4FOTV 33%UPPM
  40. Other Solution Storage

  41. OpenTSDB Kairos InfluxDB

  42. Other Solution Collector

  43. fluentd?

  44. Integration with other tools

  45. log health Infrastructure Information metrics Dynamic Static Host, Service, Role,

    Network, Location …
  46. log health Integration Viewer metrics Dynamic Static Integration View Host,

    Service, Role, Network, Location …
  47. log health Host, Service, Role, Network, Location … Integration with

    API metrics Dynamic Static Integration View JSON
  48. Monitoring Tool Requirement ✓ Scalable & Robust ✓ Pluggable ✓

    Programmable ✓ Friendly to immutable infrastructure ✓ Simple 
  49. ҟৗͳ౒ྗ͕ඞཁͳπʔϧʹݻࣥͤͣʹ ےͷྑ͍πʔϧΛબΜͰ͍͖͍ͨ

  50. Graduate from

  51. Sensu & Graphite Moden Monitoring Tool Monitoring Casual Talk In

    Kyoto id:y_uuki
  52. Thanks