Monitoring with Sensu Peter Eisentraut [email protected] @petereisentraut PGConf NYC 2014 

Life with Nagios and Munin • cumbersome to add/change hosts • not built with configuration management in mind • stale projects 

Life with Sensu • hosts register automatically • built with configuration management in mind • active project 

Sensu Architecture 

Installation • use Chef or Puppet • see documentation 

Configuration /etc/sensu/config.json { "rabbitmq": { "host": "sensu.example.net", "port": 5671, "vhost": "/sensu", "user": "sensu", "password": "password", "ssl": { "cert_chain_file": "/etc/sensu/ssl/cert.pem", "private_key_file": "/etc/sensu/ssl/key.pem" } } } 

A Check /etc/sensu/conf.d/checks/check-cpu.json { "checks": { "check-cpu": { "command": "/etc/sensu/community/plugins/system/ check-cpu.rb -w 80 -c 95", "handlers": ["email", "pagerduty"], "subscribers": ["default"], "interval": 60, "occurrences": 10, "refresh": 1800 } } } 

https://github.com/sensu/sensu-community-plugins 

Use Chef sensu_check "check-cpu" do command "/etc/sensu/community/plugins/system/check-cpu .rb -w 80 -c 95" handlers ["email", "pagerduty"] subscribers ["default"] interval 60 additional(occurrences: 10, refresh: 1800) end 

Use Puppet sensu::check { 'check-cpu': command => '/etc/sensu/community/plugins/system/ check-cpu.rb -w 80 -c 95', handlers => ['email', 'pagerduty'], subscribers => ['default'], interval => 60, custom => { 'occurrences' => 10, 'refresh' => 1800, }, } 

A Handler /etc/sensu/conf.d/handlers/email.json { "handlers": { "email": { "type": "pipe", "command": "mail -s 'sensu alert' your@address" } } } 

A Client /etc/sensu/conf.d/client.json { "client": { "name": "server01.example.net", "address": "10.10.10.10", "subscriptions": [ "default" ] } } 

Dashboard 

What about PostgreSQL? 

Use check_postgres! sensu_check "check_postgres_backends" do interval 60 command "check_postgres --action=backends" subscribers ["postgresql"] handlers ['dba'] end 

Standalone Checks 

Standalone Checks sensu_check "check_postgres_backends_5432" do interval 60 command "check_postgres --action=backends --port=5432" standalone true handlers ['dba'] end sensu_check "check_postgres_backends_5433" do interval 60 command "check_postgres --action=backends --port=5433" standalone true handlers ['dba'] end 

Metrics 

Metrics • check: returns quasi-Boolean result • metric: returns numeric result 

A Metric { "checks": { "cpu_metrics": { "command": "/etc/sensu/community/plugins/system/ cpu-pcnt-usage-metrics.rb", "handlers": ["graphite"], "subscribers": ["default"], "type": "metric", "interval": 60 } } } 

What about PostgreSQL? 

Nagios “Performance Data” $ check_postgres --action=backends POSTGRES_BACKENDS OK: DB "postgres" 1 of 100 connections (1%) | time=0.02s commitfest=0;90;95;0;100 contrib_regression=0;90;95;0;100 foo=0;90;95;0;100 postgres=1;90;95;0;100 template0=0;90;95;0;100 template1=0;90;95;0;100 test=0;90;95;0;100 test2 =0;90;95;0;100 

Mutators check (metric) -> mutator -> handler check_postgres -> ??? -> graphite 

write your own 

Idea check_postgres --output=graphite 

Other Features • REST API • port 3030 • sensu-admin • remediation • multimaster server • custom check attributes • severity filtering • check dependencies • flap detection • subduing 

Links • http://sensuapp.org/ • http://sensuapp.org/docs • https://github.com/sensu