Using Elastic to monitor anything

Short slide deck originally developed for the Berlin DigitalOcean Meetup on Aug 16, 2016

Transcript

1. Using Elastic to monitor anything
   @ChristophWurm
   August 2016
   

   View Slide

2. Unstructured Data Structured Data
   Search Logging Security Metrics
   Analytics
   Social Sensor
   Logs
   Documents User Activity Location
   

   View Slide

3. Elastic Cloud
   Security
   X-Pack
   Kibana
   User Interface
   Elasticsearch
   Store, Index,
   & Analyze
   Ingest
   Logstash Beats
   +
   Elastic Stack
   Alerting
   Monitoring
   Reporting
   Graph
   

   View Slide

4. ”Long” pipeline
   

   View Slide

5. Ingest node
   

   View Slide

6. Filebeat
   Tail files
   Packetbeat
   Capture network packets
   Metricbeat
   System (ex Topbeat)
   Apache
   MySQL
   Nginx
   PostgreSQL
   Redis
   Zookeeper
   Winlogbeat
   Windows event logs
   

   View Slide

7. apachebeat
   httpd server-status
   burrowbeat
   kafka consumer lag
   cassandrabeat
   nodetool cfstats
   cloudtrailbeat
   aws events
   dockerbeat
   container stats
   elasticbeat
   elasticsearch status
   execbeat
   shell cmd output
   flowbeat
   sflow
   factbeat
   puppet facter
   httpbeat
   poll http(s) endpoints
   jmxproxybeat
   tomcat jmx metrics
   lmsensorsbeat
   temp, voltage, fans
   mysqlbeat
   any query -> elasticsearch
   packagebeat
   dpkg, rpm
   pingbeat
   icmp ping w/ rtt
   redisbeat
   redis info
   twitterbeat
   tweets by screen name
   wmibeat
   windows monitoring
   https://www.elastic.co/guide/en/beats/libbeat/master/community-beats.html
   

   View Slide

8. Demo
   Beats & Kibana 5
   Script at https://github.com/cwurm/elastic-monitoring-meetup
   

   View Slide

9. Q&A
   @ChristophWurm
   

   View Slide