Collect All the Things with Beats

The ELK stack is widely used to collect logs with Logstash, analyze and search the data with Elasticsearch, and visualize what is happening in your system. Now there is a new platform for shipping many types of operational data: Beats. It can be used to collect, parse, and ship any type of data.

This talk provides an overview of the Beats platform and dives into gathering system metrics, tapping your wire data, and using real-time file analytics. Finally, we take a look at building custom data shippers to get a complete overview of your infrastructure.

Transcript

1. Collect all the things with Philipp Krenn̴̴̴̴̴@xeraa
6. Infrastructure | Developer Advocate
7. Who uses 10+ servers or containers?
8. How are you logging?
9. What is a log?
10. Now what?
11. You Know, for Search
14. ELK Stack
19. Elastic Stack
22. Filebeat
23. Metricbeat
24. Packetbeat
25. Winlogbeat
26. Community Beats
27. Dockbeat Docker daemon monitoring https://github.com/Ingensi/dockbeat
28. Springbeat Spring Boot /metrics & /health https://github.com/consulthys/springbeat
29. Journalbeat systemd / journald https://github.com/mheese/journalbeat
30. Execbeat execbeat: execs: - cron: "@every 10s" command: echo args: "Hello World" fields: host: testhost
31. Nagioscheckbeat input: checks: - name: "disks" cmd: "plugins/check_disk" args: "-w 80 -c 90 -x /dev" period: "1h" - name: "load" cmd: "plugins/check_load" args: "-w 5 -c 10" period: "1m"
32. Your Beat
33. libbeat https://github.com/elastic/beats/tree/master/generate/beat
34. Conclusion
36. Try it https://github.com/xeraa/vagrant-elastic-stack
37. Takk! Questions? Philipp Krenn̴̴̴̴̴@xeraa PS: Stickers & raffle
38. Schnitzel: https://flic.kr/p/9m27wm Architecture: https://flic.kr/p/6dwCAe Conchita: https://flic.kr/p/nBqSHT Elk horn: https://www.theexplora.com/the-irish- elk-megaloceros-giganteus/ Container ship: https://flic.kr/p/hjxW62 https://flic.kr/p/2AzAVJ Wooden logs: https://flic.kr/p/9vvbKE
39. Axe https://flic.kr/p/pBU2VD Tools https://flic.kr/p/5JFmTS Files https://flic.kr/p/2EFcQ Metric https://flic.kr/p/9g5h3f Packages https://flic.kr/p/cJFDLN Windows https://flic.kr/p/94Z6y Library https://flic.kr/p/fiXcBj Beats https://flic.kr/p/bWopW7