Handling Emergency Response with Logstash

I shared a use-case of Logstash, using it to process logs and redirect to a destination based on a response status. Apache Logs were processed and the output sent to PageDuty for 5xx errors and Slack for 4xx errors.

Presented at Elastic Meetup Lagos https://www.meetup.com/Nigeria-Elastic-Fantastics/events/252393468/

Transcript

1. Handling Emergency Response with Logstash Abubakar Siddiq Ango @sarki247
2. About Me Social Entrepreneur, building the community in Bauchi, Nigeria. Support Engineer @ GitLab (We’re hiring!!!, visit https://about.gitlab.com/jobs) I do DevOps, CI/CD, Kubernetes & Cloud native You can find my at https://abuango.me & twitter.com/sarki247
3. Oh No!!!
4. Logs come to the rescue ...but seriously, all those lines!
5. ELK stack is your new bff
6. logstash
7. Components of logstash
8. Using Logstash - Download or Install Logstash - Download Binary files - Install OS Packages - Create a config file - Setup plugins - Setup necessary services
9. Config file # This is a comment. You should use comments to describe # parts of your configuration. input { ... } filter { ... } output { ... }
10. Demo - Parse Apache Access Logs - Send slack notification for 4xx responses - Trigger PagerDuty for 5xx responses
11. Demo - input input { file { path => "/var/log/apache2/access.log" type => "logs" } }
12. DEMO - FILTER filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}" } } }
13. DEMO - OUTPUT output { stdout { codec => json } if [response] =~ /^5\d\d/ { pagerduty { event_type => "trigger" description => "%{host} - Internal Server Error - %{response}" details => { timestamp => "%{timestamp}" message => "%{message}" } service_key => "721f938726c64-xxxxxxxx" incident_key => "logstash/%{host}/%{type}" } } else if [response] =~ /^4\d\d/ { slack { url => "https://hooks.slack.com/services/T9A5C-xxxx/BC8TQ-xxxxx/M6ByQuyOwt-xxxxxxxxxx" username => "abuango" channel => "abuango" format => "400 - File not found Error ==> %{message}" } } }
14. Questions?
15. Thank you :)