Handling Emergency Response with Logstash Abubakar Siddiq Ango @sarki247 

About Me Social Entrepreneur, building the community in Bauchi, Nigeria. Support Engineer @ GitLab (We’re hiring!!!, visit https://about.gitlab.com/jobs) I do DevOps, CI/CD, Kubernetes & Cloud native You can find my at https://abuango.me & twitter.com/sarki247 

Oh No!!! 

Logs come to the rescue ...but seriously, all those lines! 

ELK stack is your new bff 

logstash 

Components of logstash 

Using Logstash - Download or Install Logstash - Download Binary files - Install OS Packages - Create a config file - Setup plugins - Setup necessary services 

Config file # This is a comment. You should use comments to describe # parts of your configuration. input { ... } filter { ... } output { ... } 

Demo - Parse Apache Access Logs - Send slack notification for 4xx responses - Trigger PagerDuty for 5xx responses 

Demo - input input { file { path => "/var/log/apache2/access.log" type => "logs" } } 

DEMO - FILTER filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}" } } } 

DEMO - OUTPUT output { stdout { codec => json } if [response] =~ /^5\d\d/ { pagerduty { event_type => "trigger" description => "%{host} - Internal Server Error - %{response}" details => { timestamp => "%{timestamp}" message => "%{message}" } service_key => "721f938726c64-xxxxxxxx" incident_key => "logstash/%{host}/%{type}" } } else if [response] =~ /^4\d\d/ { slack { url => "https://hooks.slack.com/services/T9A5C-xxxx/BC8TQ-xxxxx/M6ByQuyOwt-xxxxxxxxxx" username => "abuango" channel => "abuango" format => "400 - File not found Error ==> %{message}" } } } 

Questions? 

Thank you :)