Don't let your log go away

Talk given at Paris Tech Talk MeetUp

Transcript

1. DON’T LET YOUR LOG GO AWAY @odolbeau 1
2. WHO AM I? Olivier Dolbeau @odolbeau Work at BlaBlaCar 2
3. Log 3
4. Which logs are we
 talking about? 4
5. access logs 5
6. syslog syslog 6
7. application logs 7
8. Access 8
9. SSH 9
10. Analyze 10
11. tail grep cat 11
12. This is specific to its access logs My roommate uses this to colorise his access logs… 12
18. Inputs Filters Outputs 41 inputs • syslog • udp • varnishlog • gelf • … 50 filters • date • geoip • i18n • urldecode • … 55 outputs • elasticsearch • redis • email • graphite • … And there are also some codecs 18
19. Kibana 19
21. ELK 21
23. syslog syslog 23
25. *.* @127.0.0.1:514;RSYSLOG_ForwardFormat 25
26. input { udp { port => 514 type => syslog } } Logstash - Input 26
27. filter { if [type] == "syslog" { grok { match => [ "message", "<%{POSINT:syslog_pri}>% {TIMESTAMP_ISO8601:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} % {DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: % {GREEDYDATA:syslog_message}" ] add_field => [ "received_at", "%{@timestamp}" ] add_field => [ "received_from", "%{host}" ] add_tag => [ "rsyslog" ] } } } Logstash - Filter 27
28. output { elasticsearch_http { host => “my_es.blablacar.com” port => 9200 index => "logstashv1-%{+YYYY.MM.dd}" manage_template => false } } Logstash - Output 28
30. syslog 30