Don't let your log go away

Transcript

1. DON’T LET YOUR LOG GO AWAY @odolbeau 1

2. WHO AM I? Olivier Dolbeau @odolbeau Work at BlaBlaCar 2

3. Log 3

4. Which logs are we
 talking about? 4

5. access logs 5

6. syslog syslog 6

7. application logs 7

8. Access 8

9. SSH 9

10. Analyze 10

11. tail grep cat 11

12. This is specific to its access logs My roommate uses

    this to colorise his access logs… 12

13. 13

14. 14

15. 15

16. 16

17. 17

18. Inputs Filters Outputs 41 inputs • syslog • udp •

    varnishlog • gelf • … 50 filters • date • geoip • i18n • urldecode • … 55 outputs • elasticsearch • redis • email • graphite • … And there are also some codecs 18

19. Kibana 19

20. 20

21. ELK 21

22. 22

23. syslog syslog 23

24. 24

25. *.* @127.0.0.1:514;RSYSLOG_ForwardFormat 25

26. input { udp { port => 514 type => syslog

    } } Logstash - Input 26

27. filter { if [type] == "syslog" { grok { match

    => [ "message", "<%{POSINT:syslog_pri}>% {TIMESTAMP_ISO8601:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} % {DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: % {GREEDYDATA:syslog_message}" ] add_field => [ "received_at", "%{@timestamp}" ] add_field => [ "received_from", "%{host}" ] add_tag => [ "rsyslog" ] } } } Logstash - Filter 27

28. output { elasticsearch_http { host => “my_es.blablacar.com” port => 9200

    index => "logstashv1-%{+YYYY.MM.dd}" manage_template => false } } Logstash - Output 28

29. 29

30. syslog 30

31. 31

32. 32

33. None