Don't let your log go away

Don't let your log go away

Talk given at Paris Tech Talk MeetUp

418997665c4a3368515ecf9c3d746b95?s=128

Olivier Dolbeau

March 24, 2015
Tweet

Transcript

  1. 3.
  2. 9.
  3. 12.

    This is specific to its access logs My roommate uses

    this to colorise his access logs… 12
  4. 13.

    13

  5. 14.

    14

  6. 15.

    15

  7. 16.

    16

  8. 17.

    17

  9. 18.

    Inputs Filters Outputs 41 inputs • syslog • udp •

    varnishlog • gelf • … 50 filters • date • geoip • i18n • urldecode • … 55 outputs • elasticsearch • redis • email • graphite • … And there are also some codecs 18
  10. 19.
  11. 20.

    20

  12. 21.
  13. 22.

    22

  14. 24.

    24

  15. 26.
  16. 27.

    filter { if [type] == "syslog" { grok { match

    => [ "message", "<%{POSINT:syslog_pri}>% {TIMESTAMP_ISO8601:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} % {DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: % {GREEDYDATA:syslog_message}" ] add_field => [ "received_at", "%{@timestamp}" ] add_field => [ "received_from", "%{host}" ] add_tag => [ "rsyslog" ] } } } Logstash - Filter 27
  17. 28.

    output { elasticsearch_http { host => “my_es.blablacar.com” port => 9200

    index => "logstashv1-%{+YYYY.MM.dd}" manage_template => false } } Logstash - Output 28
  18. 29.

    29

  19. 30.
  20. 31.

    31

  21. 32.

    32

  22. 33.