Laisse pas trainer ton log !

Laisse pas trainer ton log !

Given at #forumphp Paris 2014.
Slides are in english, talk was given in french.

Joind.in : https://joind.in/talk/view/11942
Vidéo (in french): https://www.youtube.com/watch?v=1r1SOeaDqH4&list=PL9zDdgiGjkIeeVlrsz9A8o3HtZhvERHT-&index=7

418997665c4a3368515ecf9c3d746b95?s=128

Olivier Dolbeau

October 23, 2014
Tweet

Transcript

  1. 3.

    3

  2. 5.

    –Wikipedia “In computing, a logfile (or simply log) is a

    file that records either the events which happen while an operating system or other software runs, […].” 5
  3. 6.

    –Wikipedia “In computing, a logfile (or simply log) is a

    file that records either the events which happen while an operating system or other software runs, […].” 6
  4. 11.
  5. 12.
  6. 15.

    15

  7. 17.

    This is specific to its access logs My roommate uses

    this to colorise his access logs… 15
  8. 18.

    16

  9. 19.

    17

  10. 20.

    18

  11. 21.

    19

  12. 22.

    20

  13. 23.

    Inputs Filters Outputs 41 inputs • syslog • udp •

    varnishlog • gelf • … 50 filters • date • geoip • i18n • urldecode • … 55 outputs • elasticsearch • redis • email • graphite • … 21
  14. 24.

    Inputs Filters Outputs 41 inputs • syslog • udp •

    varnishlog • gelf • … 50 filters • date • geoip • i18n • urldecode • … 55 outputs • elasticsearch • redis • email • graphite • … And there are also some codecs 21
  15. 25.
  16. 26.

    23

  17. 27.
  18. 28.

    25

  19. 30.

    27

  20. 32.
  21. 33.

    filter { if [type] == "syslog" { grok { match

    => [ "message", "<%{POSINT:syslog_pri}>% {TIMESTAMP_ISO8601:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} % {DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: % {GREEDYDATA:syslog_message}" ] add_field => [ "received_at", "%{@timestamp}" ] add_field => [ "received_from", "%{host}" ] add_tag => [ "rsyslog" ] } } } Logstash - Filter 30
  22. 34.

    output { elasticsearch_http { host => “my_es.blablacar.com” port => 9200

    index => "logstashv1-%{+YYYY.MM.dd}" manage_template => false } } Logstash - Output 31
  23. 36.

    33

  24. 37.

    • StreamHandler • ErrorLogHandler • SwiftMailerHandler • SyslogUdpHandler • FirePHPHandler

    • FingersCrossedHandler • NullHandler • … More than 36 handlers! It’s just some outputs! 34
  25. 39.

    36

  26. 40.

    37

  27. 41.

    38

  28. 44.

    41

  29. 48.

    43

  30. 50.

    45

  31. 51.
  32. 54.
  33. 55.

    Logstash - Filter 47 This space has intentionally been left

    blank. We don’t need any filter Because logstash works well!
  34. 56.

    Logstash - Filter 47 This space has intentionally been left

    blank. We don’t need any filter Because logstash works well! With Heka you need to write a lot of Lua
  35. 58.

    output { elasticsearch_http { host => “my_es.blablacar.com” port => 9200

    index => "logstashv1-%{+YYYY.MM.dd}" manage_template => false } } Logstash - Output 49
  36. 59.

    output { elasticsearch_http { host => “my_es.blablacar.com” port => 9200

    index => "logstashv1-%{+YYYY.MM.dd}" manage_template => false } } Logstash - Output 49 It’s a duplicate slide!
  37. 60.
  38. 61.

    51

  39. 62.

    52

  40. 64.

    54

  41. 65.