Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Centralized logging

fraosug
March 17, 2013

Centralized logging

Vorrag von Jan-Piet Mens

fraosug

March 17, 2013
Tweet

More Decks by fraosug

Other Decks in Technology

Transcript

  1. some logs 179.44.34.142 - - [13/Sep/2012:02:32:49 -0400]"GET / files/logstash/logstash-1.1.0-monolithic.jar HTTP/1.1"

    200 40923996 "-" "Chef Client/0.10.10" 11-Nov-2012 09:00:33.604 transfer of 'ww.mens.de/IN' from 192.168.1.20#53: Transfer completed: 1 messages, 6 records, 320 bytes, 0.091 secs (3516 bytes/sec) Oct 22 14:54:22 hippo slapd[6829]: conn=1011 op=252 MOD attr=krbLastSuccessfulAuth krbExtraData
  2. what time is it? 1304060505 29/Apr/2011:07:05:26 +0000 Fri, 21 Nov

    1997 09:55:06 -0600 Oct 11 20:21:47 020805 13:51:24 110429.071055,118 @4000000037c219bf2ef02e94
  3. configuration input { file { type => "apache" path =>

    [ "/var/log/apache.log" ] } } filter { grok { type => "apache" pattern => "%{COMBINEDAPACHELOG}" } } output { elasticsearch { host => 'localhost' } }
  4. logstash inputs amqp, eventlog, exec, file, ganglia, gelf, gemfire, generator,

    heroku, irc, log4j, lumberjack, pipe, redis, relp, sqs, stdin, stomp, syslog, tcp, twitter, udp, xmpp, zenoss, zeromq
  5. logstash filters alter, checksum, csv, date, dns, environment, gelfify, geoip,

    grep, grok, grokdiscovery, json, kv, multiline, mutate, noop, split, syslog_pri, urldecode, xml, zeromq
  6. logstash outputs amqp, boundary, circonus, datadog, elasticsearch, elasticsearch_http, elasticsearch_river, email,

    exec, file, ganglia, gelf, gemfire, graphite, graphtastic, http, internal, irc, juggernaut, librato, loggly, lumberjack, metriccatcher, mongodb, nagios, nagios_nsca, null, opentsdb, pagerduty, pipe, redis, riak, riemann, sns, sqs, statsd, stdout, stomp, tcp, websocket, xmpp, zabbix, zeromq
  7. log shipping • syslog-ng • rsyslog • Redis / 0mq

    • lumberjack • Beaver • ...