Upgrade to Pro — share decks privately, control downloads, hide ads and more …

StatsD Workshop Monitorama 2013

StatsD Workshop Monitorama 2013

Daniel Schauenberg

March 29, 2013
Tweet

More Decks by Daniel Schauenberg

Other Decks in Technology

Transcript

  1. • What is this StatsD? • Metrics types • Backends

    • Graphite and operating StatsD • Send all the metrics • Write all the backends Friday, March 29, 13
  2. Pluggable backends function MyBackend(startupTime, config, emitter){ var self = this;

    this.lastFlush = startupTime; this.lastException = startupTime; this.config = config.mybackendconfig || {}; // what to do with metrics emitter.on('flush', function(timestamp, metrics) {}); // update your status emitter.on('status', function(callback) {}); // handle raw packet emitter.on('packet', function(packet, rinfo) {}); }; Friday, March 29, 13
  3. Management Interface > telnet 127.0.0.1 8126 > stats > counters

    > delcounters > help Friday, March 29, 13
  4. Graphite + StatsD • default backend • whisper archives should

    match flush interval • sum aggregation for raw counters • docs/graphite.md Friday, March 29, 13
  5. Tuning UDP # increase default core memory sizes net.core.rmem_default =

    16777216 net.core.wmem_default = 16777216 net.ipv4.udp_wmem_min = 67108864 net.ipv4.udp_rmem_min = 67108864 net.ipv4.udp_mem = 4648512 6198016 9297024 Friday, March 29, 13
  6. Sampling • default sampling of 0.1 on all metrics •

    most sent keys (log file) are sampled 0.01 Friday, March 29, 13
  7. Basic setup • git clone https://github.com/etsy/statsd.git • curl https://gist.github.com/mrtazz/5270376 >

    test_config.js • nc -k -l 2003 > output.log • node stats.js test_config.js Friday, March 29, 13
  8. Send some stats > echo -n "foo.bar:1|c" | nc -w0

    -u 127.0.0.1 8125 > echo -n "foo.lerl:3|g" | nc -w0 -u 127.0.0.1 8125 > echo -n "foo.lol:33|ms" | nc -w0 -u 127.0.0.1 8125 > echo -n "foo.lol:43|ms" | nc -w0 -u 127.0.0.1 8125 > echo -n "foo.lol:53|ms" | nc -w0 -u 127.0.0.1 8125 Friday, March 29, 13
  9. Some actual stats Disk space: df -m | grep "/var"

    | awk '{print "diskspace.var:"$4"|g"}' Apache HTTPD processes: echo -n "httpd.procs:$(ps aux | grep [h]ttpd | wc -l)|g" Logfile size: du -s /var/log/chef.log | awk '{print "cheflog:"$1"|g"}' Friday, March 29, 13
  10. Is this real life? • Set up a web page

    • Have it send stats to StatsD • page load time • counters when page was accessed • look in the examples/ directory for code Friday, March 29, 13
  11. Histograms • You should have timers now • Configure histograms

    for them • { graphite: { histograms: [] } Friday, March 29, 13
  12. Write a new backend • Have it HTTP POST to

    a web server • write metrics to a file • send a txt message • repeat all UDP packets to TCP Friday, March 29, 13
  13. Hackathon ideas • Metrics path validation in backend • Pluggable

    receiver frontends • Forked backends • Configurable timer metrics Friday, March 29, 13
  14. Basic setup • git clone https://github.com/etsy/statsd.git • curl https://gist.github.com/mrtazz/5270376 >

    test_config.js • nc -k -l 2003 > output.log • node stats.js test_config.js Friday, March 29, 13