Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Collecting Metrics

Collecting Metrics

Collecting metrics in your application with StatsD and librato.


Till Klampaeckel

November 06, 2012


  1. www.easybib.com @klimpong Collecting Metrics MEASURE ALL THE THINGS. 1 Thursday,

    November 8, 12
  2. www.easybib.com @klimpong Till Klampäckel @klimpong — http://github.com/till I love open

    source. I’m addicted to graphs (and effects in keynote). /Till 2 Thursday, November 8, 12
  3. www.easybib.com @klimpong Just in case — we’re (always) looking for

    PHP and frontend developers. till@imagineeasy.com /jobs 3 Thursday, November 8, 12
  4. www.easybib.com @klimpong A unit of measurement. An indicator for what

    is going on. /metrics 4 Thursday, November 8, 12
  5. www.easybib.com @klimpong Google Analytics/piwik: utm-campaign-tags Remember counter.cgi? Facebook pages. /examples

    5 Thursday, November 8, 12
  6. www.easybib.com @klimpong Visits per year 0M 10M 20M 30M 40M

    50M 60M 70M 80M 2007 2008 2009 2010 2011 Visits per year /examples/graphs 6 Thursday, November 8, 12
  7. www.easybib.com @klimpong <script>? PHP to a database, remote service? StatsD!

    /collect 7 Thursday, November 8, 12
  8. www.easybib.com @klimpong StatsD is a collection daemon by Etsy. StatsD

    is Node.js. StatsD uses UDP. It’s asynchronous and painless. Backends are pluggable. /statsd 8 Thursday, November 8, 12 Etsy’s work was inspired by Flickr. standard backends: graphite, console, reapeter available backends: librato, cloudwatch, mongodb, ...
  9. www.easybib.com @klimpong Original implementation in Node.JS. Erlang, Go, Scala, Java,

    Python. /statsd/pattern 9 Thursday, November 8, 12 Original, original (Cal Henderson) — something in Perl at Flickr. StatsD as a pattern to collect metrics.
  10. www.easybib.com @klimpong Counters. Gauges. Timings. /statsd/metrics 10 Thursday, November 8,

    12 Really simple, no PhD required. Counter: grows absolute over time (number of total downloads) Gauge: for current values (e.g. “currently online”) Timings: how long something took There is more in master.
  11. www.easybib.com @klimpong Default backend. http://graphite.wikidot.com/ /statsd/graphite 11 Thursday, November 8,

    12 We wanted a hosted service (less management). Read a couple blog posts about scaling Graphite which scared me off. Librato has less aggregation and very little transformation but it’s been rock solid.
  12. www.easybib.com @klimpong Download and install Node.js: http://nodejs.org/download/ Clone statsd from

    Github: https://github.com/etsy/statsd npm install statsd-librato-backend /statsd/install 12 Thursday, November 8, 12 Install Node.js through: homebrew, aptitude, Windows installer cd into statsd directory to npm install
  13. www.easybib.com @klimpong { port: statsd_port, mgmt_address: '', backends: ['statsd-librato-backend'], librato:

    { email: 'email_on_librato', token: 'librato_token', source: 'where_metrics_are_coming_from' } } /statsd/configure 13 Thursday, November 8, 12 Watch me type! The minimum is email and token. port: port of the statsd process librato.email/.token: credentials from librato librato.source: e.g. your cluster (application, whatever)
  14. www.easybib.com @klimpong ./bin/statsd /statsd/run 14 Thursday, November 8, 12 Maybe

    add monit for when the node process crashes (0.8.0+ recommended).
  15. www.easybib.com @klimpong Find a PHP client library. /statsd/now-what 15 Thursday,

    November 8, 12 StatsD has examples for half a dozen languages. Want something more sophisticated?
  16. www.easybib.com @klimpong <?php // https://github.com/till/php-statsd require_once 'StatsD.php'; StatsD::init(array( 'enabled' =>

    true, 'host' => 'statsd1' )); StatsD::increment('some-value'); StatsD::increment(array( 'multiple', 'counters', 'work', 'as', 'well' )); /collect 16 Thursday, November 8, 12 watch me type again forked the PHP sample code
  17. www.easybib.com @klimpong /librato 17 Thursday, November 8, 12 This is

    an excerpt of stuff we collect. And every time there’s a problem, we add to it. https://metrics.librato.com/pricing 50 metrics every 60 seconds = ~5.00 USD/month flush every 30 = ~8.50 USD a month
  18. www.easybib.com @klimpong Graph the ms it takes to pull up

    different views and _all_docs. /librato/demo 18 Thursday, November 8, 12
  19. www.easybib.com @klimpong Requests (per second) to our API. And failed

    requests also (none). /librato/demo 19 Thursday, November 8, 12
  20. www.easybib.com @klimpong Graph requests (per second) to EasyBib Research related

    APIs. /librato/demo 20 Thursday, November 8, 12
  21. www.easybib.com Fin 21 Thursday, November 8, 12