Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Collecting Metrics

Collecting Metrics

Collecting metrics in your application with StatsD and librato.

Till Klampaeckel

November 06, 2012
Tweet

More Decks by Till Klampaeckel

Other Decks in Programming

Transcript

  1. www.easybib.com @klimpong Collecting Metrics MEASURE ALL THE THINGS. 1 Thursday,

    November 8, 12
  2. www.easybib.com @klimpong Till Klampäckel @klimpong — http://github.com/till I love open

    source. I’m addicted to graphs (and effects in keynote). /Till 2 Thursday, November 8, 12
  3. www.easybib.com @klimpong Just in case — we’re (always) looking for

    PHP and frontend developers. [email protected] /jobs 3 Thursday, November 8, 12
  4. www.easybib.com @klimpong A unit of measurement. An indicator for what

    is going on. /metrics 4 Thursday, November 8, 12
  5. www.easybib.com @klimpong Google Analytics/piwik: utm-campaign-tags Remember counter.cgi? Facebook pages. /examples

    5 Thursday, November 8, 12
  6. www.easybib.com @klimpong Visits per year 0M 10M 20M 30M 40M

    50M 60M 70M 80M 2007 2008 2009 2010 2011 Visits per year /examples/graphs 6 Thursday, November 8, 12
  7. www.easybib.com @klimpong <script>? PHP to a database, remote service? StatsD!

    /collect 7 Thursday, November 8, 12
  8. www.easybib.com @klimpong StatsD is a collection daemon by Etsy. StatsD

    is Node.js. StatsD uses UDP. It’s asynchronous and painless. Backends are pluggable. /statsd 8 Thursday, November 8, 12 Etsy’s work was inspired by Flickr. standard backends: graphite, console, reapeter available backends: librato, cloudwatch, mongodb, ...
  9. www.easybib.com @klimpong Original implementation in Node.JS. Erlang, Go, Scala, Java,

    Python. /statsd/pattern 9 Thursday, November 8, 12 Original, original (Cal Henderson) — something in Perl at Flickr. StatsD as a pattern to collect metrics.
  10. www.easybib.com @klimpong Counters. Gauges. Timings. /statsd/metrics 10 Thursday, November 8,

    12 Really simple, no PhD required. Counter: grows absolute over time (number of total downloads) Gauge: for current values (e.g. “currently online”) Timings: how long something took There is more in master.
  11. www.easybib.com @klimpong Default backend. http://graphite.wikidot.com/ /statsd/graphite 11 Thursday, November 8,

    12 We wanted a hosted service (less management). Read a couple blog posts about scaling Graphite which scared me off. Librato has less aggregation and very little transformation but it’s been rock solid.
  12. www.easybib.com @klimpong Download and install Node.js: http://nodejs.org/download/ Clone statsd from

    Github: https://github.com/etsy/statsd npm install statsd-librato-backend /statsd/install 12 Thursday, November 8, 12 Install Node.js through: homebrew, aptitude, Windows installer cd into statsd directory to npm install
  13. www.easybib.com @klimpong { port: statsd_port, mgmt_address: '127.0.0.1', backends: ['statsd-librato-backend'], librato:

    { email: 'email_on_librato', token: 'librato_token', source: 'where_metrics_are_coming_from' } } /statsd/configure 13 Thursday, November 8, 12 Watch me type! The minimum is email and token. port: port of the statsd process librato.email/.token: credentials from librato librato.source: e.g. your cluster (application, whatever)
  14. www.easybib.com @klimpong ./bin/statsd /statsd/run 14 Thursday, November 8, 12 Maybe

    add monit for when the node process crashes (0.8.0+ recommended).
  15. www.easybib.com @klimpong Find a PHP client library. /statsd/now-what 15 Thursday,

    November 8, 12 StatsD has examples for half a dozen languages. Want something more sophisticated?
  16. www.easybib.com @klimpong <?php // https://github.com/till/php-statsd require_once 'StatsD.php'; StatsD::init(array( 'enabled' =>

    true, 'host' => 'statsd1' )); StatsD::increment('some-value'); StatsD::increment(array( 'multiple', 'counters', 'work', 'as', 'well' )); /collect 16 Thursday, November 8, 12 watch me type again forked the PHP sample code
  17. www.easybib.com @klimpong /librato 17 Thursday, November 8, 12 This is

    an excerpt of stuff we collect. And every time there’s a problem, we add to it. https://metrics.librato.com/pricing 50 metrics every 60 seconds = ~5.00 USD/month flush every 30 = ~8.50 USD a month
  18. www.easybib.com @klimpong Graph the ms it takes to pull up

    different views and _all_docs. /librato/demo 18 Thursday, November 8, 12
  19. www.easybib.com @klimpong Requests (per second) to our API. And failed

    requests also (none). /librato/demo 19 Thursday, November 8, 12
  20. www.easybib.com @klimpong Graph requests (per second) to EasyBib Research related

    APIs. /librato/demo 20 Thursday, November 8, 12
  21. www.easybib.com Fin 21 Thursday, November 8, 12