Collecting Metrics

Collecting metrics in your application with StatsD and librato.

Transcript

1. www.easybib.com @klimpong Collecting Metrics MEASURE ALL THE THINGS. 1 Thursday, November 8, 12
2. www.easybib.com @klimpong Till Klampäckel @klimpong — http://github.com/till I love open source. I’m addicted to graphs (and effects in keynote). /Till 2 Thursday, November 8, 12
3. www.easybib.com @klimpong Just in case — we’re (always) looking for PHP and frontend developers. till@imagineeasy.com /jobs 3 Thursday, November 8, 12
4. www.easybib.com @klimpong A unit of measurement. An indicator for what is going on. /metrics 4 Thursday, November 8, 12
5. www.easybib.com @klimpong Google Analytics/piwik: utm-campaign-tags Remember counter.cgi? Facebook pages. /examples 5 Thursday, November 8, 12
6. www.easybib.com @klimpong Visits per year 0M 10M 20M 30M 40M 50M 60M 70M 80M 2007 2008 2009 2010 2011 Visits per year /examples/graphs 6 Thursday, November 8, 12
7. www.easybib.com @klimpong <script>? PHP to a database, remote service? StatsD! /collect 7 Thursday, November 8, 12
8. www.easybib.com @klimpong StatsD is a collection daemon by Etsy. StatsD is Node.js. StatsD uses UDP. It’s asynchronous and painless. Backends are pluggable. /statsd 8 Thursday, November 8, 12 Etsy’s work was inspired by Flickr. standard backends: graphite, console, reapeter available backends: librato, cloudwatch, mongodb, ...
9. www.easybib.com @klimpong Original implementation in Node.JS. Erlang, Go, Scala, Java, Python. /statsd/pattern 9 Thursday, November 8, 12 Original, original (Cal Henderson) — something in Perl at Flickr. StatsD as a pattern to collect metrics.
10. www.easybib.com @klimpong Counters. Gauges. Timings. /statsd/metrics 10 Thursday, November 8, 12 Really simple, no PhD required. Counter: grows absolute over time (number of total downloads) Gauge: for current values (e.g. “currently online”) Timings: how long something took There is more in master.
11. www.easybib.com @klimpong Default backend. http://graphite.wikidot.com/ /statsd/graphite 11 Thursday, November 8, 12 We wanted a hosted service (less management). Read a couple blog posts about scaling Graphite which scared me off. Librato has less aggregation and very little transformation but it’s been rock solid.
12. www.easybib.com @klimpong Download and install Node.js: http://nodejs.org/download/ Clone statsd from Github: https://github.com/etsy/statsd npm install statsd-librato-backend /statsd/install 12 Thursday, November 8, 12 Install Node.js through: homebrew, aptitude, Windows installer cd into statsd directory to npm install
13. www.easybib.com @klimpong { port: statsd_port, mgmt_address: '127.0.0.1', backends: ['statsd-librato-backend'], librato: { email: 'email_on_librato', token: 'librato_token', source: 'where_metrics_are_coming_from' } } /statsd/configure 13 Thursday, November 8, 12 Watch me type! The minimum is email and token. port: port of the statsd process librato.email/.token: credentials from librato librato.source: e.g. your cluster (application, whatever)
14. www.easybib.com @klimpong ./bin/statsd /statsd/run 14 Thursday, November 8, 12 Maybe add monit for when the node process crashes (0.8.0+ recommended).
15. www.easybib.com @klimpong Find a PHP client library. /statsd/now-what 15 Thursday, November 8, 12 StatsD has examples for half a dozen languages. Want something more sophisticated?
16. www.easybib.com @klimpong <?php // https://github.com/till/php-statsd require_once 'StatsD.php'; StatsD::init(array( 'enabled' => true, 'host' => 'statsd1' )); StatsD::increment('some-value'); StatsD::increment(array( 'multiple', 'counters', 'work', 'as', 'well' )); /collect 16 Thursday, November 8, 12 watch me type again forked the PHP sample code
17. www.easybib.com @klimpong /librato 17 Thursday, November 8, 12 This is an excerpt of stuff we collect. And every time there’s a problem, we add to it. https://metrics.librato.com/pricing 50 metrics every 60 seconds = ~5.00 USD/month flush every 30 = ~8.50 USD a month
18. www.easybib.com @klimpong Graph the ms it takes to pull up different views and _all_docs. /librato/demo 18 Thursday, November 8, 12
19. www.easybib.com @klimpong Requests (per second) to our API. And failed requests also (none). /librato/demo 19 Thursday, November 8, 12
20. www.easybib.com @klimpong Graph requests (per second) to EasyBib Research related APIs. /librato/demo 20 Thursday, November 8, 12
21. www.easybib.com Fin 21 Thursday, November 8, 12