Introduction to StatsD

WebCamp KL August 2013

Transcript

1. Introduction to StatsD
   

   View Slide

2. Information
   ●
   Improving a system requires resources
   ●
   Increase return on investment by correct assignment
   of resources
   ●
   Decisions are made by analyzing information
   

   View Slide

3. Data Collection
   ●
   Accurate data helps better decisions
   ●
   Data from different aspects provide a better
   understanding
   ●
   Data collection should have least affect on
   performance of the running system
   

   View Slide

4. Measurement
   ●
   Measurement from outside simulates users better, but
   is limited to the surface layer (interface)
   ●
   Measurement from inside provides accurate results
   from all layers of the system
   

   View Slide

5. StatsD
   ●
   A solution to send data from within the system with
   non blocking behavior
   ●
   StatsD was proposed by etsy, There is no standard
   (yet). Implemented in Node.js
   ●
   System sends metrics over UDP to StatsD server
   ●
   StatsD server aggregates and flushes data to a
   persistent backend
   

   View Slide

6. StatsD
   ●
   Protocol is simple: Metrics have name, value and type
   sent as text over UDP.
   ●
   Client libraries available for many languages, easy to
   implement.
   ●
   Compatible servers implemented in other languages
   (Python, Ruby, Perl, C, …)
   ●
   A very common backend is Graphite
   

   View Slide

7. Metric Types
   ●
   Counter: Anything you count
   ●
   Timer: Duration of a task
   ●
   Gauge: Any arbitrary value
   ●
   Set: Number of unique values in each sampling period
   

   View Slide

8. Bucky
   ●
   Bucky is a StatsD server written in Python
   $ pip install bukcy
   $ pip install bukcy
   $ bucky bucky.conf
   metricsd_enabled = False
   collectd_enabled = False
   statsd_enabled = True
   statsd_ip = ""
   statsd_port = 8125
   statsd_flush_time = 1.0
   graphite_ip = "127.0.0.1"
   graphite_port = 2003
   ●
   Sample Bucky configuration file
   

   View Slide

9. Sample Code
   #!/usr/bin/env python
   # - install statsd module: pip install statsd
   # - we sleep for a random time (max 5 secs) to simulate a
   # real action
   # - statsd timers are in microseconds
   import time
   import random
   import statsd
   client = statsd.StatsClient('localhost', 8125)
   start_t = time.time()
   client.incr('action')
   time.sleep(random.random() * 5)
   client.timing('action', (time.time() - start_t) * 1000 * 1000)
   

   View Slide

10. Graphite
    ●
    A solution to collect, store and visualize data over time
    ●
    Contains 3 components
    – Carbon: Network service to receive data
    – Whisper: Time series database file
    – Graphite: Web interface
    – Simple protocol: Text overt TCP with a name, value
    and a timestamp separated by space.
    

    View Slide

11. Graphite
    

    View Slide

12. Farzad Ghanei
    www.ghanei.net
    

    View Slide