Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Measure your Rails app

Measure your Rails app

So you've done a web app, you tested it, deployed it, and your first customers begin to use your product and share it to their friends. That's great! Now, how do you know how are they using your application? Are you sure of the impacts of your deployments? What if you doubled the response time by fixing a bug and didn't see it?

In order to watch your application behavior, you need real-time metrics.

What are exactly real-time metrics? What can you use to store and watch them? What are exactly RRDtool, Graphite, StatsD? Can we improve existing solutions?

Jean-Daniel Guyot

April 12, 2012
Tweet

More Decks by Jean-Daniel Guyot

Other Decks in Programming

Transcript

  1. measure
    your rails app

    View Slide

  2. metrics
    solutions
    going further

    View Slide

  3. Jean-Daniel Guyot
    @jdguyot

    View Slide

  4. View Slide

  5. metrics
    solutions
    going further

    View Slide

  6. Statistics

    View Slide

  7. Statistics ≠ Real-time metrics

    View Slide

  8. Statistics = Play with your model

    View Slide

  9. Statistics
    SELECT AVG(price),
    date_trunc('month', created_at) d
    FROM payments
    WHERE state = ‘paid’
    GROUP BY d

    View Slide

  10. 0 €
    43 €
    85 €
    128 €
    170 €
    dec. jan. feb. mar.

    View Slide

  11. real-time metrics

    View Slide

  12. real-time trends
    metrics

    View Slide

  13. system
    application

    View Slide

  14. system
    application

    View Slide

  15. Good ol’ tools
    Syslog
    RRDtool
    Munin
    Cacti

    View Slide

  16. View Slide

  17. View Slide

  18. data value
    Minute Hour Day Month Year

    View Slide

  19. data compression
    10 sec during 1 day
    1 min during 1 week
    10 min

    View Slide

  20. issues
    No way to insert data on an
    irregular basis
    No way to insert data in the past
    Insert one value at a time (I/O
    performance issue)

    View Slide

  21. metrics
    solutions
    going further

    View Slide

  22. graphite

    View Slide

  23. graphite
    Easy to use
    Combine graphs easily
    Apply functions
    Save graphs & dashboards
    (Python)

    View Slide

  24. More than storage
    Whisper (DB)
    Carbon (Backend)
    Webapp (UI)
    inspired from RRDtool

    View Slide

  25. timestamp key value

    View Slide

  26. pimp my gaphite
    Whisper (DB)
    Carbon (Backend)
    Webapp (UI)
    Graphiti Graphene

    View Slide

  27. sending data to
    graphite

    View Slide

  28. App

    View Slide

  29. App Graphite
    timestamp key value

    View Slide

  30. issues

    View Slide

  31. App Graphite
    OMGTCP!

    View Slide

  32. App Graphite
    2 values in 10 seconds interval:
    OVERWRITE

    View Slide

  33. App Graphite
    Counters?

    View Slide

  34. flickr
    StatsD (Perl)

    View Slide

  35. etsy
    StatsD (Node.js)

    View Slide

  36. Statsd
    Ruby, JS, C++, whatever
    Counters, Timers
    Automatic simple statistics values
    computation (95% percentile,
    average, …)

    View Slide

  37. StatsD Graphite
    App

    View Slide

  38. metrics
    solutions
    going further

    View Slide

  39. issues

    View Slide

  40. StatsD Graphite
    App
    Monitoring
    Maintenance
    3 standalone apps =

    View Slide

  41. 2 solutions

    View Slide

  42. ambroise
    github.com/capitainetrain/ambroise

    View Slide

  43. App Graphite
    Unicorn + after_fork + StatsD
    Ambroise
    =

    View Slide

  44. ambroise
    Nothing to start
    Less code
    Simple

    View Slide

  45. statsd + rabbitmq

    View Slide

  46. StatsD Graphite
    App
    Classic architecture

    View Slide

  47. StatsD
    Graphite
    App
    Add RabbitMQ
    RabbitMQ

    View Slide

  48. RMQ
    Graphite
    App
    Detach StatsD
    StatsD
    MQ
    RMQ

    View Slide

  49. statsd + rabbitmq
    No latency
    Fault-resistant
    Only one running app
    Simple

    View Slide

  50. a few graphite tips

    View Slide

  51. timestamp key value

    View Slide

  52. folder.folder.key

    View Slide

  53. tip #1
    Pay attention to values overwriting
    Several machines: add a
    namespace per machine
    mystat.machine1.my_counter
    To get all stats: mystat.*.counter

    View Slide

  54. tip #2
    Use only 1 RabbitMQ for all the hosts
    (development machines/integration)
    hostname.app_name.mystat.machin
    e1.my_counter

    View Slide

  55. Thanks.

    View Slide