Measure your Rails app

So you've done a web app, you tested it, deployed it, and your first customers begin to use your product and share it to their friends. That's great! Now, how do you know how are they using your application? Are you sure of the impacts of your deployments? What if you doubled the response time by fixing a bug and didn't see it?

In order to watch your application behavior, you need real-time metrics.

What are exactly real-time metrics? What can you use to store and watch them? What are exactly RRDtool, Graphite, StatsD? Can we improve existing solutions?

Transcript

1. measure
   your rails app
   

   View Slide

2. metrics
   solutions
   going further
   

   View Slide

3. Jean-Daniel Guyot
   @jdguyot
   

   View Slide

4. View Slide

5. metrics
   solutions
   going further
   

   View Slide

6. Statistics
   

   View Slide

7. Statistics ≠ Real-time metrics
   

   View Slide

8. Statistics = Play with your model
   

   View Slide

9. Statistics
   SELECT AVG(price),
   date_trunc('month', created_at) d
   FROM payments
   WHERE state = ‘paid’
   GROUP BY d
   

   View Slide

10. 0 €
    43 €
    85 €
    128 €
    170 €
    dec. jan. feb. mar.
    

    View Slide

11. real-time metrics
    

    View Slide

12. real-time trends
    metrics
    

    View Slide

13. system
    application
    

    View Slide

14. system
    application
    

    View Slide

15. Good ol’ tools
    Syslog
    RRDtool
    Munin
    Cacti
    

    View Slide

16. View Slide

17. View Slide

18. data value
    Minute Hour Day Month Year
    

    View Slide

19. data compression
    10 sec during 1 day
    1 min during 1 week
    10 min
    

    View Slide

20. issues
    No way to insert data on an
    irregular basis
    No way to insert data in the past
    Insert one value at a time (I/O
    performance issue)
    

    View Slide

21. metrics
    solutions
    going further
    

    View Slide

22. graphite
    

    View Slide

23. graphite
    Easy to use
    Combine graphs easily
    Apply functions
    Save graphs & dashboards
    (Python)
    

    View Slide

24. More than storage
    Whisper (DB)
    Carbon (Backend)
    Webapp (UI)
    inspired from RRDtool
    

    View Slide

25. timestamp key value
    

    View Slide

26. pimp my gaphite
    Whisper (DB)
    Carbon (Backend)
    Webapp (UI)
    Graphiti Graphene
    

    View Slide

27. sending data to
    graphite
    

    View Slide

28. App
    

    View Slide

29. App Graphite
    timestamp key value
    

    View Slide

30. issues
    

    View Slide

31. App Graphite
    OMGTCP!
    

    View Slide

32. App Graphite
    2 values in 10 seconds interval:
    OVERWRITE
    

    View Slide

33. App Graphite
    Counters?
    

    View Slide

34. flickr
    StatsD (Perl)
    

    View Slide

35. etsy
    StatsD (Node.js)
    

    View Slide

36. Statsd
    Ruby, JS, C++, whatever
    Counters, Timers
    Automatic simple statistics values
    computation (95% percentile,
    average, …)
    

    View Slide

37. StatsD Graphite
    App
    

    View Slide

38. metrics
    solutions
    going further
    

    View Slide

39. issues
    

    View Slide

40. StatsD Graphite
    App
    Monitoring
    Maintenance
    3 standalone apps =
    

    View Slide

41. 2 solutions
    

    View Slide

42. ambroise
    github.com/capitainetrain/ambroise
    

    View Slide

43. App Graphite
    Unicorn + after_fork + StatsD
    Ambroise
    =
    

    View Slide

44. ambroise
    Nothing to start
    Less code
    Simple
    

    View Slide

45. statsd + rabbitmq
    

    View Slide

46. StatsD Graphite
    App
    Classic architecture
    

    View Slide

47. StatsD
    Graphite
    App
    Add RabbitMQ
    RabbitMQ
    

    View Slide

48. RMQ
    Graphite
    App
    Detach StatsD
    StatsD
    MQ
    RMQ
    

    View Slide

49. statsd + rabbitmq
    No latency
    Fault-resistant
    Only one running app
    Simple
    

    View Slide

50. a few graphite tips
    

    View Slide

51. timestamp key value
    

    View Slide

52. folder.folder.key
    

    View Slide

53. tip #1
    Pay attention to values overwriting
    Several machines: add a
    namespace per machine
    mystat.machine1.my_counter
    To get all stats: mystat.*.counter
    

    View Slide

54. tip #2
    Use only 1 RabbitMQ for all the hosts
    (development machines/integration)
    hostname.app_name.mystat.machin
    e1.my_counter
    

    View Slide

55. Thanks.
    

    View Slide