Pro Yearly is on sale from $80 to $50! »

ElixirConf 2016 - Measuring your elixir App

4569aec00cb223b3fbf484f9e7ba1256?s=47 Renan Ranelli
September 06, 2016

ElixirConf 2016 - Measuring your elixir App

4569aec00cb223b3fbf484f9e7ba1256?s=128

Renan Ranelli

September 06, 2016
Tweet

Transcript

  1. Measuring your Elixir application Renan Ranelli

  2. Senior software engineer @ Milhouse (@renanranelli)

  3. Milhouse (@renanranelli)

  4. None
  5. None
  6. Why should I care?

  7. Why should I care? We write code.

  8. Why should I care? We write code. We do it

    because it generate business value.
  9. Why should I care? We write code. We do it

    because it generate business value. But this business value is generated only when our code *runs*. *NOT* when it is written
  10. Why should I care? In order to make good decisions

    about something, we must understand it.
  11. Why should I care? In order to make good decisions

    about something, we must understand it. (And we want to make good decisions because we like getting paid)
  12. But... there is a problem

  13. Why should I care? Its hard to reason about code

  14. Which one is faster?

  15. We don’t know

  16. Which is faster?

  17. The mental model we have of our system is *always*

    flawed.
  18. But wait! There's still hope.

  19. When we measure stuff, the confusion fades:

  20. When we measure stuff, the confusion fades:

  21. This is *SCIENCE* !

  22. This is *SCIENCE* !

  23. Types of metrics & how we look at them

  24. The metrics we’re interested are **time series**:

  25. The metrics we’re interested are **time series**:

  26. Metrics – average value

  27. Metrics – average value

  28. Metrics – average value ← Time

  29. Metrics – average value First thing you should ask: Which

    time period does this average refer to ??? Averages completely destroy the **variability** and **recency**.
  30. Metrics – average value A common thing to do is

    add a “decay” component to the average calculation:
  31. Metrics – average value A common thing to do is

    add a “decay” component to the average calculation:
  32. Metrics – distribution

  33. Metrics – distribution

  34. Histograms Histograms are like averages on steroids. As such, you

    **have** to pay attention to The aggregation period it refers to
  35. Metrics – resolution

  36. Metrics – resolution

  37. How do I get started in the real world?

  38. You have to solve these parts: ❑ Collect ❑ Store

    ❑ Query, Analyze & Transform ❑ Visualize ❑ … ❑ MAKE DECISIONS (!)
  39. Our metrics collection architecture Log parsing Exometer & Elixometer Influxdb

    plugin “report api” Query & Visualize Transform & Analyze STORE Collect Store View
  40. Our (boring) choices ❑ Collect •Exometer & Elixometer are very

    easy to set up •We have used collectd extensively in the past ❑ Store (InfluxDB) •**Very** easy to set up & configure. And free. •Built-in retention policy configuration •Awesome query language & Continuous queries •Plugs seamlessly with grafana
  41. Our (boring) choices ❑ View (Grafana) •We have used it

    in the past. •It is awesome and beautiful (you will see!)
  42. Our (boring) choices ❑ You might ask: ❑ Why you’re

    not using solution like Splunk, DataDog, etc?
  43. Our (boring) choices ❑ You might ask: Why you’re not

    using solution like Splunk, DataDog, etc? ❑ Our answer: ❑ We don't like our data outside our walls. Also, NDAs and legal obligations
  44. Exometer & Elixometer ❑ Exometer Exometer Reporter Metrics buffer (pre

    aggregated ) High frequency Metric reports Store & update Aggregated view Collect & convert Low frequency Metrics formated for other backends
  45. Exometer & Elixometer ❑ This design is almost omnipresent in

    metrics world. ❑ Other tools like statsd, telegraf, etc might fit the role of exometer in your case. Use the one you’re most comfortable with. ❑ It is **very** important to be able to control the frequency which your storage receives requests. You don’t want to overload it.
  46. DEMONSTRATION • Measuring all requests to a phoenix app Step

    by step instructions here: http://milhouseonsoftware.com/2016/05/08/me asuring-your-elixir-application/
  47. Other things you should measure

  48. Other things you should measure ❑ The Erlang VM is

    **awesome** (and that’s why we’re here ;P) ❑ There are lots of resources out there on how to measure, monitor and operate it. ❑ But sadly I don’t have enough time to go in depth about it ~ Use vm_stats to gather virtual machine metrics! https://github.com/ferd/vmstats
  49. Other things you should measure ❑ The Erlang VM is

    **awesome** (and that’s why we’re here ;P) ❑ ❑ There are lots of resources out there on how to measure, monitor and operate it. ❑ ❑ But sadly I don’t have enough time to go in depth about the things the instrumentation the VM exposes ~
  50. Other things you should measure ❑ The Erlang VM is

    **awesome** (and that’s why we’re here ;P) ❑ ❑ There are lots of resources out there on how to measure, monitor and operate it.
  51. Closing remarks

  52. Closing remarks ❑ Find “seams” in your code where you

    can “plug” measurements. ❑ Try not to spread metrics reporting “everywhere”. It’s best to separat “high level” measurements from “low level” ones. ❑ This is just the beginning. After you have this infrastructure in place, you have to start doing analysis, correlation and other math stuffs. That’s where the real *gold* is.
  53. Closing remarks ❑ Bend your metrics to generate alerts. This

    is **really** awesome. ❑ Another (similar) metrics collection architecture is described here: http://tech.footballaddicts.com/blog/gathering-metrics-in-elixir- applications ❑ Zed Shaw has an awesome post about statistics: ❑https://zedshaw.com/archive/programmers-need-to-learn-statistics- or-i-will-kill-them-all/
  54. Obrigado!

  55. milhouseonsoftware.com @renanranelli /rranelli Milhouse @ (We're hiring!!!)