Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Lies, Damn Lies, and Metrics (Distill 2014)

Lies, Damn Lies, and Metrics (Distill 2014)

Metrics are great, and measuring things can provide tremendously useful insights. But there's a problem: metrics lie to you. Metrics just report the numbers that were measured. Analyzing those numbers is up to us, and that analysis can go wrong in so, so many ways. Learn how to arm yourself against human intuition, interpreter pauses, routing, instrumentation lag, and other issues. Don't get so caught up in instrumenting that you lose sight of why metrics exist! Make sure your metrics are telling you actionable information, instead of just accurate numbers.

André Arko

August 08, 2014
Tweet

More Decks by André Arko

Other Decks in Technology

Transcript

  1. “Normal” 5 -5 -4 -3 -2 -1 0 1 2

    3 4 0 0.1 0.2 0.3 0.4
  2. “Normal” 5 -5 -4 -3 -2 -1 0 1 2

    3 4 0 0.1 0.2 0.3 0.4
  3. Real Life 5 -5 -4 -3 -2 -1 0 1

    2 3 4 0 0.1 0.2 0.3 0.4
  4. 10 0 1 2 3 4 5 6 7 8

    9 250 0 50 100 150 200
  5. 10 0 1 2 3 4 5 6 7 8

    9 250 0 50 100 150 200
  6. 10 0 1 2 3 4 5 6 7 8

    9 250 0 50 100 150 200
  7. 10 0 1 2 3 4 5 6 7 8

    9 1000 0 250 500 750