Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Map & Territory: A story of visibility

Map & Territory: A story of visibility


Pierre-Yves Ritschard

April 19, 2013

More Decks by Pierre-Yves Ritschard

Other Decks in Technology


  1. Map & Territory a story of visibility

  2. Pierre-Yves @pyr https://github.com/pyr

  3. https://exoscale.ch

  4. Visibility

  5. How do we work ?

  6. None
  7. How do we improve?

  8. Avoid Shortcuts!

  9. None
  10. We want lower defect rates

  11. We want to make informed decisions

  12. Design Build Live

  13. Visibility

  14. Extracting meaningful state data from heterogeneous event sources, over time

  15. Meaningful (relates to business value)

  16. State Data (structured payload)

  17. Heterogeneous (everyone is involved)

  18. Over time (tracking)

  19. How does it help my system's lifecycle ?

  20. Map =/= Territory

  21. Break out of our mental model

  22. "I'll push this minor change, it cannot do any harm"

  23. None
  24. "I'll just add this static route"

  25. None
  26. Better lifecycle Informed decisions Better maps

  27. Systems are (increasingly) complex

  28. Web Infrastructure circa 00 (2 servers)

  29. Visibility Circa '00

  30. Web Infrastructure circa '12 (27 nodes)

  31. None
  32. Visibility Circa '12

  33. Q: how is business doing today ? A:

  34. Q: how is business doing today ? A: based on

    these key metrics we're looking good
  35. Figure out those key metrics

  36. We need appropriate tooling

  37. events across: system, components, software

  38. The event stream approach

  39. Plenty of small producers Few big consumers

  40. Production: Anything that happens or moves (logs too!): Normalize &

  41. Consumption: Aggregate Correlate Decide

  42. Aggregation compute compound metrics (ratios, sums)

  43. Correlation

  44. Decision track, alert, ignore, scale

  45. Implementing on premise, saas or in between ?

  46. SaaS loggly, papertrail, librato, datadog, ...

  47. On Premise collectd, logstash, graphite, statsd, riemann

  48. The path to visibility: Find key metrics Find the right

    tools Rely on an event stream Involve everyone Challenge your mental model Hopefully, improve quality and lower defect rates in the process!
  49. Questions ?