Map & Territory: A story of visibility

Transcript

1. Map & Territory a story of visibility

2. Pierre-Yves @pyr https://github.com/pyr

3. https://exoscale.ch

4. Visibility

5. How do we work ?

6. None

7. How do we improve?

8. Avoid Shortcuts!

9. None

10. We want lower defect rates

11. We want to make informed decisions

12. Design Build Live

13. Visibility

14. Extracting meaningful state data from heterogeneous event sources, over time

15. Meaningful (relates to business value)

16. State Data (structured payload)

17. Heterogeneous (everyone is involved)

18. Over time (tracking)

19. How does it help my system's lifecycle ?

20. Map =/= Territory

21. Break out of our mental model

22. "I'll push this minor change, it cannot do any harm"

23. None

24. "I'll just add this static route"

25. None

26. Better lifecycle Informed decisions Better maps

27. Systems are (increasingly) complex

28. Web Infrastructure circa 00 (2 servers)

29. Visibility Circa '00

30. Web Infrastructure circa '12 (27 nodes)

31. None

32. Visibility Circa '12

33. Q: how is business doing today ? A:

34. Q: how is business doing today ? A: based on

    these key metrics we're looking good

35. Figure out those key metrics

36. We need appropriate tooling

37. events across: system, components, software

38. The event stream approach

39. Plenty of small producers Few big consumers

40. Production: Anything that happens or moves (logs too!): Normalize &

    Stream

41. Consumption: Aggregate Correlate Decide

42. Aggregation compute compound metrics (ratios, sums)

43. Correlation

44. Decision track, alert, ignore, scale

45. Implementing on premise, saas or in between ?

46. SaaS loggly, papertrail, librato, datadog, ...

47. On Premise collectd, logstash, graphite, statsd, riemann

48. The path to visibility: Find key metrics Find the right

    tools Rely on an event stream Involve everyone Challenge your mental model Hopefully, improve quality and lower defect rates in the process!

49. Questions ?