UX Design and Education for Effective Monitoring Tools (TechSummit Berlin 2017)

I, like many of us, chose to work in infrastructure because I thought it meant I could avoid talking to other people as much as possible. I was wrong. It turns out that a huge amount of work in monitoring is about empowering the rest of your engineering organization to use the tools you develop correctly, quickly, and effectively. We spend so much time explaining how to interpret timeseries data and why averaging percentiles is a bad idea. After feedback from our engineers, we embarked on a journey to redesign our internal monitoring tools and understand where people were struggling with the existing system.

In this talk, I will explain how we approached the problem of making concepts like interpolation, aggregation, and alerting more intuitive and how we identified pain points for new users. I will outline common misconceptions our users have about monitoring and how we cleared up this confusion in our UI without forcing everyone to spend hours on documentation. Rather than copying and pasting existing UX design principles onto our monitoring problems, we will see how we can reinterpret these ideas and apply them to our unique situation to create a better experience for everyone.

Transcript

1. UX Design and Education for Effective Monitoring Tools Amy Nguyen @amyngyn April 12, 2017 or: one way to become a 10x engineer
2. Hi!
3. Hi! • Joined Pinterest in 2015, now on Visibility team
4. Hi! • Joined Pinterest in 2015, now on Visibility team • Recent projects: network tracing, D3 data visualizations, caching OpenTSDB timeseries data
5. Hi! • Joined Pinterest in 2015, now on Visibility team • Recent projects: network tracing, D3 data visualizations, caching OpenTSDB timeseries data • Most importantly: overhauling the user experience of our monitoring stack
6. Hi! • Joined Pinterest in 2015, now on Visibility team • Recent projects: network tracing, D3 data visualizations, caching OpenTSDB timeseries data • Most importantly: overhauling the user experience of our monitoring stack • amynguyen.net / @amyngyn
7. 7 About Pinterest the world's first visual discovery engine
8. 8 About Pinterest • 150 million monthly active users the world's first visual discovery engine
9. 9 About Pinterest • 150 million monthly active users • 100 billion pins the world's first visual discovery engine
10. 10 About Pinterest • 150 million monthly active users • 100 billion pins • 2 billion boards the world's first visual discovery engine
11. 11 About Pinterest Monitoring the world's greatest my favorite monitoring team
12. 12 About Pinterest Monitoring • Over 150,000 requests per second the world's greatest my favorite monitoring team
13. 13 About Pinterest Monitoring • Over 150,000 requests per second • 20 terabytes logged per day the world's greatest my favorite monitoring team
14. 14 About Pinterest Monitoring • Over 150,000 requests per second • 20 terabytes logged per day • 2.5 million metrics per second the world's greatest my favorite monitoring team
15. 15 About Pinterest Monitoring • Over 150,000 requests per second • 20 terabytes logged per day • 2.5 million metrics per second • Over 400 engineers the world's greatest my favorite monitoring team
16. Our tools: Dashboards
17. Our tools: Exploration
18. Our Tools: Alerting
19. Storytime
20. How to become a 10x engineer
21. How to become a 10x engineer 1. Help 10 other engineers do their jobs better.
22. How to become a 10x engineer 1. Help 10 other engineers do their jobs better. 2. Profit!
23. How to help others do their jobs better
24. How to help others do their jobs better 1. Improve the experience of working with your documentation. 2. Improve the experience of working with your team. 3. Improve the experience of working with your tools.
25. How to help others do their jobs better 1. Improve the experience of working with your documentation. 2. Improve the experience of working with your team. 3. Improve the experience of working with your tools.
26. • Is your documentation split across Google Drive, internal wikis, and old emails that get forwarded around?
27. • Is your documentation split across Google Drive, internal wikis, and old emails that get forwarded around? • Does your documentation assume prior knowledge that a new engineer wouldn't have?
28. • Is your documentation split across Google Drive, internal wikis, and old emails that get forwarded around? • Does your documentation assume prior knowledge that a new engineer wouldn't have? • When was the last time anyone on your team followed the instructions in your documentation and checked that they still worked?
29. How can we maintain good documentation?
30. How can we maintain good documentation? • Add common questions to the docs as they come up.
31. How can we maintain good documentation? • Add common questions to the docs as they come up. • Always link to the docs even if you can answer it yourself.
32. How can we maintain good documentation? • Add common questions to the docs as they come up. • Always link to the docs even if you can answer it yourself. • Encourage everyone to contribute to the docs.
33. How to help others do their jobs better 1. Improve the experience of working with your documentation. 2. Improve the experience of working with your team. 3. Improve the experience of working with your tools.
34. • How do people contact your team if they have questions?
35. • How do people contact your team if they have questions? • How long does it usually take for someone to have their question answered?
36. • How do people contact your team if they have questions? • How long does it usually take for someone to have their question answered? • What happens when someone sends you a complaint or request?
37. • How do people contact your team if they have questions? • How long does it usually take for someone to have their question answered? • What happens when someone sends you a complaint or request? • How do people feel after they interact with your team?
38. • How do people contact your team if they have questions? • How long does it usually take for someone to have their question answered? • What happens when someone sends you a complaint or request? • How do people feel after they interact with your team? ◦ Stupid? Confused? Unsatisfied? Unwilling to return in the future?
39. • How do people contact your team if they have questions? • How long does it usually take for someone to have their question answered? • What happens when someone sends you a complaint or request? • How do people feel after they interact with your team? ◦ Stupid? Confused? Unsatisfied? Unwilling to return in the future? ◦ Heard! Supported! Understood! Learned something new!
40. How to help others do their jobs better 1. Improve the experience of working with your documentation. 2. Improve the experience of working with your team. 3. Improve the experience of working with your tools.
41. Reward exploration.
45. Set reasonable defaults.
50. What you want is (probably) not what your users want.
53. What you know is (probably) not what your users know.
54. How to help others do their jobs better 1. Improve the experience of working with your documentation. 2. Improve the experience of working with your team. 3. Improve the experience of working with your tools.
55. Thanks! @amyngyn