UX Design and Education for Effective Monitoring Tools (TechSummit Berlin 2017)

I, like many of us, chose to work in infrastructure because I thought it meant I could avoid talking to other people as much as possible. I was wrong. It turns out that a huge amount of work in monitoring is about empowering the rest of your engineering organization to use the tools you develop correctly, quickly, and effectively. We spend so much time explaining how to interpret timeseries data and why averaging percentiles is a bad idea. After feedback from our engineers, we embarked on a journey to redesign our internal monitoring tools and understand where people were struggling with the existing system.

In this talk, I will explain how we approached the problem of making concepts like interpolation, aggregation, and alerting more intuitive and how we identified pain points for new users. I will outline common misconceptions our users have about monitoring and how we cleared up this confusion in our UI without forcing everyone to spend hours on documentation. Rather than copying and pasting existing UX design principles onto our monitoring problems, we will see how we can reinterpret these ideas and apply them to our unique situation to create a better experience for everyone.

Transcript

1. UX Design and Education
   for Effective Monitoring Tools
   Amy Nguyen
   @amyngyn
   April 12, 2017
   or: one way to become a 10x engineer
   

   View Slide

2. Hi!
   

   View Slide

3. Hi!
   ● Joined Pinterest in 2015, now on
   Visibility team
   

   View Slide

4. Hi!
   ● Joined Pinterest in 2015, now on
   Visibility team
   ● Recent projects: network tracing,
   D3 data visualizations, caching
   OpenTSDB timeseries data
   

   View Slide

5. Hi!
   ● Joined Pinterest in 2015, now on
   Visibility team
   ● Recent projects: network tracing,
   D3 data visualizations, caching
   OpenTSDB timeseries data
   ● Most importantly: overhauling the user
   experience of our monitoring stack
   

   View Slide

6. Hi!
   ● Joined Pinterest in 2015, now on
   Visibility team
   ● Recent projects: network tracing,
   D3 data visualizations, caching
   OpenTSDB timeseries data
   ● Most importantly: overhauling the user
   experience of our monitoring stack
   ● amynguyen.net / @amyngyn
   

   View Slide

7. 7
   About Pinterest
   the world's first visual discovery engine
   

   View Slide

8. 8
   About Pinterest
   ● 150 million monthly active users
   the world's first visual discovery engine
   

   View Slide

9. 9
   About Pinterest
   ● 150 million monthly active users
   ● 100 billion pins
   the world's first visual discovery engine
   

   View Slide

10. 10
    About Pinterest
    ● 150 million monthly active users
    ● 100 billion pins
    ● 2 billion boards
    the world's first visual discovery engine
    

    View Slide

11. 11
    About Pinterest Monitoring
    the world's greatest my favorite monitoring team
    

    View Slide

12. 12
    About Pinterest Monitoring
    ● Over 150,000 requests per second
    the world's greatest my favorite monitoring team
    

    View Slide

13. 13
    About Pinterest Monitoring
    ● Over 150,000 requests per second
    ● 20 terabytes logged per day
    the world's greatest my favorite monitoring team
    

    View Slide

14. 14
    About Pinterest Monitoring
    ● Over 150,000 requests per second
    ● 20 terabytes logged per day
    ● 2.5 million metrics per second
    the world's greatest my favorite monitoring team
    

    View Slide

15. 15
    About Pinterest Monitoring
    ● Over 150,000 requests per second
    ● 20 terabytes logged per day
    ● 2.5 million metrics per second
    ● Over 400 engineers
    the world's greatest my favorite monitoring team
    

    View Slide

16. Our tools: Dashboards
    

    View Slide

17. Our tools: Exploration
    

    View Slide

18. Our Tools: Alerting
    

    View Slide

19. Storytime
    

    View Slide

20. How to become a 10x engineer
    

    View Slide

21. How to become a 10x engineer
    1. Help 10 other engineers do their jobs better.
    

    View Slide

22. How to become a 10x engineer
    1. Help 10 other engineers do their jobs better.
    2. Profit!
    

    View Slide

23. How to help others do their jobs better
    

    View Slide

24. How to help others do their jobs better
    1. Improve the experience of working with your documentation.
    2. Improve the experience of working with your team.
    3. Improve the experience of working with your tools.
    

    View Slide

25. How to help others do their jobs better
    1. Improve the experience of working with your documentation.
    2. Improve the experience of working with your team.
    3. Improve the experience of working with your tools.
    

    View Slide

26. ● Is your documentation split across Google Drive, internal wikis, and old
    emails that get forwarded around?
    

    View Slide

27. ● Is your documentation split across Google Drive, internal wikis, and old
    emails that get forwarded around?
    ● Does your documentation assume prior knowledge that a new engineer
    wouldn't have?
    

    View Slide

28. ● Is your documentation split across Google Drive, internal wikis, and old
    emails that get forwarded around?
    ● Does your documentation assume prior knowledge that a new engineer
    wouldn't have?
    ● When was the last time anyone on your team followed the instructions in
    your documentation and checked that they still worked?
    

    View Slide

29. How can we maintain good documentation?
    

    View Slide

30. How can we maintain good documentation?
    ● Add common questions to the docs as they come up.
    

    View Slide

31. How can we maintain good documentation?
    ● Add common questions to the docs as they come up.
    ● Always link to the docs even if you can answer it yourself.
    

    View Slide

32. How can we maintain good documentation?
    ● Add common questions to the docs as they come up.
    ● Always link to the docs even if you can answer it yourself.
    ● Encourage everyone to contribute to the docs.
    

    View Slide

33. How to help others do their jobs better
    1. Improve the experience of working with your documentation.
    2. Improve the experience of working with your team.
    3. Improve the experience of working with your tools.
    

    View Slide

34. ● How do people contact your team if they have questions?
    

    View Slide

35. ● How do people contact your team if they have questions?
    ● How long does it usually take for someone to have their question
    answered?
    

    View Slide

36. ● How do people contact your team if they have questions?
    ● How long does it usually take for someone to have their question
    answered?
    ● What happens when someone sends you a complaint or request?
    

    View Slide

37. ● How do people contact your team if they have questions?
    ● How long does it usually take for someone to have their question
    answered?
    ● What happens when someone sends you a complaint or request?
    ● How do people feel after they interact with your team?
    

    View Slide

38. ● How do people contact your team if they have questions?
    ● How long does it usually take for someone to have their question
    answered?
    ● What happens when someone sends you a complaint or request?
    ● How do people feel after they interact with your team?
    ○ Stupid? Confused? Unsatisfied? Unwilling to return in the future?
    

    View Slide

39. ● How do people contact your team if they have questions?
    ● How long does it usually take for someone to have their question
    answered?
    ● What happens when someone sends you a complaint or request?
    ● How do people feel after they interact with your team?
    ○ Stupid? Confused? Unsatisfied? Unwilling to return in the future?
    ○ Heard! Supported! Understood! Learned something new!
    

    View Slide

40. How to help others do their jobs better
    1. Improve the experience of working with your documentation.
    2. Improve the experience of working with your team.
    3. Improve the experience of working with your tools.
    

    View Slide

41. Reward exploration.
    

    View Slide

42. View Slide

43. View Slide

44. View Slide

45. Set reasonable defaults.
    

    View Slide

46. View Slide

47. View Slide

48. View Slide

49. View Slide

50. What you want is (probably)
    not what your users want.
    

    View Slide

51. View Slide

52. View Slide

53. What you know is (probably)
    not what your users know.
    

    View Slide

54. How to help others do their jobs better
    1. Improve the experience of working with your documentation.
    2. Improve the experience of working with your team.
    3. Improve the experience of working with your tools.
    

    View Slide

55. Thanks!
    @amyngyn
    

    View Slide