UX Design and Education for Effective Monitoring Tools (LISA17)

The fastest way to become a 10X engineer is by enabling 10 other engineers to do their jobs better. As infrastructure engineers, part of our mission is to empower the rest of our engineering organization to use the tools we develop correctly, quickly, and independently. Yet we often fall short of that mission in unexpected ways. In this talk, I will explain ways to make concepts like interpolation, aggregation, and alerting more intuitive and how to identify pain points for new users. I'll go over common misconceptions users have about monitoring and how you can clear up this confusion with improved training and UI design.

Transcript

1. Amy Nguyen @amyngyn LISA17
   UX Design and Education for
   Effective Monitoring Tools
   

   View Slide

2. Amy Nguyen @amyngyn LISA17
   Hi!
   - I'm Amy!
   - Observability engineer!
   - Also on the plenary panel
   tomorrow at 4pm!
   - amynguyen.net
   @amyngyn
   

   View Slide

3. Amy Nguyen @amyngyn LISA17
   Agenda
   - Motivation
   - Sharing what you know
   - Designing what your users want
   - Recap
   

   View Slide

4. Amy Nguyen @amyngyn LISA17
   - Motivation
   - Sharing what you know
   - Designing what your users want
   - Recap
   Agenda
   

   View Slide

5. Amy Nguyen @amyngyn LISA17
   UX and Your Situation
   

   View Slide

6. Amy Nguyen @amyngyn LISA17
   UX and Your Situation
   team
   

   View Slide

7. Amy Nguyen @amyngyn LISA17
   UX and Your Situation
   team
   documentation
   

   View Slide

8. Amy Nguyen @amyngyn LISA17
   UX and Your Situation
   team
   documentation
   tools
   

   View Slide

9. Amy Nguyen @amyngyn LISA17
   UX and Your Situation
   team
   documentation
   tools
   probably paying for
   vendors TBH?
   

   View Slide

10. Amy Nguyen @amyngyn LISA17
    UX and Your Situation
    team
    documentation
    tools
    probably paying for
    vendors TBH?
    this talk
    

    View Slide

11. Amy Nguyen @amyngyn LISA17
    UX and Your Situation
    team
    documentation
    tools
    probably paying for
    vendors TBH?
    this talk
    things we can
    all control
    

    View Slide

12. Amy Nguyen @amyngyn LISA17
    Why should we care about
    user experience?
    

    View Slide

13. Amy Nguyen @amyngyn LISA17
    

    View Slide

14. Amy Nguyen @amyngyn LISA17
    

    View Slide

15. Amy Nguyen @amyngyn LISA17
    

    View Slide

16. Amy Nguyen @amyngyn LISA17
    Why should we care about user experience?
    

    View Slide

17. Amy Nguyen @amyngyn LISA17
    Why should we care about user experience?
    - Prevent misunderstandings: not everyone is (or should
    have to be) an expert at interpreting monitoring data
    

    View Slide

18. Amy Nguyen @amyngyn LISA17
    Why should we care about user experience?
    - Prevent misunderstandings: not everyone is (or should
    have to be) an expert at interpreting monitoring data
    - Developer velocity: help people reach conclusions
    faster, help your company move faster
    

    View Slide

19. Amy Nguyen @amyngyn LISA17
    Why should we care about user experience?
    - Prevent misunderstandings: not everyone is (or should
    have to be) an expert at interpreting monitoring data
    - Developer velocity: help people reach conclusions
    faster, help your company move faster
    - Data democracy: you don't know what questions
    people want to answer with their own data
    

    View Slide

20. Amy Nguyen @amyngyn LISA17
    Why should we care about user experience?
    - Prevent misunderstandings: not everyone is (or should
    have to be) an expert at interpreting timeseries data
    - Developer velocity: help people reach conclusions
    faster, help your company move faster
    - Data democracy: you don't know what questions
    people want to answer with their own data
    

    View Slide

21. Amy Nguyen @amyngyn LISA17
    Why should we care about user experience?
    Because we can help engineers work correctly, quickly,
    and independently.
    

    View Slide

22. Amy Nguyen @amyngyn LISA17
    "The fastest way to become a 10x engineer is
    to help 10 other engineers do their jobs better.
    - Wayne Gretzky"
    - Michael Scott
    

    View Slide

23. Amy Nguyen @amyngyn LISA17
    - Motivation
    - Sharing what you know
    - Designing what your users want
    - Recap
    Agenda
    

    View Slide

24. Amy Nguyen @amyngyn LISA17
    Sharing what you know
    

    View Slide

25. Amy Nguyen @amyngyn LISA17
    Sharing what you know
    1. Education vs intuition: Don't overload people with too
    much information.
    

    View Slide

26. Amy Nguyen @amyngyn LISA17
    stats.example.metric.errors > 5
    

    View Slide

27. Amy Nguyen @amyngyn LISA17
    

    View Slide

28. Amy Nguyen @amyngyn LISA17
    

    View Slide

29. Amy Nguyen @amyngyn LISA17
    Sharing what you know
    1. Education vs intuition: Don't overload people with too
    much information.
    2. Best practices: Use your expertise to determine the
    most helpful default behavior.
    

    View Slide

30. Amy Nguyen @amyngyn LISA17
    

    View Slide

31. Amy Nguyen @amyngyn LISA17
    Sharing what you know
    1. Education vs intuition: Don't overload people with too
    much information.
    2. Best practices: Use your expertise to determine the
    most helpful default behavior.
    3. Potential pitfalls: Make it hard to do the wrong thing.
    

    View Slide

32. Amy Nguyen @amyngyn LISA17
    

    View Slide

33. Amy Nguyen @amyngyn LISA17
    It looks like you're trying
    to alert on the most
    recent data.
    Are you sure you want
    to do that?
    

    View Slide

34. Amy Nguyen @amyngyn LISA17
    - Motivation
    - Sharing what you know
    - Designing what your users want
    - Recap
    Agenda
    

    View Slide

35. Amy Nguyen @amyngyn LISA17
    Designing what your users want
    

    View Slide

36. Amy Nguyen @amyngyn LISA17
    Designing what your users want
    1. Performance: Do whatever it takes to make it fast.
    

    View Slide

37. Amy Nguyen @amyngyn LISA17
    Performance: Cool Ideas!
    

    View Slide

38. Amy Nguyen @amyngyn LISA17
    Performance: Cool Ideas!
    - Backend
    

    View Slide

39. Amy Nguyen @amyngyn LISA17
    Performance: Cool Ideas!
    - Backend
    - Roll-up data over long time ranges
    

    View Slide

40. Amy Nguyen @amyngyn LISA17
    Performance: Cool Ideas!
    - Backend
    - Roll-up data over long time ranges
    - Store latest data in memory (e.g., Facebook's Gorilla paper and
    Beringei project)
    

    View Slide

41. Amy Nguyen @amyngyn LISA17
    Performance: Cool Ideas!
    - Backend
    - Roll-up data over long time ranges
    - Store latest data in memory (e.g., Facebook's Gorilla paper and
    Beringei project)
    - Add a cache layer (e.g., Turn's Splicer project)
    

    View Slide

42. Amy Nguyen @amyngyn LISA17
    Performance: Cool Ideas!
    - Backend
    - Roll-up data over long time ranges
    - Store latest data in memory (e.g., Facebook's Gorilla paper and
    Beringei project)
    - Add a cache layer (e.g., Turn's Splicer project)
    yeah sure get back to
    us in 6 months
    

    View Slide

43. Amy Nguyen @amyngyn LISA17
    Performance: Cool Ideas!
    - Backend
    - Roll-up data over long time ranges
    - Store latest data in memory (e.g., Facebook's Gorilla paper and
    Beringei project)
    - Add a cache layer (e.g., Turn's Splicer project)
    - Frontend
    

    View Slide

44. Amy Nguyen @amyngyn LISA17
    Performance: Cool Ideas!
    - Backend
    - Roll-up data over long time ranges
    - Store latest data in memory (e.g., Facebook's Gorilla paper and
    Beringei project)
    - Add a cache layer (e.g., Turn's Splicer project)
    - Frontend
    - Don't reload existing data if the user changes the time window
    

    View Slide

45. Amy Nguyen @amyngyn LISA17
    Performance: Cool Ideas!
    - Backend
    - Roll-up data over long time ranges
    - Store latest data in memory (e.g., Facebook's Gorilla paper and
    Beringei project)
    - Add a cache layer (e.g., Turn's Splicer project)
    - Frontend
    - Don't reload existing data if the user changes the time window
    - Prevent the user from requesting the data incessantly
    

    View Slide

46. Amy Nguyen @amyngyn LISA17
    Performance: Cool Ideas!
    - Backend
    - Roll-up data over long time ranges
    - Store latest data in memory (e.g., Facebook's Gorilla paper and
    Beringei project)
    - Add a cache layer (e.g., Turn's Splicer project)
    - Frontend
    - Don't reload existing data if the user changes the time window
    - Prevent the user from requesting the data incessantly
    - Lazy-load graphs on a dashboard
    

    View Slide

47. Amy Nguyen @amyngyn LISA17
    Designing what your users want
    1. Performance: Do whatever it takes to make it fast.
    2. Exploration: Make it easy to try things without fear.
    

    View Slide

48. Amy Nguyen @amyngyn LISA17
    don't mess with this
    

    View Slide

49. Amy Nguyen @amyngyn LISA17
    don't mess with this
    

    View Slide

50. Amy Nguyen @amyngyn LISA17
    don't mess with this
    or this
    

    View Slide

51. Amy Nguyen @amyngyn LISA17
    don't mess with this
    don't change it
    or this
    

    View Slide

52. Amy Nguyen @amyngyn LISA17
    don't mess with this
    don't change it
    or this
    It doesn't deserve to be treated this way
    it's just trying to help
    

    View Slide

53. Amy Nguyen @amyngyn LISA17
    Accessibility is for everyone
    

    View Slide

54. Amy Nguyen @amyngyn LISA17
    Accessibility is for everyone
    - Color contrast
    

    View Slide

55. Amy Nguyen @amyngyn LISA17
    Accessibility is for everyone
    - Color contrast
    

    View Slide

56. Amy Nguyen @amyngyn LISA17
    Accessibility is for everyone
    - Color contrast
    - Customizable color schemes
    

    View Slide

57. Amy Nguyen @amyngyn LISA17
    Accessibility is for everyone
    - Color contrast
    - Customizable color schemes
    - Navigation bar that isn't entirely controlled by JavaScript
    

    View Slide

58. Amy Nguyen @amyngyn LISA17
    Accessibility is for everyone
    - Color contrast
    - Customizable color schemes
    - Navigation bar that isn't entirely controlled by JavaScript
    - Keyboard navigation
    

    View Slide

59. Amy Nguyen @amyngyn LISA17
    Accessibility is for everyone
    - Color contrast
    - Customizable color schemes
    - Navigation bar that isn't entirely controlled by JavaScript
    - Keyboard navigation
    - Obvious focus on what needs attention
    

    View Slide

60. Amy Nguyen @amyngyn LISA17
    Exploration through previews
    

    View Slide

61. Amy Nguyen @amyngyn LISA17
    Exploration through previews: terraform plan
    

    View Slide

62. Amy Nguyen @amyngyn LISA17
    Exploration through previews: alert previews
    

    View Slide

63. Amy Nguyen @amyngyn LISA17
    Exploration recap
    1. Don't break basic browser features.
    2. Design for accessibility.
    3. Make it easy to try things without making permanent
    changes.
    

    View Slide

64. Amy Nguyen @amyngyn LISA17
    Designing what your users want
    1. Performance: Do whatever it takes to make it fast.
    2. Exploration: Make it easy to try things without fear.
    3. Simplicity: Make it easy to figure out what to do.
    

    View Slide

65. Amy Nguyen @amyngyn LISA17
    

    View Slide

66. Amy Nguyen @amyngyn LISA17
    

    View Slide

67. Amy Nguyen @amyngyn LISA17
    

    View Slide

68. Amy Nguyen @amyngyn LISA17
    manually type in your metric if
    you know the name somehow
    

    View Slide

69. Amy Nguyen @amyngyn LISA17
    manually type in your metric if
    you know the name somehow
    aggregator
    

    View Slide

70. Amy Nguyen @amyngyn LISA17
    manually type in your metric if
    you know the name somehow
    aggregator hope you remember what tags are
    available for this metric lol
    

    View Slide

71. Amy Nguyen @amyngyn LISA17
    manually type in your metric if
    you know the name somehow
    aggregator
    secret bonus:
    you can downsample?!
    hope you remember what tags are
    available for this metric lol
    

    View Slide

72. Amy Nguyen @amyngyn LISA17
    manually type in your metric if
    you know the name somehow
    aggregator
    secret bonus:
    you can downsample?!
    hope you remember what tags are
    available for this metric lol
    

    View Slide

73. Amy Nguyen @amyngyn LISA17
    

    View Slide

74. Amy Nguyen @amyngyn LISA17
    available options
    you probably don't
    need to touch
    

    View Slide

75. Amy Nguyen @amyngyn LISA17
    manual entry
    for power users
    

    View Slide

76. Amy Nguyen @amyngyn LISA17
    relevant information
    needed to create a query!
    

    View Slide

77. Amy Nguyen @amyngyn LISA17
    - Motivation
    - Sharing what you know
    - Designing what your users want
    - Recap
    Agenda
    

    View Slide

78. Amy Nguyen @amyngyn LISA17
    "The fastest way to become a 10x engineer is
    to help 10 other engineers do their jobs better.
    - Wayne Gretzky"
    - Michael Scott
    

    View Slide

79. Amy Nguyen @amyngyn LISA17
    UX and Your Situation
    team
    documentation
    tools
    probably paying for
    vendors TBH?
    this talk
    things we can
    all control
    

    View Slide

80. Amy Nguyen @amyngyn LISA17
    Sharing what you know
    1. Education vs intuition: Don't overload people with too
    much information.
    2. Best practices: Use your expertise to determine the
    most helpful default behavior.
    3. Potential pitfalls: Make it hard to do the wrong thing.
    

    View Slide

81. Amy Nguyen @amyngyn LISA17
    Designing what your users want
    1. Performance: Do whatever it takes to make it fast.
    2. Exploration: Make it easy to try things without fear.
    3. Simplicity: Make it easy to figure out what to do.
    

    View Slide

82. Amy Nguyen @amyngyn LISA17
    Thanks!
    (also check out the plenary panel! 4pm tomorrow!)
    

    View Slide