Empowering Users with Site Reliability Engineering

Users of our systems care very little about the product or service itself and almost nothing about the underlying technology in which we, as engineers, invest our efforts. What they care about is realizing tasks that are important to them. They care about results. Results that improve their lives. By shifting our perspective to better align what users find important (that is, results that improve their lives), we as engineers can offer a more reliable and resilient system. We can create more successful users. Those who will not only remain long-term customers, but proponents and advisors to guide and grow the business. In this lecture we will look at what users really want. We will discuss ways in which we can adjust our monitoring efforts to obtain that information. And finally, we will talk about how we can create a valid feedback loop to provide insight into how we can fulfill those desires.

1. NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH NEW TECH 1/99
2. ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES ACTIVITIES Photo by Andre Hunter on Unsplash 2/99
3. SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? SERVERLESS? 3/99
4.             4/99
5. KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES KUBERNETES                   Photo by Lee Campbell on Unsplash 5/99
6. Photo by Thought Catalog on Unsplash 6/99
7. Arti cial INTELLIGENCE 7/99
8. 8/99
9. 9/99
10. Photo by bruce mars on Unsplash 10/99
11. HOW DID YOU LEARN ABOUT IT IT IT IT IT IT IT IT IT IT IT IT IT IT IT IT IT IT? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Photo by NESA by Makers on Unsplash 11/99
12. "yo. check this out." Photo by Oleg Magni on Unsplash 12/99
13. WHAT PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM did it solve? 13/99
14. I AM NOW BETTER BETTER BETTER BETTER BETTER BETTER BETTER BETTER BETTER BETTER BETTER BETTER BETTER BETTER BETTER BETTER BETTER BETTER AT... AT... AT... AT... AT... AT... AT... AT... AT... AT... AT... AT... AT... AT... AT... AT... AT... AT... 14/99
15. Photo by Glenn Carstens-Peters on Unsplash 15/99
16. How users feels about THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF THEMSELF? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Photo by Hammer & Tusk on Unsplash 16/99
17. DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? DO & BE? (What or how do we enable them?) 17/99
18. EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING EMPOWERING USERS THROUGH SITE RELIABILITY ENGINEERING 18/99
19. JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASONHAND MICROSOFT 19/99
20. 20/99
21. MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS MUSIC TO MY EARS                    I can't believe you were able to do that ... You are amazing! - My Current Best Friend 21/99
22. We share NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR NEW TECH WITH OUR FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS BECAUSE... 22/99
23. WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR WE LIKE OUR FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS FRIENDS Photo by Dai KE on Unsplash 23/99
24. 24/99
25. 25/99
26. 26/99
27. 27/99
28. BETTER VERSION OF ME ME ME ME ME ME ME ME ME ME ME ME ME ME ME ME ME ME 28/99
29. 29/99
30. 30/99
31. JEN WHOLNER (FASTLY) @ #SRECON 31/99
32. 32/99
33. PRODUCT MANAGEMENT 101 WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? WHAT? & WHY? WHY? WHY? WHY? WHY? WHY? WHY? WHY? WHY? WHY? WHY? WHY? WHY? WHY? WHY? WHY? WHY? WHY? 33/99
34. WHAT PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM PROBLEM does it solve? 34/99
35. RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 RULE #1 KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS KNOW YOUR USERS 35/99
36. UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND UNDERSTAND  THEM ..as best as possible.. 36/99
37. IT'S NOT NOT RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE RELIABLE if it can't deliver on the promise to enable user results 37/99
38. 38/99
39. MONITOR FOR WHAT USERS CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT CARE ABOUT 39/99
40. WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT DO THEY CARE ABOUT? 40/99
41. Photo by Thomas Jensen on Unsplash 41/99
42. 42/99
43.  Traditionally, we monitored our systems for problems related to... 43/99
44. KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWN KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS KNOWNS Logical cause & effect scenarios 44/99
45. Photo by Chris Liverani on Unsplash 45/99
46. We've all seen some WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF WEIRD STUFF 46/99
47.  47/99
48. ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE ADVICE FROM ME .. FOR ME  "Don't get too comfortable." - you 48/99
49. Like life , WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE WE KNOW SO LITTLE  ..  ..  below the surface of the ocean 49/99
50. UNKNOWN UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS UNKNOWNS  You don't know ... what you don't know! 50/99
51.                   + + + + + + + + + + + + + + + + + +                   = = = = = = = = = = = = = = = = = =                   51/99
52. Which has a bigger impact to BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? BUSINESS VALUE? 52/99
53. SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY SITE RELIABILITY ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING ENGINEERING An engineering discipline devoted to helping an organization sustainably achieve the appropriate level of reliability in their systems, services, and products. 53/99
54. RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY Availability Latency Coverage Throughput Correctness Fidelity Freshness Durability 54/99
55. RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY of BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE                   55/99
56. CREATING CREATING CREATING CREATING CREATING CREATING CREATING CREATING CREATING CREATING CREATING CREATING CREATING CREATING CREATING CREATING CREATING CREATING VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS VIRTUOUS FEEDBACK LOOPS 56/99
57. SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL SERVICE LEVEL INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS INDICATORS "how are things?" 57/99
58. CALCULATING THE SLI SLI SLI SLI SLI SLI SLI SLI SLI SLI SLI SLI SLI SLI SLI SLI SLI SLI 58/99
59. Over the course of ...                   AND Measured from ...                   59/99
60. AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY 60/99
61. 100,000 total requests 2,000 failed requests 98,000 successful requests 61/99
62. 98,000 98,000 98,000 98,000 98,000 98,000 98,000 98,000 98,000 98,000 98,000 98,000 98,000 98,000 98,000 98,000 98,000 98,000/ / / / / / / / / / / / / / / / / /100,000 100,000 100,000 100,000 100,000 100,000 100,000 100,000 100,000 100,000 100,000 100,000 100,000 100,000 100,000 100,000 100,000 100,000 62/99
63. 98 98 98 98 98 98 98 98 98 98 98 98 98 98 98 98 98 98/ / / / / / / / / / / / / / / / / /100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 63/99
64. 98% 98% 98% 98% 98% 98% 98% 98% 98% 98% 98% 98% 98% 98% 98% 98% 98% 98% AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY AVAILABILITY 64/99
65. AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI AVAILABILITY SLI 65/99
66. SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE SERVICE LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL LEVEL OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE OBJECTIVE 66/99
67. Photo by Raúl Nájera on Unsplash 67/99
68. 68/99
69.                   + + + + + + + + + + + + + + + + + +                   = = = = = = = = = = = = = = = = = =                   69/99
70. TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER TOTAL NUMBER OF ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS ABANDONED CARTS 70/99
71. REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE REASONABLE NUMBER OF ABANDONED CARTS in last 6 hours 71/99
72. YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE YOU MAY BE THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... THINKING... 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 72/99
73.                   BEEP BEEP BEEP BEEP BEEP BEEP BEEP BEEP BEEP BEEP BEEP BEEP BEEP BEEP BEEP BEEP BEEP BEEP 73/99
74.                   ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 74/99
75.                   DATA DRIVEN 75/99
76. ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS ABANDONDED CARTS 76/99
77.                   ARE WE DELIVERING ON THE VALUE? 77/99
78. RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY RELIABILITY of BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE BUSINESS VALUE                   78/99
79. ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ARE YOU WORRIED ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. ABOUT.. 79/99
80. A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER A COMPUTER  OR 80/99
81. YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS? YOUR USERS?  (read: Your business) 81/99
82. WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP KEEPS YOU UP AT AT AT AT AT AT AT AT AT AT AT AT AT AT AT AT AT AT NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? NIGHT? Photo by Niklas Hamann on Unsplash 82/99
83. WHAT ARE YOU CURRENTLY MONITORING? Photo by Victoria Heath on Unsplash 83/99
84. WHAT IS THE VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? VALUE? 84/99
85. WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A WHAT IS IT LIKE AS A USER USER USER USER USER USER USER USER USER USER USER USER USER USER USER USER USER USER? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 85/99
86. WHAT PARTS OF YOUR SYSTEM ARE LESS "OBSERVABLE"? 86/99
87. WHAT ENGINEERING EFFORT IS NEEDED TO FILL BLIND SPOTS? 87/99
88. RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE RESPONSE PLAN PLAN PLAN PLAN PLAN PLAN PLAN PLAN PLAN PLAN PLAN PLAN PLAN PLAN PLAN PLAN PLAN PLAN Photo by Connor Betts on Unsplash 88/99
89. WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S WHERE'S the  URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? URGENCY? 89/99
90. HOW? HOW? HOW? HOW? HOW? HOW? HOW? HOW? HOW? HOW? HOW? HOW? HOW? HOW? HOW? HOW? HOW? HOW? DO WE DO WE DO WE DO WE DO WE DO WE DO WE DO WE DO WE DO WE DO WE DO WE DO WE DO WE DO WE DO WE DO WE DO WE DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? DO THIS? 90/99
91. IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS IT DEPENDS                   91/99
92. EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT EXPERIMENT Reliability is everyone's job 92/99
93. ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS ASK QUESTIONS internally & externally 93/99
94. INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE INCREASE OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY OBSERVABILITY 94/99
95. ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH ESTABLISH SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO SLI/SLO 95/99
96. CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY CONTINOUSLY IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE IMPROVE 96/99
97.  I can't believe you were able to do that ... You are amazing! - Self 97/99
98. 98/99
99. JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASON HAND JASONHAND MICROSOFT 99/99