Site Reliability Tips

Five Tips to getting started with Site Reliability Engineering.

Transcript

1. Jason Hand Senior Cloud Advocate jasonhand
2. @jasonhand | #Slush18 Devs Ops
3. #Slush18 @jasonhand | #Slush18
4. #Slush18 @jasonhand | #Slush18
5. Site Reliability Engineering @jasonhand | #Slush18
6. Site Reliability Engineering engineering appropriate @jasonhand | #Slush18
7. Site Reliability Engineering engineering appropriate reliability @jasonhand | #Slush18
8. Tips 5 Essential SRE @jasonhand | #Slush18
9. @jasonhand | #Slush18
10. Unknown @jasonhand | #Slush18
11. Tip#1 • Observability = Reality • Set Service Level Expectations • Measure & report against specific expectations @jasonhand | #Slush18
12. #Slush18 @jasonhand | #Slush18 Modern Monitoring
13. Observability Observability Monitoring @jasonhand | #Slush18
14. Expectations SLIs, SLOs, and SLAs … oh my! @jasonhand | #Slush18
16. Latency Traffic Errors Duration Saturation Utilization Rate The percentage of time a resource is in use The amount of work the resource must perform The number of failed requests The number of requests per second The amount of time to process a request The time it takes to service a request A measure of how much demand on the system @jasonhand | #Slush18
17. Service Level Indicator (SLI) (represented as a ratio or …% proportion) # of successful HTTP calls / # of HTTP calls # of operations that completed in < 10ms / # of operations # of “full quality responses” / # of responses # of records processed / # of records ratio X 100 = % proportion @jasonhand | #Slush18
18. @jasonhand | #Slush18
19. requests errors How many requests? Here is everything I know. @jasonhand | #Slush18
20. requests errors How many failed packets? Here is everything I know. NOT NEVER @jasonhand | #Slush18
21. Service Level Objective (SLO) HTTP requests (as reported by the load balancer) 95% 30-day (example) SLI @jasonhand | #Slush18
22. 9’s appropriate? 99.999% ? 99.99% ? 99.9% ? @jasonhand | #Slush18
23. / year / quarter / month / week / day / hour 99% 3.65 days 21.6 hours 7.2 hours 1.68 hours 14.4 minutes 36 seconds 99.9% 8.76 hours 2.16 minutes 43.2 seconds 10.1 minutes 1.44 minutes 3.6 seconds 99.99% 52.6 minutes 12.96 minutes 4.32 minutes 60.5 seconds 8.64 seconds 0.36 seconds 99.999% 5.26 minutes 1.30 minutes 25.9 seconds 6.05 seconds 0.87 seconds 0.04 seconds 9’s appropriate? @jasonhand | #Slush18
24. Error Budget Measurable, objective and self-policing mechanism to balance innovation versus reliability @jasonhand | #Slush18
25. Engineering REAL LIFE? @jasonhand | #Slush18
26. Game Days Using knowledge and structured plans routinely to rehearse incident response. Expanding and improving the current baseline. Prepare - Rehearse - Respond @jasonhand | #Slush18
27. Tip#2 • Improve Observability • Increase Mental Models • Test hypothesis in Production @jasonhand | #Slush18
28. @jasonhand | #Slush18 ü Reduce Unknown Unknowns ü Test Limitations of Systems
29. Tip#3 • PIRs for ALL customer facing disruptions • PIRs are blameless @jasonhand | #Slush18
30. Failure @jasonhand | #Slush18
31. post-incident review + psychological safety @jasonhand | #Slush18
32. Tip#4 @jasonhand | #Slush18
33. @jasonhand | #Slush18
34. Tip#5 Mind the silos @jasonhand | #Slush18
36. @jasonhand | #microsoftxslush
37. jhand.co/SREBook culture @jasonhand | #microsoftxslush
38. @jasonhand | #Slush18
39. Microsoft Learn @jasonhand | #microsoftxslush
40. |
41. Jason Hand Senior Cloud Advocate @jasonhand