Promoting Chaos

Given @PagerDuty's Failure Friday meetup.

How we automate our failures in the cloud on the Reliability Engineering team at Netflix.

Transcript

1. Corey Bertram site reliability engineer ! cbertram@netflix.com
 @0x71
9. promoting chaos
10. reliability engineering
11. unreliability engineering
12. freedom responsibility &
13. anything that can go wrong
 will go wrong.
15. 923incidents
16. living in an unreliable
 world
17. plan for the worst
18. 40 million +
19. architecture d i s t r i b u t e d
22. XX,000 instance count
23. evolving
24. self
 service
25. static
26. testing
 :(
28. start small.
29. keep it
 simple.
30. log.
 everything.
31. alerting
32. trust
34. opt-in
35. runs
 daily
36. Chaos Gorilla
37. get. out.
38. …but
39. ELB ASG EC2
40. opt-out
41. room war
42. quarterly bi-weekly
43. christmas 2012
44. ELB wide region
45. welp :(
46. active active
48. active $$$$
49. Chaos Kong
50. good
 luck.
51. loldns
52. [gatekeeper]
53. testing
54. reality
55. 9999?
56. balance
57. keep it
 simple.
58. thanks! questions?