Berlin 2013 - Session - Ryan Smith

Transcript

1. Predictable Failure Building systems that fail in predictable ways.
2. Failure is the inability to handle failure.
3. Southern Airways Flight 242
5. What went wrong • Radar misguided pilots into the storm • Pilots applied thrust to a stalling engine • Close landing field was not suggested
6. How will the system fail?
7. How will Redis fail?
9. British Airways Flight 009
10. What went wrong • All engines failed • St. Elmos fire but nothing on the radar • Radio to tower was not 100% • 1st officers oxygen mask broke
11. What happens when the system fails
12. No Redundancy - Simple func Query(c Conn, query string) Result { return c.DoQuery(query) }
13. Redundancy - Complex func Query(conns []Conn, query string) Result { ch := make(chan Result, len(conns)) for _, conn := range conns { go func(c Conn) { ch <- c.DoQuery(query): }(conn) } return <-ch }
14. Independence
15. Same function Different implementation
16. How many redundancies?
17. A component that just works
18. Dormant Failures
19. Don't wait until disaster strikes to find out that Your secondary RDMS has a full disk
20. Propagation
21. Danke! @ryandotsmith