Berlin 2013 - Session - Ryan Smith

Transcript

1. Predictable Failure Building systems that fail in predictable ways.

2. Failure is the inability to handle failure.

3. Southern Airways Flight 242

4. None

5. What went wrong • Radar misguided pilots into the storm

   • Pilots applied thrust to a stalling engine • Close landing field was not suggested

6. How will the system fail?

7. How will Redis fail?

8. None

9. British Airways Flight 009

10. What went wrong • All engines failed • St. Elmos

    fire but nothing on the radar • Radio to tower was not 100% • 1st officers oxygen mask broke

11. What happens when the system fails

12. No Redundancy - Simple func Query(c Conn, query string) Result

    { return c.DoQuery(query) }

13. Redundancy - Complex func Query(conns []Conn, query string) Result {

    ch := make(chan Result, len(conns)) for _, conn := range conns { go func(c Conn) { ch <- c.DoQuery(query): }(conn) } return <-ch }

14. Independence

15. Same function Different implementation

16. How many redundancies?

17. A component that just works

18. Dormant Failures

19. Don't wait until disaster strikes to find out that Your

    secondary RDMS has a full disk

20. Propagation

21. Danke! @ryandotsmith