Release It!

I read this book by Michael Nygard, and here are some of my takes.

1. Release It! Design and Deploy Production-Ready Software Jake Trent 18 Aug 2009 by Michael T. Nygard
2. The Book Is thoughtful reading •Prompt evaluation: ◦Self ◦Project ◦Company •Pragprog.com
3. Motivation For quality, production-ready software •Go home at night w/o calls to your cell phone •Avoid un-needed cost: ◦Down time costs ◦Opportunity costs ◦Operational costs ◦Legal costs •Respectable work
4. The Need For periodic reminder of these issues •Code that passes QA: ◦Can still fail miserably ◦Can still give users bad impressions ◦Can still inflict avoidable costs •Problems will happen
5. Stability Antipatterns
6. Integration Points Antipattern •Modern display of coupling: Systems talking to systems •Every contact point is a possible failure point •Things not under your control ◦Network reliability ◦Data availability ◦External system correctness
7. Chain Reactions Antipattern •One failure often triggers another •Resource availability often the catalyst •Turn into the system attacking itself
8. Cascading Failures Antipattern •Failure in one layer causes problems in callers •Insufficiently paranoid integration points
9. Users Antipattern •"Users of a system have this knack for creative destruction." •Each user consumes more memory •Some are a burden, others plain malicious
10. Blocked Threads Antipattern •"Adding complexity to solve on problem creates the risk of entirely new failure modes." •Resource pool contention •Beware 3rd party API •Timeouts
11. Attacks of Self-Denial Antipattern •Plan for your own success •"Good marketing can kill you at any time."
12. Scaling Effects Antipattern •Horizontal scale communication •Shared resource bottleneck
13. Unbalanced Capacities Antipattern •Performance will depend on your most constrained resource •Not often discovered by QA •Consider proportions of types of transactions
14. Slow Responses Antipattern •Better to fail fast than to hog resources only to eventually fail
15. SLA Inversion Antipattern •"When calling third parties, service levels only decrease." •Consider real need and real cost •Service level can only be as high as the lowest subsystem
16. Unbounded Result Sets Antipattern •Test uses unrealistically small data sets •Use limits on all queries
17. Stability Patterns
18. Use Timeouts Pattern •Prevent integration points from becoming blocked threads •Retry for potential transient timeouts •Ability to move on without return (fail fast)
19. Circuit Breaker Pattern •Prevent operations rather than re-execute them •Note each failure until switch is flipped •Use with timeouts - try again eventually •Visible to operations
20. Bulkheads Pattern •Find natural partitions ◦Thread groups ◦Resource pools ◦Hardware
21. Steady State Pattern •System should run w/o manual intervention •Human fiddling leads to error •Purge data •Roll logs •At least move out of production environment
22. Fail Fast Pattern •Check availability before attempted use •Basic parameter checking before loading expensive objects •"Don't do useless work"
23. Handshaking Pattern •Allow integration points to throttle themselves
24. Test Harness Pattern •Box independent of the "norms" of the environment •As devious as possible, esp at network level •Out-of-spec •Stress
25. Decoupling Middleware Pattern •Decide on the plumbing at the "last responsible moment" •Hardest to change later
26. "Paranoia is just good thinking." "It's unlikely that anyone will notice your system's lack of downtime." Michal T. Nygard
27. Das Ende aprilandjake.com/content/release-it-stability-review/