Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Resilient Software Design
Search
Swanand Pagnis
January 21, 2015
Technology
480
1
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Resilient Software Design
Lessons learned from Michal Nygard's "Release It!"
Swanand Pagnis
January 21, 2015
Transcript
77 slides
Resilient Software Design Building software that doesn’t give up!
1. What’s all this about? 2. Anti-patterns: What not to
do! 3. Patterns: Make your life easier.
Find the odd one 1. :cloud_factory 2. "CloudFactory" 3. CloudFactory
4. CLOUD_FACTORY
What will this code print? if fork puts "I won
the lottery!" else puts "I am bankrupt!" end
What’s all this about?
Stability
It Just Works™
It Should Work™
Bad things happen
Good things happen as well
But those are rare!
Anything that can go wrong, will go wrong. - Murphy’s
Law
Developers think positive
Too much so!
Be negative.
Our goal is to build software
Our goal is also to minimise pain
Our goal is also to save money
Resilient software saves money by not breaking when needed
Resilient software saves money by using optimum infrastructure
Resilient software saves money by keeping developers happy
Anti-patterns
1. Integration points
Integration is not what you think™
Database is an integration
Third party services are integration.
Your cache layer is an integration
None
Networks fail more often
Socket based protocols have a special way of failing
Refused connection is bad.
Hanged connection is worse.
Micro-services that talk to each other, will stop talking abruptly
2. Unbalanced Capacities
Specially applicable to micro-services
3. Slow responses
4. Unbounded Result Sets
Major anti-pattern, overlooked by many
What is the size of an HTTP cookie?
Patterns
1. Use timeouts!
What is the default timeout on Ruby’s net/http?
Now and forever, networks will always be unreliable. - Michael
T Nygard
Every network call in your system must have a timeout
This includes database calls
This includes API calls
This includes cache lookups
What to do when the timeout occurs depends on where
it occurred
Highly context specific, so the dev team should make that
decision
ProTip: Do not use Ruby’s “timeout” module
Instead, depend on libraries for the timeout
If you’re a library author, just use net/http’s timeout
2. Circuit Breaker
None
3. Bulkheads
A ship is divided into several water- tight compartments
In case there is a leakage in one section, water
doesn’t flood into other sections
Same principle!
Use resource pools
Use rate limiting
Consideration: Capacity
Bulkheading often conflicts increasing or variable capacity
Consideration: Performance
Bulkheading often results in slightly reduced performance
It’s worth it, trust me™
4. Fail fast
Remember guard clauses in Ruby?
# class Event def closest_event return unless self.location # …
# … end
Same principle!
Any Ruby libraries?
Not a lot :(
shopify/semian
Thank you!
Questions?
Swanand Pagnis Principal Engineer @ First
Swanand Pagnis @_swanand on Twitter
Swanand Pagnis @swanandp on GitHub
More Decks by Swanand Pagnis
See All by Swanand Pagnis
Principles of Project Management Flow Part 1 and 2
swanandp
0
400
The Road to Continuous Deployments
swanandp
0
760
Fearless Joins
swanandp
0
430
The Deep Work Divide: Restoring the balance between collaboration and deep work
swanandp
1
1.1k
The Well Rounded Engineer
swanandp
24
15k
The Deep Work Mentality
swanandp
7
2.4k
Combinator Parsing
swanandp
0
990
It's 2017, and I still want to sell you a graph database
swanandp
2
1.1k
Architecture Over Framework: Rethink Your App Structure
swanandp
1
320
Other Decks in Technology
See All in Technology
個人ASを取得して、自宅Kubernetes上のVMをグローバルIPで公開した話
munenick
0
210
Agent Routerで作るマルチテナント 推論基盤 #CNDW2026 / Building Multi-Tenant Inference Platform with Agent Router #CNDW2026
everpeace
4
560
お風呂場をサーバルームにしよう
yusuke427
0
380
MCPでネットワーク障害調査を自動化する前に直すべき3つのこと
yama3133
1
650
AWS Client VPNの設定でハマったポイントとその過程での気づき
amarelo_n24
1
420
攻撃と守りは何が変わり、何が変わらないのか~最近の不正アクセスから考える、リスク評価と標準の使いどころ
oidfj
2
270
OpenTelemetryで変わった 新時代のCloudWatchメトリクス監視
tsukuboshi
0
480
エンタープライズIAMについて - ID・権限をどう統制するか
oidfj
1
260
ノードが消える前提で設計する!Karpenterで金融システムを止めないためのノウハウ大全
daitak
1
170
認知の質も文化醸成も。数年取り組んで見えてきた技術広報のKPI設計 / Tech PR KPIs: Lessons on Awareness Quality and Culture After Several Years
bitkey
PRO
0
150
AIエージェントのアーキテクチャ設計
nsatoh
0
370
AWS Blocksで AIエージェント作れるの知ってます?
kmiya84377
1
130
Featured
See All Featured
Tell your own story through comics
letsgokoyo
1
1.1k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
23k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Self-Hosted WebAssembly Runtime for Runtime-Neutral Checkpoint/Restore in Edge–Cloud Continuum
chikuwait
0
960
RailsConf 2023
tenderlove
30
1.6k
The Illustrated Children's Guide to Kubernetes
chrisshort
51
53k
The Language of Interfaces
destraynor
162
27k
Heart Work Chapter 1 - Part 1
lfama
PRO
11
37k
Hiding What from Whom? A Critical Review of the History of Programming languages for Music
tomoyanonymous
3
1.5k
Measuring Dark Social's Impact On Conversion and Attribution
stephenakadiri
2
330
Embracing the Ebb and Flow
colly
88
5.2k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
33
5.9k