Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Resilient Software Design
Swanand Pagnis
January 21, 2015
Technology
1
200
Resilient Software Design
Lessons learned from Michal Nygard's "Release It!"
Swanand Pagnis
January 21, 2015
Tweet
Share
More Decks by Swanand Pagnis
See All by Swanand Pagnis
The Road to Continuous Deployments
swanandp
0
280
Fearless Joins
swanandp
0
190
The Deep Work Divide: Restoring the balance between collaboration and deep work
swanandp
0
660
The Well Rounded Engineer
swanandp
21
8.2k
The Deep Work Mentality
swanandp
5
1.5k
Combinator Parsing
swanandp
0
410
It's 2017, and I still want to sell you a graph database
swanandp
2
560
Architecture Over Framework: Rethink Your App Structure
swanandp
1
180
Let's get real (time): SSE, WebSockets and WebRTC for the soul
swanandp
2
730
Other Decks in Technology
See All in Technology
データベースの発表には RDBMS 以外もありますよ
maroon1st
0
230
cdk deployに必要な権限ってなんだ?
kinyok
0
160
マイクロサービス宣言から8年 振り返りとこれから / Eight Years After the Microservices Declaration A Look Back and A Look Ahead
eisuke
2
150
Exploring MapStore Release 2022.02: improved 3DTiles support and more
simboss
PRO
0
340
地方自治体業務あるある ーアナログ最適化編-
y150saya
1
250
2年で10→70人へ! スタートアップの 情報セキュリティ課題と施策
miekobayashi
1
470
オンプレk8sとEKSの並行運用の実際
ch1aki
0
260
Oktaの管理者権限を適切に移譲してみた
shimosyan
2
260
立ち止まっても、寄り道しても / even if I stop, even if I take a detour
katoaz
0
400
Cloudflare Workersで動くOG画像生成器
aiji42
1
490
OCIコンテナサービス関連の技術詳細 /oke-ocir-details
oracle4engineer
PRO
0
770
Hatena Engineer Seminar #23 「チームとプロダクトを育てる Mackerel 開発合宿」
arthur1
0
360
Featured
See All Featured
A designer walks into a library…
pauljervisheath
199
16k
Designing with Data
zakiwarfel
91
4.2k
The Mythical Team-Month
searls
210
40k
What's in a price? How to price your products and services
michaelherold
233
9.7k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
152
13k
StorybookのUI Testing Handbookを読んだ
zakiyama
8
3.2k
Design by the Numbers
sachag
271
18k
The Cult of Friendly URLs
andyhume
69
5.1k
How GitHub (no longer) Works
holman
298
140k
Faster Mobile Websites
deanohume
295
29k
It's Worth the Effort
3n
177
26k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
13
1.1k
Transcript
Resilient Software Design Building software that doesn’t give up!
1. What’s all this about? 2. Anti-patterns: What not to
do! 3. Patterns: Make your life easier.
Find the odd one 1. :cloud_factory 2. "CloudFactory" 3. CloudFactory
4. CLOUD_FACTORY
What will this code print? if fork puts "I won
the lottery!" else puts "I am bankrupt!" end
What’s all this about?
Stability
It Just Works™
It Should Work™
Bad things happen
Good things happen as well
But those are rare!
Anything that can go wrong, will go wrong. - Murphy’s
Law
Developers think positive
Too much so!
Be negative.
Our goal is to build software
Our goal is also to minimise pain
Our goal is also to save money
Resilient software saves money by not breaking when needed
Resilient software saves money by using optimum infrastructure
Resilient software saves money by keeping developers happy
Anti-patterns
1. Integration points
Integration is not what you think™
Database is an integration
Third party services are integration.
Your cache layer is an integration
None
Networks fail more often
Socket based protocols have a special way of failing
Refused connection is bad.
Hanged connection is worse.
Micro-services that talk to each other, will stop talking abruptly
2. Unbalanced Capacities
Specially applicable to micro-services
3. Slow responses
4. Unbounded Result Sets
Major anti-pattern, overlooked by many
What is the size of an HTTP cookie?
Patterns
1. Use timeouts!
What is the default timeout on Ruby’s net/http?
Now and forever, networks will always be unreliable. - Michael
T Nygard
Every network call in your system must have a timeout
This includes database calls
This includes API calls
This includes cache lookups
What to do when the timeout occurs depends on where
it occurred
Highly context specific, so the dev team should make that
decision
ProTip: Do not use Ruby’s “timeout” module
Instead, depend on libraries for the timeout
If you’re a library author, just use net/http’s timeout
2. Circuit Breaker
None
3. Bulkheads
A ship is divided into several water- tight compartments
In case there is a leakage in one section, water
doesn’t flood into other sections
Same principle!
Use resource pools
Use rate limiting
Consideration: Capacity
Bulkheading often conflicts increasing or variable capacity
Consideration: Performance
Bulkheading often results in slightly reduced performance
It’s worth it, trust me™
4. Fail fast
Remember guard clauses in Ruby?
# class Event def closest_event return unless self.location # …
# … end
Same principle!
Any Ruby libraries?
Not a lot :(
shopify/semian
Thank you!
Questions?
Swanand Pagnis Principal Engineer @ First
Swanand Pagnis @_swanand on Twitter
Swanand Pagnis @swanandp on GitHub