$30 off During Our Annual Pro Sale. View Details »
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
How Square Stays Up
Search
pui
July 18, 2012
Technology
3
280
How Square Stays Up
I talk I gave on the tools and processes Square uses to stay stable and available
pui
July 18, 2012
Tweet
Share
Other Decks in Technology
See All in Technology
その意思決定、まだ続けるんですか? ~痛みを超えて未来を作る、AI時代の撤退とピボットの技術~
applism118
45
25k
AI駆動開発2025年振り返りとTips集
knr109
1
140
Modern Data Stack大好きマンが語るSnowflakeの魅力
sagara
0
180
.NET 10 のパフォーマンス改善
nenonaninu
2
3.2k
ページの可視領域を算出する方法について整理する
yamatai1212
0
140
Introduction to Bill One Development Engineer
sansan33
PRO
0
320
Capture Checking / Separation Checking 入門
tanishiking
0
110
Kill the Vibe?Architecture in the age of AI
stoth
1
140
生成AIシステムとAIエージェントに関する性能や安全性の評価
shibuiwilliam
2
290
mablでリグレッションテストをデイリー実行するまで #mablExperience
bengo4com
0
440
Digital omtanke på Internetdagarna 2025
axbom
PRO
0
150
確実に伝えるHealth通知 〜半自動システムでほどよく漏れなく / JAWS-UG 神戸 #9 神戸へようこそ!LT会
genda
0
170
Featured
See All Featured
jQuery: Nuts, Bolts and Bling
dougneiner
65
8k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
132
19k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
Become a Pro
speakerdeck
PRO
30
5.7k
The Invisible Side of Design
smashingmag
302
51k
4 Signs Your Business is Dying
shpigford
186
22k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
9
1.1k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
48
9.8k
Writing Fast Ruby
sferik
630
62k
How GitHub (no longer) Works
holman
316
140k
Why Our Code Smells
bkeepers
PRO
340
57k
Side Projects
sachag
455
43k
Transcript
How Square Stays Up Tools and Processes Square Uses to
Maintain Stability and Availability
@pui_ling Erica Kwan
1 2 3 4 Developing Deploying Monitoring On-calling
Developing 1
We pair program (sometimes)
We solo, then get a code review (other times)
Why?
PCI Compliance Read all about it: http://en.wikipedia.org/wiki/Payment_Card_Industry_Data_Security_Standard
It is also good practice
git checkout -b topic-branch do work* git checkout master git
merge --no-ff topic-branch
A clean commit history helps
A super good git workflow: http://sandofsky.com/blog/git-workflow.html
git rebase --interactive
git rebase protip: config rebase.autosquash = true
git commit -m “squash! Monkeys”
pick 8374d8e Monkeys squash 8374d8e squash! Monkeys pick 259a7e6 Better
monkeys
Deploying 2
We deploy lots
but there are processes around deploys
Some history
We do canary deploys
None
Our full deploys do rolling restarts
And automatically run integration tests
Monitoring 3
We use common monitoring tools
We have application level checks
We have custom metrics dashboards
Graphite (whisper) + Cubism.js http://square.github.com/cubism/ http://d3js.org/ More info:
Horizon Graph http://vis.berkeley.edu/papers/horizon/
None
On-Calling 4
Engineers are responsible for their work
Ad-hoc at first
First real on-call rotations were simple
Original escalation path:
Engineer 1
Engineer 2
@jack
General on-call could not be responsible for everything
Now, every engineering team has an on-call rotation
Process is still evolving
Do these 4 things well all the time
@pui_ling /pui