Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Rapid Release at Massive Scale
Search
Proko Chou
November 06, 2017
Technology
0
79
Rapid Release at Massive Scale
The study of Facebook's release architect: Rapid Release at Massive Scale
Proko Chou
November 06, 2017
Tweet
Share
More Decks by Proko Chou
See All by Proko Chou
Software Testing
prokochou
0
70
Introduce ISTQB Test
prokochou
0
66
Context Driven Testing
prokochou
0
74
Mobile App Testing
prokochou
0
54
Mining Patterns from Lyrics of Chinese Popular Music
prokochou
0
150
Other Decks in Technology
See All in Technology
Node-REDのFunctionノードでMCPサーバーの実装を試してみた / Node-RED × MCP 勉強会 vol.1
you
PRO
0
110
Yamla: Rustでつくるリアルタイム性を追求した機械学習基盤 / Yamla: A Rust-Based Machine Learning Platform Pursuing Real-Time Capabilities
lycorptech_jp
PRO
3
120
PostgreSQL 18 cancel request key長の変更とRailsへの関連
yahonda
0
120
生成AI時代 文字コードを学ぶ意義を見出せるか?
hrsued
1
280
Observability infrastructure behind the trillion-messages scale Kafka platform
lycorptech_jp
PRO
0
140
「Chatwork」の認証基盤の移行とログ活用によるプロダクト改善
kubell_hr
1
150
Кто отправит outbox? Валентин Удальцов, автор канала Пых
lamodatech
0
340
AIエージェント最前線! Amazon Bedrock、Amazon Q、そしてMCPを使いこなそう
minorun365
PRO
14
5.1k
米国国防総省のDevSecOpsライフサイクルをAWSのセキュリティサービスとOSSで実現
syoshie
2
1.1k
急成長を支える基盤作り〜地道な改善からコツコツと〜 #cre_meetup
stefafafan
0
120
Postman AI エージェントビルダー最新情報
nagix
0
110
AWS Summit Japan 2025 Community Stage - App workflow automation by AWS Step Functions
matsuihidetoshi
1
260
Featured
See All Featured
Code Reviewing Like a Champion
maltzj
524
40k
Speed Design
sergeychernyshev
32
1k
Optimizing for Happiness
mojombo
379
70k
Scaling GitHub
holman
459
140k
Faster Mobile Websites
deanohume
307
31k
We Have a Design System, Now What?
morganepeng
53
7.7k
The Cult of Friendly URLs
andyhume
79
6.5k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
657
60k
Documentation Writing (for coders)
carmenintech
72
4.9k
Art, The Web, and Tiny UX
lynnandtonic
299
21k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Bash Introduction
62gerente
614
210k
Transcript
Rapid Release at Massive Scale 20171106 Proko
About Author
Agenda • Canary Deployment • The reasons why adopt rapid
release at massive scale • Solutions • Limitations
Canary Deployment
Canary Deployment (cont.) • Canaries were once regularly used in
coal mining as an early warning system • Distributed to a small number of users • As an early warning indicator with less impact on downtime
Canary Deployment • Deploy to one or more canary servers
• Test, or wait until satisfied • Deploy to the remaining servers
Why
Previous deployment process Three times a day Cherry-picks
Weekly Web Branch
Time Zone Issue Release duration: 14 hours every single week
London New York Menlo Park
How
Quasi-continuous “push from master” system
Tool and Process
Improve tools and process
Continuous Integration
Gatekeeper System
Test Tool Deployment (cont.)
Test Tool Deployment (cont.)
Test Tool Deployment (cont.)
Test Tool Deployment (cont.)
Test Tool Deployment
Continuous Delivery at Scale
Continuous Delivery at Scale (cont.)
Continuous Delivery at Scale: C1 (internal)
Canary Deployment Continuous Delivery at Scale: C2 (production)
Continuous Delivery at Scale: C3
Result
System
No More Hotfix
Deploy anytime, anywhere All engineers everywhere in the world can
develop and deliver their code when it makes sense for them.
Forcing Function
Make the user experience better, faster
Limitation
None
None
Android OS Version
Solution
What’s more • Use Puffin for Facebook (For Facebook) •
Add more checking point which can be executed automatically • The result can be checked by developers if it is failed
Reference 1. Article: https://code.facebook.com/posts/270314900139291/rapid-release-at-mas sive-scale/ 2. Video: https://atscaleconference.com/videos/rapid-release-at-massive-scale/ 3. Canary
Deployment: https://blog.ccjeng.com/2015/06/canary-deployment.html https://octopus.com/docs/patterns/canary-deployments
Q&A