Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Performance and Maintainability with Continuous...
Search
Sponsored
·
Ship Features Fearlessly
Turn features on and off without deploys. Used by thousands of Ruby developers.
→
Seth Walker
April 24, 2014
Technology
1.4k
2
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Performance and Maintainability with Continuous Experimentation
Seth Walker
April 24, 2014
Transcript
90 slides
Performance and Maintainability with Continuous Experimentation @sethwalker sethwalker.me/talks/continuous-experimentation
@sethwalker Seth Walker Allison McKnight Jonathan Klein @sethwalker @aemcknig @jonathanklein
Daniel Espeset @danielespeset @dxna Daniel Na Lara Swanson @laraswanson
None
@sethwalker
@sethwalker 175 committers
@sethwalker 100 committers touch CSS & JS
@sethwalker
@sethwalker Confidence?
Operations
@sethwalker Feature Flags
@sethwalker $cfg['save_everything'] = 'staff'; // ... snip ... if (isEnabled('save_everything'))
{ writeToDB($everything); } else { throwAway($everything); }
@sethwalker $cfg['save_everything'] = '1%'; // ... snip ... if (isEnabled('save_everything'))
{ writeToDB($everything); } else { throwAway($everything); }
@sethwalker $cfg['save_everything'] = '5%'; // ... snip ... if (isEnabled('save_everything'))
{ writeToDB($everything); } else { throwAway($everything); }
@sethwalker $cfg['save_everything'] = '1%'; // ... snip ... if (isEnabled('save_everything'))
{ writeToDB($everything); } else { throwAway($everything); }
@sethwalker $cfg['save_everything'] = '5%'; // ... snip ... if (isEnabled('save_everything'))
{ writeToDB($everything); } else { throwAway($everything); }
@sethwalker $cfg['save_everything'] = '10%'; // ... snip ... if (isEnabled('save_everything'))
{ writeToDB($everything); } else { throwAway($everything); }
@sethwalker $cfg['jquery_1_11'] = 'staff'; // ... snip ... if (isEnabled("jquery_1_11"))
{ <script src="/js/lib/jquery-1.11.0.js"></script> } else { <script src="/js/lib/jquery-1.8.2.js"></script> }
@sethwalker $cfg['jquery_1_11'] = '5%'; // ... snip ... if (isEnabled("jquery_1_11"))
{ <script src="/js/lib/jquery-1.11.0.js"></script> } else { <script src="/js/lib/jquery-1.8.2.js"></script> }
@sethwalker $cfg['jquery_1_11'] = '50%'; // ... snip ... if (isEnabled("jquery_1_11"))
{ <script src="/js/lib/jquery-1.11.0.js"></script> } else { <script src="/js/lib/jquery-1.8.2.js"></script> }
None
None
None
None
None
@sethwalker $cfg['jquery_1_11'] = '5%'; // ... snip ... if (isEnabled("jquery_1_11"))
{ <script src="/js/lib/jquery-1.11.0.js"></script> } else { <script src="/js/lib/jquery-1.8.2.js"></script> }
None
None
None
None
@sethwalker $cfg['jquery_1_11'] = '50%'; // ... snip ... if (isEnabled("jquery_1_11"))
{ <script src="/js/lib/jquery-1.11.0.js"></script> } else { <script src="/js/lib/jquery-1.8.2.js"></script> }
@sethwalker $cfg['jquery_1_11'] = '100%'; // ... snip ... if (isEnabled("jquery_1_11"))
{ <script src="/js/lib/jquery-1.11.0.js"></script> } else { <script src="/js/lib/jquery-1.8.2.js"></script> }
None
@sethwalker
@sethwalker window.Etsy.Context = { // ... snip ... "feature":{ "conversations.rejuvination":true,
"activity.dynamic_load_on_scroll":true, "overlays.buttery_smooth":true, "collections.perf_render_fix":true, } };
@sethwalker if (Etsy.Context.feature["overlays.buttery_smooth"]) { doSmooth(); } else { doJanky(); }
@sethwalker TODO: Feature Flags for the asset pipeline
@sethwalker $cfg['js_debug_build'] = 'staff'; // ... snip ... if (isEnabled("js_debug_build"))
{ <script src="/js-debug/main.js"></script> } else { <script src="/js/main.js"></script> }
@sethwalker + Analytics
Continuous Experimentation
@sethwalker Small, measurable changes Dan McKinley - http://mcfunley.com/design-for-continuous-experimentation
@sethwalker Validate hypotheses
@sethwalker Invalidate hypotheses
None
None
@sethwalker
None
:(
@sethwalker 100+ active flags
@sethwalker 60+ active experiments
@sethwalker 14 experimental iterations on listing page redesign
None
None
None
None
None
None
None
None
None
None
None
None
None
@sethwalker N inactive experiments
@sethwalker N inactive experiments N > 0
@sethwalker Cruft
@sethwalker if (isEnabled("new_listing_page")) { <script src="/js/listing/new-listing.js"></script> } if (isEnabled("new_favorite_button")) {
<script src="/js/new-favorites.js"></script> } if (isEnabled("cart2")) { <script src="/js/add-to-cart2.js"></script> }
@sethwalker $cfg["new_listing_page"] = "100%"; $cfg["new_favorite_button"] = "5%"; $cfg["cart2"] = "50%";
@sethwalker define("foo", ["logger"], function(Logger) { Logger.warn("foo is deprecated!"); doFoo(); });
@sethwalker <link rel="stylesheet" href="/css/base.css" /> <link rel="stylesheet" href="/css/listing.css" /> if
(isEnabled("new_listing_page")) { <link rel="stylesheet" href="/css/new-listing.css" /> } if (isEnabled("new_favorite_button")) { <link rel="stylesheet" href="/css/new-favorites.css" /> } if (isEnabled("cart2")) { <link rel="stylesheet" href="/css/add-to-cart2.css" /> }
@sethwalker Let's clean up!
Fear
@sethwalker Throw away
@sethwalker Throw away ability
@sethwalker Design for Throwaway-ability Bill Scott - http://www.looksgoodworkswell.com/the-experimentation-layer/
@sethwalker Tools
@sethwalker Needs More Visibility
None
None
None
None
None
None
None
@sethwalker selectorText = document.styleSheets[i].cssRules[j].selectorText; selected = document.querySelector(selectorText); results.push({ selector: selectorText,
found: !!selected ? 1 : 0}); // ... snip ... log(results);
@sethwalker Existing code - Seen in the wild = SUSPECT
@sethwalker Instrumentation
@sethwalker Increasing confidence
@sethwalker Adapting tools and process
Thanks! @sethwalker sethwalker.me/talks/continuous-experimentation
More Decks by Seth Walker
See All by Seth Walker
A Public Commitment to Performance
sethwalker
0
3.8k
Other Decks in Technology
See All in Technology
自動計装なしでECSサービスをApplication Signalsに載せるまで
ota1022
2
460
専門家に使ってもらうAIプロダクトをつくる - 病院の医事課向けDPCコーディング支援システムの設計と評価
hagino3000
4
1k
ノードが消える前提で設計する!Karpenterで金融システムを止めないためのノウハウ大全
daitak
1
220
HTTP通信に使おうとすると実はめんどくさいfile_get_contents
maguroalternative
0
250
Is the Website Dying?
miriamschwab
0
120
AWS DevOps Agentの構造から読み解く運用 AI エージェントの設計原則/aws-devops-agent-design-principles
tomoki10
0
240
Web Performance Is a Journey [PerfNow 2026]
tammyeverts
2
120
みん強Lightning! データ基盤ひとりでできるかな
endonanana
0
170
パスワードのない鍵で、 データを手元に置く
ginjihayashi
0
400
AWS re:Invent 2026 FSI Pre-Event Networking Night
junjikoide
0
140
Gov-JAWS#9 統合監視機能のマネージドサービス化+AIによる障害時自動調査
fj_yoshimura_kosuke
0
230
[関西Kaggler会 20261113] 続・コミュニティコンペのススメ
nejumi
0
190
Featured
See All Featured
HDC tutorial
michielstock
2
1k
Paper Plane (Part 1)
katiecoart
PRO
3
12k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Designing for Timeless Needs
cassininazir
1
570
sira's awesome portfolio website redesign presentation
elsirapls
0
520
Music & Morning Musume
bryan
48
7.5k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
31
3.4k
Navigating Algorithm Shifts & AI Overviews - #SMXNext
aleyda
1
1.7k
Why Mistakes Are the Best Teachers: Turning Failure into a Pathway for Growth
auna
0
360
Marketing Yourself as an Engineer | Alaka | Gurzu
gurzu
0
390
Getting science done with accelerated Python computing platforms
jacobtomlinson
2
580
Highjacked: Video Game Concept Design
rkendrick25
PRO
2
590