Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Performance and Maintainability with Continuous...
Search
Seth Walker
April 24, 2014
Technology
2
1.3k
Performance and Maintainability with Continuous Experimentation
Seth Walker
April 24, 2014
Tweet
Share
More Decks by Seth Walker
See All by Seth Walker
A Public Commitment to Performance
sethwalker
0
3.7k
Other Decks in Technology
See All in Technology
AI人生苦節10年で会得したAIがやること_人間がやること.pdf
shibuiwilliam
1
250
Microsoft Learn MCP/Fabric データエージェント/Fabric MCP/Copilot Studio-簡単・便利なAIエージェント作ってみた -"Building Simple and Powerful AI Agents with Microsoft Learn MCP, Fabric Data Agent, Fabric MCP, and Copilot Studio"-
reireireijinjin6
1
210
【CEDEC2025】現場を理解して実現!ゲーム開発を効率化するWebサービスの開発と、利用促進のための継続的な改善
cygames
PRO
0
650
20250728 MCP, A2A and Multi-Agents in the future
yoshidashingo
1
190
地域コミュニティへの「感謝」と「恩返し」 / 20250726jawsug-tochigi
kasacchiful
0
120
AIに全任せしないコーディングとマネジメント思考
kikuchikakeru
0
380
生成AI時代におけるAI・機械学習技術を用いたプロダクト開発の深化と進化 #BetAIDay
layerx
PRO
1
790
AWS re:Inforce 2025 re:Cap Update Pickup & AWS Control Tower の運用における考慮ポイント
htan
1
140
【CEDEC2025】『ウマ娘 プリティーダービー』における映像制作のさらなる高品質化へ!~ 豊富な素材出力と制作フローの改善を実現するツールについて~
cygames
PRO
0
170
反脆弱性(アンチフラジャイル)とデータ基盤構築
cuebic9bic
2
140
AIエージェントを支える設計
tkikuchi1002
12
2.9k
「育てる」サーバーレス 〜チーム開発研修で学んだ、小さく始めて大きく拡張するAWS設計〜
yu_kod
1
230
Featured
See All Featured
Reflections from 52 weeks, 52 projects
jeffersonlam
351
21k
Rails Girls Zürich Keynote
gr2m
95
14k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
50
5.5k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
35
2.5k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Designing for humans not robots
tammielis
253
25k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.9k
Transcript
Performance and Maintainability with Continuous Experimentation @sethwalker sethwalker.me/talks/continuous-experimentation
@sethwalker Seth Walker Allison McKnight Jonathan Klein @sethwalker @aemcknig @jonathanklein
Daniel Espeset @danielespeset @dxna Daniel Na Lara Swanson @laraswanson
None
@sethwalker
@sethwalker 175 committers
@sethwalker 100 committers touch CSS & JS
@sethwalker
@sethwalker Confidence?
Operations
@sethwalker Feature Flags
@sethwalker $cfg['save_everything'] = 'staff'; // ... snip ... if (isEnabled('save_everything'))
{ writeToDB($everything); } else { throwAway($everything); }
@sethwalker $cfg['save_everything'] = '1%'; // ... snip ... if (isEnabled('save_everything'))
{ writeToDB($everything); } else { throwAway($everything); }
@sethwalker $cfg['save_everything'] = '5%'; // ... snip ... if (isEnabled('save_everything'))
{ writeToDB($everything); } else { throwAway($everything); }
@sethwalker $cfg['save_everything'] = '1%'; // ... snip ... if (isEnabled('save_everything'))
{ writeToDB($everything); } else { throwAway($everything); }
@sethwalker $cfg['save_everything'] = '5%'; // ... snip ... if (isEnabled('save_everything'))
{ writeToDB($everything); } else { throwAway($everything); }
@sethwalker $cfg['save_everything'] = '10%'; // ... snip ... if (isEnabled('save_everything'))
{ writeToDB($everything); } else { throwAway($everything); }
@sethwalker $cfg['jquery_1_11'] = 'staff'; // ... snip ... if (isEnabled("jquery_1_11"))
{ <script src="/js/lib/jquery-1.11.0.js"></script> } else { <script src="/js/lib/jquery-1.8.2.js"></script> }
@sethwalker $cfg['jquery_1_11'] = '5%'; // ... snip ... if (isEnabled("jquery_1_11"))
{ <script src="/js/lib/jquery-1.11.0.js"></script> } else { <script src="/js/lib/jquery-1.8.2.js"></script> }
@sethwalker $cfg['jquery_1_11'] = '50%'; // ... snip ... if (isEnabled("jquery_1_11"))
{ <script src="/js/lib/jquery-1.11.0.js"></script> } else { <script src="/js/lib/jquery-1.8.2.js"></script> }
None
None
None
None
None
@sethwalker $cfg['jquery_1_11'] = '5%'; // ... snip ... if (isEnabled("jquery_1_11"))
{ <script src="/js/lib/jquery-1.11.0.js"></script> } else { <script src="/js/lib/jquery-1.8.2.js"></script> }
None
None
None
None
@sethwalker $cfg['jquery_1_11'] = '50%'; // ... snip ... if (isEnabled("jquery_1_11"))
{ <script src="/js/lib/jquery-1.11.0.js"></script> } else { <script src="/js/lib/jquery-1.8.2.js"></script> }
@sethwalker $cfg['jquery_1_11'] = '100%'; // ... snip ... if (isEnabled("jquery_1_11"))
{ <script src="/js/lib/jquery-1.11.0.js"></script> } else { <script src="/js/lib/jquery-1.8.2.js"></script> }
None
@sethwalker
@sethwalker window.Etsy.Context = { // ... snip ... "feature":{ "conversations.rejuvination":true,
"activity.dynamic_load_on_scroll":true, "overlays.buttery_smooth":true, "collections.perf_render_fix":true, } };
@sethwalker if (Etsy.Context.feature["overlays.buttery_smooth"]) { doSmooth(); } else { doJanky(); }
@sethwalker TODO: Feature Flags for the asset pipeline
@sethwalker $cfg['js_debug_build'] = 'staff'; // ... snip ... if (isEnabled("js_debug_build"))
{ <script src="/js-debug/main.js"></script> } else { <script src="/js/main.js"></script> }
@sethwalker + Analytics
Continuous Experimentation
@sethwalker Small, measurable changes Dan McKinley - http://mcfunley.com/design-for-continuous-experimentation
@sethwalker Validate hypotheses
@sethwalker Invalidate hypotheses
None
None
@sethwalker
None
:(
@sethwalker 100+ active flags
@sethwalker 60+ active experiments
@sethwalker 14 experimental iterations on listing page redesign
None
None
None
None
None
None
None
None
None
None
None
None
None
@sethwalker N inactive experiments
@sethwalker N inactive experiments N > 0
@sethwalker Cruft
@sethwalker if (isEnabled("new_listing_page")) { <script src="/js/listing/new-listing.js"></script> } if (isEnabled("new_favorite_button")) {
<script src="/js/new-favorites.js"></script> } if (isEnabled("cart2")) { <script src="/js/add-to-cart2.js"></script> }
@sethwalker $cfg["new_listing_page"] = "100%"; $cfg["new_favorite_button"] = "5%"; $cfg["cart2"] = "50%";
@sethwalker define("foo", ["logger"], function(Logger) { Logger.warn("foo is deprecated!"); doFoo(); });
@sethwalker <link rel="stylesheet" href="/css/base.css" /> <link rel="stylesheet" href="/css/listing.css" /> if
(isEnabled("new_listing_page")) { <link rel="stylesheet" href="/css/new-listing.css" /> } if (isEnabled("new_favorite_button")) { <link rel="stylesheet" href="/css/new-favorites.css" /> } if (isEnabled("cart2")) { <link rel="stylesheet" href="/css/add-to-cart2.css" /> }
@sethwalker Let's clean up!
Fear
@sethwalker Throw away
@sethwalker Throw away ability
@sethwalker Design for Throwaway-ability Bill Scott - http://www.looksgoodworkswell.com/the-experimentation-layer/
@sethwalker Tools
@sethwalker Needs More Visibility
None
None
None
None
None
None
None
@sethwalker selectorText = document.styleSheets[i].cssRules[j].selectorText; selected = document.querySelector(selectorText); results.push({ selector: selectorText,
found: !!selected ? 1 : 0}); // ... snip ... log(results);
@sethwalker Existing code - Seen in the wild = SUSPECT
@sethwalker Instrumentation
@sethwalker Increasing confidence
@sethwalker Adapting tools and process
Thanks! @sethwalker sethwalker.me/talks/continuous-experimentation