Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Making YouTube Fast (Velocity 2014)

Avatar for Alex Nicksay Alex Nicksay
June 26, 2014
Programming
4
290

Making YouTube Fast (Velocity 2014)

Each month on YouTube, almost an hour of video is watched for every person on Earth. This scale presents some unique challenges, but we’re also working on many of the same problems that most developers face. Getting all that video to viewers as soon as possible is especially important to us, so we’ve taken on an initiative to make YouTube fast.

In this talk, we’re going to present some of the projects we’ve been working on to speed up your viewing experience. In particular, we’ll introduce SPF (spfjs), a new HTML5 JS framework that now forms the foundation of YouTube navigation, and discuss the challenges of integrating that with an aggressive, bandwidth-hungry HTML5 video player.

Topics will include how we use and test the framework to implement dynamic page loads, persistent video players, partial page updates, and dynamic page and resource prefetching, while maintaining fast server-side content rendering at scale. We will discuss both the quantitative and qualitative improvements to the YouTube viewing experience and how you can achieve them.

Alex Nicksay @webdevjesus
Rick Viscomi @rick_viscomi
Steven Robertson @_srobertson

Presented at Velocity 2014 — June 26, 2014:
https://www.oreilly.com/library/view/velocity-conference-santa/9781491900383/oreillyvideos2068560.html

Learn more about SPF:
https://youtube.github.io/spfjs/
Avatar for Alex Nicksay

Alex Nicksay

June 26, 2014
Tweet

More Decks by Alex Nicksay

See All by Alex Nicksay
Faster Navigation at YouTube with SPF (October 2014)
Avatar for Alex Nicksay nicksay
0
440
Front-end Performance Improvements at YouTube (Velocity China 2010)
Avatar for Alex Nicksay nicksay
0
130

Other Decks in Programming

See All in Programming
SwiftDataのカスタムデータストアを試してみた
Avatar for 1mash0 1mash0
0
140
Ruby's Line Breaks
Avatar for yui-knk yui_knk
4
2.8k
音声プラットフォームのアーキテクチャ変遷から学ぶ、クラウドネイティブなバッチ処理 (20250422_CNDS2025_Batch_Architecture)
Avatar for Koki Senda thousanda
0
400
Jakarta EE Meets AI
Avatar for ivargrimstad ivargrimstad
0
800
AIコーディングの理想と現実
Avatar for Tomohisa Takaoka tomohisa
35
37k
Empowering Developers with HTML-Aware ERB Tooling @ RubyKaigi 2025, Matsuyama, Ehime
Avatar for Marco Roth marcoroth
2
960
プロフェッショナルとしての成長「問題の深掘り」が導く真のスキルアップ / issue-analysis-and-skill-up
Avatar for MinoDriven minodriven
8
1.9k
Laravel × Clean Architecture
Avatar for Takayuki bumptakayuki
PRO
0
140
Boost Your Performance and Developer Productivity with Jakarta EE 11
Avatar for ivargrimstad ivargrimstad
0
790
Memory API : Patterns, Performance et Cas d'Utilisation
Avatar for José josepaumard
1
170
20250429 - CNTUG Meetup #67 / DevOps Taiwan Meetup #69 - Deep Dive into Tetragon: Building Runtime Security and Observability with eBPF
Avatar for ChengHao Yang tico88612
0
170
個人開発の学生アプリが企業譲渡されるまで
Avatar for akidon0000 akidon0000
2
1.2k

Featured

See All Featured
Design and Strategy: How to Deal with People Who Don’t "Get" Design
Avatar for Morgane Peng morganepeng
129
19k
Rails Girls Zürich Keynote
Avatar for Gregor Martynus gr2m
94
13k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
227
22k
Practical Tips for Bootstrapping Information Extraction Pipelines
Avatar for Matthew Honnibal honnibal
PRO
19
1.2k
Docker and Python
Avatar for Tania Allard trallard
44
3.4k
JavaScript: Past, Present, and Future - NDC Porto 2020
Avatar for David Neal reverentgeek
47
5.4k
Speed Design
Avatar for Sergey Chernyshev sergeychernyshev
29
930
Bash Introduction
Avatar for André Augusto Costa Santos 62gerente
612
210k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
Avatar for Eileen M. Uchitelle eileencodes
23
2.7k
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
31
1.2k
Documentation Writing (for coders)
Avatar for Carmen Chung carmenintech
71
4.7k
4 Signs Your Business is Dying
Avatar for Josh Pigford shpigford
183
22k

Transcript

  1. +StevenRobertson @_srobertson Making YouTube Fast +AlexNicksay @webdevjesus +RickViscomi @rick_viscomi

  2. +AlexNicksay @webdevjesus Faster Navigation at YouTube with SPF

  3. data: http://www.youtube.com/yt/press/ image: http://commons.wikimedia.org/wiki/File:Earth_Western_Hemisphere.jpg ! Over 6 billion hours of

    video are watched each month on YouTube.

  4. image: ! That’s almost one hour for every person on

    Earth. !

  5. ! We want to get that video to you as

    fast as possible. !

  6. click play Last Year This Year Time to Playback Desktop

  7. click Last Year This Year Time to Playback play Desktop

  8. click Last Year This Year Time to Playback play Desktop

    40% faster

  9. Where did we start? ! !

  10. Where did we start? ! Real User Monitoring + Analysis

  11. click play Critical Stages of Playback

  12. fetch info init player load video click Critical Stages of

    Playback Desktop Last Year play

  13. Critical Stages of Playback init player

  14. What if we reused the same player? Critical Stages of

    Playback init player

  15. Critical Stages of Playback fetch info init player load video

    click play Desktop Last Year

  16. Critical Stages of Playback fetch info init player load video

    click play Desktop This Year

  17. Persistent = Less Work ! Less Work = Faster

  18. How did we do it? ! !

  19. How did we do it? ! A new JS framework

    for page navigation.

  20. Structured Page Fragments

  21. SPF ! !

  22. SPF is a lightweight framework for navigation and page updates.

  23. Initial: Static Loading

  24. Subsequent: Dynamic Loading with SPF

  25. What do we gain? ! !

  26. Static fast first render existing performance Benefits

  27. SPF Static Dynamic fast first render persistent interface existing performance

    same rendering code Benefits

  28. What do we gain? ! Better User Experience

  29. What do we gain? !

  30. What do we gain? !

  31. And development? ! !

  32. Use the same code for static + dynamic. Productivity

  33. Use the same code for static + dynamic. Productivity Use

    any server-side language + templates. Or, send data and use client-side templates. Flexibility

  34. How do we implement? ! !

  35. <a href="/destination">Go!</a> Standard Link Request

  36. <a class="spf-link" href="/destination">Go!</a> SPF Link Request

  37. <a href="/destination">Go!</a> Standard Link <a class="spf-link" href="/destination">Go!</a> SPF Link Progressive

    Enhancement Request class=“spf-link”

  38. <html> <head> <!-- Styles --> </head> <body> <div id="masthead">…</div> <div

    id="content"> <!-- Content --> </div> <div id="footer">…</div> <!-- Scripts --> </body> </html> StaticResponse.html Response

  39. { ! "head": "<!-- Styles -->", ! ! "body": {

    "content": "<!-- Content -->", }, ! "foot": "<!-- Scripts -->" ! } DynamicResponse.spf.json Response

  40. DynamicResponse.spf.json { ! "head": "<!-- Styles -->", ! ! "body":

    { "content": "<!-- Content -->", }, ! "foot": "<!-- Scripts -->" ! } <html> <head> <!-- Styles --> </head> <body> <div id="masthead">…</div> <div id="content"> <!-- Content --> </div> <div id="footer">…</div> <!-- Scripts --> </body> </html> StaticResponse.html Response

  41. But wait, there’s more… ! !

  42. Optional Features Custom Caching Reuse Responses Conditionally

  43. Custom Caching Prefetching Get Responses Before Requests Optional Features

  44. Custom Caching Multipart Responses Prefetching On-the-Fly Processing Optional Features

  45. Just how fast is it? ! !

  46. data: YouTube real user monitoring * data for playbacks without

    ads 2013-08-14 2014-05-14 Time to Playback Desktop Median

  47. data: YouTube real user monitoring * data for playbacks without

    ads Composite HTML5 Flash Time to Playback Desktop Median

  48. data: YouTube real user monitoring Time to Playback Desktop Median

    * data for playbacks without ads

  49. data: YouTube real user monitoring No Persistence Users Reported Site

    Felt Faster Time to Playback Desktop Median * data for playbacks without ads

  50. data: YouTube real user monitoring Video Pages: Chrome Only ~8.5%

    Improvement Time to Playback Desktop Median * data for playbacks without ads

  51. data: YouTube real user monitoring Video Pages: All Browsers ~9.0%

    Improvement Time to Playback Desktop Median * data for playbacks without ads

  52. data: YouTube real user monitoring On-the-Fly over HTTP ~5.0% Improvement

    Time to Playback Desktop Median * data for playbacks without ads

  53. data: YouTube real user monitoring Default HTML5: No Ads ~8.0%

    Improvement Time to Playback Desktop Median * data for playbacks without ads

  54. data: YouTube real user monitoring All Major Pages ~7.5% Improvement

    Time to Playback Desktop Median * data for playbacks without ads

  55. data: YouTube real user monitoring Default HTML5: Ads ~2.0% Improvement

    Time to Playback Desktop Median * data for playbacks without ads

  56. data: YouTube real user monitoring On-the-Fly over SPDY ~4.5% Improvement

    Time to Playback Desktop Median * data for playbacks without ads

  57. data: YouTube real user monitoring Time to Playback Desktop Median

    * data for playbacks without ads 40% faster

  58. data: YouTube real user monitoring Time to Playback Desktop Median

    * data for playbacks without ads Dynamic Navigation 40% faster

  59. data: YouTube real user monitoring Time to Playback Desktop Median

    * data for playbacks without ads Dynamic Navigation Persistent Player 40% faster

  60. data: YouTube real user monitoring Time to Playback Desktop Median

    * data for playbacks without ads Dynamic Navigation Persistent Player Default HTML5 40% faster

  61. data: YouTube real user monitoring Time to Playback Desktop Median

    * data for playbacks without ads Dynamic Navigation Persistent Player Default HTML5 On-the-Fly 40% faster

  62. What about “regular” pages? ! !

  63. data: YouTube real user monitoring 2013-08-14 2014-05-14 Time to Load

    Search Results Desktop Median

  64. data: YouTube real user monitoring Results Time to Load Search

    Results Desktop Median

  65. data: YouTube real user monitoring Time to Load Search Results

    Desktop Median

  66. data: YouTube real user monitoring Time to Load Search Results

    Desktop Median Video Pages Only

  67. data: YouTube real user monitoring Time to Load Search Results

    Desktop Median All Major Pages

  68. data: YouTube real user monitoring Time to Load Search Results

    Desktop Median Search Results ~22.5% Improvement

  69. data: YouTube real user monitoring Time to Load Search Results

    Desktop Median 22.5% faster

  70. What’s next?

  71. What’s next? Get the code github.com/youtube/spfjs

  72. +AlexNicksay @webdevjesus Thank you! #webperf #spfjs Get the code github.com/youtube/spfjs

  73. Web Performance Testing
 at YouTube +RickViscomi @rick_viscomi

  74. SPF

  75. None

  76. ! Only captures static navigation

  77. None

  78. logData 0 navigate http://www.youtube.com/ logData 1 navigate http://www.youtube.com/watch?v=ussCHoQttyQ WPT Script

    Flow View

  79. logData 0 navigate http://www.youtube.com/ logData 1 navigate http://www.youtube.com/watch?v=ussCHoQttyQ WPT Script

    Flow View

  80. logData 0 navigate http://www.youtube.com/ logData 1 navigate http://www.youtube.com/watch?v=ussCHoQttyQ WPT Script

    Flow View

  81. logData 0 navigate http://www.youtube.com/ logData 1 navigate http://www.youtube.com/watch?v=ussCHoQttyQ WPT Script

    Flow View

  82. logData 0 navigate http://www.youtube.com/ logData 1 navigate http://www.youtube.com/watch?v=ussCHoQttyQ WPT Script

    Flow View

  83. The cache is warm,
 but it’s still static navigation

  84. logData 0 navigate http://www.youtube.com/ logData 1 navigate http://www.youtube.com/watch?v=ussCHoQttyQ WPT Script

    Flow View

  85. WPT Script SPF Transition

  86. logData 0 navigate http://www.youtube.com/ logData 1 WPT Script SPF Transition

  87. logData 0 navigate http://www.youtube.com/ logData 1 /* click a link

    to a video */ WPT Script SPF Transition <a class="spf-link" href="/watch?v=dQw4w9WgXcQ">Go!</a>

  88. logData 0 navigate http://www.youtube.com/ logData 1 execAndWait WPT Script SPF

    Transition <a class="spf-link" href="/watch?v=dQw4w9WgXcQ">Go!</a>

  89. logData 0 navigate http://www.youtube.com/ logData 1 execAndWait (function(){ var a

    = document.querySelector(“a[href^=‘/watch?v']"); a.click(); })() WPT Script SPF Transition <a class="spf-link" href="/watch?v=dQw4w9WgXcQ">Go!</a>
  90. None

  91. logData 0 navigate http://www.youtube.com/ logData 1 execAndWait (function(){ var a

    = document.querySelector(“a[href^=‘/watch?v’]"); a.href = “/watch?v=ussCHoQttyQ"; a.click(); })() WPT Script SPF Transition <a class="spf-link" href="/watch?v=dQw4w9WgXcQ">Go!</a>

  92. http://www.webpagetest.org/result/140624_18_5Q5/

  93. http://www.webpagetest.org/result/140624_18_5Q5/ MISSION ACCOMPLISHED:
 Instrumenting dynamic navigations

  94. logData 0 navigate http://www.youtube.com/ logData 1 execAndWait (function(){ var a

    = document.querySelector(“a[href^=‘/watch?v’]"); a.href = “/watch?v=ussCHoQttyQ"; a.click(); })() WPT Script SPF Transition

  95. logData 0 navigate {starting page} logData 1 execAndWait (function(){ var

    a = document.querySelector(“a[href^=‘{ending page}’]”); a.href = “/watch?v={video ID}“; a.click(); })() WPT Script SPF Transition

  96. SPF No SPF No Ads Timecoded Watch Home Search Ads

  97. No SPF SPF No Ads Timecoded Watch Home Search Ads

  98. Chrome Firefox DSL Cable Watch Home Search IE Dial-up MISSION

    ACCOMPLISHED:
 Testing dynamic navigations

  99. Document Complete First Paint DOM Loaded On Load

  100. Video Content

  101. Playback Start

  102. performance.mark('aft'); main.js Application-Specific Metrics

  103. None

  104. v=2&s=youtube&action=watch,watch7_html5&yt_nav=1&yt_pft=none& docid=ussCHoQttyQ&ei=6Q2pU72mMuiI0QHcm4GgDQ&cpn=l2khPNrU oSFcVuQs&fmt=243&tds=4&cmt=0.067&yt_pt=html5&yt_lt=warm&e=9 10133,927625,930008,934030,949401&yt_ref=&yt_spf=1&yt_ad=0&yt_ pl=0&yt_li=0&srt=98&rt=nc0.1,nc1.56,cfg.56,gv.97,nc2.348,je.412,jl. 413,ol.426,tdl.723,fvb.999,plev.1370,vfp.1375,l2s.1378,pbs.1412,arr. 623,vrr.626,msa.667,vda.1278,vri.671,vdns.887,vreq.924,vrc.1274,ari. 668,adns.887,areq.924,fab.983,arc.1258,aft.1412&it=st.89,req.97,dns_. 27,tcp_.35,req_.72,rcv_.332

  105. Playback Start http://www.webpagetest.org/google/view_csi.php?test=140624_18_5Q5

  106. Playback Start http://www.webpagetest.org/google/view_csi.php?test=140624_18_5Q5 MISSION ACCOMPLISHED:
 Analyzing dynamic navigations

  107. Buy the book bit.ly/usingwebpagetest

  108. Thank you! Buy the book bit.ly/usingwebpagetest +RickViscomi @rick_viscomi

  109. Delivering Media for Interactive Applications +StevenRobertson @_srobertson

  110. Pick one.

  111. chrome://tracing chrome://net-internals

  112. OH RIGHT TRACING

  113. None

  114. TCP is affecting your app’s performance.

  115. Parallelize loading of small resources.

  116. … how about big ones?

  117. TCP slow start.

  118. None

  119. Exponential growth

  120. First packet loss

  121. Linear growth

  122. DOUBLE THE WOW

  123. None

  124. In the long run, parallelized loading can increase throughput…

  125. … at the cost of variability.

  126. Wireshark whenever you can.

  127. None

  128. Wireshark whenever you can.

  129. Which is basically never.

  130. You can get pretty far with progress events.

  131. Which is basically never.

  132. None
  133. None
  134. None

  135. Sometimes, things just suck.

  136. None
  137. None

  138. So we turned it off.

  139. Rebuffer stats got way worse (of course).

  140. Watch-time was neutral.

  141. One of the most divisive experiments we’ve run.

  142. None

  143. We cheated.

  144. Give users a way to show you how you could

    have done better.
  145. None

  146. Thank you! #end144p Check out the website youtube.com +StevenRobertson @_srobertson

  147. Thank you! #webperf #spfjs #end144p Buy the book bit.ly/usingwebpagetest +StevenRobertson

    @_srobertson +AlexNicksay @webdevjesus Get the SPF code github.com/youtube/spfjs Check out the website youtube.com +RickViscomi @rick_viscomi