Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Making YouTube Fast (Velocity 2014)

Alex Nicksay
June 26, 2014
Programming
4
270

Making YouTube Fast (Velocity 2014)

Each month on YouTube, almost an hour of video is watched for every person on Earth. This scale presents some unique challenges, but we’re also working on many of the same problems that most developers face. Getting all that video to viewers as soon as possible is especially important to us, so we’ve taken on an initiative to make YouTube fast.

In this talk, we’re going to present some of the projects we’ve been working on to speed up your viewing experience. In particular, we’ll introduce SPF (spfjs), a new HTML5 JS framework that now forms the foundation of YouTube navigation, and discuss the challenges of integrating that with an aggressive, bandwidth-hungry HTML5 video player.

Topics will include how we use and test the framework to implement dynamic page loads, persistent video players, partial page updates, and dynamic page and resource prefetching, while maintaining fast server-side content rendering at scale. We will discuss both the quantitative and qualitative improvements to the YouTube viewing experience and how you can achieve them.

Alex Nicksay @webdevjesus
Rick Viscomi @rick_viscomi
Steven Robertson @_srobertson

Presented at Velocity 2014 — June 26, 2014:
https://www.oreilly.com/library/view/velocity-conference-santa/9781491900383/oreillyvideos2068560.html

Learn more about SPF:
https://youtube.github.io/spfjs/

Alex Nicksay

June 26, 2014
Tweet

More Decks by Alex Nicksay

See All by Alex Nicksay
Faster Navigation at YouTube with SPF (October 2014)
nicksay
0
430
Front-end Performance Improvements at YouTube (Velocity China 2010)
nicksay
0
120

Other Decks in Programming

See All in Programming
Site Reliability Engineering for GMO
pyama86
8
1k
VSCodeでのDatabricks開発もお勧めしたい/I would also recommend Databricks development with VSCode.
kazumain
0
260
TYPO3 v13 – The road to LTS: What's new and new APIs
luisasofie_xoxo
0
210
スクラムガイドのスプリントレトロスペクティブを改めて読みかえしてみた / Re-reading the Sprint Retrospective Section in the Scrum Guide
mackey0225
3
430
検証も兼ねて個人開発でHonoとかと向き合った話
hanetsuki
1
1.1k
Snowflakeで眠ったデータを起こそう！
estie
0
120
Tailwind CSSを本気でカスタマイズする方法
fsubal
13
5.3k
大規模UIKitベースアプリへのTCAの段階的導入/gradual-adoption-of-tca-in-a-large-scale-uikit-based-app
takehilo
1
180
Node.js v22 で変わること
yosuke_furukawa
PRO
9
3.5k
if constexpr文はテンプレート世界のラムダ式である
faithandbrave
3
650
Hanami and htmx
bkuhlmann
0
210
Azure OpenAI Serviceのプロンプトエンジニアリング入門
tomokusaba
3
710

Featured

See All Featured
The Brand Is Dead. Long Live the Brand.
mthomps
49
29k
YesSQL, Process and Tooling at Scale
rocio
164
13k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
30
6k
Docker and Python
trallard
34
2.7k
Atom: Resistance is Futile
akmur
259
25k
StorybookのUI Testing Handbookを読んだ
zakiyama
13
4.6k
Fashionably flexible responsive web design (full day workshop)
malarkey
398
65k
Into the Great Unknown - MozCon
thekraken
10
990
How GitHub Uses GitHub to Build GitHub
holman
468
290k
Making the Leap to Tech Lead
cromwellryan
124
8.5k
Fireside Chat
paigeccino
21
2.6k
Unsuck your backbone
ammeep
663
57k

Transcript

  1. +StevenRobertson @_srobertson Making YouTube Fast +AlexNicksay @webdevjesus +RickViscomi @rick_viscomi

  2. +AlexNicksay @webdevjesus Faster Navigation at YouTube with SPF

  3. data: http://www.youtube.com/yt/press/ image: http://commons.wikimedia.org/wiki/File:Earth_Western_Hemisphere.jpg ! Over 6 billion hours of

    video are watched each month on YouTube.

  4. image: ! That’s almost one hour for every person on

    Earth. !

  5. ! We want to get that video to you as

    fast as possible. !

  6. click play Last Year This Year Time to Playback Desktop

  7. click Last Year This Year Time to Playback play Desktop

  8. click Last Year This Year Time to Playback play Desktop

    40% faster

  9. Where did we start? ! !

  10. Where did we start? ! Real User Monitoring + Analysis

  11. click play Critical Stages of Playback

  12. fetch info init player load video click Critical Stages of

    Playback Desktop Last Year play

  13. Critical Stages of Playback init player

  14. What if we reused the same player? Critical Stages of

    Playback init player

  15. Critical Stages of Playback fetch info init player load video

    click play Desktop Last Year

  16. Critical Stages of Playback fetch info init player load video

    click play Desktop This Year

  17. Persistent = Less Work ! Less Work = Faster

  18. How did we do it? ! !

  19. How did we do it? ! A new JS framework

    for page navigation.

  20. Structured Page Fragments

  21. SPF ! !

  22. SPF is a lightweight framework for navigation and page updates.

  23. Initial: Static Loading

  24. Subsequent: Dynamic Loading with SPF

  25. What do we gain? ! !

  26. Static fast first render existing performance Benefits

  27. SPF Static Dynamic fast first render persistent interface existing performance

    same rendering code Benefits

  28. What do we gain? ! Better User Experience

  29. What do we gain? !

  30. What do we gain? !

  31. And development? ! !

  32. Use the same code for static + dynamic. Productivity

  33. Use the same code for static + dynamic. Productivity Use

    any server-side language + templates. Or, send data and use client-side templates. Flexibility

  34. How do we implement? ! !

  35. <a href="/destination">Go!</a> Standard Link Request

  36. <a class="spf-link" href="/destination">Go!</a> SPF Link Request

  37. <a href="/destination">Go!</a> Standard Link <a class="spf-link" href="/destination">Go!</a> SPF Link Progressive

    Enhancement Request class=“spf-link”

  38. <html> <head> <!-- Styles --> </head> <body> <div id="masthead">…</div> <div

    id="content"> <!-- Content --> </div> <div id="footer">…</div> <!-- Scripts --> </body> </html> StaticResponse.html Response

  39. { ! "head": "<!-- Styles -->", ! ! "body": {

    "content": "<!-- Content -->", }, ! "foot": "<!-- Scripts -->" ! } DynamicResponse.spf.json Response

  40. DynamicResponse.spf.json { ! "head": "<!-- Styles -->", ! ! "body":

    { "content": "<!-- Content -->", }, ! "foot": "<!-- Scripts -->" ! } <html> <head> <!-- Styles --> </head> <body> <div id="masthead">…</div> <div id="content"> <!-- Content --> </div> <div id="footer">…</div> <!-- Scripts --> </body> </html> StaticResponse.html Response

  41. But wait, there’s more… ! !

  42. Optional Features Custom Caching Reuse Responses Conditionally

  43. Custom Caching Prefetching Get Responses Before Requests Optional Features

  44. Custom Caching Multipart Responses Prefetching On-the-Fly Processing Optional Features

  45. Just how fast is it? ! !

  46. data: YouTube real user monitoring * data for playbacks without

    ads 2013-08-14 2014-05-14 Time to Playback Desktop Median

  47. data: YouTube real user monitoring * data for playbacks without

    ads Composite HTML5 Flash Time to Playback Desktop Median

  48. data: YouTube real user monitoring Time to Playback Desktop Median

    * data for playbacks without ads

  49. data: YouTube real user monitoring No Persistence Users Reported Site

    Felt Faster Time to Playback Desktop Median * data for playbacks without ads

  50. data: YouTube real user monitoring Video Pages: Chrome Only ~8.5%

    Improvement Time to Playback Desktop Median * data for playbacks without ads

  51. data: YouTube real user monitoring Video Pages: All Browsers ~9.0%

    Improvement Time to Playback Desktop Median * data for playbacks without ads

  52. data: YouTube real user monitoring On-the-Fly over HTTP ~5.0% Improvement

    Time to Playback Desktop Median * data for playbacks without ads

  53. data: YouTube real user monitoring Default HTML5: No Ads ~8.0%

    Improvement Time to Playback Desktop Median * data for playbacks without ads

  54. data: YouTube real user monitoring All Major Pages ~7.5% Improvement

    Time to Playback Desktop Median * data for playbacks without ads

  55. data: YouTube real user monitoring Default HTML5: Ads ~2.0% Improvement

    Time to Playback Desktop Median * data for playbacks without ads

  56. data: YouTube real user monitoring On-the-Fly over SPDY ~4.5% Improvement

    Time to Playback Desktop Median * data for playbacks without ads

  57. data: YouTube real user monitoring Time to Playback Desktop Median

    * data for playbacks without ads 40% faster

  58. data: YouTube real user monitoring Time to Playback Desktop Median

    * data for playbacks without ads Dynamic Navigation 40% faster

  59. data: YouTube real user monitoring Time to Playback Desktop Median

    * data for playbacks without ads Dynamic Navigation Persistent Player 40% faster

  60. data: YouTube real user monitoring Time to Playback Desktop Median

    * data for playbacks without ads Dynamic Navigation Persistent Player Default HTML5 40% faster

  61. data: YouTube real user monitoring Time to Playback Desktop Median

    * data for playbacks without ads Dynamic Navigation Persistent Player Default HTML5 On-the-Fly 40% faster

  62. What about “regular” pages? ! !

  63. data: YouTube real user monitoring 2013-08-14 2014-05-14 Time to Load

    Search Results Desktop Median

  64. data: YouTube real user monitoring Results Time to Load Search

    Results Desktop Median

  65. data: YouTube real user monitoring Time to Load Search Results

    Desktop Median

  66. data: YouTube real user monitoring Time to Load Search Results

    Desktop Median Video Pages Only

  67. data: YouTube real user monitoring Time to Load Search Results

    Desktop Median All Major Pages

  68. data: YouTube real user monitoring Time to Load Search Results

    Desktop Median Search Results ~22.5% Improvement

  69. data: YouTube real user monitoring Time to Load Search Results

    Desktop Median 22.5% faster

  70. What’s next?

  71. What’s next? Get the code github.com/youtube/spfjs

  72. +AlexNicksay @webdevjesus Thank you! #webperf #spfjs Get the code github.com/youtube/spfjs

  73. Web Performance Testing
 at YouTube +RickViscomi @rick_viscomi

  74. SPF

  75. None

  76. ! Only captures static navigation

  77. None

  78. logData 0 navigate http://www.youtube.com/ logData 1 navigate http://www.youtube.com/watch?v=ussCHoQttyQ WPT Script

    Flow View

  79. logData 0 navigate http://www.youtube.com/ logData 1 navigate http://www.youtube.com/watch?v=ussCHoQttyQ WPT Script

    Flow View

  80. logData 0 navigate http://www.youtube.com/ logData 1 navigate http://www.youtube.com/watch?v=ussCHoQttyQ WPT Script

    Flow View

  81. logData 0 navigate http://www.youtube.com/ logData 1 navigate http://www.youtube.com/watch?v=ussCHoQttyQ WPT Script

    Flow View

  82. logData 0 navigate http://www.youtube.com/ logData 1 navigate http://www.youtube.com/watch?v=ussCHoQttyQ WPT Script

    Flow View

  83. The cache is warm,
 but it’s still static navigation

  84. logData 0 navigate http://www.youtube.com/ logData 1 navigate http://www.youtube.com/watch?v=ussCHoQttyQ WPT Script

    Flow View

  85. WPT Script SPF Transition

  86. logData 0 navigate http://www.youtube.com/ logData 1 WPT Script SPF Transition

  87. logData 0 navigate http://www.youtube.com/ logData 1 /* click a link

    to a video */ WPT Script SPF Transition <a class="spf-link" href="/watch?v=dQw4w9WgXcQ">Go!</a>

  88. logData 0 navigate http://www.youtube.com/ logData 1 execAndWait WPT Script SPF

    Transition <a class="spf-link" href="/watch?v=dQw4w9WgXcQ">Go!</a>

  89. logData 0 navigate http://www.youtube.com/ logData 1 execAndWait (function(){ var a

    = document.querySelector(“a[href^=‘/watch?v']"); a.click(); })() WPT Script SPF Transition <a class="spf-link" href="/watch?v=dQw4w9WgXcQ">Go!</a>
  90. None

  91. logData 0 navigate http://www.youtube.com/ logData 1 execAndWait (function(){ var a

    = document.querySelector(“a[href^=‘/watch?v’]"); a.href = “/watch?v=ussCHoQttyQ"; a.click(); })() WPT Script SPF Transition <a class="spf-link" href="/watch?v=dQw4w9WgXcQ">Go!</a>

  92. http://www.webpagetest.org/result/140624_18_5Q5/

  93. http://www.webpagetest.org/result/140624_18_5Q5/ MISSION ACCOMPLISHED:
 Instrumenting dynamic navigations

  94. logData 0 navigate http://www.youtube.com/ logData 1 execAndWait (function(){ var a

    = document.querySelector(“a[href^=‘/watch?v’]"); a.href = “/watch?v=ussCHoQttyQ"; a.click(); })() WPT Script SPF Transition

  95. logData 0 navigate {starting page} logData 1 execAndWait (function(){ var

    a = document.querySelector(“a[href^=‘{ending page}’]”); a.href = “/watch?v={video ID}“; a.click(); })() WPT Script SPF Transition

  96. SPF No SPF No Ads Timecoded Watch Home Search Ads

  97. No SPF SPF No Ads Timecoded Watch Home Search Ads

  98. Chrome Firefox DSL Cable Watch Home Search IE Dial-up MISSION

    ACCOMPLISHED:
 Testing dynamic navigations

  99. Document Complete First Paint DOM Loaded On Load

  100. Video Content

  101. Playback Start

  102. performance.mark('aft'); main.js Application-Specific Metrics

  103. None

  104. v=2&s=youtube&action=watch,watch7_html5&yt_nav=1&yt_pft=none& docid=ussCHoQttyQ&ei=6Q2pU72mMuiI0QHcm4GgDQ&cpn=l2khPNrU oSFcVuQs&fmt=243&tds=4&cmt=0.067&yt_pt=html5&yt_lt=warm&e=9 10133,927625,930008,934030,949401&yt_ref=&yt_spf=1&yt_ad=0&yt_ pl=0&yt_li=0&srt=98&rt=nc0.1,nc1.56,cfg.56,gv.97,nc2.348,je.412,jl. 413,ol.426,tdl.723,fvb.999,plev.1370,vfp.1375,l2s.1378,pbs.1412,arr. 623,vrr.626,msa.667,vda.1278,vri.671,vdns.887,vreq.924,vrc.1274,ari. 668,adns.887,areq.924,fab.983,arc.1258,aft.1412&it=st.89,req.97,dns_. 27,tcp_.35,req_.72,rcv_.332

  105. Playback Start http://www.webpagetest.org/google/view_csi.php?test=140624_18_5Q5

  106. Playback Start http://www.webpagetest.org/google/view_csi.php?test=140624_18_5Q5 MISSION ACCOMPLISHED:
 Analyzing dynamic navigations

  107. Buy the book bit.ly/usingwebpagetest

  108. Thank you! Buy the book bit.ly/usingwebpagetest +RickViscomi @rick_viscomi

  109. Delivering Media for Interactive Applications +StevenRobertson @_srobertson

  110. Pick one.

  111. chrome://tracing chrome://net-internals

  112. OH RIGHT TRACING

  113. None

  114. TCP is affecting your app’s performance.

  115. Parallelize loading of small resources.

  116. … how about big ones?

  117. TCP slow start.

  118. None

  119. Exponential growth

  120. First packet loss

  121. Linear growth

  122. DOUBLE THE WOW

  123. None

  124. In the long run, parallelized loading can increase throughput…

  125. … at the cost of variability.

  126. Wireshark whenever you can.

  127. None

  128. Wireshark whenever you can.

  129. Which is basically never.

  130. You can get pretty far with progress events.

  131. Which is basically never.

  132. None
  133. None
  134. None

  135. Sometimes, things just suck.

  136. None
  137. None

  138. So we turned it off.

  139. Rebuffer stats got way worse (of course).

  140. Watch-time was neutral.

  141. One of the most divisive experiments we’ve run.

  142. None

  143. We cheated.

  144. Give users a way to show you how you could

    have done better.
  145. None

  146. Thank you! #end144p Check out the website youtube.com +StevenRobertson @_srobertson

  147. Thank you! #webperf #spfjs #end144p Buy the book bit.ly/usingwebpagetest +StevenRobertson

    @_srobertson +AlexNicksay @webdevjesus Get the SPF code github.com/youtube/spfjs Check out the website youtube.com +RickViscomi @rick_viscomi