Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Improving the Video Delivery at The New York Times

Improving the Video Delivery at The New York Times

This talk presents the improvements we made on the on-demand stack and live streaming platform at The New York Times.

5dc4abe3dc14c25021ff2295fc380d73?s=128

Flávio Ribeiro

November 03, 2017
Tweet

Transcript

  1. Improving the Video Delivery at Streaming Media West, 2017.

  2. Flavio Ribeiro Lead Engineer
 { , } /flavioribeiro http://flavioribeiro.com

  3. Flavio Ribeiro Lead Engineer
 { , } /flavioribeiro http://flavioribeiro.com Maxwell

    Da Silva Executive Director of Engineering
 { , } /dayvson http://dayvson.me
  4. agenda

  5. agenda • recap of 2016’s SMWest

  6. agenda • recap of 2016’s SMWest • on-demand stack improvements

  7. agenda • recap of 2016’s SMWest • on-demand stack improvements

    • live streaming stack improvements
  8. agenda • recap of 2016’s SMWest • on-demand stack improvements

    • live streaming stack improvements • what’s coming next
  9. Media Factory Client Acquisition API Media Factory API Transcoding API

    Database Distribution API CDN Storage / Swap Area recap
  10. Media Factory Client Acquisition API Media Factory API Transcoding API

    Database Distribution API CDN Storage / Swap Area recap
  11. None
  12. None
  13. some numbers

  14. some numbers • > 20k encoding/publishing jobs

  15. some numbers • > 20k encoding/publishing jobs • ~ 12

    different outputs each (H264/MP4, HLS levels, VP8/WebM)
  16. some numbers • > 20k encoding/publishing jobs • ~ 12

    different outputs each (H264/MP4, HLS levels, VP8/WebM) • 4 external pull requests on the transcoding API
  17. some numbers • > 20k encoding/publishing jobs • ~ 12

    different outputs each (H264/MP4, HLS levels, VP8/WebM) • 4 external pull requests on the transcoding API • 6 encoding services supported
  18. on-demand stack improvements

  19. None
  20. cdn migration: our video library

  21. • migrate assets to Google Cloud Storage 
 cdn migration:

    our video library
  22. • migrate assets to Google Cloud Storage except HLS •

    generate adaptive formats on-the-fly 
 cdn migration: our video library
  23. • migrate assets to Google Cloud Storage except HLS •

    generate adaptive formats on-the-fly • less storage (~40%) • more formats delivered (MPEG-Dash for free) • faster transcoding process • help on closed captions support
 cdn migration: our video library
  24. cdn migration: our video library

  25. cdn migration: our video library GCS H264/MP4 files

  26. cdn migration: our video library GCS Helper GCS H264/MP4 files

  27. cdn migration: our video library GCS Helper + GCS H264/MP4

    files
  28. cdn migration: our video library GCS Helper + Cache GCS

    H264/MP4 files
  29. cdn migration: our video library Before and after. Less storage,

    more formats delivered.
  30. None
  31. None
  32. Media Factory Client Acquisition API Media Factory API Transcoding API

    Database Distribution API CDN Storage / Swap Area pipeline integration
  33. Media Factory Client Acquisition API Media Factory API Transcoding API

    Database Distribution API Storage / Swap Area pipeline integration
  34. Media Factory Client Acquisition API Media Factory API Transcoding API

    Database Distribution API Storage / Swap Area pipeline integration GCS GCS Helper + Cache
  35. closed captions

  36. closed captions

  37. closed captions • accessibility

  38. closed captions • accessibility • follow web standards

  39. closed captions • accessibility • follow web standards • muted

    autoplay
  40. closed captions • accessibility • follow web standards • muted

    autoplay • first step for internationalization
  41. closed captions • accessibility • follow web standards • muted

    autoplay • first step for internationalization • integrate with the pipeline
  42. closed captions • accessibility • follow web standards • muted

    autoplay • first step for internationalization • integrate with the pipeline • avoid vendor dependency
  43. closed captions • accessibility • follow web standards • muted

    autoplay • first step for internationalization • integrate with the pipeline • avoid vendor dependency • multiple generation options
  44. closed captions • accessibility • follow web standards • muted

    autoplay • first step for internationalization • integrate with the pipeline • avoid vendor dependency • multiple generation options • Integrated with our CMS
  45. closed captions Captions API 3Play Google Speech API Amazon Lex

    Generation Client Abstraction etc...
  46. Media Factory Client Acquisition API Media Factory API Transcoding API

    Database Distribution API Storage / Swap Area GCS GCS Helper + Cache pipeline integration
  47. Media Factory Client Acquisition API Media Factory API Transcoding API

    Database Distribution API Storage / Swap Area GCS GCS Helper + Cache Captions API pipeline integration
  48. FOTO DO CMS

  49. FOTO DO CMS

  50. #EXTM3U #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP- ID="subs0",LANGUAGE="en",NAME="English",AUTOSELECT=YES,DEFAULT=YES,URI="index-f6.m3u8" #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=3778274,RESOLUTION=854x480,FRAME- RATE=29.970,CODECS="avc1.4d401f,mp4a.40.2",SUBTITLES="subs0" index-f1-v1-a1.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=365226,RESOLUTION=426x240,FRAME- RATE=29.970,CODECS="avc1.4d401f,mp4a.40.2",SUBTITLES="subs0" index-f2-v1-a1.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=967515,RESOLUTION=640x360,FRAME-

    RATE=29.970,CODECS="avc1.4d401f,mp4a.40.2",SUBTITLES="subs0" index-f3-v1-a1.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1568184,RESOLUTION=854x480,FRAME- RATE=29.970,CODECS="avc1.4d401f,mp4a.40.2",SUBTITLES="subs0" index-f4-v1-a1.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2573061,RESOLUTION=854x480,FRAME- RATE=29.970,CODECS="avc1.4d401f,mp4a.40.2",SUBTITLES="subs0" index-f5-v1-a1.m3u8 segmented WebVTT on HLS
  51. player integration

  52. • hls.js player integration

  53. • hls.js • Google ExoPlayer player integration

  54. • hls.js • Google ExoPlayer • Apple AVPlayer player integration

  55. • hls.js • Google ExoPlayer • Apple AVPlayer • Apple

    Safari player integration
  56. player integration

  57. on-demand stack improvements recap!

  58. on-demand stack improvements recap! • we are shutting down our

    datacenters
  59. on-demand stack improvements recap! • we are shutting down our

    datacenters • serving adaptive formats on-the-fly (less storage, more formats, speed up encoding)
  60. on-demand stack improvements recap! • we are shutting down our

    datacenters • serving adaptive formats on-the-fly (less storage, more formats, speed up encoding) • a/b tests to avoid hurting playback experience
  61. on-demand stack improvements recap! • we are shutting down our

    datacenters • serving adaptive formats on-the-fly (less storage, more formats, speed up encoding) • a/b tests to avoid hurting playback experience • closed captions API is just a wrapper for vendors
  62. on-demand stack improvements recap! • we are shutting down our

    datacenters • serving adaptive formats on-the-fly (less storage, more formats, speed up encoding) • a/b tests to avoid hurting playback experience • closed captions API is just a wrapper for vendors • on-the-fly packager takes care of serving the captions too
  63. on-demand stack improvements recap! • we are shutting down our

    datacenters • serving adaptive formats on-the-fly (less storage, more formats, speed up encoding) • a/b tests to avoid hurting playback experience • closed captions API is just a wrapper for vendors • on-the-fly packager takes care of serving the captions too • WebVTT on HLS for the win!
  64. live streaming stack improvements

  65. some background

  66. some background • we had a basic support for live

  67. some background • we had a basic support for live

    • newsroom expressed interest in doing more live streaming events
  68. some background • we had a basic support for live

    • newsroom expressed interest in doing more live streaming events • Presidential Election debates and correlated events
  69. live streaming stack @ 2016

  70. live streaming stack @ 2016

  71. live streaming stack @ 2016

  72. live streaming stack @ 2016

  73. problem 1: delivery

  74. problem 1: delivery • tons of HTTP PUT requests through

    open internet
  75. problem 1: delivery • tons of HTTP PUT requests through

    open internet • our deal with the CDN was coming to an end
  76. live streaming stack @ 2016

  77. our new live streaming stack

  78. our new live streaming stack

  79. our new live streaming stack

  80. our new live streaming stack

  81. our new live streaming stack

  82. problem 2: management

  83. problem 2: management • manual setup of the live events

  84. problem 2: management • manual setup of the live events

    • no clear/automated way for live-to-VoD
  85. problem 2: management • manual setup of the live events

    • no clear/automated way for live-to-VoD • no clear/automated way to ingest the stream on partners
  86. our new live streaming stack

  87. our new live streaming stack

  88. our new live streaming stack

  89. None
  90. None
  91. None
  92. what’s next?

  93. what’s next

  94. what’s next • thumbnails service

  95. what’s next • thumbnails service • A/B test: fragmented MP4’s

    versus MPEG-TS
  96. what’s next • thumbnails service • A/B test: fragmented MP4’s

    versus MPEG-TS • A/B test: variable segments size for faster start
  97. what’s next • thumbnails service • A/B test: fragmented MP4’s

    versus MPEG-TS • A/B test: variable segments size for faster start • A/B test: motion covers for click-to-play engagement
  98. what’s next • thumbnails service • A/B test: fragmented MP4’s

    versus MPEG-TS • A/B test: variable segments size for faster start • A/B test: motion covers for click-to-play engagement • closed captions feeding recommendations / search
  99. what’s next • thumbnails service • A/B test: fragmented MP4’s

    versus MPEG-TS • A/B test: variable segments size for faster start • A/B test: motion covers for click-to-play engagement • closed captions feeding recommendations / search • closed captions integration with partners
  100. what’s next • thumbnails service • A/B test: fragmented MP4’s

    versus MPEG-TS • A/B test: variable segments size for faster start • A/B test: motion covers for click-to-play engagement • closed captions feeding recommendations / search • closed captions integration with partners • DVR for live streaming
  101. + thumbnails service Application nginx-thumbextractor Google Cloud Storage

  102. wanna see more?

  103. http://github.com/nytimes/video-transcoding-api Transcoding API http://github.com/nytimes/video-captions-api Captions API http://github.com/nytimes/gcs-helper GCS Helper http://github.com/nytimes/elemental-live-client

    http://github.com/nytimes/nginx-vod-module-docker +
  104. We’re hiring nyti.ms/technology @NYTDevs | developers.nytimes.com

  105. Stay updated open.nytimes.com

  106. Connect with us on Slack! http://video-dev.org

  107. thank you! { , } /flavioribeiro

  108. • https://speakerdeck.com/flavioribeiro/building-a-closed-captions-ecosystem-at-the-new-york-times • http://blog.flavioribeiro.com/using-open-source-to-create-a-video-thumbnails-service/ • https://open.nytimes.com/improving-our-video-experience-part-one-our-on-demand-video-platform-cf818e03353d • https://medium.com/ellation-tech/just-in-time-vod-packaging-in-an-aws-environment-with-role-based-credentials-4e33202a4450 • https://media.netflix.com/en/company-blog/the-power-of-a-picture

    more links