Improving the Video Delivery at The New York Times

Slide 1

Slide 1 text

Improving the Video Delivery at Streaming Media West, 2017.

Slide 2

Slide 2 text

Flavio Ribeiro Lead Engineer  { , } /flavioribeiro http://flavioribeiro.com

Slide 3

Slide 3 text

Flavio Ribeiro Lead Engineer  { , } /flavioribeiro http://flavioribeiro.com Maxwell Da Silva Executive Director of Engineering  { , } /dayvson http://dayvson.me

Slide 4

Slide 4 text

agenda

Slide 5

Slide 5 text

agenda • recap of 2016’s SMWest

Slide 6

Slide 6 text

agenda • recap of 2016’s SMWest • on-demand stack improvements

Slide 7

Slide 7 text

agenda • recap of 2016’s SMWest • on-demand stack improvements • live streaming stack improvements

Slide 8

Slide 8 text

agenda • recap of 2016’s SMWest • on-demand stack improvements • live streaming stack improvements • what’s coming next

Slide 9

Slide 9 text

Media Factory Client Acquisition API Media Factory API Transcoding API Database Distribution API CDN Storage / Swap Area recap

Slide 10

Slide 10 text

Media Factory Client Acquisition API Media Factory API Transcoding API Database Distribution API CDN Storage / Swap Area recap

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

some numbers

Slide 14

Slide 14 text

some numbers • > 20k encoding/publishing jobs

Slide 15

Slide 15 text

some numbers • > 20k encoding/publishing jobs • ~ 12 different outputs each (H264/MP4, HLS levels, VP8/WebM)

Slide 16

Slide 16 text

some numbers • > 20k encoding/publishing jobs • ~ 12 different outputs each (H264/MP4, HLS levels, VP8/WebM) • 4 external pull requests on the transcoding API

Slide 17

Slide 17 text

some numbers • > 20k encoding/publishing jobs • ~ 12 different outputs each (H264/MP4, HLS levels, VP8/WebM) • 4 external pull requests on the transcoding API • 6 encoding services supported

Slide 18

Slide 18 text

on-demand stack improvements

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

cdn migration: our video library

Slide 21

Slide 21 text

• migrate assets to Google Cloud Storage   cdn migration: our video library

Slide 22

Slide 22 text

• migrate assets to Google Cloud Storage except HLS • generate adaptive formats on-the-fly   cdn migration: our video library

Slide 23

Slide 23 text

• migrate assets to Google Cloud Storage except HLS • generate adaptive formats on-the-fly • less storage (~40%) • more formats delivered (MPEG-Dash for free) • faster transcoding process • help on closed captions support  cdn migration: our video library

Slide 24

Slide 24 text

cdn migration: our video library

Slide 25

Slide 25 text

cdn migration: our video library GCS H264/MP4 files

Slide 26

Slide 26 text

cdn migration: our video library GCS Helper GCS H264/MP4 files

Slide 27

Slide 27 text

cdn migration: our video library GCS Helper + GCS H264/MP4 files

Slide 28

Slide 28 text

cdn migration: our video library GCS Helper + Cache GCS H264/MP4 files

Slide 29

Slide 29 text

cdn migration: our video library Before and after. Less storage, more formats delivered.

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

Media Factory Client Acquisition API Media Factory API Transcoding API Database Distribution API CDN Storage / Swap Area pipeline integration

Slide 33

Slide 33 text

Media Factory Client Acquisition API Media Factory API Transcoding API Database Distribution API Storage / Swap Area pipeline integration

Slide 34

Slide 34 text

Media Factory Client Acquisition API Media Factory API Transcoding API Database Distribution API Storage / Swap Area pipeline integration GCS GCS Helper + Cache

Slide 35

Slide 35 text

closed captions

Slide 36

Slide 36 text

closed captions

Slide 37

Slide 37 text

closed captions • accessibility

Slide 38

Slide 38 text

closed captions • accessibility • follow web standards

Slide 39

Slide 39 text

closed captions • accessibility • follow web standards • muted autoplay

Slide 40

Slide 40 text

closed captions • accessibility • follow web standards • muted autoplay • first step for internationalization

Slide 41

Slide 41 text

closed captions • accessibility • follow web standards • muted autoplay • first step for internationalization • integrate with the pipeline

Slide 42

Slide 42 text

closed captions • accessibility • follow web standards • muted autoplay • first step for internationalization • integrate with the pipeline • avoid vendor dependency

Slide 43

Slide 43 text

closed captions • accessibility • follow web standards • muted autoplay • first step for internationalization • integrate with the pipeline • avoid vendor dependency • multiple generation options

Slide 44

Slide 44 text

Slide 45

Slide 45 text

closed captions Captions API 3Play Google Speech API Amazon Lex Generation Client Abstraction etc...

Slide 46

Slide 46 text

Media Factory Client Acquisition API Media Factory API Transcoding API Database Distribution API Storage / Swap Area GCS GCS Helper + Cache pipeline integration

Slide 47

Slide 47 text

Media Factory Client Acquisition API Media Factory API Transcoding API Database Distribution API Storage / Swap Area GCS GCS Helper + Cache Captions API pipeline integration

Slide 48

Slide 48 text

FOTO DO CMS

Slide 49

Slide 49 text

FOTO DO CMS

Slide 50

Slide 50 text

#EXTM3U #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP- ID="subs0",LANGUAGE="en",NAME="English",AUTOSELECT=YES,DEFAULT=YES,URI="index-f6.m3u8" #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=3778274,RESOLUTION=854x480,FRAME- RATE=29.970,CODECS="avc1.4d401f,mp4a.40.2",SUBTITLES="subs0" index-f1-v1-a1.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=365226,RESOLUTION=426x240,FRAME- RATE=29.970,CODECS="avc1.4d401f,mp4a.40.2",SUBTITLES="subs0" index-f2-v1-a1.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=967515,RESOLUTION=640x360,FRAME- RATE=29.970,CODECS="avc1.4d401f,mp4a.40.2",SUBTITLES="subs0" index-f3-v1-a1.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1568184,RESOLUTION=854x480,FRAME- RATE=29.970,CODECS="avc1.4d401f,mp4a.40.2",SUBTITLES="subs0" index-f4-v1-a1.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2573061,RESOLUTION=854x480,FRAME- RATE=29.970,CODECS="avc1.4d401f,mp4a.40.2",SUBTITLES="subs0" index-f5-v1-a1.m3u8 segmented WebVTT on HLS

Slide 51

Slide 51 text

player integration

Slide 52

Slide 52 text

• hls.js player integration

Slide 53

Slide 53 text

• hls.js • Google ExoPlayer player integration

Slide 54

Slide 54 text

• hls.js • Google ExoPlayer • Apple AVPlayer player integration

Slide 55

Slide 55 text

• hls.js • Google ExoPlayer • Apple AVPlayer • Apple Safari player integration

Slide 56

Slide 56 text

player integration

Slide 57

Slide 57 text

on-demand stack improvements recap!

Slide 58

Slide 58 text

on-demand stack improvements recap! • we are shutting down our datacenters

Slide 59

Slide 59 text

on-demand stack improvements recap! • we are shutting down our datacenters • serving adaptive formats on-the-fly (less storage, more formats, speed up encoding)

Slide 60

Slide 60 text

Slide 61

Slide 61 text

on-demand stack improvements recap! • we are shutting down our datacenters • serving adaptive formats on-the-fly (less storage, more formats, speed up encoding) • a/b tests to avoid hurting playback experience • closed captions API is just a wrapper for vendors

Slide 62

Slide 62 text

Slide 63

Slide 63 text

Slide 64

Slide 64 text

live streaming stack improvements

Slide 65

Slide 65 text

some background

Slide 66

Slide 66 text

some background • we had a basic support for live

Slide 67

Slide 67 text

some background • we had a basic support for live • newsroom expressed interest in doing more live streaming events

Slide 68

Slide 68 text

some background • we had a basic support for live • newsroom expressed interest in doing more live streaming events • Presidential Election debates and correlated events

Slide 69

Slide 69 text

live streaming stack @ 2016

Slide 70

Slide 70 text

live streaming stack @ 2016

Slide 71

Slide 71 text

live streaming stack @ 2016

Slide 72

Slide 72 text

live streaming stack @ 2016

Slide 73

Slide 73 text

problem 1: delivery

Slide 74

Slide 74 text

problem 1: delivery • tons of HTTP PUT requests through open internet

Slide 75

Slide 75 text

problem 1: delivery • tons of HTTP PUT requests through open internet • our deal with the CDN was coming to an end

Slide 76

Slide 76 text

live streaming stack @ 2016

Slide 77

Slide 77 text

our new live streaming stack

Slide 78

Slide 78 text

our new live streaming stack

Slide 79

Slide 79 text

our new live streaming stack

Slide 80

Slide 80 text

our new live streaming stack

Slide 81

Slide 81 text

our new live streaming stack

Slide 82

Slide 82 text

problem 2: management

Slide 83

Slide 83 text

problem 2: management • manual setup of the live events

Slide 84

Slide 84 text

problem 2: management • manual setup of the live events • no clear/automated way for live-to-VoD

Slide 85

Slide 85 text

problem 2: management • manual setup of the live events • no clear/automated way for live-to-VoD • no clear/automated way to ingest the stream on partners

Slide 86

Slide 86 text

our new live streaming stack

Slide 87

Slide 87 text

our new live streaming stack

Slide 88

Slide 88 text

our new live streaming stack

Slide 89

Slide 89 text

No content

Slide 90

Slide 90 text

No content

Slide 91

Slide 91 text

No content

Slide 92

Slide 92 text

what’s next?

Slide 93

Slide 93 text

what’s next

Slide 94

Slide 94 text

what’s next • thumbnails service

Slide 95

Slide 95 text

what’s next • thumbnails service • A/B test: fragmented MP4’s versus MPEG-TS

Slide 96

Slide 96 text

what’s next • thumbnails service • A/B test: fragmented MP4’s versus MPEG-TS • A/B test: variable segments size for faster start

Slide 97

Slide 97 text

what’s next • thumbnails service • A/B test: fragmented MP4’s versus MPEG-TS • A/B test: variable segments size for faster start • A/B test: motion covers for click-to-play engagement

Slide 98

Slide 98 text

what’s next • thumbnails service • A/B test: fragmented MP4’s versus MPEG-TS • A/B test: variable segments size for faster start • A/B test: motion covers for click-to-play engagement • closed captions feeding recommendations / search

Slide 99

Slide 99 text

Slide 100

Slide 100 text

Slide 101

Slide 101 text

+ thumbnails service Application nginx-thumbextractor Google Cloud Storage

Slide 102

Slide 102 text

wanna see more?

Slide 103

Slide 103 text

http://github.com/nytimes/video-transcoding-api Transcoding API http://github.com/nytimes/video-captions-api Captions API http://github.com/nytimes/gcs-helper GCS Helper http://github.com/nytimes/elemental-live-client http://github.com/nytimes/nginx-vod-module-docker +

Slide 104

Slide 104 text

We’re hiring nyti.ms/technology @NYTDevs | developers.nytimes.com

Slide 105

Slide 105 text

Stay updated open.nytimes.com

Slide 106

Slide 106 text

Connect with us on Slack! http://video-dev.org

Slide 107

Slide 107 text

thank you! { , } /flavioribeiro

Slide 108

Slide 108 text

• https://speakerdeck.com/flavioribeiro/building-a-closed-captions-ecosystem-at-the-new-york-times • http://blog.flavioribeiro.com/using-open-source-to-create-a-video-thumbnails-service/ • https://open.nytimes.com/improving-our-video-experience-part-one-our-on-demand-video-platform-cf818e03353d • https://medium.com/ellation-tech/just-in-time-vod-packaging-in-an-aws-environment-with-role-based-credentials-4e33202a4450 • https://media.netflix.com/en/company-blog/the-power-of-a-picture more links