The Nitty Gritty of Adaptive Video Transmuxing in JS

A talk on video transmuxing using MSE in Javascript, given at EmpireJS 2016.

Transcript

1. Adaptive Video Transmuxing Donato Borrello @CodeAssault Lead Engineer, JW Player The Nitty Gritty of In JavaScript
2. Take MPEG-TS and make it MP4
3. What’s goin’ on wit client-side video Like this ------------>
4. A 10,000 foot view
5. What’s a Provider?
6. Why HTTP Live Streaming? HLS is the most widely used ABR streaming format
7. Research
8. Take it? or Make it?
9. Opportunities for improvement ➔ Faster time to first frame ◆ Incremental transmuxing instead of batch ◆ “Jump-start” downloads ◆ Clever Parsing using FSM ➔ More feature-rich ◆ Multi-track audio (HLSv4) ➔ More streams supported ◆ A frustratingly thorough test suite Ultimately, we can take the best parts from each
10. Caterpillar is born!
11. Caterpillar Shaped Architecture! Adaptive Video Streaming over HTTP
12. Transmuxing with 
 Incremental Computation
15. Transmuxing Engine
16. Events or FRP ➔ RxJS using Observable objects ➔ Event based stream philosophy (push streams) ◆ Equates to batch processing ➔ We want Pull Streams ◆ Backpressure with buffers ➔ Quite complex - unused features and hard to debug ➔ Profiling was challenging - trampoline what?
17. Communicating Sequential Processes ➔ Enter CSP ◆ Channels, processes and buffers ◆ Using generators ➔ Does exactly as much work as needed ➔ Cleaner code ◆ Easier profiling and debugging Process Channel Channel
18. “Jump-start” Downloading
20. A 10,793 Line 
 Media Manifest 248 kb file size At 1 Mbps, that’s 250ms of added latency until download completes
21. Manifest Parsing ➔ There is a big async dependency
22. More typical examples DVR Streams - worst offenders Live Streams - rarely a problem VOD Streams - longer movies see improvement The media playlist for Captain America - Winter Soldier is
 1500 lines, 27kb
23. NALU Parsing 
 with Finite State Machines
24. PES > NALU > H264 > MP4
25. That extra step for video PES Packets
26. I Started Dreaming in Hex TS Packet Starts 0x47 PES Packet Starts NALU starts 0x000001
27. Finding NAL Units They don’t line up with anything. They can start and end anywhere. All we know is 0x001 or 0x0001
29. Miscellanea
30. Adaptive Switching ➔ Time to download ➔ Time to Transmux ➔ Network Volatility ➔ Manifest accuracy ➔ Available Qualities ➔ Individual stream status
31. ➔ Generators screwed us (blame v8) ◆ Babel-Regenerator saved us ➔ UglifyJS is only partially compatible with ES6 ➔ Webpack + Babel + Typescript was a hassle Impact of using ES6
32. Thanks! @JWDevelopers Rob Walch Kyle Aurand Jong Park Evol Greaves Danny Finkelstein Mohammed Siddique Sid Patel Eric Boyd Andrew Prelusky Pooja Maden @Google Joey Parrish @Chromium Chris Cunningham Dale Curtis @gliese1337/HLS.js Logan Kearsley