EVERYTHING THEY DON’T TELL YOU ABOUT VIDEO IN THE BROWSER @philnash

Phil Nash @philnash http://philna.sh [email protected]

WEB PLATFORM @philnash

WEBRTC @philnash

PEER TO PEER @philnash

WebRTC Support ✅ ✅ ✅ ✅ ✅ @philnash

SORT OF... @philnash

THE ARCHITECTURE OF WEBRTC @philnash

WEBRTC TODAY @philnash

getUserMedia @philnash

SELECTING CAMERAS @philnash

THIS SHOULD BE EASY @philnash

Selecting cameras navigator.mediaDevices.getUserMedia({ video: true }) 01. 02. 03. @philnash

Selecting cameras navigator.mediaDevices.enumerateDevices() .then(devices => { // woo! }) 01. 02. 03. 04. @philnash

BUT... @philnash

MediaConstraints @philnash

Selecting cameras navigator.mediaDevices.getUserMedia({ video: ??? }) 01. 02. 03. @philnash

facingMode @philnash

Selecting cameras navigator.mediaDevices.getUserMedia({ video: { facingMode: ['user' | 'environment' | 'left' | 'right'] } }) 01. 02. 03. 04. 05. @philnash

deviceId @philnash

Selecting cameras navigator.mediaDevices.enumerateDevices().then(devices => { navigator.mediaDevices.getUserMedia({ video: { deviceId: devices[0].deviceId } }) }); 01. 02. 03. 04. 05. 06. 07. @philnash

BUT... @philnash

Selecting cameras navigator.mediaDevices.enumerateDevices().then(devices => { navigator.mediaDevices.getUserMedia({ video: { deviceId: { exact: devices[0].deviceId } } }) }); 01. 02. 03. 04. 05. 06. 07. @philnash

More constraints const constraints = { width: { min: 640, ideal: 1920, max: 1920 }, height: { min: 400, ideal: 1080 }, aspectRatio: 1.777777778, frameRate: { max: 30 }, facingMode: { exact: "user" } }; 01. 02. 03. 04. 05. 06. 07. @philnash

OverconstrainedError @philnash

SCREEN SHARING @philnash

Screen capture support ✅ @philnash

✨getDisplayMedia✨ @philnash

Screen capture navigator.getDisplayMedia() .then(stream => { // the stream is the display }); 01. 02. 03. 04. @philnash

PLUGINS @philnash

Screen capture - Chrome chrome.runtime.onMessageExternal.addListener((message, sender, sendResponse) =>; { const sources = message.sources; const tab = sender.tab; chrome.desktopCapture.chooseDesktopMedia(sources, tab, (streamId) => { if (!streamId) { sendResponse({ type: 'error', message: 'Failed to get stream ID' }); } else { sendResponse({ type: 'success', streamId: streamId }); } }); return true; }); 01. 02. 03. 04. 05. 06. 07. 08. 09. 10. 11. 12. 13. 14. 15. 16. 17. 18. @philnash

Screen capture - Chrome chrome.runtime.sendMessage(EXTENSION_ID, request, response => { if (response && response.type === 'success') { navigator.mediaDevices .getUserMedia({ video: { mandatory: { chromeMediaSource: 'desktop', chromeMediaSourceId: response.streamId } } }) } 01. 02. 03. 04. 05. 06. 07. 08. 09. 10. 11. 12. @philnash

NONSTANDARD BEHAVIOUR @philnash

Screen capture - Firefox navigator.mediaDevices.getUserMedia({ video: { mediaSource: 'screen' } }).then(returnedStream => { // use the stream }); 01. 02. 03. 04. 05. 06. 07. @philnash

Screen capture - Firefox function isFirefox() { var mediaSourceSupport = !!navigator.mediaDevices.getSupportedConstraints() .mediaSource; var matchData = navigator.userAgent.match(/Firefox\/(\d+)/); var firefoxVersion = 0; if (matchData && matchData[1]) { firefoxVersion = parseInt(matchData[1], 10); } return mediaSourceSupport && firefoxVersion >= 52; } 01. 02. 03. 04. 05. 06. 07. 08. 09. @philnash

STANDARDS COMING SOON @philnash

WEB AUDIO @philnash

Web audio const ctx = new AudioContext(); const oscillator = ctx.createOscillator(); oscillator.frequency = 400; oscillator.connect(ctx.destination); oscillator.start() 01. 02. 03. 04. 05. @philnash

Web audio const ctx = new AudioContext(); const analyser = ctx.createAnalyser(); 01. 02. @philnash

WEB PLATFORM++ @philnash

YOUR SKILLS++ @philnash

EVERYTHING THEY DON’T TELL YOU ABOUT VIDEO IN THE BROWSER @philnash

Thanks! @philnash http://philna.sh [email protected]