Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Everything they don’t tell you about video in the browser

Phil Nash
October 27, 2018

Everything they don’t tell you about video in the browser

So, you want to make a video call in the browser? There are libraries available to handle the WebRTC for you, but what about everything else you want in that call? The browsers aren’t going to make that as easy for you I’m afraid.

This talk will be a deep dive into the features you want to see in a video call. We’ll start with a video chat and look into screen sharing, switching cameras, the web audio API and the exciting world of media constraints. You’ll leave this talk understanding the full promise of getUserMedia!

--

Links:

Screen capture in Chrome: https://www.twilio.com/blog/2017/10/screen-capture-in-google-chrome.html
Screen capture in Firefox: https://www.twilio.com/blog/2017/10/screen-capture-in-firefox.html
Screen capture in Edge: https://www.twilio.com/blog/2018/05/screen-capture-in-microsoft-edge.html
Add screen sharing to a Twilio Video application https://www.twilio.com/blog/2018/01/screen-sharing-twilio-video.html
Choosing cameras in JavaScript with the mediaDevices API: https://www.twilio.com/blog/2018/04/choosing-cameras-javascript-mediadevices-api.html
Switching cameras during a video chat with Twilio Video: https://www.twilio.com/blog/2018/06/switching-cameras-twilio-video-chat.html
Audio visualisation with the Web Audio API and React: https://www.twilio.com/blog/audio-visualisation-web-audio-api--react

Phil Nash

October 27, 2018
Tweet

More Decks by Phil Nash

Other Decks in Programming

Transcript

  1. More constraints const constraints = { width: { min: 640,

    ideal: 1920, max: 1920 }, height: { min: 400, ideal: 1080 }, aspectRatio: 1.777777778, frameRate: { max: 30 }, facingMode: { exact: "user" } }; 01. 02. 03. 04. 05. 06. 07. @philnash
  2. Screen capture - Chrome chrome.runtime.onMessageExternal.addListener((message, sender, sendResponse) =>; { const

    sources = message.sources; const tab = sender.tab; chrome.desktopCapture.chooseDesktopMedia(sources, tab, (streamId) => { if (!streamId) { sendResponse({ type: 'error', message: 'Failed to get stream ID' }); } else { sendResponse({ type: 'success', streamId: streamId }); } }); return true; }); 01. 02. 03. 04. 05. 06. 07. 08. 09. 10. 11. 12. 13. 14. 15. 16. 17. 18. @philnash
  3. Screen capture - Chrome chrome.runtime.sendMessage(EXTENSION_ID, request, response => { if

    (response && response.type === 'success') { navigator.mediaDevices .getUserMedia({ video: { mandatory: { chromeMediaSource: 'desktop', chromeMediaSourceId: response.streamId } } }) } 01. 02. 03. 04. 05. 06. 07. 08. 09. 10. 11. 12. @philnash
  4. Screen capture - Firefox navigator.mediaDevices.getUserMedia({ video: { mediaSource: 'screen' }

    }).then(returnedStream => { // use the stream }); 01. 02. 03. 04. 05. 06. 07. @philnash
  5. Screen capture - Firefox function isFirefox() { var mediaSourceSupport =

    !!navigator.mediaDevices.getSupportedConstraints() .mediaSource; var matchData = navigator.userAgent.match(/Firefox\/(\d+)/); var firefoxVersion = 0; if (matchData && matchData[1]) { firefoxVersion = parseInt(matchData[1], 10); } return mediaSourceSupport && firefoxVersion >= 52; } 01. 02. 03. 04. 05. 06. 07. 08. 09. @philnash
  6. Web audio const ctx = new AudioContext(); const oscillator =

    ctx.createOscillator(); oscillator.frequency = 400; oscillator.connect(ctx.destination); oscillator.start() 01. 02. 03. 04. 05. @philnash
  7. Web audio const ctx = new AudioContext(); const analyser =

    ctx.createAnalyser(); 01. 02. @philnash