Tweet
Tweet

Slide 1

Slide 1 text

YOU'RE ON MUTE! WEBRTC AND OUR LIVES ON SCREEN @philnash

Slide 2

Slide 2 text

Phil Nash @philnash https://philna.sh [email protected]

Slide 3

Slide 3 text

YOU'RE ON MUTE! WEBRTC AND OUR LIVES ON SCREEN @philnash

Slide 4

Slide 4 text

OUR LIVES ON SCREEN @philnash

Slide 5

Slide 5 text

WEBRTC VS @philnash

Slide 6

Slide 6 text

GREAT FOR WORK, FRIENDS OR FAMILY @philnash

Slide 7

Slide 7 text

NOT SO GREAT FOR ONE OFFS @philnash

Slide 8

Slide 8 text

NOT SO GREAT FOR SPECIALIST APPS @philnash

Slide 9

Slide 9 text

WEBRTC IS FOR ALL THE OTHER APPLICATIONS @philnash

Slide 10

Slide 10 text

REAL-TIME VIDEO, AUDIO AND DATA EXPERIENCES FOR YOUR USERS @philnash

Slide 11

Slide 11 text

Today 1. Basics of WebRTC 2. WebRTC Usage 3. Controlling the user experience 4. Tools and takeaways @philnash

Slide 12

Slide 12 text

BASICS OF WEBRTC @philnash

Slide 13

Slide 13 text

@philnash

Slide 14

Slide 14 text

Signalling @philnash

Slide 15

Slide 15 text

Peer to peer @philnash

Slide 16

Slide 16 text

SFU @philnash

Slide 17

Slide 17 text

ICE • ICE — Interactive Connectivity Establishment • NAT — Network Address Translation • STUN — Session Traversal Utilities for NAT • TURN — Traversal Using Relays around NAT @philnash

Slide 18

Slide 18 text

THE APIS @philnash

Slide 19

Slide 19 text

getUserMedia navigator.mediaDevices.getUserMedia({ audio: true, video: true }).then(stream => { // ... }); 01. 02. 03. 04. @philnash

Slide 20

Slide 20 text

RTCPeerConnection const pc = new RTCPeerConnection({ iceServers }); pc.addEventListener("icecandidate", (candidate) => { // send candidate to signalling }); 01. 02. 03. 04. @philnash

Slide 21

Slide 21 text

RTCPeerConnection pc.addStream(stream); pc.createOffer().then(offer => { pc.setLocalDescription(offer); // send offer to peer }); 01. 02. 03. 04. 05. @philnash

Slide 22

Slide 22 text

RTCPeerConnection pc.setRemoteDescription(offer); pc.createAnswer().then(answer => { pc.setLocalDescription(answer); // send answer to peer }); 01. 02. 03. 04. 05. @philnash

Slide 23

Slide 23 text

RTCPeerConnection pc.setRemoteDescription(answer); pc.addEventListener("track", event => { // Connected! // show remote video in UI }); 01. 02. 03. 04. 05. @philnash

Slide 24

Slide 24 text

RTCDataChannel const dataChannel = pc.createDataChannel("my-data-channel"); dataChannel.addEventListener("message", event => { console.log(event.data); }); 01. 02. 03. 04. @philnash

Slide 25

Slide 25 text

RTCDataChannel pc.addEventListener("datachannel", event => { event.channel.addEventListener("open", () => { event.channel.send("Hi!"); }); }); 01. 02. 03. 04. 05. @philnash

Slide 26

Slide 26 text

WEBRTC USAGE @philnash

Slide 27

Slide 27 text

@philnash

Slide 28

Slide 28 text

@philnash

Slide 29

Slide 29 text

IMPROVING THE USER EXPERIENCE @philnash

Slide 30

Slide 30 text

MEDIA @philnash

Slide 31

Slide 31 text

getUserMedia navigator.mediaDevices.getUserMedia({ audio: true, video: true }).then(stream => { // ... }); 01. 02. 03. 04. @philnash

Slide 32

Slide 32 text

getUserMedia - Permissions navigator.mediaDevices.getUserMedia({ audio: true, video: true }) .then(stream => { // success }) .catch(error => { // permission denied }); 01. 02. 03. 04. 05. 06. @philnash

Slide 33

Slide 33 text

getUserMedia - Media Constraints { audio: true, video: true } 01. 02. 03. 04. @philnash

Slide 34

Slide 34 text

Media Constraints - facingMode { audio: true, video: { facingMode: "user" // "environment", "right", "left" } } 01. 02. 03. 04. 05. 06. @philnash

Slide 35

Slide 35 text

Media Constraints - deviceId { audio: true, video: { deviceId: { exact: device.id } } } 01. 02. 03. 04. 05. 06. @philnash

Slide 36

Slide 36 text

enumerateDevices navigator.mediaDevices.enumerateDevices() .then(devices => { devices.forEach(device => { console.log(`${device.id}: ${device.label}`); }); }); 01. 02. 03. 04. 05. 06. @philnash

Slide 37

Slide 37 text

enumerateDevices - Permissions navigator.mediaDevices.enumerateDevices() .then(devices => { devices.forEach(device => { console.log(device.label); // ??? }); }); 01. 02. 03. 04. 05. 06. @philnash

Slide 38

Slide 38 text

Media Constraints - resolution { audio: true, video: { width: 1920, height: 1080 } } 01. 02. 03. 04. 05. 06. 07. @philnash

Slide 39

Slide 39 text

Media Constraints - resolution { video: { width: { min: 320, max: 1920, ideal: 1920 }, height: { min: 240, max: 1080, ideal: 1080 } } } 01. 02. 03. 04. 05. 06. @philnash

Slide 40

Slide 40 text

Media Constraints - resolution { video: { width: { exact: 1920 }, height: { exact: 1080 } } } // => 🚨 OverconstrainedError 01. 02. 03. 04. 05. 06. 07. @philnash

Slide 41

Slide 41 text

Media Constraints - torch 🔦😄 { video: { advanced: [{ torch: true }], } } 01. 02. 03. 04. 05. @philnash

Slide 42

Slide 42 text

Media Constraints - torch 🔦😄 { video: { advanced: [{ torch: false }], // doesn't work 🤨 } } 01. 02. 03. 04. 05. @philnash

Slide 43

Slide 43 text

Applying media constraints videoTrack.applyConstraints({ width: 1920, height: 1080 }); 01. 02. 03. 04. @philnash

Slide 44

Slide 44 text

Applying media constraints videoTrack.applyConstraints({ deviceId: { exact: device.id } }); ❌ Don't do this! ❌ 01. 02. 03. @philnash

Slide 45

Slide 45 text

Changing cameras videoTrack.stop(); navigator.mediaDevices.getUserMedia({ video: { deviceId: { exact: device.id } } }); 01. 02. 03. 04. 05. 06. @philnash

Slide 46

Slide 46 text

DEMO @philnash

Slide 47

Slide 47 text

RECOMMENDATION: ALWAYS PROVIDE A WAY TO CHECK AND CHANGE INPUTS @philnash

Slide 48

Slide 48 text

RECOMMENDATION: UNDERSTAND THE AVAILABLE MEDIA CONSTRAINTS @philnash

Slide 49

Slide 49 text

SCREEN SHARE @philnash

Slide 50

Slide 50 text

Screen share navigator.mediaDevices.getDisplayMedia() .then(stream => { // add stream track to peer connection }) 01. 02. 03. 04. @philnash

Slide 51

Slide 51 text

DEMO @philnash

Slide 52

Slide 52 text

RECOMMENDATION: LET PEOPLE SHARE SCREENS WITHOUT AN EXTENSION @philnash

Slide 53

Slide 53 text

WEB AUDIO API @philnash

Slide 54

Slide 54 text

Web Audio API Not part of WebRTC Vital part of displaying audio to the user @philnash

Slide 55

Slide 55 text

DEMO @philnash

Slide 56

Slide 56 text

YOU'RE ON MUTE! @philnash

Slide 57

Slide 57 text

DEMO @philnash

Slide 58

Slide 58 text

DATA CHANNEL @philnash

Slide 59

Slide 59 text

RTCDataChannel const dataChannel = pc.createDataChannel("my-data-channel"); dataChannel.addEventListener("message", event => { console.log(event.data); }); dataChannel.send("Hi!"); 01. 02. 03. 04. 05. @philnash

Slide 60

Slide 60 text

DEMO @philnash

Slide 61

Slide 61 text

USEFUL TOOLS @philnash

Slide 62

Slide 62 text

OPEN SOURCE DIAGNOSTICS @philnash

Slide 63

Slide 63 text

DEMO @philnash

Slide 64

Slide 64 text

RTC Diagnostics https://github.com/twilio/rtc-diagnostics/ @philnash

Slide 65

Slide 65 text

OUR LIVES ON SCREEN @philnash

Slide 66

Slide 66 text

WEBRTC IS EVERYWHERE @philnash

Slide 67

Slide 67 text

WEBRTC IS POWERFUL BECAUSE IT IS ON THE WEB @philnash

Slide 68

Slide 68 text

WE HAVE THE TOOLS TO BUILD GREAT WEBRTC EXPERIENCES @philnash

Slide 69

Slide 69 text

Thanks! @philnash https://philna.sh [email protected]