Slide 1

Slide 1 text

WebRTC 101 for Developers How to get started building your first WebRTC application

Slide 2

Slide 2 text

Dan Jenkins @dan_jenkins Google Developer Expert (Web Technologies specialising in WebRTC) Founder Nimble Ape Web Developer - not a typical telecommunications developer General Geek Lego Geek

Slide 3

Slide 3 text

Nimble Ape @nimbleapeltd WebRTC Development & Consulting IoT | Microservices | Docker Web APIs & Scalable Web Services Asterisk Applications Node.js Development & Consulting

Slide 4

Slide 4 text

Real Time Communication What is WebRTC? Web

Slide 5

Slide 5 text

Who's built something with WebRTC?

Slide 6

Slide 6 text

Draft or Spec? It's currently a Working Draft and not a Specification. Expect a Specification soon. Don't let this stop you building things with it. Both the IETF & W3C involved. https://www.w3.org/TR/webrtc/

Slide 7

Slide 7 text

Peer to Peer P2P?

Slide 8

Slide 8 text

Encrypted by design Secure!

Slide 9

Slide 9 text

Media Channel

Slide 10

Slide 10 text

Data Channel { "code": 200, "status": "Ok", "copyright": "Ž 2016 MARVEL", "attributionText": "Data provided by Marvel. Ž 2016 MARVEL", "attributionHTML": "Data provided by Marvel. Ž 2016 MARVEL", "etag": "051a4391a8d0f1ca2c9a8da7bf3345e2e8a47f0f", "data": { "offset": 0, "limit": 20, "total": 1, "count": 1, "results": [ { "id": 1009220, "name": "Captain America", "description": "Vowing to serve his country any way he could, young Steve Rogers took the super soldier serum to become America's one-man army. Fighting for the red, white and blue for over 60 years, Captain America is the living, breathing symbol of freedom and liberty.", "modified": "2014-06-10T16:13:04-0400", "thumbnail": { "path": "http://i.annihil.us/u/prod/marvel/i/mg/3/50/537ba56d31087", "extension": "jpg" }, "resourceURI": "http://gateway.marvel.com/v1/public/characters/1009220" } ] } } Strings

Slide 11

Slide 11 text

Data Channel Files

Slide 12

Slide 12 text

Data Channel Pretty much anything...

Slide 13

Slide 13 text

getUserMedia RTCPeerConnection RTCDataChannel Core APIs

Slide 14

Slide 14 text

Signalling Server

Slide 15

Slide 15 text

How your WebRTC endpoints communicate vital bits of information to one another WebRTC is a NEW Web API so you'd think you'd have a nice JSON blob explaining all the important things so a connection can be made. SDP (Session Description Protocol)

Slide 16

Slide 16 text

WRONG! SDP (Session Description Protocol)

Slide 17

Slide 17 text

a=candidate:4022866446 1 udp 2113937151 192.168.0.197 36768 typ host generation 0 a=candidate:4022866446 2 udp 2113937151 192.168.0.197 36768 typ host generation 0 a=candidate:2706108158 1 tcp 1509957375 192.168.0.197 0 typ host generation 0 a=candidate:2706108158 2 tcp 1509957375 192.168.0.197 0 typ host generation 0 a=candidate:1853887674 1 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0 a=candidate:1853887674 2 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0 a=candidate:2157334355 1 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0 a=candidate:2157334355 2 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0 SDP (Session Description Protocol)

Slide 18

Slide 18 text

Offer pc.createOffer(function(offer) { pc.setLocalDescription(new RTCSessionDescription(offer), function() { // send the offer to a server to be forwarded to the friend you're calling. }, error); }, error);

Slide 19

Slide 19 text

Answer pc.setRemoteDescription(new RTCSessionDescription(offer), function() { pc.createAnswer(function(answer) { pc.setLocalDescription(new RTCSessionDescription(answer), function() { // send the answer to a server to be forwarded back to the caller (you) }, error); }, error); }, error);

Slide 20

Slide 20 text

Audio Codecs We Care About OPUS G711 (PSTN interop)

Slide 21

Slide 21 text

Video Codecs We Care About VP8 VP9 H.264

Slide 22

Slide 22 text

ICE (Interactive Connectivity Establishment) (ICE Candidates) How to talk to me

Slide 23

Slide 23 text

STUN (Session Traversal Utilities for NAT) What's my public IP address?

Slide 24

Slide 24 text

TURN (Traversal Using Relay around NAT) Bugger, a nasty firewall is stopping us connecting P2P ... Let's relay our media instead

Slide 25

Slide 25 text

No Plugins...

Slide 26

Slide 26 text

Browsers

Slide 27

Slide 27 text

Apple iPhone Native Apps only (Cordova/Phonegap Supported)

Slide 28

Slide 28 text

Android Native Apps & Web View

Slide 29

Slide 29 text

Other Devices Anything that can run a WebRTC stack (or subset of it)

Slide 30

Slide 30 text

An open source adapter to level out all of the browsers to the "standard" Use it somewhere within your codebase https://github.com/webrtc/adapter adapter.js

Slide 31

Slide 31 text

You'll hear about ORTC But don't worry about it. There's an adapter! (https://github.com/twilio/ortc-adapter) But Microsoft have committed to introducing WebRTC "1.0" into Edge... so expect that soon! ORTC (Object Real Time Communications)

Slide 32

Slide 32 text

getUserMedia() now requires a trusted origin Localhost or HTTPS (But not self-signed TLS certs - letsencrypt.org) Trusted Origins

Slide 33

Slide 33 text

Getting Started with WebRTC

Slide 34

Slide 34 text

What can I use WebRTC for?

Slide 35

Slide 35 text

Or rather... What should I not use WebRTC for? What can I use WebRTC for?

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

Don't build phone calling

Slide 38

Slide 38 text

you can...

Slide 39

Slide 39 text

But don't... Build something awesome

Slide 40

Slide 40 text

Platform as a Service Use one! PaaS

Slide 41

Slide 41 text

Twilio var accessManager = new Twilio.AccessManager(token); conversationsClient = new Twilio.Conversations.Client(accessManager); conversationsClient.listen().then(clientConnected, function (error) { log('Could not connect to Twilio: ' + error.message); }); function clientConnected() { conversationsClient.on('invite', function (invite) { invite.accept().then(conversationStarted); }); } function conversationStarted(conversation) { activeConversation = conversation; if (!previewMedia) { conversation.localMedia.attach('#local-media'); } conversation.on('participantConnected', function (participant) { participant.media.attach('#remote-media'); }); conversation.on('disconnected', function (conversation) { conversation.localMedia.stop(); conversation.disconnect(); activeConversation = null; }); } https://www.twilio.com/docs/api/video/guide/quickstart-js

Slide 42

Slide 42 text

Tokbox var apiKey = 'YOUR-API-KEY'; var sessionId = 'YOUR-SESSION-ID'; var token = 'YOUR-TOKEN'; var session = OT.initSession(apiKey, sessionId) .on('streamCreated', function(event) { session.subscribe(event.stream); }) .connect(token, function(error) { var publisher = OT.initPublisher(); session.publish(publisher); }); https://tokbox.com/developer/quickstart/

Slide 43

Slide 43 text

Respoke // App ID from the Respoke Dashboard for your App var appId = "c10a2075-3f3d-466f-82f9-d2285e64c5d4"; // The unique username identifying the user var endpointId = "[email protected]"; // Create an instance of the Respoke client using your App ID var client = respoke.createClient({ appId: appId, developmentMode: true }); // "connect" event fired after successful connection to Respoke client.listen("connect", function(e) { console.log("Connected to Respoke!", e); }); // Execute some signin event, then connect to Respoke with client.connect({ endpointId: endpointId }); https://docs.respoke.io/client/javascript/getting-started.html

Slide 44

Slide 44 text

Xura (Was Forge, Acision, Crocodile? I lose track...) apidaze plivo kandy.io sinch Cisco Frozen Mountain Voxbone And many more...

Slide 45

Slide 45 text

Vanila JS 'use strict'; var startButton = document.getElementById('startButton'); var callButton = document.getElementById('callButton'); var hangupButton = document.getElementById('hangupButton'); callButton.disabled = true; hangupButton.disabled = true; startButton.onclick = start; callButton.onclick = call; hangupButton.onclick = hangup; var startTime; var localVideo = document.getElementById('localVideo'); var remoteVideo = document.getElementById('remoteVideo'); localVideo.addEventListener('loadedmetadata', function() { trace('Local video videoWidth: ' + this.videoWidth + 'px, videoHeight: ' + this.videoHeight + 'px'); }); remoteVideo.addEventListener('loadedmetadata', function() { trace('Remote video videoWidth: ' + this.videoWidth + 'px, videoHeight: ' + this.videoHeight + 'px'); }); remoteVideo.onresize = function() { trace('Remote video size changed to ' + remoteVideo.videoWidth + 'x' + remoteVideo.videoHeight); // We'll use the first onsize callback as an indication that video has started // playing out. if (startTime) { var elapsedTime = window.performance.now() - startTime; trace('Setup time: ' + elapsedTime.toFixed(3) + 'ms'); startTime = null; } }; var localStream; var pc1; var pc2; var offerOptions = { offerToReceiveAudio: 1, offerToReceiveVideo: 1 }; function getName(pc) { return (pc === pc1) ? 'pc1' : 'pc2'; } function getOtherPc(pc) { return (pc === pc1) ? pc2 : pc1; } function gotStream(stream) { trace('Received local stream'); localVideo.srcObject = stream; localStream = stream; callButton.disabled = false; } function start() { trace('Requesting local stream'); startButton.disabled = true; navigator.mediaDevices.getUserMedia({ audio: true, video: true }) .then(gotStream) .catch(function(e) { alert('getUserMedia() error: ' + e.name); }); } function call() { callButton.disabled = true; hangupButton.disabled = false; trace('Starting call'); startTime = window.performance.now(); var videoTracks = localStream.getVideoTracks(); var audioTracks = localStream.getAudioTracks(); if (videoTracks.length > 0) { trace('Using video device: ' + videoTracks[0].label); } if (audioTracks.length > 0) { trace('Using audio device: ' + audioTracks[0].label); } var servers = null; pc1 = new RTCPeerConnection(servers); trace('Created local peer connection object pc1'); pc1.onicecandidate = function(e) { onIceCandidate(pc1, e); }; pc2 = new RTCPeerConnection(servers); trace('Created remote peer connection object pc2'); pc2.onicecandidate = function(e) { onIceCandidate(pc2, e); }; pc1.oniceconnectionstatechange = function(e) { onIceStateChange(pc1, e); }; pc2.oniceconnectionstatechange = function(e) { onIceStateChange(pc2, e); }; pc2.onaddstream = gotRemoteStream; pc1.addStream(localStream); trace('Added local stream to pc1'); trace('pc1 createOffer start'); pc1.createOffer(onCreateOfferSuccess, onCreateSessionDescriptionError, offerOptions); } 209 lines of code function onCreateSessionDescriptionError(error) { trace('Failed to create session description: ' + error.toString()); } function onCreateOfferSuccess(desc) { trace('Offer from pc1\n' + desc.sdp); trace('pc1 setLocalDescription start'); pc1.setLocalDescription(desc, function() { onSetLocalSuccess(pc1); }, onSetSessionDescriptionError); trace('pc2 setRemoteDescription start'); pc2.setRemoteDescription(desc, function() { onSetRemoteSuccess(pc2); }, onSetSessionDescriptionError); trace('pc2 createAnswer start'); // Since the 'remote' side has no media stream we need // to pass in the right constraints in order for it to // accept the incoming offer of audio and video. pc2.createAnswer(onCreateAnswerSuccess, onCreateSessionDescriptionError); } function onSetLocalSuccess(pc) { trace(getName(pc) + ' setLocalDescription complete'); } function onSetRemoteSuccess(pc) { trace(getName(pc) + ' setRemoteDescription complete'); } function onSetSessionDescriptionError(error) { trace('Failed to set session description: ' + error.toString()); } function gotRemoteStream(e) { remoteVideo.srcObject = e.stream; trace('pc2 received remote stream'); } function onCreateAnswerSuccess(desc) { trace('Answer from pc2:\n' + desc.sdp); trace('pc2 setLocalDescription start'); pc2.setLocalDescription(desc, function() { onSetLocalSuccess(pc2); }, onSetSessionDescriptionError); trace('pc1 setRemoteDescription start'); pc1.setRemoteDescription(desc, function() { onSetRemoteSuccess(pc1); }, onSetSessionDescriptionError); } function onIceCandidate(pc, event) { if (event.candidate) { getOtherPc(pc).addIceCandidate(new RTCIceCandidate(event.candidate), function() { onAddIceCandidateSuccess(pc); }, function(err) { onAddIceCandidateError(pc, err); } ); trace(getName(pc) + ' ICE candidate: \n' + event.candidate.candidate); } } function onAddIceCandidateSuccess(pc) { trace(getName(pc) + ' addIceCandidate success'); } function onAddIceCandidateError(pc, error) { trace(getName(pc) + ' failed to add ICE Candidate: ' + error.toString()); } function onIceStateChange(pc, event) { if (pc) { trace(getName(pc) + ' ICE state: ' + pc.iceConnectionState); console.log('ICE state change event: ', event); } } function hangup() { trace('Ending call'); pc1.close(); pc2.close(); pc1 = null; pc2 = null; hangupButton.disabled = true; callButton.disabled = false; } https://github.com/webrtc/samples/blob/gh-pages/src/content/peerconnection/pc1/js/main.js

Slide 46

Slide 46 text

Vanila JS OK, slight exaggeration You can probably do it in around 70 lines of really bad JavaScript...

Slide 47

Slide 47 text

Resources https://webrtchacks.com/sdp-anatomy/ https://webrtc.github.io/samples/ https://www.webrtc-experiment.com/ https://github.com/webrtc/apprtc https://appr.tc/ http://iswebrtcreadyyet.com/ https://webrtcweekly.com/ https://bloggeek.me/

Slide 48

Slide 48 text

IoT Workshop Come join the IoT workshop tomorrow! It's FREE!

Slide 49

Slide 49 text

Thanks! [email protected] @dan_jenkins @nimbleapeltd Link to slides will be tweeted soon!

Slide 50

Slide 50 text

images 
 used http://img14.deviantart.net/9aad/i/2015/147/5/3/the_flash_silhouette_wallpaper_by_speedaroo-d8v1jo7.jpg https://upload.wikimedia.org/wikipedia/commons/thumb/3/31/ProhibitionSign2.svg/2000px-ProhibitionSign2.svg.png https://upload.wikimedia.org/wikipedia/commons/e/e2/Captain-america_serial_poster.jpg