Share
Embed
Enabling Voice Applications with WebRTC and ORTC in Microsoft Edge
Mark Roberts, Software Engineer on Twilio’s SDK Team, will describe how Twilio brought ORTC support to their WebRTC SDK, lessons learned along the way, and Twilio’s ORTC roadmap.
Mark Roberts
April 04, 2016
Transcript
66 slides
-
ENABLING VOICE APPLICATIONS WITH WEBRTC & ORTC IN MICROSOFT EDGE
MARK ROBERTS SOFTWARE ENGINEER [email protected] -
© 2009 - 2014 Twilio, Inc. All rights reserved. Voice
Applications PSTN -
© 2009 - 2014 Twilio, Inc. All rights reserved. Voice
Applications Ready Not Ready -
© 2009 - 2014 Twilio, Inc. All rights reserved. Voice
Applications Ready Not Ready -
© 2009 - 2014 Twilio, Inc. All rights reserved. Voice
Applications Ready Not Ready -
© 2009 - 2014 Twilio, Inc. All rights reserved. Voice
Applications Ready Not Ready -
© 2009 - 2014 Twilio, Inc. All rights reserved. Four
Basic Tasks Four basic tasks for a voice application: -
© 2009 - 2014 Twilio, Inc. All rights reserved. Four
Basic Tasks Four basic tasks for a voice application: 1. Capture audio from a microphone -
© 2009 - 2014 Twilio, Inc. All rights reserved. Four
Basic Tasks Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint -
© 2009 - 2014 Twilio, Inc. All rights reserved. Four
Basic Tasks Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint 3. Receive audio from an endpoint -
© 2009 - 2014 Twilio, Inc. All rights reserved. Four
Basic Tasks Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint 3. Receive audio from an endpoint 4.Playback received audio -
© 2009 - 2014 Twilio, Inc. All rights reserved. Browser
Support <script src="twilio.js"></script> -
© 2009 - 2014 Twilio, Inc. All rights reserved. WebRT-what⁇
• Simple JavaScript APIs • RTCPeerConnection • RTCIceCandidate • RTCSessionDescription • First-class browser support • Built on open standards (STUN, TURN, ICE, DTLS, RTP) -
© 2009 - 2014 Twilio, Inc. All rights reserved. Four
Basic Tasks Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint 3. Receive audio from an endpoint 4.Playback received audio -
© 2009 - 2014 Twilio, Inc. All rights reserved. Four
Basic Tasks Four basic tasks for a voice application: 1. Capture audio from a microphone 2.Send audio to an endpoint 3.Receive audio from an endpoint 4.Playback received audio -
WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security
(DTLS) WebR establish connectivity secure connectivity -
WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security
(DTLS) WebR establish connectivity secure connectivity send and receive media -
WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security
(DTLS) Secure Real-time Transport Protocol (SRTP) WebR establish connectivity secure connectivity send and receive media -
© 2009 - 2014 Twilio, Inc. All rights reserved. WebRT-what⁇
• Simple JavaScript APIs • RTCPeerConnection • RTCIceCandidate • RTCSessionDescription • First-class browser support • Built on open standards (STUN, TURN, ICE, DTLS, RTP) -
© 2009 - 2014 Twilio, Inc. All rights reserved. WebRT-what⁇
• Simple JavaScript APIs • RTCPeerConnection • RTCIceCandidate • RTCSessionDescription • First-class browser support • Built on open standards (STUN, TURN, ICE, DTLS, RTP) • Interoperability • No plugins required! • Bring-your-own signaling -
WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security
(DTLS) Secure Real-time Transport Protocol (SRTP) WebR establish connectivity secure connectivity send and receive media -
© 2009 - 2014 Twilio, Inc. All rights reserved. Browser
Support <script src="twilio.js"></script> -
© 2009 - 2014 Twilio, Inc. All rights reserved. Browser
Support <script src="twilio.js"></script> -
© 2009 - 2014 Twilio, Inc. All rights reserved. Browser
Support <script src="twilio.js"></script> -
© 2009 - 2014 Twilio, Inc. All rights reserved. Object
RTC (ORTC) • Built on the same open standards as WebRTC (STUN, TURN, ICE, DTLS, RTP) • Bring-your-own signaling -
© 2009 - 2014 Twilio, Inc. All rights reserved. Object
RTC (ORTC) • Built on the same open standards as WebRTC (STUN, TURN, ICE, DTLS, RTP) • Bring-your-own signaling • Does not use Session Description Protocol (SDP) -
WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security
(DTLS) Secure Real-time Transport Protocol (SRTP) WebR establish connectivity secure connectivity send and receive media -
WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security
(DTLS) Secure Real-time Transport Protocol (SRTP) SDP WebR -
© 2009 - 2014 Twilio, Inc. All rights reserved. Session
Description Protocol (SDP) v=0 o=- 7419095041521713450 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS BcUNfqxMTfmJ2aPDEipCTK4BnvabR7xKeTil m=audio 35898 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126 c=IN IP4 54.244.51.11 a=rtcp:39742 IN IP4 54.244.51.11 a=sendrecv a=rtcp-mux a=rtpmap:111 opus/48000/2 a=fmtp:111 minptime=10; useinbandfec=1 a=rtpmap:103 ISAC/16000 a=rtpmap:104 ISAC/32000 a=rtpmap:9 G722/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:106 CN/32000 a=rtpmap:105 CN/16000 a=fingerprint:sha-256 F1:A4:DB:A3:AA:CC: 1C:B5:2F:91:A0:2F:E8:FD:0B:80:F7:02:1F: 9E:23:49:1C:06:1F:4F:AB:96:44:F1:CB:D6 a=setup:actpass m=video 51034 UDP/TLS/RTP/SAVPF 100 101 116 117 96 c=IN IP4 54.244.51.11 a=rtcp:27538 IN IP4 54.244.51.11 a=ice-ufrag:mKHEfIR34arV95jd a=ice-pwd:YdwDVrwNPLwIsFF2sJtHZDSk a=fingerprint:sha-256 F1:A4:DB:A3:AA:CC: 1C:B5:2F:91:A0:2F:E8:FD:0B:80:F7:02:1F: 9E:23:49:1C:06:1F:4F:AB:96:44:F1:CB:D6 a=setup:actpass a=extmap:2 urn:ietf:params:rtp- hdrext:toffset a=extmap:3 http://www.webrtc.org/ experiments/rtp-hdrext/abs-send-time a=extmap:4 urn:3gpp:video-orientation a=sendrecv a=rtcp-mux a=rtpmap:100 VP8/90000 a=rtcp-fb:100 ccm fir -
© 2009 - 2014 Twilio, Inc. All rights reserved. Session
Description Protocol (SDP) v=0 o=- 7419095041521713450 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS BcUNfqxMTfmJ2aPDEipCTK4BnvabR7xKeTil m=audio 35898 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126 c=IN IP4 54.244.51.11 a=rtcp:39742 IN IP4 54.244.51.11 a=sendrecv a=rtcp-mux a=rtpmap:111 opus/48000/2 a=fmtp:111 minptime=10; useinbandfec=1 a=rtpmap:103 ISAC/16000 a=rtpmap:104 ISAC/32000 a=rtpmap:9 G722/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:106 CN/32000 a=rtpmap:105 CN/16000 -
© 2009 - 2014 Twilio, Inc. All rights reserved. Session
Description Protocol (SDP) v=0 o=- 7419095041521713450 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS BcUNfqxMTfmJ2aPDEipCTK4BnvabR7xKeTil m=audio 35898 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126 c=IN IP4 54.244.51.11 a=rtcp:39742 IN IP4 54.244.51.11 a=sendrecv a=rtcp-mux a=rtpmap:111 opus/48000/2 a=fmtp:111 minptime=10; useinbandfec=1 a=rtpmap:103 ISAC/16000 a=rtpmap:104 ISAC/32000 a=rtpmap:9 G722/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:106 CN/32000 a=rtpmap:105 CN/16000 -
© 2009 - 2014 Twilio, Inc. All rights reserved. Object
RTC (ORTC) • Built on the same open standards as WebRTC (STUN, TURN, ICE, DTLS, RTP) • Bring-your-own signaling • Does not use Session Description Protocol (SDP) • Lower-level JavaScript APIs -
WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security
(DTLS) RTCIceTransport RTCIceGatherer RTCDtlsTransport -
WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security
(DTLS) Secure Real-time Transport Protocol (SRTP) RTCIceTransport RTCIceGatherer RTCDtlsTransport RTCRtpReceiver RTCRtpSender -
© 2009 - 2014 Twilio, Inc. All rights reserved. Object
RTC (ORTC) • Built on the same open standards as WebRTC (STUN, TURN, ICE, DTLS, RTP) • Bring-your-own signaling • Does not use Session Description Protocol (SDP) • Lower-level JavaScript APIs • RTCIceGatherer • RTCIceTransport • RTCDtlsTransport • RTCRtpSender • RTCRtpReceiver -
© 2009 - 2014 Twilio, Inc. All rights reserved. Object
RTC (ORTC) • Built on the same open standards as WebRTC (STUN, TURN, ICE, DTLS, RTP) • Bring-your-own signaling • Does not use Session Description Protocol (SDP) • Lower-level JavaScript APIs • RTCIceGatherer • RTCIceTransport • RTCDtlsTransport • RTCRtpSender • RTCRtpReceiver • Power and flexibility over “plain” WebRTC -
© 2009 - 2014 Twilio, Inc. All rights reserved. Bridging
the Gap ORTC • RTCIceGatherer • RTCIceTransport • RTCDtlsTransport • RTCRtpSender • RTCRtpReceiver WebRTC • RTCPeerConnection • RTCIceCandidate • RTCSessionDescription -
© 2009 - 2014 Twilio, Inc. All rights reserved. Browser
Support <script src="twilio.js"></script> -
© 2009 - 2014 Twilio, Inc. All rights reserved. Browser
Support ORTC <script src="twilio.js"></script> -
© 2009 - 2014 Twilio, Inc. All rights reserved. Review
• Flash is no longer the only real-time communications solution for browsers. -
© 2009 - 2014 Twilio, Inc. All rights reserved. Review
• Flash is no longer the only real-time communications solution for browsers. • WebRTC and ORTC enable interoperable, plugin-free communication. -
© 2009 - 2014 Twilio, Inc. All rights reserved. Review
• Flash is no longer the only real-time communications solution for browsers. • WebRTC and ORTC enable interoperable, plugin-free communication. • Real-time communications is complex, but developing voice applications does not have to be complicated. -
© 2009 - 2014 Twilio, Inc. All rights reserved. Voice
Applications Ready Not Ready