Enabling Voice Applications with WebRTC and ORTC in Microsoft Edge

Mark Roberts, Software Engineer on Twilio’s SDK Team, will describe how Twilio brought ORTC support to their WebRTC SDK, lessons learned along the way, and Twilio’s ORTC roadmap.

Transcript

1. ENABLING VOICE APPLICATIONS WITH WEBRTC & ORTC IN MICROSOFT EDGE MARK ROBERTS SOFTWARE ENGINEER MROBERTS@TWILIO.COM
2. © 2009 - 2014 Twilio, Inc. All rights reserved. Voice Applications
3. © 2009 - 2014 Twilio, Inc. All rights reserved. Voice Applications
4. © 2009 - 2014 Twilio, Inc. All rights reserved. Voice Applications
5. © 2009 - 2014 Twilio, Inc. All rights reserved. Voice Applications PSTN
6. © 2009 - 2014 Twilio, Inc. All rights reserved. Voice Applications Ready Not Ready
7. © 2009 - 2014 Twilio, Inc. All rights reserved. Voice Applications Ready Not Ready
8. © 2009 - 2014 Twilio, Inc. All rights reserved. Voice Applications Ready Not Ready
9. © 2009 - 2014 Twilio, Inc. All rights reserved. Voice Applications Ready Not Ready
10. + =
11. <script src="twilio.js"></script>
12. © 2009 - 2014 Twilio, Inc. All rights reserved. Four Basic Tasks Four basic tasks for a voice application:
13. © 2009 - 2014 Twilio, Inc. All rights reserved. Four Basic Tasks Four basic tasks for a voice application: 1. Capture audio from a microphone
14. © 2009 - 2014 Twilio, Inc. All rights reserved. Four Basic Tasks Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint
15. © 2009 - 2014 Twilio, Inc. All rights reserved. Four Basic Tasks Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint 3. Receive audio from an endpoint
16. © 2009 - 2014 Twilio, Inc. All rights reserved. Four Basic Tasks Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint 3. Receive audio from an endpoint 4.Playback received audio
17. FIRST, THERE WAS FLASH…
18. © 2009 - 2014 Twilio, Inc. All rights reserved. Browser Support <script src="twilio.js"></script>
19. ALONG CAME WEBRTC…
20. © 2009 - 2014 Twilio, Inc. All rights reserved. WebRT-what⁇
21. © 2009 - 2014 Twilio, Inc. All rights reserved. WebRT-what⁇ • Simple JavaScript APIs • RTCPeerConnection • RTCIceCandidate • RTCSessionDescription • First-class browser support • Built on open standards
 (STUN, TURN, ICE, DTLS, RTP)
22. © 2009 - 2014 Twilio, Inc. All rights reserved. Four Basic Tasks Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint 3. Receive audio from an endpoint 4.Playback received audio
23. © 2009 - 2014 Twilio, Inc. All rights reserved. Four Basic Tasks Four basic tasks for a voice application: 1. Capture audio from a microphone 2.Send audio to an endpoint 3.Receive audio from an endpoint 4.Playback received audio
24. WebRTC Endpoint WebRTC Endpoint
25. WebRTC Endpoint WebR establish connectivity
26. WebRTC Endpoint Interactive Connectivity Establishment (ICE) WebR establish connectivity
27. WebRTC Endpoint Interactive Connectivity Establishment (ICE) WebR establish connectivity secure connectivity
28. WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security (DTLS) WebR establish connectivity secure connectivity
29. WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security (DTLS) WebR establish connectivity secure connectivity send and receive media
30. WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security (DTLS) Secure Real-time Transport Protocol (SRTP) WebR establish connectivity secure connectivity send and receive media
31. © 2009 - 2014 Twilio, Inc. All rights reserved. WebRT-what⁇ • Simple JavaScript APIs • RTCPeerConnection • RTCIceCandidate • RTCSessionDescription • First-class browser support • Built on open standards
 (STUN, TURN, ICE, DTLS, RTP)
32. © 2009 - 2014 Twilio, Inc. All rights reserved. WebRT-what⁇ • Simple JavaScript APIs • RTCPeerConnection • RTCIceCandidate • RTCSessionDescription • First-class browser support • Built on open standards
 (STUN, TURN, ICE, DTLS, RTP) • Interoperability • No plugins required! • Bring-your-own signaling
33. WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security (DTLS) Secure Real-time Transport Protocol (SRTP) WebR establish connectivity secure connectivity send and receive media
34. WebRTC Endpoint WebR
35. © 2009 - 2014 Twilio, Inc. All rights reserved. Browser Support <script src="twilio.js"></script>
36. © 2009 - 2014 Twilio, Inc. All rights reserved. Browser Support <script src="twilio.js"></script>
37. © 2009 - 2014 Twilio, Inc. All rights reserved. Browser Support
38. © 2009 - 2014 Twilio, Inc. All rights reserved. Browser Support <script src="twilio.js"></script>
39. EDGE & ORTC ANNOUNCED…
40. © 2009 - 2014 Twilio, Inc. All rights reserved. Object RTC (ORTC)
41. © 2009 - 2014 Twilio, Inc. All rights reserved. Object RTC (ORTC) • Built on the same open standards as WebRTC
 (STUN, TURN, ICE, DTLS, RTP) • Bring-your-own signaling
42. © 2009 - 2014 Twilio, Inc. All rights reserved. Object RTC (ORTC) • Built on the same open standards as WebRTC
 (STUN, TURN, ICE, DTLS, RTP) • Bring-your-own signaling • Does not use Session Description Protocol (SDP)
43. WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security (DTLS) Secure Real-time Transport Protocol (SRTP) WebR establish connectivity secure connectivity send and receive media
44. WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security (DTLS) Secure Real-time Transport Protocol (SRTP) SDP WebR
45. © 2009 - 2014 Twilio, Inc. All rights reserved. Session Description Protocol (SDP) v=0 o=- 7419095041521713450 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS BcUNfqxMTfmJ2aPDEipCTK4BnvabR7xKeTil m=audio 35898 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126 c=IN IP4 54.244.51.11 a=rtcp:39742 IN IP4 54.244.51.11 a=sendrecv a=rtcp-mux a=rtpmap:111 opus/48000/2 a=fmtp:111 minptime=10; useinbandfec=1 a=rtpmap:103 ISAC/16000 a=rtpmap:104 ISAC/32000 a=rtpmap:9 G722/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:106 CN/32000 a=rtpmap:105 CN/16000 a=fingerprint:sha-256 F1:A4:DB:A3:AA:CC: 1C:B5:2F:91:A0:2F:E8:FD:0B:80:F7:02:1F: 9E:23:49:1C:06:1F:4F:AB:96:44:F1:CB:D6 a=setup:actpass m=video 51034 UDP/TLS/RTP/SAVPF 100 101 116 117 96 c=IN IP4 54.244.51.11 a=rtcp:27538 IN IP4 54.244.51.11 a=ice-ufrag:mKHEfIR34arV95jd a=ice-pwd:YdwDVrwNPLwIsFF2sJtHZDSk a=fingerprint:sha-256 F1:A4:DB:A3:AA:CC: 1C:B5:2F:91:A0:2F:E8:FD:0B:80:F7:02:1F: 9E:23:49:1C:06:1F:4F:AB:96:44:F1:CB:D6 a=setup:actpass a=extmap:2 urn:ietf:params:rtp- hdrext:toffset a=extmap:3 http://www.webrtc.org/ experiments/rtp-hdrext/abs-send-time a=extmap:4 urn:3gpp:video-orientation a=sendrecv a=rtcp-mux a=rtpmap:100 VP8/90000 a=rtcp-fb:100 ccm fir
46. © 2009 - 2014 Twilio, Inc. All rights reserved. Session Description Protocol (SDP) v=0 o=- 7419095041521713450 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS BcUNfqxMTfmJ2aPDEipCTK4BnvabR7xKeTil m=audio 35898 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126 c=IN IP4 54.244.51.11 a=rtcp:39742 IN IP4 54.244.51.11 a=sendrecv a=rtcp-mux a=rtpmap:111 opus/48000/2 a=fmtp:111 minptime=10; useinbandfec=1 a=rtpmap:103 ISAC/16000 a=rtpmap:104 ISAC/32000 a=rtpmap:9 G722/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:106 CN/32000 a=rtpmap:105 CN/16000
47. © 2009 - 2014 Twilio, Inc. All rights reserved. Session Description Protocol (SDP) v=0 o=- 7419095041521713450 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS BcUNfqxMTfmJ2aPDEipCTK4BnvabR7xKeTil m=audio 35898 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126 c=IN IP4 54.244.51.11 a=rtcp:39742 IN IP4 54.244.51.11 a=sendrecv a=rtcp-mux a=rtpmap:111 opus/48000/2 a=fmtp:111 minptime=10; useinbandfec=1 a=rtpmap:103 ISAC/16000 a=rtpmap:104 ISAC/32000 a=rtpmap:9 G722/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:106 CN/32000 a=rtpmap:105 CN/16000
48. © 2009 - 2014 Twilio, Inc. All rights reserved. Object RTC (ORTC) • Built on the same open standards as WebRTC
 (STUN, TURN, ICE, DTLS, RTP) • Bring-your-own signaling • Does not use Session Description Protocol (SDP) • Lower-level JavaScript APIs
49. WebRTC Endpoint
50. WebRTC Endpoint Interactive Connectivity Establishment (ICE) RTCIceTransport RTCIceGatherer
51. WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security (DTLS) RTCIceTransport RTCIceGatherer RTCDtlsTransport
52. WebRTC Endpoint Interactive Connectivity Establishment (ICE) Datagram Transport Layer Security (DTLS) Secure Real-time Transport Protocol (SRTP) RTCIceTransport RTCIceGatherer RTCDtlsTransport RTCRtpReceiver RTCRtpSender
53. © 2009 - 2014 Twilio, Inc. All rights reserved. Object RTC (ORTC) • Built on the same open standards as WebRTC
 (STUN, TURN, ICE, DTLS, RTP) • Bring-your-own signaling • Does not use Session Description Protocol (SDP) • Lower-level JavaScript APIs • RTCIceGatherer • RTCIceTransport • RTCDtlsTransport • RTCRtpSender • RTCRtpReceiver
54. © 2009 - 2014 Twilio, Inc. All rights reserved. Object RTC (ORTC) • Built on the same open standards as WebRTC
 (STUN, TURN, ICE, DTLS, RTP) • Bring-your-own signaling • Does not use Session Description Protocol (SDP) • Lower-level JavaScript APIs • RTCIceGatherer • RTCIceTransport • RTCDtlsTransport • RTCRtpSender • RTCRtpReceiver • Power and flexibility over “plain” WebRTC
55. © 2009 - 2014 Twilio, Inc. All rights reserved. Bridging the Gap ORTC • RTCIceGatherer • RTCIceTransport • RTCDtlsTransport • RTCRtpSender • RTCRtpReceiver WebRTC • RTCPeerConnection • RTCIceCandidate • RTCSessionDescription
56. © 2009 - 2014 Twilio, Inc. All rights reserved. Browser Support
57. © 2009 - 2014 Twilio, Inc. All rights reserved. Browser Support <script src="twilio.js"></script>
58. © 2009 - 2014 Twilio, Inc. All rights reserved. Browser Support ORTC <script src="twilio.js"></script>
59. © 2009 - 2014 Twilio, Inc. All rights reserved. Review
60. © 2009 - 2014 Twilio, Inc. All rights reserved. Review • Flash is no longer the only real-time communications solution for browsers.
61. © 2009 - 2014 Twilio, Inc. All rights reserved. Review • Flash is no longer the only real-time communications solution for browsers. • WebRTC and ORTC enable interoperable, plugin-free communication.
62. © 2009 - 2014 Twilio, Inc. All rights reserved. Review • Flash is no longer the only real-time communications solution for browsers. • WebRTC and ORTC enable interoperable, plugin-free communication. • Real-time communications is complex, but developing voice applications does not have to be complicated.
63. © 2009 - 2014 Twilio, Inc. All rights reserved. Voice Applications Ready Not Ready
64. <script src="twilio.js"></script>
65. twilio.js
66. THANK YOU MARK ROBERTS SOFTWARE ENGINEER MROBERTS@TWILIO.COM