WebRTC. Real-time communication for mobile devices.

Transcript

1. WebRTC Real-time communication
2. Who am I? • Sergey Zenchenko @ Techery • sergeizenchenko@gmail.com • http://techery.io • @sergeyzenchenko
3. Examples of Real-time • VOIP • Instant messaging • Video conferencing • Real-time collaboration
4. Bob Alice
5. Bob Alice
6. Tasks • Record audio from the mic • Encode audio • Send audio to Bob • Decode audio • Send audio to the speaker
7. Send audio to Bob
8. Send audio to Bob with lowest possible latency
9. Protocols TCP vs UDP
10. TCP • Used almost everywhere • Connection based • Reliable • Error detection • Flow control • Congestion control
11. TCP problems • High latency • We don’t need reliability for real-time
12. User Datagram Protocol (UDP) A self-contained, independent entity of data carrying sufficient information to be routed from the source to the destination nodes without reliance on earlier exchanges between the nodes and the transporting network. Datagram is:
13. Datagram structure
14. UDP good parts • Low overhead • We are not waiting for packet receiving acknowledgement! • Optimal for real-time
16. UDP problems • No guarantee of message delivery • No guarantee of order of delivery • No connection state tracking • No congestion control
17. What should we use?
18. RTP FTW!
19. Real-time Transmission Protocol • Industrial standard for real-time communication • Designed for stream delivery over IP network • http://en.wikipedia.org/wiki/Real-time_Transport_Protocol • http://www.ietf.org/rfc/rfc3550.txt
20. RTP/RTCP • Works over UDP or TCP • RTCP for feedback and QoS • Guarantee of order of delivery • Connection state tracking • Congestion control • Can provide packet retransmission
21. RTP packet
22. RTP packet • Sequence number • Timestamp • Payload • Payload type
23. Order of delivery 1 2 3 4 5 6 7 8 Process packets
24. Network topology
25. Typical app Amazing app server Bob Alice
26. Problems • Additional delays • You need dedicated server which can handle high load traffic
27. Peer-to-peer Bob Alice
28. How to find way to connect?
29. Local Network 192.169.0.2 192.169.0.3 Bob Alice
30. What about devices outside local network? 192.169.0.2 10.15.0.6 Router A Router B 193.110.176.232 88.110.56.23 Bob Alice
31. Network Address Translation (NAT)
34. Interactive Connectivity Establishment (ICE) ICE is a protocol, and a set of methods, that seek to establish the most efficient tunnel between the participants
35. STUN • http://en.wikipedia.org/wiki/STUN • Protocol that allows the host application to discover the presence of a network address translator on the network and obtain the allocated public IP and port tuple for the current connection • Free to use STUN servers available in the internet Session Traversal Utilities for NAT
36. How it works
37. How it works • The application discovers its public IP and port tuple and is then able to use this information as part of its application data when communicating with its peers. • The outbound binding request to the STUN server establishes NAT routing entries along the path, such that the inbound packets arriving at the public IP and port tuple can now find their way back to the host application on the internal network. • The STUN protocol defines a simple mechanism for keepalive pings to keep the NAT routing entries from timing out.
38. What if UPD is blocked?
39. TURN Traversal Using Relays around NAT
40. IcE
41. WebRTC • http://www.webrtc.org • https://www.youtube.com/watch?v=p2HzZkd2A40 WebRTC is a free, open project that enables web browsers with Real-Time Communications (RTC) capabilities via simple JavaScript APIs. The WebRTC components have been optimized to best serve this purpose.
42. Usage • http://tokbox.com • http://webrtc.co • http://pubnub.github.io/rtc-pubnub-fileshare/guide/ • http://www.zingaya.com • https://vline.com
43. WebRTC features • ICE, STUN, TURN protocols implementation • Audio/Video codecs • Session management • RTP stack • Security
45. WebRTC native api • http://www.webrtc.org/webrtc-native-code-package • http://www.webrtc.org/reference/native-apis • Designed for browser developers • Available for iOS, Android. • http://ninjanetic.com/how-to-get-started-with-webrtc- and-ios-without-wasting-10-hours-of-your-life/
46. Main parts PeerConnection Audio Stream Video Stream Signalling Channel Application Data Stream
47. RTCPeerConnection • RTCPeerConnection manages the full ICE workflow for NAT traversal. • RTCPeerConnection sends automatic (STUN) keepalives between peers. • RTCPeerConnection keeps track of local/remote streams. • RTCPeerConnection triggers automatic stream renegotiation as required. • RTCPeerConnection provides necessary APIs to generate the connection offer, accept the answer, allows us to query the connection for its current state, and more.
48. • HTTP api • WebSocket • TCP socket • Email • Twitter • Pigeons
49. RTCPeerConnection Setup 1. Create peer connection 2. Create local stream 3. Add audio/video tracks to stream 4. Add stream to peer connection
50. Workflow Signalling Server Bob Alice Router Router Offer SDP
51. m=audio 1 RTP/SAVPF 111 ... 1 a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=candidate:1862263974 1 udp 2113937151 192.168.1.73 60834 typ host ... 2 a=mid:audio a=rtpmap:111 opus/48000/2 3 a=fmtp:111 minptime=10 Session Description Protocol (SDP)
52. Workflow Signalling Server Bob Alice Router Router Offer SDP
53. Workflow Signalling Server Bob Alice Router Router Offer SDP
54. Workflow Signalling Server Bob Alice Router Router Offer SDP
55. Workflow Signalling Server Bob Alice Router Router Offer SDP
56. Workflow Signalling Server Bob Alice Router Router Answer SDP
57. Workflow Signalling Server Bob Alice Router Router Answer SDP
58. Workflow Signalling Server Bob Alice Router Router Answer SDP
59. Workflow Signalling Server Bob Alice Router Router Answer SDP
60. Workflow Signalling Server Bob Alice Router Router Answer SDP
61. Workflow Signalling Server Bob Alice Router Router STUN Server What’s my IP??
62. Workflow Signalling Server Bob Alice Router Router STUN Server What’s my IP??
63. Workflow Signalling Server Bob Alice Router Router ICE candidate
64. ICE Candidate a=candidate:1862263974 1 udp 2113937151 192.168.1.73 60834 typ host
65. Workflow Signalling Server Bob Alice Router Router ICE candidate
66. Workflow Signalling Server Bob Alice Router Router ICE candidate
67. Workflow Signalling Server Bob Alice Router Router ICE candidate
68. Workflow Signalling Server Bob Alice Router Router ICE candidate
69. Workflow Signalling Server Bob Alice Router Router ICE candidate
70. Workflow Signalling Server Bob Alice Router Router ICE candidate ICE candidate ICE candidate ICE candidate
71. Code sample https://github.com/Techery/WebRTC-iOS
72. Demo http://crowdmics.com
73. Data Channel • Application data transferring • Stream Control Transmission Protocol (SCTP)
74. TCP vs UDP vs SCTP
75. References • http://chimera.labs.oreilly.com/books/1230000000545/ index.html • http://www.webrtc.org/reference/native-apis • www.webrtc.org/webrtc-native-code-package • https://github.com/Techery/WebRTC-iOS