Upgrade to Pro — share decks privately, control downloads, hide ads and more …

WebRTC basic and workshop

2836c80ba2ab0c0a0fcd820c713b106d?s=47 mganeko
October 15, 2013

WebRTC basic and workshop

This explains about basic of webrtc, such as userMedia, MediaStream, PeerConnection, and signaling process.
This presentation is used for workshop in my company.

2836c80ba2ab0c0a0fcd820c713b106d?s=128

mganeko

October 15, 2013
Tweet

Transcript

  1. WebRTC Basic & Workshop 2013.10.11 INFOCOM CORPORATION https://www.infocom.co.jp/english/aboutus/group/index.html m.ganeko@infocom.co.jp 1

  2. References • This document is referring following web sites. –

    WebRTC official site • http://www.webrtc.org/ – HTML5 ROCKS • http://www.html5rocks.com/en/tutorials/webrtc/basics/ – WebRTC for Beginners Muaz Khan • https://www.webrtc-experiment.com/docs/webrtc-for-beginners.html • https://github.com/muaz-khan/WebRTC-Experiment – WebRTC Conference 2013 Atlanta • http://www.webrtcworld.com/conference/west/default.aspx • http://images.tmcnet.com/expo/webrtc- conference/pdfs/WebRTC%20SG13AtlantaRe-CapSml.pdf – And so many blogs. • All rights of each product is reserved by original right holder. – ®, TM are omitted. 2
  3. What’s WebRTC • Web Real-Time Communication – Frameworks for realizing

    Real-Time Communication on Web Browser – A Part of HTML5 • Many technologies are combined – Do you know what are included? 3
  4. WebRTC elements • User Media – Camera – Microphone –

    Screen capture • Stream (MediaStream) • P2P communication (RTCPeerConnection) • Data transfer (DataChannel) • Related API, HTML features – JavaScript (most important) – Video, Audio – WebScoket – WebAudio API – Canvas – WebGL – …, etc. 4
  5. WebRTC ready browsers • Windows / Mac – Chrome 26

    - (Camera, microphone. Configuration for screen capture) – FireFox 22 - (Camera, microphone. NO screen capture) – Opera 15 - (Camera, microphone. NO screen capture) *different prefix for JavaScript Chrome: webkit, FireFox: moz • Android – Chrome 29 - (Camera, microphone. NO screen capture) • iOS – Not supported yet – Google says, it is on progress. coming soon. • At WebRTC Conference 2013 Atlanta • * I hope it will be include in Chrome for iOS near future. 5
  6. Why so interested in WebRTC? Because WebRTC is disruptive. Career

    fixed telephone mobile phone (TV broadcast) method Over The Top Skype, WebEx (YouTube, USTREAM) Web Browser WebRTC can speak to world- wide (with cost) for users can speak to world- wide with low cost no application, free to use only careers with infrastructure can participate market independent of careers, few venders can participate without special infrastructure, everyone can participate none for venders limited API, limited combination fully programmable, use as component single use usage user can combine built in to products / services call center, e-commerce, information site, etc.
  7. Combination VS. Built in Combination  making estimate sheet with

    WORD and Calculator Total: 568,900 Automatic calculation Built in  making estimate sheet with EXCEL Total: 568,900 Calculate sum by hand, and type into WORD by hand
  8. Media 8

  9. Let’s try with Chrome • Prepare – OS: Windows, Mac

    OS X – Browser:Chrome 26 or later – Web cam, microphone – editor app, as you like • Contents – Handling User Media – Handling stream 9
  10. Self camera (for Chrome) <html> <head><title>WebRTC Camera</title></head> <body> <video id="video1"

    autoplay="1"></video> </body> <script> var video1 = document.getElementById('video1'); navigator.webkitGetUserMedia({audio:false, video:true}, function(stream) { // success video1.src = window.webkitURL.createObjectURL(stream); }, function(err) { // error console.log(err); } ); </script> </html> 10
  11. Goal 11

  12. Try in local machine (file:// ) • Nothing must be

    shown • Watch JavaScript console … – NavigatorUserMediaError {code: 1, PERMISSION_DENIED: 1} … – PERMISSION ERROR • Local HTML (file://) can not access to User Media •  So, please put it into your Web Server – Apache, nginx, IIS, as you like. 12
  13. Goal, again 13

  14. Sepia camera <html> <head><title>WebRTC Camera</title><head> <body> <video id="video1" autoplay="1" style="-webkit-filter:

    sepia(80%); "></video> </body> <script> var video1 = document.getElementById('video1'); navigator.webkitGetUserMedia({audio:false, video:true}, function(stream) { // success video1.src = window.webkitURL.createObjectURL(stream); }, function(err) { // error console.log(err); } ); </script> </html> 14
  15. Goal 15

  16. Mirror Camera <html> <head><title>WebRTC Camera</title><head> <body> <video id="video1" autoplay="1" ></video>

    <video id="video2" autoplay="1" style="-webkit-transform: scaleX(-1);"></video> </body> <script> var video1 = document.getElementById('video1'); var video2 = document.getElementById('video2'); navigator.webkitGetUserMedia({audio:false, video:true}, function(stream) { // success video1.src = window.webkitURL.createObjectURL(stream); video2.src = window.webkitURL.createObjectURL(stream); }, function(err) { // error console.log(err); } ); </script> </html> 16
  17. Goal 17

  18. Other variations Thin camera <video id="video2" autoplay="1" style="-webkit-transform: scaleX(0.5);"></video> Heavy

    camera <video id="video2" autoplay="1" style="-webkit-transform: scaleX(1.5);"></video> Hang down camera <video id="video2" autoplay="1" style="-webkit-transform: scaleY(-1);"></video> Black and White <video id="video2" autoplay="1" style="-webkit-filter: grayscale(100%);"></video> Lean camera <video id="video6" autoplay="1" style="-webkit-transform: rotate(-30deg);"></video> 18
  19. 2 video tag and MediaStream (in mirror camera) 19 1

    MediaStream can connect to several outputs. MediaStream : output = 1 : n
  20. Howling <html> <head><title>WebRTC Camera</title><head> <body> <video id="video1" autoplay="1"></video> </body> <script>

    var video1 = document.getElementById('video1'); navigator.webkitGetUserMedia({audio:true, video:true}, function(stream) { // success video1.src = window.webkitURL.createObjectURL(stream); }, function(err) { // error console.log(err); } ); </script> </html> 20
  21. How howling happens (local) 21 Most of samples are implemented

    like this  Headphone is necessary
  22. summary • User Media (microphone, camera) – navigator.webkitGetUserMedia( {audio:true, video:true},

    … • User Media (Screen Capture) – navigator.webkitGetUserMedia( video : { mandatory : { chromeMediaSource : "screen" } }, • Security (Access permission) • Media Stream – MediaStream (video, audio) 22
  23. WebRTC Security Access permission 23

  24. Media layer 24

  25. Communication 25

  26. Communication • RTCPeerConnection – Transfer root for MediaStream ( video

    and audio ) – P2P (Peer to Peer)  browser to browser – use UDP/IP 26 RTCPeerConnection RTCPeerConnection UDP/IP
  27. Before starting P2P communication • Have to know IP address

    of each other • Have to know UPD port number to use – UDP port is decided dynamically • Negotiation process is necessary, before RTCPeerConnection communication – This process is called as “Signaling” 27 RTCPeerConnection RTCPeerConnection UDP/IP IP address of each other UDP port number to use
  28. Signaling before P2P • Broker is needed, which is known

    by both browser – → Signaling server is prepared usually • No standard of signaling protocol – Your own way • WebSocket (TCP/IP) • Ajax (HTTP, HTTPS) – Existing protocol • SIP(for VoIP) with WebSocket(TCP/IP) • XMPP(for IM)with WebSocket(TCP/IP) 28
  29. Exchanged information while signaling • Session Description Protocol (SDP) –

    Information of session contents, such as media type (video, audio), codec, etc. – IP address, port number – P2P data transfer protocol → Secure RTP is used in WebRTC – Band width – Session property (name, id, active time, etc.) • Interactive Connectivity Establishment (ICE) – Lists of possible transfer roots • Direct P2P • Using STUN to go through NAT port mapping • Using TRUN, with packet relay server – Shortest (less hop) root will be chosen 29
  30. Direct P2p in same network 30 Dynamic UDP port (50000~65535)

  31. IP addr / UDP port exchange via STUN Then P2P

    over NAT 31
  32. Packet relay via TURN server (go through Firewall) 32

  33. Sequence before P2P (1) with WebSocket 33 PeerConnection socket Application

    Signaling Server PeerConnection socket Application connect() connect connect() connect createOffer() offer sdp setLocalDescription(sdp) send(sdp) send sdp send sdp onMessage(sdp) setRemoteDescription(sdp) createAnswer() answer sdp send sdp send(sdp) send sdp onMessage(sdp) setRemoteDescription(sdp) setLocalDescription(sdp)
  34. Sequence before P2P (2) with WebSocket 34 PeerConnection socket Application

    Signaling Server PeerConnection socket Application onIceCandidate(ice) send(ice) send ice send ice onMessage(ice) addIceCandidate(ice) send ice send(ice) send ice onMessage(ice) addIceCandidate(ice) onIceCandidate(ice) onIceCandidate() : end of candidate onIceCandidate() : end of candidate P2P stream transfer At last, P2P starts.
  35. Advantage / disadvantage of P2P • Advantage – No server

    transfer, no overhead – Without high spec sever • Disadvantage – Communication path glows exponentially, for many participants • 2 node → 1 path • 3 node → 3 path • 4 node → 6 path • 5 node → 10 path • 6 node → 15 path 35 P2P Server-client P2P for 4 Server-client for 4
  36. WebRTC communication types • bidirectional communication – 1 to 1

    audio – 1 to 1 video (& audio) – 1 to 1 screen capture (& audio) – n to n audio – n to n video (& audio) – n to n screen capture (& audio) 36
  37. WebRTC communication types • one-way communication – 1 → 1

    audio – 1 → 1 video (& audio) – 1 → 1 screen capture (& audio) – 1 → n audio – 1 → n video (& audio) – 1 → n screen capture (& audio) 37
  38. WebRTC communication types • standalone – microphone capture → speaker

    – camera capture → view in screen – screen capture – audio recording – video recording – Image capture (with Canvas) 38
  39. WebRTC class relation 39

  40. Howling prevention (local) 40 It seems to work…

  41. …still happens with partner Howling with partner 41 Auto mute,

    or auto band filter should be needed. Is it possible in JavaScript? (I don’t know)
  42. Screen capture & audio 42 2 MediaStream into 1 PeerConnection

  43. Data transfer (DataChannel) • Data transfer over RTCPeerConnection • Using

    UDP/IP, so packet may be lost – → DataChannel has retry feature (not packet, but whole data) • Events of onOpen, onClose, onMessage • For text chat, file transfer, game • DataChannel VS. WebScoket – DataChannel is P2P, WebSocket is server-client – DataChannel is UDP/IP, WebSocket is TCP/IP • Specification is not determined yet. 43
  44. Conclusion • WebRTC is powerful – Expand web browser potential.

    Camera, microphone, screen capture. – Combination with other HTML5 features • Canvas, CSS, WebAudio API, WebGL • WebRTC is flexible – Local, 1 to 1, n to n – Bidirectional, single-way • WebRTC is disruptive – Any one can participate • WebRTC is programmable – Built into your own products / services 44
  45. WebRTC has no limit. Your imagination will define limit. END

    45