WebRTC official site • http://www.webrtc.org/ – HTML5 ROCKS • http://www.html5rocks.com/en/tutorials/webrtc/basics/ – WebRTC for Beginners Muaz Khan • https://www.webrtc-experiment.com/docs/webrtc-for-beginners.html • https://github.com/muaz-khan/WebRTC-Experiment – WebRTC Conference 2013 Atlanta • http://www.webrtcworld.com/conference/west/default.aspx • http://images.tmcnet.com/expo/webrtc-conference/pdfs/WebRTC%20SG13AtlantaRe- CapSml.pdf – Socket.io • Official http://socket.io/ Japanese http://jxck.github.io/socket.io/ – And so many blogs. • All rights of each product is reserved by original right holder. – ®, TM are omitted. 2
Blog entries – http://t3n.de/news/cube-slam-pong-klon-live-video-473325/ – http://japanese.engadget.com/2013/06/12/google-cube-slam- webrtc-pong/ 4 WebRTC here
fixed telephone mobile phone (TV broadcast) method Over The Top Skype, WebEx (YouTube, USTREAM) Web Browser WebRTC can speak to world- wide (with cost) for users can speak to world- wide with low cost no application, free to use only careers with infrastructure can participate market independent of careers, few venders can participate without special infrastructure, everyone can participate none for venders limited API, limited combination fully programmable, use as component single use usage user can combine built in to products / services call center, e-commerce, information site, etc.
WORD and Calculator Total: 568,900 Automatic calculation Built in making estimate sheet with EXCEL Total: 568,900 Calculate sum by hand, and type into WORD by hand
Screen capture • Stream (MediaStream) • P2P communication (RTCPeerConnection) • Data transfer (DataChannel) • Related API, HTML features – JavaScript (most important) – Video, Audio – WebScoket – WebAudio API – Canvas – WebGL – …, etc. 7
- (Camera, microphone. Configuration for screen capture) – FireFox 22 - (Camera, microphone. NO screen capture) – Opera 15 - (Camera, microphone. NO screen capture) *different prefix for JavaScript Chrome: webkit, FireFox: moz • Android – Chrome 29 - (Camera, microphone. NO screen capture) • iOS – Not supported yet – Google says, it is on progress. coming soon. • At WebRTC Conference 2013 Atlanta • * I hope it will be include in Chrome for iOS near future. 14
of each other • Have to know UPD port number to use – UDP port is decided dynamically • Negotiation process is necessary, before RTCPeerConnection communication – This process is called as “Signaling” 25 RTCPeerConnection RTCPeerConnection UDP/IP IP address of each other UDP port number to use
by both browser – → Signaling server is prepared usually • No standard of signaling protocol – Your own way • WebSocket (TCP/IP) • Ajax (HTTP, HTTPS) – Existing protocol • SIP(for VoIP) with WebSocket(TCP/IP) • XMPP(for IM）with WebSocket(TCP/IP) 26
by both browser – → Signaling server is prepared usually • No standard of signaling protocol – Your own way • WebSocket (TCP/IP) • Ajax (HTTP, HTTPS) – Existing protocol • SIP(for VoIP) with WebSocket(TCP/IP) • XMPP(for IM）with WebSocket(TCP/IP) 27 Node.js ( + socket.io )
Information of session contents, such as media type (video, audio), codec, etc. – IP address, port number – P2P data transfer protocol → Secure RTP is used in WebRTC – Band width – Session property (name, id, active time, etc.) • Interactive Connectivity Establishment (ICE) – Lists of possible transfer roots • Direct P2P • Using STUN to go through NAT port mapping • Using TRUN, with packet relay server – Shortest (less hop) root will be chosen 28
start communication function connect() { sendOffer(); } // end of communication function hangUp() { socket.json.send({type: "bye"}); stop(); } function stop() { peerConn.close(); peerConn = null; }
40 var peerConn = null; // RTCPeerConnection function sendOffer() { peerConn = prepareNewConnection(); peerConn.createOffer(function (sessionDescription) { // in case of success peerConn.setLocalDescription(sessionDescription); socket.json.send(sessionDescription); }, function () { // in case of error console.log("Create Offer failed"); }, mediaConstraints); } function sendAnswer(evt) { peerConn = prepareNewConnection(); peerConn.setRemoteDescription(new RTCSessionDescription(evt)); peerConn.createAnswer(function (sessionDescription) { // in case of success peerConn.setLocalDescription(sessionDescription); socket.json.send(sessionDescription); }, function () { // in case of error console.log("Create Answer failed"); }, mediaConstraints); } Signaling SDP Signaling SDP
{ var pc_config = {"iceServers":[]}; var peer = new webkitRTCPeerConnection(pc_config); peer.onicecandidate = function (evt) { if (evt.candidate) { socket.json.send({type: "candidate", sdpMLineIndex: evt.candidate.sdpMLineIndex, sdpMid: evt.candidate.sdpMid, candidate: evt.candidate.candidate}); } else { } }; peer.addStream(localStream); peer.addEventListener("addstream", onRemoteStreamAdded, false); peer.addEventListener("removestream", onRemoteStreamRemoved, false) function onRemoteStreamAdded(event) { remoteVideo.src = window.webkitURL.createObjectURL(event.stream); // set stream to video element } function onRemoteStreamRemoved(event) { remoteVideo.src = ""; // clear stream of video element } return peer; } Signaling ICE
bring a WebSocket-like API to many browsers and devices • with some specific features to help with the creation of real-world real-time applications and games – Multiple transport support (old user agents, mobile browsers, etc.). 'websocket' , 'flashsocket' , 'htmlfile' , 'xhr-polling' , 'jsonp-polling‘ – Multiple sockets under the same connection (namespaces). • Why do I need to write like this? – var socket = io.connect('http://localhost:9001/'); • I want to use WebScoket, so why not like this? – io.connect('localhost:9001/'); 43
Socket.IO client first decides on a transport to utilize to connect. – A simple HTTP(or HTTPS) handshake takes place at the beginning of a Socket.IO connection. – The handshake, if successful, results in the client receiving: • A session id that will be given for the transport to open connections. • A number of seconds within which a heartbeat is expected (heartbeat timeout) • A number of seconds after the transport connection is closed when the socket is considered disconnected if the transport connection is not reopened (close timeout). 44
{ console.log('Channel opened.'); channelReady = true; var roomname = getRoomName(); socket.emit('enter', roomname); } function getRoomName() { // for example, set ?roomname in URL var url = document.location.href; var args = url.split('?'); if (args.length > 1) { var room = args[1]; if (room != "") { return room; } } return "_defaultroom"; }
B send offer sdp setRemoteDescription(sdp) setRemoteDescription(sdp) send answer sdp send ice addIceCandidate(ice) send ice addIceCandidate(ice) send ice addIceCandidate(ice) send ice addIceCandidate(ice) end of candidate end of candidate P2P stream transfer createNewPeer createNewPeer send Ice successive from one side send Ice successive from the other side
B send offer sdp setRemoteDescription(sdp) setRemoteDescription(sdp) send answer sdp send ice addIceCandidate(ice) send ice addIceCandidate(ice) send ice addIceCandidate(ice) send ice addIceCandidate(ice) end of candidate end of candidate P2P stream transfer createNewPeer createNewPeer Accept ICE before create Answer SDP This is not an error case. This is acceptable.
1 audio – 1 to 1 video (& audio) – 1 to 1 screen capture (& audio) – n to n audio – n to n video (& audio) – n to n screen capture (& audio) 52 Multiple people in 1 room, multiple rooms
A User B send offer sdp (broadcast) setRemoteDescription(sdp) setRemoteDescription(sdp) send answer sdp User C setRemoteDescription(sdp) send offer sdp (broadcast) send answer sdp createNewPeer createNewPeer createNewPeer no peer! → createNewPeer setRemoteDescription(sdp) send offer sdp corrupt! OK OK OK
A User B send “response” setRemoteDescription(sdp) setRemoteDescription(sdp) send answer sdp User C setRemoteDescription(sdp) send “call” (broadcast) send answer sdp createNewPeer createNewPeer createNewPeer setRemoteDescription(sdp) send offer sdp send offer sdp OK OK send “response” createNewPeer OK setRemoteDescription(sdp) OK send “call” (broadcast)
var connections = {}; // Connection hash function Connection() { // Connection Class var self = this; var id = ""; // socket.id of partner var peerconnection = null; // RTCPeerConnection instance var established = false; // is Already Established var iceReady = false; } function getConnection(id) { var con = null; con = connections[id]; return con; } function addConnection(id, connection) { connections[id] = connection; } getConnectionCount(), isConnectPossible(), deleteConnection(id), …, etc.
server by yourself? – http://code.google.com/p/rfc5766-turn-server/ • How to configure firewall, NAT, Proxy ?? 72 Please tell me, any information or advice!
JavaScript? – To subtract sound waveform – To invert sound waveform and add • Something like noise canceling headphone 76 Please tell me, any information or advice!
Camera, microphone, screen capture. – Combination with other HTML5 features • Canvas, CSS, WebAudio API, WebGL • WebRTC is flexible – Local, 1 to 1, n to n – Bidirectional, single-way • WebRTC is disruptive – Any one can participate • WebRTC is programmable – Built into your own products / services 77
mganeko
thomasvitale
foursue
tatsuo48
ma3tk
bengo4com
aiqlab
oracle4engineer
scotthsieh825
kazuya_araki_tokyo
k5k
coconala_engineer
konifar
62gerente
lara
geoffreycrofte
holman
destraynor
garrettdimon
rasmusluckow
pauljervisheath
mojombo
kneath
addyosmani
brianwarren
