Share
Embed
Playing with OSS WebRTC SFU meidasoup
English translation of the slides for WebRTC Meetup Tokyo 13
Un-offical document for how to use mediasoup from your Node.js App
mganeko
January 28, 2017
Transcript
20 slides
-
WebRTC Meetup Tokyo #13 Playing with OSS SFU meidasoup
INFOCOM CORPORATION Masashi Ganeko @massie_g Jan. 27th, 2017 1 -
About me • Masashi Ganeko / @massie_g
– working for a research team of INFOCOM CORPORATION • WriQng a series of WebRTC on HTML5Experts.jp (HTML5 tech site) – hZps://html5experts.jp/series/webrtc2016/ • English PresentaQon for WebRTC (2013, 2016) – hZps://speakerdeck.com/mganeko • Japanese PresentaQon for WebRTC (2013-‐2016) – hZp://www.slideshare.net/mganeko 2 -
Topic of today • SFU : SelecQve forwarding Unit
• mediasoup : – Open Source WebRTC SFU module for Node.js 3 -
P2P vs. SFU: SelecQve Forwarding Unit 4 Browser A
Browser B Browser D Browser C P2P • No media server • Browser side: • CPU load: Very High • Network load: Very High 4 Browser A Browser B Browser D Browser C SFU Dispatch audio/video SFU • User SFU server, CPU load: Low • Browser side: low CPU load • Browser side: network load: middle Able to choose display layout on member side ◎ -
MCU vs. SFU 5 Browser A Browser B
Browser D Browser C SFU Dispatch audio/video SFU • User SFU server, CPU load: Low • Browser side: low CPU load • Browser side: network load: middle Able to choose display layout on member side ◎ Browser A Browser B Browser D Browser C MCU Mix video / audio MCU • Use MCU server, CPU load: Super High • Browser side: very low CPU/Network load -
Encoding and EncrypQng 6 Browser Thanks to いらすとや hZp://www.irasutoya.com/
Browser P2P Encode Encrypt Transmit Decrypt Decode TURN Browser TURN Browser SFU MCU Browser SFU Browser Browser MCU Browser -
What is mediasoup ? • Open Source WebRTC SFU
– Web Site hZps://mediasoup.org/ – GitHub hZps://github.com/ibc/mediasoup/ • Not a Server App, but components for a Server – Instead of creaQng yet another opinionated server, mediasoup is a Node.js module which can be integrated into a larger applicaQon or made standalone with just a few lines of JavaScript. • Provided as Node.js module – You have to prepare Web Server and Signaling method 7 -
What is mediasoup ? (2) • Minimum … Only
Media (No DataChannel), No Codecs • Powerful … Main part is build with C++, on libuv – Node.js invokes child worker process – Node.js and worker communicate via interprocess communicaQon • JavaScript API for Node.js – WebRTC 1.0 … RTCPeerConnecQon – ORTC … Transport, RtpSender, RtpReceiver 8 -
DEMO of mediasoup sample 9 • MulQple parQcipants video chat
sample – Video and Audio – Looks same as P2P, but … • if you stop SFU Sever process, then Remote video/audio will stop • Source codes on GitHub – hZps://github.com/mganeko/mediasoup_sample -
How to install mediasoup • PreparaQon – Node.js v4.0.0
以上 – Linux / Mac OS X (Windows not supported yet) – Python 2.x (NOT Python 3.x !!) – make – gcc with g++ , or clang with C++11 • InstallaQon – $ npm install mediasoup • C++ compile may take a few minutes 10 -
Object structure of mediasoup 11 Server Room Room Room Peer
Peer Peer Transport Transport RtpSender RtpSender RtpSender RtpReceiver RtpReceiver RtpReceiver ・1 Server has Many Rooms ・1 Room has Many Peers ・1 Peer has mulQple RtpSender ・1 Peer has mulQple RtpReceiver ・1 Peer has mulQple のTransport Correspond to MediaTrack (video/audio) Correspond to MediaTrack Correspond to UDP/TCP port 1 Transport is used by -‐ mulQple RtpSender -‐ mulQple RtpReceiver BUNDLE, rtcp-‐mux mulQ-‐stream -
Inside of mediasoup Room (Guess) 12 Room Peer RtpReceiver RtpSender
Browser Browser Peer RtpReceiver RtpSender Peer Browser RtpReceiver RtpSender RtpSender RtpSender RtpSender Streams are connected automaQcally in the room (may be) Transport Transport Transport -
worker process 13 Default : Core number Case of
5 Peers in 1 Room All peers in the same Room are assigned to the same worker ? -
Rough Sketch to use let server = mediasoup.Server(); let
peer1; let receiver1; return server.createRoom(roomOpQons) .then( (room) => { peer1 = room.Peer('alice'); return peer.setCapabiliQes(peerCapabiliQes); }) .then(() => { return peer.createTransport({ tcp: false }); }) .then((transport) => { rtpReceiver1 = peer1.RtpReceiver('audio', transport); }) 14 • Compose objects step by step • Use Promise • Do not create RtpSender expressly • Will be created automaQcally, passed by event (I do not understand well ..) -
Another Object structure 15 Server Room Room Room Peer Peer
Peer RTCPeerConnecQon • 1 RTCPeerConnecQon corresponds to 1 Peer • Same interface as Browser Offer/Answer RTCPeerConnecQon RTCPeerConnecQon -
Flow with RTCPeerConencQon 16 Browser A RTCPeerConnecQon mediasoup
RTCPeerConnecQon A Browser B send offer SDP send offer SDP send answer SDP send answer SDP onnegoQaQonneeded() fired onnegoQaQonneeded() fired RTCPeerConnecQon B RTCPeerConnecQon send answer SDP onaddstream() / ontrack() fired send offer SDP send offer SDP send answer SDP onaddstream() / ontrack() fired createOffer() createAnswer() createOffer() createAnswer() createOffer() createAnswer() createOffer() createAnswer() -
LimitaQon of RTCPeerConnecQon • Need iniQal Offer from Remote (Browser)
– Currently, the mediasoup implementaQon of RTCPeerConnecQon requires that the iniQal offer comes from the remote endpoint, • Aser onnegoQaQonneeded() event, create Offer in mediasoup • Not allowed to re-‐creaQng Offer in Remote (Browser) – Add/Remove of Remote stream/track is not supported • It seems like Plan B support is tentaQve. 17 LimitaQon of RTCPeerConnecQon, NOT limitaQon of mediasoup itself. It must be possible if you compose Transport, RtpReceiver, SDP manually. -
Other limitaQon of mediasoup • mediasoup should be outside of
NAT/Firewall. Need global IP address. – mediasoup does not look for STUN/TURN server • Browser inside of NAT/Firewall is OK – When Browoser looks for STUN/TURN server, it is possible to connect. 18 mediasoup Browser STUN Browser TURN NAT Firewall -
Conclusion • Trends of WebRTC is changing as P2P →
MCU → SFU • Keep watching mediasoup, because it is very acQve – First choice if you want to built in SFU in your applicaQon. • If you want high funcQonal Server, there are commercial products (such as Siguredo Sora) • Sample source on GitHub – hZps://github.com/mganeko/mediasoup_sample 19