Playing with OSS WebRTC SFU meidasoup

English translation of the slides for WebRTC Meetup Tokyo 13
Un-offical document for how to use mediasoup from your Node.js App

Transcript

1. WebRTC  Meetup  Tokyo  #13   Playing  with  OSS  SFU  meidasoup INFOCOM  CORPORATION   Masashi  Ganeko   @massie_g   Jan.  27th,  2017 1
2. About  me •  Masashi  Ganeko      /  @massie_g   –  working  for  a  research  team  of  INFOCOM  CORPORATION   •  WriQng  a  series  of  WebRTC  on  HTML5Experts.jp  (HTML5  tech  site)   –  hZps://html5experts.jp/series/webrtc2016/   •  English  PresentaQon  for  WebRTC  (2013,  2016)   –  hZps://speakerdeck.com/mganeko   •  Japanese  PresentaQon  for  WebRTC  (2013-­‐2016)   –  hZp://www.slideshare.net/mganeko   2
3. Topic  of  today •  SFU  :  SelecQve  forwarding  Unit   •  mediasoup  :     – Open  Source  WebRTC  SFU  module  for  Node.js   3
4. P2P  vs.  SFU: SelecQve  Forwarding  Unit 4 Browser   A Browser   B Browser   D Browser   C P2P   •  No  media  server     •  Browser  side:   •  CPU  load:  Very  High   •  Network  load:  Very  High 4 Browser   A Browser   B Browser   D Browser   C SFU   Dispatch   audio/video   SFU   •  User  SFU  server,  CPU  load:  Low   •  Browser  side:  low  CPU  load   •  Browser  side:  network  load:  middle   Able  to  choose   display  layout    on  member  side  ◎
5. MCU  vs.  SFU 5 Browser   A Browser   B Browser   D Browser   C SFU   Dispatch   audio/video   SFU   •  User  SFU  server,  CPU  load:  Low   •  Browser  side:  low  CPU  load   •  Browser  side:  network  load:  middle   Able  to  choose   display  layout    on  member  side  ◎ Browser   A Browser   B Browser   D Browser   C MCU       Mix     video  /  audio MCU   •  Use  MCU  server,    CPU  load:  Super  High   •  Browser  side:  very  low  CPU/Network  load  
6. Encoding  and  EncrypQng 6 Browser   Thanks  to  いらすとや hZp://www.irasutoya.com/ Browser   P2P Encode Encrypt Transmit Decrypt Decode TURN Browser   TURN   Browser   SFU MCU Browser   SFU   Browser   Browser   MCU   Browser  
7. What  is  mediasoup  ? •  Open  Source  WebRTC  SFU   –  Web  Site  hZps://mediasoup.org/   –  GitHub  hZps://github.com/ibc/mediasoup/   •  Not  a  Server  App,  but  components  for  a  Server   –  Instead  of  creaQng  yet  another  opinionated  server,  mediasoup  is  a   Node.js  module  which  can  be  integrated  into  a  larger  applicaQon  or   made  standalone  with  just  a  few  lines  of  JavaScript.   •  Provided  as  Node.js  module   –  You  have  to  prepare  Web  Server  and  Signaling  method   7
8. What  is  mediasoup  ?    (2) •  Minimum  …  Only  Media  (No  DataChannel),  No  Codecs   •  Powerful  …  Main  part  is  build  with  C++,  on  libuv   –  Node.js  invokes  child  worker  process   –  Node.js  and  worker  communicate  via  interprocess  communicaQon     •  JavaScript  API  for  Node.js   –  WebRTC  1.0  …  RTCPeerConnecQon   –  ORTC  …  Transport,  RtpSender,  RtpReceiver   8
9. DEMO  of  mediasoup  sample 9 •  MulQple  parQcipants  video  chat  sample   –  Video  and  Audio   –  Looks  same  as  P2P,  but  …   •  if  you  stop  SFU  Sever  process,  then  Remote  video/audio  will  stop   •  Source  codes  on  GitHub   –  hZps://github.com/mganeko/mediasoup_sample  
10. How  to  install  mediasoup •  PreparaQon   –  Node.js  v4.0.0  以上   –  Linux  /  Mac  OS  X  　(Windows  not  supported  yet)   –  Python  2.x　(NOT  Python  3.x  !!)   –  make   –  gcc  with  g++  ,      or  clang　with  C++11   •  InstallaQon   –  $  npm  install  mediasoup   •  C++  compile  may  take  a  few  minutes 10
11. Object  structure  of  mediasoup 11 Server Room Room Room Peer Peer Peer Transport Transport RtpSender RtpSender RtpSender RtpReceiver RtpReceiver RtpReceiver ・1  Server  has  Many  Rooms   ・1  Room  has  Many  Peers     ・1  Peer  has  mulQple  RtpSender   ・1  Peer  has  mulQple  RtpReceiver   ・1  Peer  has  mulQple  のTransport   Correspond  to  MediaTrack  (video/audio) Correspond  to  MediaTrack Correspond  to   UDP/TCP  port 1  Transport  is  used  by    -­‐  mulQple  RtpSender    -­‐  mulQple  RtpReceiver   BUNDLE,  rtcp-­‐mux   mulQ-­‐stream
12. Inside  of  mediasoup  Room  (Guess) 12 Room Peer RtpReceiver RtpSender Browser   Browser   Peer RtpReceiver RtpSender Peer Browser   RtpReceiver RtpSender RtpSender RtpSender RtpSender Streams  are   connected   automaQcally  in  the   room  (may  be) Transport Transport Transport
13. worker  process 13 Default  :   Core  number Case  of  5  Peers   in  1  Room All  peers  in  the  same   Room  are  assigned  to   the  same  worker  ?  
14. Rough  Sketch  to  use let  server  =  mediasoup.Server();   let  peer1;   let  receiver1;   return  server.createRoom(roomOpQons)   .then(  (room)  =>  {      peer1  =  room.Peer('alice');      return  peer.setCapabiliQes(peerCapabiliQes);   })   .then(()  =>  {      return  peer.createTransport({  tcp:  false  });   })   .then((transport)  =>  {      rtpReceiver1  =  peer1.RtpReceiver('audio',  transport);   })   14 •  Compose  objects  step  by  step   •  Use  Promise   •  Do  not  create  RtpSender  expressly   •  Will  be  created  automaQcally,  passed  by  event   (I  do  not   understand  well  ..)
15. Another  Object  structure 15 Server Room Room Room Peer Peer Peer RTCPeerConnecQon •  1  RTCPeerConnecQon  corresponds  to  1  Peer   •  Same  interface  as  Browser  Offer/Answer     RTCPeerConnecQon RTCPeerConnecQon
16. Flow  with  RTCPeerConencQon 16 Browser  A   RTCPeerConnecQon   mediasoup   RTCPeerConnecQon  A   Browser  B   send  offer  SDP send  offer  SDP send  answer  SDP send  answer  SDP onnegoQaQonneeded()  fired   onnegoQaQonneeded()  fired   RTCPeerConnecQon  B   RTCPeerConnecQon   send  answer  SDP onaddstream()  /  ontrack()  fired   send  offer  SDP send  offer  SDP send  answer  SDP onaddstream()  /  ontrack()  fired   createOffer()   createAnswer()   createOffer()   createAnswer()   createOffer()   createAnswer()   createOffer()   createAnswer()  
17. LimitaQon  of  RTCPeerConnecQon •  Need  iniQal  Offer  from  Remote  (Browser)   –  Currently,  the  mediasoup  implementaQon  of  RTCPeerConnecQon  requires  that   the  iniQal  offer  comes  from  the  remote  endpoint,     •  Aser  onnegoQaQonneeded()  event,  create  Offer  in  mediasoup   •  Not  allowed  to  re-­‐creaQng  Offer  in  Remote  (Browser)   –  Add/Remove  of  Remote  stream/track  is  not  supported   •  It  seems  like  Plan  B  support  is  tentaQve.   17 LimitaQon  of  RTCPeerConnecQon,  NOT  limitaQon  of  mediasoup  itself.   It  must  be  possible  if  you  compose  Transport,  RtpReceiver,  SDP  manually.
18. Other  limitaQon  of  mediasoup •  mediasoup  should  be  outside  of  NAT/Firewall.  Need  global  IP  address.   –  mediasoup  does  not  look  for  STUN/TURN  server   •  Browser  inside  of  NAT/Firewall  is  OK   –  When  Browoser  looks  for  STUN/TURN  server,  it  is  possible  to  connect.   18 mediasoup Browser STUN Browser TURN NAT Firewall
19. Conclusion •  Trends  of  WebRTC  is  changing  as  P2P  →  MCU  →  SFU   •  Keep  watching  mediasoup,  because  it  is  very  acQve   –  First  choice  if  you  want  to  built  in  SFU  in  your  applicaQon.   •  If  you  want  high  funcQonal  Server,  there  are  commercial  products   (such  as  Siguredo  Sora)   •  Sample  source  on  GitHub   –  hZps://github.com/mganeko/mediasoup_sample   19
20. Thank  you! 20