Playing with OSS WebRTC SFU meidasoup

English translation of the slides for WebRTC Meetup Tokyo 13
Un-offical document for how to use mediasoup from your Node.js App

Transcript

1. WebRTC  Meetup  Tokyo  #13  
   Playing  with  OSS  SFU  meidasoup
   INFOCOM  CORPORATION  
   Masashi  Ganeko  
   @massie_g  
   Jan.  27th,  2017
   1
   

   View Slide

2. About  me
   •  Masashi  Ganeko      /  @massie_g  
   –  working  for  a  research  team  of  INFOCOM  CORPORATION  
   •  WriQng  a  series  of  WebRTC  on  HTML5Experts.jp  (HTML5  tech  site)  
   –  hZps://html5experts.jp/series/webrtc2016/  
   •  English  PresentaQon  for  WebRTC  (2013,  2016)  
   –  hZps://speakerdeck.com/mganeko  
   •  Japanese  PresentaQon  for  WebRTC  (2013-­‐2016)  
   –  hZp://www.slideshare.net/mganeko  
   
   2
   

   View Slide

3. Topic  of  today
   •  SFU  :  SelecQve  forwarding  Unit  
   •  mediasoup  :    
   – Open  Source  WebRTC  SFU  module  for  Node.js  
   3
   

   View Slide

4. P2P  vs.  SFU: SelecQve  Forwarding  Unit
   4
   Browser  
   A
   Browser  
   B
   Browser  
   D
   Browser  
   C
   P2P  
   •  No  media  server    
   •  Browser  side:  
   •  CPU  load:  Very  High  
   •  Network  load:  Very  High
   4
   Browser  
   A
   Browser  
   B
   Browser  
   D
   Browser  
   C
   SFU  
   Dispatch  
   audio/video  
   SFU  
   •  User  SFU  server,  CPU  load:  Low  
   •  Browser  side:  low  CPU  load  
   •  Browser  side:  network  load:  middle  
   Able  to  choose  
   display  layout  
    on  member  side  ◎
   

   View Slide

5. MCU  vs.  SFU
   5
   Browser  
   A
   Browser  
   B
   Browser  
   D
   Browser  
   C
   SFU  
   Dispatch  
   audio/video  
   SFU  
   •  User  SFU  server,  CPU  load:  Low  
   •  Browser  side:  low  CPU  load  
   •  Browser  side:  network  load:  middle  
   Able  to  choose  
   display  layout  
    on  member  side  ◎
   Browser  
   A
   Browser  
   B
   Browser  
   D
   Browser  
   C
   MCU  
    
    
   Mix    
   video  /  audio
   MCU  
   •  Use  MCU  server,    CPU  load:  Super  High  
   •  Browser  side:  very  low  CPU/Network  load  
   

   View Slide

6. Encoding  and  EncrypQng
   6
   Browser  
   Thanks  to  いらすとや hZp://www.irasutoya.com/
   Browser  
   P2P
   Encode
   Encrypt
   Transmit
   Decrypt
   Decode
   TURN Browser  
   TURN   Browser  
   SFU
   MCU
   Browser   SFU   Browser  
   Browser   MCU   Browser  
   

   View Slide

7. What  is  mediasoup  ?
   •  Open  Source  WebRTC  SFU  
   –  Web  Site  hZps://mediasoup.org/  
   –  GitHub  hZps://github.com/ibc/mediasoup/  
   •  Not  a  Server  App,  but  components  for  a  Server  
   –  Instead  of  creaQng  yet  another  opinionated  server,  mediasoup  is  a  
   Node.js  module  which  can  be  integrated  into  a  larger  applicaQon  or  
   made  standalone  with  just  a  few  lines  of  JavaScript.  
   •  Provided  as  Node.js  module  
   –  You  have  to  prepare  Web  Server  and  Signaling  method  
   7
   

   View Slide

8. What  is  mediasoup  ?    (2)
   •  Minimum  …  Only  Media  (No  DataChannel),  No  Codecs  
   •  Powerful  …  Main  part  is  build  with  C++,  on  libuv  
   –  Node.js  invokes  child  worker  process  
   –  Node.js  and  worker  communicate  via  interprocess  communicaQon    
   •  JavaScript  API  for  Node.js  
   –  WebRTC  1.0  …  RTCPeerConnecQon  
   –  ORTC  …  Transport,  RtpSender,  RtpReceiver  
   8
   

   View Slide

9. DEMO  of  mediasoup  sample
   9
   •  MulQple  parQcipants  video  chat  sample  
   –  Video  and  Audio  
   –  Looks  same  as  P2P,  but  …  
   •  if  you  stop  SFU  Sever  process,  then  Remote  video/audio  will  stop  
   •  Source  codes  on  GitHub  
   –  hZps://github.com/mganeko/mediasoup_sample  
   

   View Slide

10. How  to  install  mediasoup
    •  PreparaQon  
    –  Node.js  v4.0.0  以上  
    –  Linux  /  Mac  OS  X  　(Windows  not  supported  yet)  
    –  Python  2.x　(NOT  Python  3.x  !!)  
    –  make  
    –  gcc  with  g++  ,      or  clang　with  C++11  
    •  InstallaQon  
    –  $  npm  install  mediasoup  
    •  C++  compile  may  take  a  few  minutes
    10
    

    View Slide

11. Object  structure  of  mediasoup
    11
    Server
    Room
    Room
    Room
    Peer
    Peer
    Peer Transport
    Transport
    RtpSender
    RtpSender
    RtpSender
    RtpReceiver
    RtpReceiver
    RtpReceiver
    ・1  Server  has  Many  Rooms  
    ・1  Room  has  Many  Peers  
     
    ・1  Peer  has  mulQple  RtpSender  
    ・1  Peer  has  mulQple  RtpReceiver  
    ・1  Peer  has  mulQple  のTransport  
    Correspond  to  MediaTrack  (video/audio)
    Correspond  to  MediaTrack
    Correspond  to  
    UDP/TCP  port
    1  Transport  is  used  by  
     -­‐  mulQple  RtpSender  
     -­‐  mulQple  RtpReceiver  
    BUNDLE,  rtcp-­‐mux  
    mulQ-­‐stream
    

    View Slide

12. Inside  of  mediasoup  Room  (Guess)
    12
    Room
    Peer
    RtpReceiver
    RtpSender
    Browser   Browser  
    Peer
    RtpReceiver
    RtpSender
    Peer
    Browser  
    RtpReceiver RtpSender
    RtpSender RtpSender
    RtpSender
    Streams  are  
    connected  
    automaQcally  in  the  
    room  (may  be)
    Transport
    Transport
    Transport
    

    View Slide

13. worker  process
    13
    Default  :  
    Core  number
    Case  of  5  Peers  
    in  1  Room
    All  peers  in  the  same  
    Room  are  assigned  to  
    the  same  worker  ?  
    

    View Slide

14. Rough  Sketch  to  use
    let  server  =  mediasoup.Server();  
    let  peer1;  
    let  receiver1;  
    return  server.createRoom(roomOpQons)  
    .then(  (room)  =>  {  
       peer1  =  room.Peer('alice');  
       return  peer.setCapabiliQes(peerCapabiliQes);  
    })  
    .then(()  =>  {  
       return  peer.createTransport({  tcp:  false  });  
    })  
    .then((transport)  =>  {  
       rtpReceiver1  =  peer1.RtpReceiver('audio',  transport);  
    })  
    14
    •  Compose  objects  step  by  step  
    •  Use  Promise  
    •  Do  not  create  RtpSender  expressly  
    •  Will  be  created  automaQcally,  passed  by  event  
    (I  do  not  
    understand  well  ..)
    

    View Slide

15. Another  Object  structure
    15
    Server
    Room
    Room
    Room
    Peer
    Peer
    Peer
    RTCPeerConnecQon
    •  1  RTCPeerConnecQon  corresponds  to  1  Peer  
    •  Same  interface  as  Browser  Offer/Answer    
    RTCPeerConnecQon
    RTCPeerConnecQon
    

    View Slide

16. Flow  with  RTCPeerConencQon
    16
    Browser  A  
    RTCPeerConnecQon  
    mediasoup  
    RTCPeerConnecQon  A  
    Browser  B  
    send  offer  SDP
    send  offer  SDP
    send  answer  SDP
    send  answer  SDP
    onnegoQaQonneeded()  fired  
    onnegoQaQonneeded()  fired  
    RTCPeerConnecQon  B   RTCPeerConnecQon  
    send  answer  SDP
    onaddstream()  /  ontrack()  fired  
    send  offer  SDP send  offer  SDP
    send  answer  SDP
    onaddstream()  /  ontrack()  fired  
    createOffer()  
    createAnswer()  
    createOffer()  
    createAnswer()  
    createOffer()  
    createAnswer()  
    createOffer()  
    createAnswer()  
    

    View Slide

17. LimitaQon  of  RTCPeerConnecQon
    •  Need  iniQal  Offer  from  Remote  (Browser)  
    –  Currently,  the  mediasoup  implementaQon  of  RTCPeerConnecQon  requires  that  
    the  iniQal  offer  comes  from  the  remote  endpoint,    
    •  Aser  onnegoQaQonneeded()  event,  create  Offer  in  mediasoup  
    •  Not  allowed  to  re-­‐creaQng  Offer  in  Remote  (Browser)  
    –  Add/Remove  of  Remote  stream/track  is  not  supported  
    •  It  seems  like  Plan  B  support  is  tentaQve.  
    17
    LimitaQon  of  RTCPeerConnecQon,  NOT  limitaQon  of  mediasoup  itself.  
    It  must  be  possible  if  you  compose  Transport,  RtpReceiver,  SDP  manually.
    

    View Slide

18. Other  limitaQon  of  mediasoup
    •  mediasoup  should  be  outside  of  NAT/Firewall.  Need  global  IP  address.  
    –  mediasoup  does  not  look  for  STUN/TURN  server  
    •  Browser  inside  of  NAT/Firewall  is  OK  
    –  When  Browoser  looks  for  STUN/TURN  server,  it  is  possible  to  connect.  
    18
    mediasoup
    Browser
    STUN
    Browser
    TURN
    NAT
    Firewall
    

    View Slide

19. Conclusion
    •  Trends  of  WebRTC  is  changing  as  P2P  →  MCU  →  SFU  
    •  Keep  watching  mediasoup,  because  it  is  very  acQve  
    –  First  choice  if  you  want  to  built  in  SFU  in  your  applicaQon.  
    •  If  you  want  high  funcQonal  Server,  there  are  commercial  products  
    (such  as  Siguredo  Sora)  
    •  Sample  source  on  GitHub  
    –  hZps://github.com/mganeko/mediasoup_sample  
    19
    

    View Slide

20. Thank  you!
    20
    

    View Slide