last N: Relevance-Based Selectivity for Forwarding Video in Multimedia Conferences

Presented by Boris Grozev, jitsi.org at NOSSDAV 2015.

Transcript

1. Last N: Relevance-Based Selectivity for Forwarding Video in Multimedia Conferences 1 Boris Grozev, Jitsi.org, University of Strasbourg Lyubomir Marinov, Jitsi.org Varun Singh, Aalto University, Finland Emil Ivov, Jitsi.org 20/03/15
2. SCALING VIDEO CONFERENCES •  Goals: – Bigger conferences – More on one server 2 20/03/15 •  What kind of conf.: – Centralized – WebRTC – RTP – Meetings, Presentations – Dynamic, Interactive
3. 3 C A B videobridge jitsi MIX Forwarding (SFU) Mixing (MCU) 16/03/15
4. 4 C A B videobridge jitsi SFU 15/03/15 JITSI VIDEOBRIDGE •  WebRTC-compatible video router •  ICE; DTLS-SRTP; SRTP; SCTP; • RTCP Termination •  RR, REMB •  SR • SRTP Termination •  (Relatively) Expensive
5. PROBLEMS(1): ON THE CLIENTS 15/03/15 5 1.  Bandwidth (down): 99 streams * 1Mbps 2.  CPU usage: live decoding of 99 streams 3.  User interface: no space for 99 video elements For a conf of 100:
6. 17/03/15 6 1.  Downstream bandwidth •  Proportional to K •  100 endpoints = 100Mbps 2.  Upstream bandwidth •  Proportional to K2 •  100 endpoints = 9900 streams = 9.9Gbps PROBLEMS (2): ON THE BRIDGE C A B videobridge jitsi 3. CPU •  Proportional to the total bitrate •  Proportional to K2 For a conf of 100
7. 7 LAST N 16/03/15 videobridge jitsi N=2
8. 8 LAST N: EFFECT 16/03/15 C B videobridge jitsi K = 100; N = 5; Clients: Receive/Decode/Render 5 streams (was 99) SFU: Downstream: still 100 streams (100Mbps) Upstream: K * N 500Mbps (was 9.9Gbps) N is constant: Linear with K (was K2)
9. 9 LAST N: PAUSING 16/03/15 videobridge jitsi Clients: No encoding, no upstream SFU: Downstream: N+1 instead of K
10. DOMINANT SPEAKER IDENTIFICATION •  Requirement for Last N •  The naïve approach doesn’t work – Different microphones / configuration – Different sound levels in the environment – Coughs 20/03/15 10
11. DOMINANT SPEAKER IDENTIFICATION •  SotA: I. Volfin and I. Cohen 2013[1] •  Maintains a dominant speaker (DS) –  Others compete –  Detects changes to the DS •  Computes scores over intervals with different length –  Short, medium, long –  Thresholds at each interval •  Works with audio in the frequency domain –  Requires decoding [1] Dominant speaker identification for multipoint videoconferencing, Computer Speech and Language Volume 27 Issue 4, June, 2013 20/03/15 11
12. DOMINANT SPEAKER IDENTIFICATION •  RFC6464: Client-to-Mixer Audio Level Indication – RTP header extension – 7 bits that indicate the level of the audio in an RTP packet •  Adapt Volfin and Cohen 2013 – Same competition model – Same intervals •  Short (20ms), medium (100ms) and long (1000ms) – Same division in sub-bands 20/03/15 12
13. TESTBED 16/03/15 13 videobridge jitsi RECV ONLY JITSI HAMMER QUAD-CORE XEON 3.7Ghz … K
14. K = 10, 15, 20, 25, 29, 33 0 5 10 15 20 25 0 100 200 300 400 500 600 0 5 10 15 20 25 CPU usage (%) Bitrate (Mbps) (47.6Mbps, 3.1%) (110.3Mbps, 5.1%) (199.4Mbps, 8.0%) (314.7Mbps, 11.7%) (425.5Mbps, 15.7%) (550.4Mbps, 20.3%) 15/03/15 14
15. 0 50 100 150 200 250 300 350 400 450 10 15 20 25 30 0 50 100 150 200 250 300 350 400 450 Mbps outbound Number of endpoints (K) n=3 n=5 n=8 n=-1 15/03/15 15
16. Pausing 0 50 100 150 200 250 300 350 400 450 0 5 10 15 20 25 30 0 50 100 150 200 250 300 350 400 450 Bitrate (Mbps) Last N video not paused video paused 15/03/15 16
17. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 0 5 10 15 20 25 30 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 CPU usage (%) Last N video not paused video paused 16/03/15 17 Pausing
18. CONCLUSION •  Conferences with forwarding use a lot of bandwidth •  The bottleneck at the server is either the CPU or the network –  In most cases the network •  Cutting down the number of forwarded streams to a constant works as expected •  DSI can be used to maintain interactivity in the conference •  DSI can be performed without decoding audio •  Future work –  Adaptive Last N 15/03/15 18
19. THANK YOU! 16/03/15 19