WebRTC - ultra low latency delivery of sub second

Introduction
Expectations for Sub-Second
Chunked Streaming
Zoom vs WebRTC
Browser Support

WebRTC Technology
Why realtime?
Framework
Connection Method
Various Protocol Used

Performance of WebRTC
How low latency
Quality - Encoding Challenges
TEST - Quality, Resolution and Bit-rate
Stability
Smartphone viewing
Number of concurrent viewers
Interactivity
Troubleshoot
Latest Case

TIPS
Delivery using CDN
Encoder
Verification

Transcript

1. 1 WebRTC - ultra low latency delivery of sub second Solutions Engineer – Daisuke Doi　Limelight Networks, Inc. 24 September 2020
2. 2 Daisuke Doi (Limelight Networks, Inc.) Engaged in consulting and project management as an engineer who proposes solution construction for over 10 years in the broadcasting, post-production and entertainment industries at domestic / global companies. Video editing environment and media asset management, industrial VR, Engaged in introducing video solutions for entertainment facilities, etc. both in Japan and overseas. Live interactive streaming is the future of video expression Believing in the development of communication, I joined Limelight Networks Japan Co., Ltd. in early 2019. Hobbies are video editing of my band. We are considering implementing live streaming for business verification. Self-Introduction solutions engineer who translates customer business requirements into concrete solutions.
3. 3 Introduction • Expectations for Sub-Second • Chunked Streaming • Zoom vs WebRTC • Browser Support WebRTC Technology • Why realtime? • Framework • Connection Method • Various Protocol Used Performance of WebRTC • How low latency • Quality - Encoding Challenges • TEST - Quality, Resolution and Bit-rate • Stability • Smartphone viewing • Number of concurrent viewers • Interactivity • Troubleshoot • Latest Case TIPS • Delivery using CDN • Encoder • Verification Agenda What is a sub-second real-time video delivery service using WebRTC?
4. 4 Expectations for Sub-Second Expectations for a live distribution business created by ultra-low latency or interactive communication Live Sports & Event • No gap against match progress • Watching experience from multiple viewpoints News • Contribution to promptness • Election bulletins and disaster reports • News for regions that cannot be covered by broadcasting Interactive Streaming • Live events such as games and e-sports • Comment online, chat, vote, poll, and more • Second screen • Live experience from multiple viewpoints with no gap between real and streaming Live & Event Venue
5. 5 Various live streaming solutions 1 Sec 60 Sec 10 Sec 5 Sec Normal HTTP Live Latency Ultra-Low-Latency 30 Sec Traditional HTTP Live Streaming Low Latency CMAF WebRTC Ultra-Low Latency Delivery Small Chunk Streaming Latency of Cable TV Low Latency Chunk streaming is now mainstream-live and real-time delivery are different
6. 6 zoom • Specialized in Video conference • 100 - 1000 viewer (until 10,000 by webinar mode) • The delay increases as the number of viewers increases • Video receive is difficult in the 1.2 Mbps or lower band zoom and WebRTC • Suitable for web conferences • Realtime • Challenges for large-scale distribution • Already stable technology WebRTC • Distribution for viewers and customization to individual UI • Constant ultra-low latency • Support ABR • No plugin required • Can be delivered only by browser and camera zoom vs WebRTC Video conference system zoom and WebRTC
7. 7 WebRTC Conference Service Encoder CDN RTMP Upload (H.264) WebRTC Viewer Service Mainly used and customized for two services WebRTC Workflow Web Browser Web Browser Web Browser Web Browser H.264 or VP8/Opus WebRTC (H.264/Opus) Web Browser
8. 8 8 Required Codec： VP8 and H.264 (IETF - RFC7742) Opus (Audio) 4K support？： VP9 -> AV1 H.265 has a lots of patents…. Safari: • Supports VP8 last year (2019) • VP9 is not supported MS Edge supports by Chromium version IE Doesn’t support Browser Support Supported by major browsers, no plugin or app installation required
9. 9 Basics of WebRTC Basics of WebRTC - Why Ultra-Low-Latency? Why Realtime？
10. 10 Why realtime? Communication method and transmission method 3sec 6sec 9sec Chunk streaming creates a delimiter with the set number of seconds and then sends it out sequentially. UDP Communication WebRTC keeps sending anyway HLS / DASH WebRTC
11. 11 Connection • UDP • ICE • STUN • TURN • SDP • Signaling Original Framework • Media Channel • Data Channel • Congestion Control • Retransmission control Encrypted communication • DTLS • SRTP • SRTCP • SCTP Framework of WebRTC Ultra-low latency and communication basics
12. 12 STUN • Required for NAT TURN • Relay Server (not required) WebRTC connection establishment method STUN 、TURN、and ICE SDP SDP ICE • Confirmation of routes and connection candidates Connectivity Check 　takes 5 second
13. 13 WebRTC Protocols: a collection of different protocols Various protocols are used to enable communication on UDP + security Complicated....
14. 14 WebRTC Protocols：DTLS - SRTP and SRTCP SRTCP • Used for signaling • Control and track individual packet delivery • Multiplexed on the same port as SRTP Data encryption and signaling on UDP + security communication SRTP • Encrypt video and audio data body • Exchange keys for SRTP using DTLS • He doesn't care if packet loss happens.. DTLS • Ensures security similar to that offered by TLS DTLS SRTP SRTCP UDP
15. 15 WebRTC data channel runs over SCTP • Data channel is used when calling javascript API required for WebRTC. • Tunnel over DTLS • Provides flow and congestion control mechanisms not found in UDP, but not as good as TCP Table: bloggeek.me WebRTC Protcols：SCTP Hybrid protocol of UDP and TCP
16. 16 WebRTC Protocol expected in the future：QUIC Research and development by Google as an alternative to the current protocol QUIC • Positioned between UDP and TCP, UDP provides similar functionality to TCP • Better congestion control • The purpose of improving Web performance, not just WebRTC • Responsible for the transport layer of HTTP / 3 X
17. 17 Current status of WebRTC Performance？ What is the performance of WebRTC? -Verification using Limelight Networks distribution equipment
18. 18 •How low latency •Quality - Encoding Challenges •TEST - Quality, Resolution and Bit-rate •Stability •Smartphone viewing •Number of concurrent viewers •Interactivity •Troubleshoot •Latest Case Current status of WebRTC What is the performance of WebRTC? -Verification using Limelight Networks distribution equipment Test Equipments Encoder : OBS Studio Hardware : DELL Latitude 7390 (2019) Windows 10 64bit, 16GB Memory, Core i5 7300U 2.6Ghz Delivery System : Limelight Networks Camera : Logicool webcam C920
19. 19 Current status of WebRTC 250ms-300ms always ultra low latency •How low latency •Quality - Encoding Challenges •TEST - Quality, Resolution and Bit-rate •Stability •Smartphone viewing •Number of concurrent viewers •Interactivity •Troubleshoot •Latest Case What is the performance of WebRTC? -Verification using Limelight Networks distribution equipment
20. 20 Current status of WebRTC Video clarity • High Bit-Rate • The right resolution for devices • Camera performance and settings Unsuitable video (Camera and encoder challenges) • Video of the entire camera moving • An image of a person moving quickly at a short distance • Live video with lots of lighting •How low latency •Quality - Encoding Challenges •TEST - Quality, Resolution and Bit-rate •Stability •Smartphone viewing •Number of concurrent viewers •Interactivity •Troubleshoot •Latest Case What is the performance of WebRTC? -Verification using Limelight Networks distribution equipment
21. 21 Current status of WebRTC FHD / HD / SD, 3Mbps - 9Mbps 15fps - 60fps, 64 - 128Kbps audio 1. Demand for desktop distribution is increasing 2. There is no way to avoid dropping video frames due to WebRTC spec 3. Since there is almost no lack of voice, there is no discomfort 4. 1080p was heavy processing. Encoder limit at 4.5Mbps 60fps 5. Encoder limit at 720p 9Mbps 60fps 6. Quality that can withstand most services with 3 Mbps. Distribute Media playback from desktop using WebRTC TEST •How low latency •Quality - Encoding Challenges •TEST - Quality, Resolution and Bit-rate •Stability •Smartphone viewing •Number of concurrent viewers •Interactivity •Troubleshoot •Latest Case What is the performance of WebRTC? -Verification using Limelight Networks distribution equipment
22. 22 Current status of WebRTC The video is affected by the network condition and loss occurs, but it is unavoidable due to the specifications. However, the sound does not stop, so there is little discomfort There is no discomfort in viewing because the audio does not almost stop due to FEC implemented in Opus. Audio Video •How low latency •Quality - Encoding Challenges •TEST - Quality, Resolution and Bit-rate •Stability •Smartphone viewing •Number of concurrent viewers •Interactivity •Troubleshoot •Latest Case What is the performance of WebRTC? -Verification using Limelight Networks distribution equipment
23. 23 Current status of WebRTC 4G / iPhone 8 & iPhone 11s HD (720p) 1Mbps - 6Mbps 30fps or 60fps Smartphone viewing looks more beautiful than a PC. There is little discomfort in dropping frames. •How low latency •Quality - Encoding Challenges •TEST - Quality, Resolution and Bit-rate •Stability •Smartphone viewing •Number of concurrent viewers •Interactivity •Troubleshoot •Latest Case What is the performance of WebRTC? -Verification using Limelight Networks distribution equipment
24. 24 Current status of WebRTC Challenge in expanding the number of concurrent connections •How low latency •Quality - Encoding Challenges •TEST - Quality, Resolution and Bit-rate •Stability •Smartphone viewing •Number of concurrent viewers •Interactivity •Troubleshoot •Latest Case What is the performance of WebRTC? -Verification using Limelight Networks distribution equipment
25. 25 Current status of WebRTC Chat with Websocket Betting function Voice communication Online Drone Racing (e-Sports) Deliver video from drone to supporters (passengers) Operate the actual drone in several regions, Distribution of e-Sports with game elements added in AR / VR Passengers help pilots •How low latency •Quality - Encoding Challenges •TEST - Quality, Resolution and Bit-rate •Stability •Smartphone viewing •Number of concurrent viewers •Interactivity •Troubleshoot •Latest Case What is the performance of WebRTC? -Verification using Limelight Networks distribution equipment
26. 26 Current status of WebRTC Most of the troubles are caused by the sending / receiving network or encoder trouble. Network for ingest Encoder backup recommended. •How low latency •Quality - Encoding Challenges •TEST - Quality, Resolution and Bit-rate •Stability •Smartphone viewing •Number of concurrent viewers •Interactivity •Troubleshoot •Latest Case What is the performance of WebRTC? -Verification using Limelight Networks distribution equipment
27. 27 Use Case: Post Production Realtime streaming creates a high quality remote editing system After the COVID-19, many business and service got to adopt an online/remote solution. One of the solution which replace traditional Post Production Studio service to all online. Editors, Directors and Sponsors can join in editing place from all remote site. Challenge Solution Benefit • To ensure interactivity, a real-time delivery platform without delay • Encoder settings • Player implementation • Secure distribution • Subsecond streaming • 6 - 7 sec latency HLS • Ondemand • Director can direct a video editing with qualified video quality. Realtime streaming provides interactivity for remote editing on a post production work. Editing Studio Directors, Producers and Sponsors Solution: for a remote Editing
28. 28 WebRTC WebRTC WebRTC Edge Global Edge Network Video Source Encoder RTMP Up load WebRTC Edge WebRTC Edge Streaming Distribution Streaming Distribution Streaming Distribution WebRTC Primary Backup ex)：WebRTC distribution Workflow using CDN Overview of real-time streaming and CDN distribution
29. 29 Verified encoder • AWS - Elemental Live • Haivison - Kurabyte ◦ Commercial encoder • OBS Project - OBS Studio ◦ Free Software (Single Ingest) • Telestream - Wirecast ◦ Paid Software (Multi Ingest ready) • CEREVO - LiveShell.x ◦ Small Box (Multi Ingest ready) Encoder: RTMP From free to appliance servers, depending on the application Encoder settings • RTMP • SD - FHD • 15fps - 30fps • Video : 500kbps 〜 3Mbps • Audio : 64kbps 〜 128kbps • CBR : 1 (WebRTC recommended) • Keyframe : 1 (WebRTC recommended) • Profile : Baseline (WebRTC recommended)
30. 30 Encoder Configuration OBS Studio, the most used encoder for personals and businesses 1. Delivery URL & Stream Key 2. Output Settings 3. Video Settings 4. Audio Settings ① ② ③ ④
31. 31 WebRTC Viewing environment verification Prepare a delivery environment, delivery contents, and viewing environment that match the actual service case Trade-off between low latency and continuous playback The point of service provision is the use case and the provision of benefits due to ultra-low latency
32. 32 linkedin.com/company/limelight-networks twitter.com/llnw facebook.com/LimelightNetworks info@llnw. com