Video Quality Assessment in Streaming

看影片時，偶爾會發生「影像畫質不好」讓人無法專注，甚至是想換片的情況，此議程將會介紹評估影片畫質的量化方式，以及畫質與串流服務間密不可分的關係，並拓展至未來可能的應用方向。

Sometimes, when watching a movie, there are situations where we want to switch because the video quality is not good. This session will introduce the Video Quality Assessment (VQA), the close relationship between video quality and streaming services, as well as future application direction.

Transcript

1. Video Quality Assessment in Streaming 論影片品質評估對串流服務成功的重要性 Yvonne Kao
2. • What is Video Quality Assessment? • Why is it important? • How to score? • SB-VQA for CVPR 2023 challenge • Analysis Agenda
3. Source: http://valiber.com/article/fruit_juices-eng
4. Source: MIDSOMMAR Trailer https://www.youtube.com/watch?v=1Vnghdsjmd0 Let’s Take A Look First
5. Source: https://blog.video.ibm.com/streaming-video-tips/what-is-video-encoding-codecs-compression-techniques/
6. Source: 27th PyData Cambridge https://www.youtube.com/watch?v=9xsSR_ERJvk
7. © 2022 KKStream. All Rights Reserved. Subjective VQA Source: https://www.epfl.ch/labs/mmspg/downloads/3dvqa/, https://tech.ebu.ch/news/2022/03/automatic-monitoring-of-perceived-quality-for-audiovisual-content
8. Netflix VMAF (Video Multimethod Assessment Fusion) • Consistent across content types PSNR = 29.1 VMAF = 20.4 Source: AOMedia 2019 Research Symposium https://www.youtube.com/watch?v=EGxNnAf71Q8 PSNR = 29.3 VMAF = 69.8
9. © 2022 KKStream. All Rights Reserved. Apple AVQT (Advanced Video Quality Tool) • Optimized quality for HLS tiers Source: WWDC 21 https://developer.apple.com/videos/play/wwdc2021/10145/ both 11600 kbps 11600 kbps/12760 kbps Animation Sport
10. © 2022 KKStream. All Rights Reserved. Source: https://reurl.cc/910O38 Could be used for recommendations Youtube Video Quality
11. CVPR The Computer Vision and Pattern Recognition Conference (CVPR) is widely considered to be one of the three most significant international conferences in the field of computer vision. Source: https://cvpr2023.thecvf.com/Conferences/2023/Sponsors, https://voxel51.com/blog/cvpr-2023-and-the-state-of-computer-vision/ Word cloud for 2023 CVPR accepted paper titles 2023 spnsors
12. CVPR 2023 No-Reference VQA Challenge Score: 66 Score: 31 • How to accurately estimate the video quality score? video source: NTIRE Workshop and Challenges @ CVPR 2023
13. Haoning Wu et. al, FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling, 2022 Fragment Sampling
14. Different Score Weight over Region Sidi Yang* et. al, MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment, 2022
15. Ensemble (Stacking-Based VQA) Source: https://www.analyticsvidhya.com/blog/2021/08/ensemble-stacking-for-machine-learning-and-deep-learning/ Ding-Jiun Huang et. al, SB-VQA: A Stack-Based Video Quality Assessment Framework for Video Enhancement, 2023
16. Experiment Design • Multi-task training strategy Source: https://www.hksilicon.com/articles/2273091
17. Quality score is related to the bitrate? • Score is positively correlated to bitrate but NOT linear • Room for cost-effective optimization
18. Score is related to human interest? • Human interests do NOT correlate positively with video quality • Content plays critical role as well heatmap-based content appeal predicted quality score Source: MIDSOMMAR Trailer https://www.youtube.com/watch?v=1Vnghdsjmd0
19. Predicted Video Quality Over Time Source: MIDSOMMAR Trailer https://www.youtube.com/watch?v=1Vnghdsjmd0
20. • VQA: score video by quality • How to do? • Netflix VMAF; Apple AVQT; Youtube YouVQ • KKC ARC SB-VQA • End-user experience-driven • Assist in cost-effective bitrate ladder selection • Help evaluating which shot worth to be improved with quality Takeaway
21. Future Video compression • Encoding • Transcoding • Monitoring • Live streaming • …… Video enhancement • Super resolution • Old film restoration • Denoise • Generation • ……
22. Thank you. Questions?