LINE Call: 8 year journey to the top level quality

Slide 1

Slide 1 text

2019 DevDay LINE Call: 8 Year Journey to the Top Level Quality > Dongwon Lee > LINE Plus Dev 3 Software Engineer

Slide 2

Slide 2 text

Agenda > What Defines the Best Call Quality? > How to Assess Call Quality > Global Quality Monitoring System

Slide 3

Slide 3 text

The Best Call Quality

Slide 4

Slide 4 text

Tele-communication

Slide 5

Slide 5 text

The New Paradigm Communication

Slide 6

Slide 6 text

The New Paradigm Communication Connecting to Each Other

Slide 7

Slide 7 text

Close Distance: 45 cm Communication Distance Social Distance: 1.2 m ~ 3.6 m Personal Distance: 45 cm ~ 1.2 m [openclipart.org] 1 m

Slide 8

Slide 8 text

Assessment of Quality

Slide 9

Slide 9 text

Reference Signal Digitalize Compress Degraded Signal Audio Converter Packet Network Time Stamp IP Address Add Address UDP header #5 #4 #3 #2 #1 IP Receiver Time Sequence #5 #3 #4 #2 #1 #5 #4 #3 #2 #1 Voice Over Internet

Slide 10

Slide 10 text

MOS: Mean Opinion Score 5 EXCELLENT 4 GOOD 3 FAIR 2 POOR 1 BAD

Slide 11

Slide 11 text

Listening test Original Voice 1 Voice 2 5 EXCELLENT 4 GOOD 3 FAIR 2 POOR 1 BAD

Slide 12

Slide 12 text

Objective measurements POLQA v3 Perceptual Objective Listening Quality Assessment PAMS Perceptual Analysis Measurement System PESQ Perceptual Evaluation of Speech Quality PSQM Perceptual Speech Quality Measure 1997 2001 2011 1998 2018 POLQA Full Band

Slide 13

Slide 13 text

? Call quality over the world? 200+ Countries Expedition Team

Slide 14

Slide 14 text

Expedition Team

Slide 15

Slide 15 text

Global Monitoring System

Slide 16

Slide 16 text

> To measure the call quality of not only LINE but also competing apps > Easy to monitor and compare the measured metrics > To measure the objective metrics obtained by the expedition team Global Quality Monitoring System

Slide 17

Slide 17 text

Global Quality Monitoring System

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

Analysis - Good Voice

Slide 20

Slide 20 text

Analysis - Bad Voice

Slide 21

Slide 21 text

Encoding Packetization Input Signal Network Depacketization Decoding Reference Signal Output Signal Full Reference (FR) Degraded Signal No Reference (NR) Media Stream Media Stream Video Quality Measurement

Slide 22

Slide 22 text

Nature of Video Code: Blockiness [Original] [Blocky]

Slide 23

Slide 23 text

VMAF PSNR MS-SSIM PSNR-HVS FR Methods

Slide 24

Slide 24 text

[Practical Evaluation of VMAF Perceptual Video Quality for WebRTC Applications, Boni García,, Luis López-Fernández, 31,Jul,2019,MDPI electrinics] Performance Comparison

Slide 25

Slide 25 text

[Practical Evaluation of VMAF Perceptual Video Quality for WebRTC Applications, Boni García,, Luis López-Fernández, 31,Jul,2019,MDPI electrinics] Comparison with Human Score

Slide 26

Slide 26 text

Video Data Acquisition [Sender] [Receiver] 1 2 3 1 3

Slide 27

Slide 27 text

3 1 4 2 1 4 3 2 Video Data Acquisition [Sender] [Receiver] FR Method

Slide 28

Slide 28 text

Result of Video Assessment VMAF Freezing Skipness https://www.youtube.com/watch?v=6ArXfcKeZWY