2019 DevDay
LINE Call: 8 Year Journey to the
Top Level Quality
> Dongwon Lee
> LINE Plus Dev 3 Software Engineer
Slide 2
Slide 2 text
Agenda > What Defines the Best Call Quality?
> How to Assess Call Quality
> Global Quality Monitoring System
Slide 3
Slide 3 text
The Best Call Quality
Slide 4
Slide 4 text
Tele-communication
Slide 5
Slide 5 text
The New Paradigm
Communication
Slide 6
Slide 6 text
The New Paradigm
Communication
Connecting
to Each Other
Slide 7
Slide 7 text
Close Distance: 45 cm
Communication Distance
Social Distance: 1.2 m ~ 3.6 m
Personal Distance: 45 cm ~ 1.2 m
[openclipart.org]
1 m
Slide 8
Slide 8 text
Assessment of Quality
Slide 9
Slide 9 text
Reference
Signal
Digitalize
Compress
Degraded
Signal
Audio
Converter
Packet Network
Time
Stamp
IP
Address
Add Address
UDP header
#5 #4 #3 #2 #1
IP
Receiver
Time
Sequence
#5 #3 #4 #2 #1
#5 #4 #3 #2 #1
Voice Over Internet
Slide 10
Slide 10 text
MOS: Mean Opinion Score
5 EXCELLENT
4 GOOD
3 FAIR
2 POOR
1 BAD
Slide 11
Slide 11 text
Listening test
Original Voice 1 Voice 2
5 EXCELLENT
4 GOOD
3 FAIR
2 POOR
1 BAD
Slide 12
Slide 12 text
Objective measurements
POLQA v3
Perceptual Objective
Listening Quality
Assessment
PAMS
Perceptual Analysis
Measurement System
PESQ
Perceptual Evaluation
of Speech Quality
PSQM
Perceptual Speech
Quality Measure
1997 2001 2011
1998 2018
POLQA
Full Band
Slide 13
Slide 13 text
?
Call quality
over the world?
200+ Countries
Expedition Team
Slide 14
Slide 14 text
Expedition Team
Slide 15
Slide 15 text
Global Monitoring System
Slide 16
Slide 16 text
> To measure the call quality of not only LINE but also competing apps
> Easy to monitor and compare the measured metrics
> To measure the objective metrics obtained by the expedition team
Global Quality Monitoring System
Slide 17
Slide 17 text
Global Quality Monitoring System
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
Analysis - Good Voice
Slide 20
Slide 20 text
Analysis - Bad Voice
Slide 21
Slide 21 text
Encoding
Packetization
Input
Signal
Network Depacketization
Decoding
Reference
Signal
Output
Signal
Full Reference (FR)
Degraded
Signal
No Reference (NR)
Media Stream Media Stream
Video Quality Measurement
Slide 22
Slide 22 text
Nature of Video Code: Blockiness
[Original] [Blocky]
Slide 23
Slide 23 text
VMAF
PSNR
MS-SSIM
PSNR-HVS
FR Methods
Slide 24
Slide 24 text
[Practical Evaluation of VMAF Perceptual Video Quality for WebRTC Applications, Boni García,, Luis López-Fernández, 31,Jul,2019,MDPI electrinics]
Performance Comparison
Slide 25
Slide 25 text
[Practical Evaluation of VMAF Perceptual Video Quality for WebRTC Applications, Boni García,, Luis López-Fernández, 31,Jul,2019,MDPI electrinics]
Comparison with Human Score
Slide 26
Slide 26 text
Video Data Acquisition
[Sender] [Receiver]
1
2
3
1
3
Slide 27
Slide 27 text
3
1 4
2
1 4
3
2
Video Data Acquisition
[Sender] [Receiver]
FR Method
Slide 28
Slide 28 text
Result of Video Assessment
VMAF
Freezing
Skipness
https://www.youtube.com/watch?v=6ArXfcKeZWY
Slide 29
Slide 29 text
Subtitle
Title 120pt
Bad Quality Video
Slide 30
Slide 30 text
Auditory System
Pinna
Ear Canal
Cochlea
Slide 31
Slide 31 text
Measured Like the Human Ear
HATS: Head And Torso Simulator