ELMAR 2010

Voice over IP (VoIP) Speech Quality Measurement with Open-Source Software

Transcript

1. Motivation Practical Measurement Summary Voice over IP (VoIP) Speech Quality Measurement with Open-Source Software Bastian Huntgeburth1 Sebastian Schumann2 Juraj Londák2 1Institut für Telekommunikationsinformatik Deutsche Telekom AG, Hochschule für Telekommunikation Leipzig, Germany 2Faculty of Electrical Engineering and Information Technology Slovak University of Technology, Bratislava, Slovakia 52. International Symposium ELMAR 2010, 15.-17. September, Zadar, Croatia Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
2. Motivation Practical Measurement Summary Outline 1 Motivation Introduction Task Theory 2 Practical Measurement Environment Evaluation Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
3. Motivation Practical Measurement Summary Introduction Task Theory Outline 1 Motivation Introduction Task Theory 2 Practical Measurement Environment Evaluation Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
4. Motivation Practical Measurement Summary Introduction Task Theory Problem Software based media processing is expensive Influence of NGNlab.eu infrastructure utilization on correct media processing has to be measured The platform hosts not only lab traffic, but is used as communication platform within several projects Task Measure the voice quality the NGNlab.eu infrastructure delivers under a certain load to determine the capabilities of the current set-up. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
5. Motivation Practical Measurement Summary Introduction Task Theory Infrastructure NGNlab.eu Next-generation communication infrastructure in the laboratory of STU Playground to implement results of research in the area of IP based services Almost solely based on open-source software Communication services logic uses low resource-intense signaling protocols (e.g., SIP) High resource-intense media streams (e.g., RTP) have to be handled (e.g., transcoding, conferencing) Virtualization and cost-efficiency in terms of hardware usage require proper dimensioning Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
6. Motivation Practical Measurement Summary Introduction Task Theory Requirements Have a fully automated test environment that is easy to set up by anybody who wants to test a VoIP infrastructure can be used for long term testing and creating statistics Follow NGNlab.eu concept of using mostly open-source applications. Commercial analyzers were out of scope. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
7. Motivation Practical Measurement Summary Introduction Task Theory QoS measurement theory Speech quality can in general be measured by comparing the signal of the sender with the receiver The difference of these signals is the loss caused by the system (infrastructure, transcoding, network etc.) Objective way to determine quality is required for the tests PESQ The Perceptual Evaluation of Speech Quality (PESQ) describes an objective method to predict the subjective quality of narrow-band speech codecs. The output is a prediction of real Mean Opinion Score (MOS) tests. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
8. Motivation Practical Measurement Summary Introduction Task Theory QoS measurement theory Speech quality can in general be measured by comparing the signal of the sender with the receiver The difference of these signals is the loss caused by the system (infrastructure, transcoding, network etc.) Objective way to determine quality is required for the tests PESQ The Perceptual Evaluation of Speech Quality (PESQ) describes an objective method to predict the subjective quality of narrow-band speech codecs. The output is a prediction of real Mean Opinion Score (MOS) tests. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
9. Motivation Practical Measurement Summary Environment Evaluation Outline 1 Motivation Introduction Task Theory 2 Practical Measurement Environment Evaluation Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
10. Motivation Practical Measurement Summary Environment Evaluation Components Two VoIP end points (i.e., sender and receiver of the test signal) A test signal that will be compared in the end Evaluator that does the signal comparison according PESQ System under test (e.g., PBX, VoIP infrastructure) Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
11. Motivation Practical Measurement Summary Environment Evaluation Tools Sender SIPp SIPp is a free Open Source test tool / traffic generator for the SIP protocol. Test signal ITU-T P.50 The recommendation provides test files to simulate voices. Evaluator ITU-T P.862 The recommendation delivers a ready-to-compile implementation. Receiver Asterisk Asterisk is a PBX and will be used to generate the file for comparison. Execution environment for all applications is GNU/Linux. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
12. Motivation Practical Measurement Summary Environment Evaluation Tools Sender SIPp SIPp is a free Open Source test tool / traffic generator for the SIP protocol. Test signal ITU-T P.50 The recommendation provides test files to simulate voices. Evaluator ITU-T P.862 The recommendation delivers a ready-to-compile implementation. Receiver Asterisk Asterisk is a PBX and will be used to generate the file for comparison. Execution environment for all applications is GNU/Linux. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
13. Motivation Practical Measurement Summary Environment Evaluation Tools Sender SIPp SIPp is a free Open Source test tool / traffic generator for the SIP protocol. Test signal ITU-T P.50 The recommendation provides test files to simulate voices. Evaluator ITU-T P.862 The recommendation delivers a ready-to-compile implementation. Receiver Asterisk Asterisk is a PBX and will be used to generate the file for comparison. Execution environment for all applications is GNU/Linux. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
14. Motivation Practical Measurement Summary Environment Evaluation Tools Sender SIPp SIPp is a free Open Source test tool / traffic generator for the SIP protocol. Test signal ITU-T P.50 The recommendation provides test files to simulate voices. Evaluator ITU-T P.862 The recommendation delivers a ready-to-compile implementation. Receiver Asterisk Asterisk is a PBX and will be used to generate the file for comparison. Execution environment for all applications is GNU/Linux. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
15. Motivation Practical Measurement Summary Environment Evaluation Preparation The test signal has to be transformed from WAV to PCAP SIPp and Asterisk will be configured with a test account The calls with the test signal are performed A file containing the received signal is created The received and the original signal are compared and evaluated Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
16. Motivation Practical Measurement Summary Environment Evaluation Setup WAV PCAP WAV System under test R O ∆ PESQ MOSLQO SIPp Asterisk Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
17. Motivation Practical Measurement Summary Environment Evaluation Process SIPp and the Asterisk register on the SUT SIPp calls the Asterisk extension and plays the PCAP file Asterisk answers the call and executes the recording function The recorded file is trimmed to a comparable length (SoX) PESQ is run on both files to compare them and calculate MOSLQO Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
18. Motivation Practical Measurement Summary Environment Evaluation MOS under varying loads Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
19. Motivation Practical Measurement Summary 1 Motivation Introduction Task Theory 2 Practical Measurement Environment Evaluation Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
20. Motivation Practical Measurement Summary Summary The proposed utilities are a very flexible alternative to expensive measurement devices The system can not only measure MOS but also generate load on the SUT It is suitable also for long-term measurement to generate statistical data Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS
21. Motivation Practical Measurement Summary Contact Sebastian Schumann Department of Telecommunications Slovak University of Technology Bratislava, Slovakia schumann@ktl.elf.stuba.sk Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS