ELMAR 2010

Transcript

1. Motivation Practical Measurement Summary Voice over IP (VoIP) Speech Quality

   Measurement with Open-Source Software Bastian Huntgeburth1 Sebastian Schumann2 Juraj Londák2 1Institut für Telekommunikationsinformatik Deutsche Telekom AG, Hochschule für Telekommunikation Leipzig, Germany 2Faculty of Electrical Engineering and Information Technology Slovak University of Technology, Bratislava, Slovakia 52. International Symposium ELMAR 2010, 15.-17. September, Zadar, Croatia Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

2. Motivation Practical Measurement Summary Outline 1 Motivation Introduction Task Theory

   2 Practical Measurement Environment Evaluation Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

3. Motivation Practical Measurement Summary Introduction Task Theory Outline 1 Motivation

   Introduction Task Theory 2 Practical Measurement Environment Evaluation Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

4. Motivation Practical Measurement Summary Introduction Task Theory Problem Software based

   media processing is expensive Influence of NGNlab.eu infrastructure utilization on correct media processing has to be measured The platform hosts not only lab traffic, but is used as communication platform within several projects Task Measure the voice quality the NGNlab.eu infrastructure delivers under a certain load to determine the capabilities of the current set-up. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

5. Motivation Practical Measurement Summary Introduction Task Theory Infrastructure NGNlab.eu Next-generation

   communication infrastructure in the laboratory of STU Playground to implement results of research in the area of IP based services Almost solely based on open-source software Communication services logic uses low resource-intense signaling protocols (e.g., SIP) High resource-intense media streams (e.g., RTP) have to be handled (e.g., transcoding, conferencing) Virtualization and cost-efficiency in terms of hardware usage require proper dimensioning Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

6. Motivation Practical Measurement Summary Introduction Task Theory Requirements Have a

   fully automated test environment that is easy to set up by anybody who wants to test a VoIP infrastructure can be used for long term testing and creating statistics Follow NGNlab.eu concept of using mostly open-source applications. Commercial analyzers were out of scope. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

7. Motivation Practical Measurement Summary Introduction Task Theory QoS measurement theory

   Speech quality can in general be measured by comparing the signal of the sender with the receiver The difference of these signals is the loss caused by the system (infrastructure, transcoding, network etc.) Objective way to determine quality is required for the tests PESQ The Perceptual Evaluation of Speech Quality (PESQ) describes an objective method to predict the subjective quality of narrow-band speech codecs. The output is a prediction of real Mean Opinion Score (MOS) tests. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

8. Motivation Practical Measurement Summary Introduction Task Theory QoS measurement theory

   Speech quality can in general be measured by comparing the signal of the sender with the receiver The difference of these signals is the loss caused by the system (infrastructure, transcoding, network etc.) Objective way to determine quality is required for the tests PESQ The Perceptual Evaluation of Speech Quality (PESQ) describes an objective method to predict the subjective quality of narrow-band speech codecs. The output is a prediction of real Mean Opinion Score (MOS) tests. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

9. Motivation Practical Measurement Summary Environment Evaluation Outline 1 Motivation Introduction

   Task Theory 2 Practical Measurement Environment Evaluation Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

10. Motivation Practical Measurement Summary Environment Evaluation Components Two VoIP end

    points (i.e., sender and receiver of the test signal) A test signal that will be compared in the end Evaluator that does the signal comparison according PESQ System under test (e.g., PBX, VoIP infrastructure) Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

11. Motivation Practical Measurement Summary Environment Evaluation Tools Sender SIPp SIPp

    is a free Open Source test tool / traffic generator for the SIP protocol. Test signal ITU-T P.50 The recommendation provides test files to simulate voices. Evaluator ITU-T P.862 The recommendation delivers a ready-to-compile implementation. Receiver Asterisk Asterisk is a PBX and will be used to generate the file for comparison. Execution environment for all applications is GNU/Linux. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

12. Motivation Practical Measurement Summary Environment Evaluation Tools Sender SIPp SIPp

    is a free Open Source test tool / traffic generator for the SIP protocol. Test signal ITU-T P.50 The recommendation provides test files to simulate voices. Evaluator ITU-T P.862 The recommendation delivers a ready-to-compile implementation. Receiver Asterisk Asterisk is a PBX and will be used to generate the file for comparison. Execution environment for all applications is GNU/Linux. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

13. Motivation Practical Measurement Summary Environment Evaluation Tools Sender SIPp SIPp

    is a free Open Source test tool / traffic generator for the SIP protocol. Test signal ITU-T P.50 The recommendation provides test files to simulate voices. Evaluator ITU-T P.862 The recommendation delivers a ready-to-compile implementation. Receiver Asterisk Asterisk is a PBX and will be used to generate the file for comparison. Execution environment for all applications is GNU/Linux. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

14. Motivation Practical Measurement Summary Environment Evaluation Tools Sender SIPp SIPp

    is a free Open Source test tool / traffic generator for the SIP protocol. Test signal ITU-T P.50 The recommendation provides test files to simulate voices. Evaluator ITU-T P.862 The recommendation delivers a ready-to-compile implementation. Receiver Asterisk Asterisk is a PBX and will be used to generate the file for comparison. Execution environment for all applications is GNU/Linux. Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

15. Motivation Practical Measurement Summary Environment Evaluation Preparation The test signal

    has to be transformed from WAV to PCAP SIPp and Asterisk will be configured with a test account The calls with the test signal are performed A file containing the received signal is created The received and the original signal are compared and evaluated Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

16. Motivation Practical Measurement Summary Environment Evaluation Setup WAV PCAP WAV

    System under test R O ∆ PESQ MOSLQO SIPp Asterisk Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

17. Motivation Practical Measurement Summary Environment Evaluation Process SIPp and the

    Asterisk register on the SUT SIPp calls the Asterisk extension and plays the PCAP file Asterisk answers the call and executes the recording function The recorded file is trimmed to a comparable length (SoX) PESQ is run on both files to compare them and calculate MOSLQO Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

18. Motivation Practical Measurement Summary Environment Evaluation MOS under varying loads

    Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

19. Motivation Practical Measurement Summary 1 Motivation Introduction Task Theory 2

    Practical Measurement Environment Evaluation Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

20. Motivation Practical Measurement Summary Summary The proposed utilities are a

    very flexible alternative to expensive measurement devices The system can not only measure MOS but also generate load on the SUT It is suitable also for long-term measurement to generate statistical data Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS

21. Motivation Practical Measurement Summary Contact Sebastian Schumann Department of Telecommunications

    Slovak University of Technology Bratislava, Slovakia schumann@ktl.elf.stuba.sk Huntgeburth, Schumann, Londák VoIP Speech Quality Measurements with OSS