Simulated Field Environment - Test Automation & Continuous Regression Testing Framework

Simulated Field Environment - Test Automation & Continuous Regression Testing Framework

Best of the Best Award winning presentation by Nokia Siemens Networks @STC 2012.
Authors - Ethiraj Alwar & Matthew Houghton

Presentation Abstract

Key Challenge of Test Organization is Execution productivity for which automation is sought after as the first option. There is traditional approach to automation where test cases are scripted individually (and then batched) and there is Simulated Field Environment approach. Test case level automation may suffice the needs at unit/component level, but system level verification needs a much more robust framework that can trigger the asynchronous events under varying conditions that leads to multiple unforeseen interactions which yield defects that are often not found until field deployment.
Simulated Field Environment is an innovative implementation of the Continuous integration and testing concept at the System Verification level which includes end-to-end verification of the system with multiple Network Elements. This framework was initially developed and institutionalized for testing the CDMA Network infrastructure (within the former Motorola Networks) and is now being used for LTE Infrastructure testing as well.
There are 4 key components of this framework – Continuous Traffic Profile which triggers both CP and OM traffic very much like field conditions, Continuous collection of both internal and external logs of all the involved network elements and interfaces, Continuous analysis of the logs to monitor the health of the system and Web based Key Performance Indicator page which the test engineer can monitor to identify any failures/issues and further debug/root-cause the issue.
This paper describes the Simulated Field environment approach, practice and experiences of LTE System Verification team. It describes the remarkable difference this environment enables in terms of test effectiveness.

About the Authors

Ethiraj Alwar has 20 years of experience in the Software industry, 15 years in Telecom industry in various positions related to System Engineering, Software development and System Verification in different domains such iDEN, CDMA, WiMax and LTE. Currently holding the position of System Verification Architect in the LTE System Verification Group. Has a B.E degree in Computer Science and Engineering from Madurai Kamaraj University, Tamilnad, India.

Matthew Houghton has 27 years of industry experience, 23 years in Mobile Broadband R&D with positions in SW Development, System Engineering, and System Verification. Originator of the Simulated Field Environment (SFE) approach to continuous regression, and architect of the SFE test automation framework. The SFE framework is designed to cost effectively transform a typical functional regression test environment into a field-like test environment yielding significantly more effective results.