Business Component and XML driven framework - BX-D

Best of the Best Award Winning Presentation by NeST @STC 2012.
Authors - Balamurali L., Pradeep P. P., RathishM. M. & Samiya Salim

Presentation Abstract

Test Automation is an accepted technique which is adapted by the industry for increasing the effectiveness of the testing phase. The recurring tasks are being automated by the tools thus simplifying the human efforts and results in increased quality of product under test. A study of test automation programmes in the industry reveals the fact that a good percentage of them fail to find the intended results. A pareto analysis revels that majority of the failures are the result of lack of capability overlap of test resources in the domain and test automation. In order to have success in a test automation program, the test engineers should be knowledgeable in the test automation tool as well as the domain under test. In practical scenario it is seen that, many a times, the test engineers who are experts in the domain are not fully capable of test automation due to lack of their awareness in testing tools.
The paper explains the experience of authors in creating an XML Driven business component test automation Framework, which simplifies test automation by concealing the complexity of script development. Test automation of single product can be done using different test automation tools based on customer demands using BX-D framework. It also ensures that multiple tools can be used for test automation of the same product which inturn reduces the limitations of the tools. The horizons of BX-D are wide open with the opportunities of cloud testing which is going to conquer the software testing world.

About the Authors

Balamurali is working as Group Manager SQA at Network Systems and Technologies (NeST). He took his M.Tech from Indian Institute of Technology (IIT) Kanpur and has more than 14 years of experience in the industry. Has played a lead role in implementing the models like CMMI1.1, CMMI 1.2, and ISO9001:2008, Automotive SPICE, ISO 27001, IEC61508 and ISO 13485. He has worked as an Assessment Team Member for CMMI V 1.2. A CSQA & CQSPE from QAI, he is also a Certified Tester (advanced level- Test Management) by ISTQB, he is involved in activities of Indian Testing Board and plays an active role in the core group of the Trivandrum chapter of Software Process Improvement Network (SPIN). He also a certified Test Manager, Information Security Professional and Internal Auditor by Department of IT, Govt. of India.

Pradeep P P is working as Senior Lead Engineer with Network Systems and Technologies Pvt Ltd. He is managing testing activities for a delivery unit. He has got 12 years of experience in software testing industry. He is an Engineering graduate from College of Engineering Trivandrum under Kerala University. He is a Certified Test Manager by STQC and Certified Tester by ISTQB - Advance Level Test Manager. He gives consultancy for various testing tools like Selenium, J-Meter, QTP, Rational Functional Tester and Test Complete.

Rathish M M is bearing the role of Test Lead with Network systems and Technologies Pvt Ltd. He received his B Tech in Computer Science and engineering from College of Engineering Trivandrum under Kerala University. He got certifications such as Microsoft certified Technology Specialist (MCTS), IBM Rational certified tester for Rational Functional Tester (RFT) , IBM Rational certified Tester for Rational Performance tester (RPT), Certified Tester by ISEB-ISTQB, Nest certified professional for Selenium, J-meter, QTP, Load runner and Test complete. With more than 8 years of extensive experience in Software Testing industry, he gained sound knowledge in test automation and performance testing.. He got expertise in test automation framework designing, virtualization, continuous integration and bug work flow designing using various tools .He extends consultancy for various testing tools like Selenium, J-Meter, QTP, Rational Functional Tester, Test Complete and Load Runner.

Samiya Salim is working as Lead Engineer QA at Network Systems and Technologies (NeST). She took her B.Tech from Cochin University and has more than 6 years of experience in software and manufacturing industry. She played a key role in implementing CMMI v1.2 and IEC 61508. She is a Certified Software Quality Professional by STQC, Ministry for Information Technology; Govt. of India and ISTQB foundation level certified.

Transcript

1. Business Component and XML driven framework – BX-D driven framework – BX-D Balamurali L Group Manager SQA Pradeep P P Senior Lead Engineer - Testing Rathish M M Lead Engineer - Testing Samiya Salim Lead Engineer - QA
2. Contents • Industry Practice • Lacunae • Solution • BX-D • BX-D • Advantages • Case Study • Future Enhancements
3. Industry Scenario Test Automation improves productivity Estimation done Trainings are conducted Still should I go for test automation?
4. What is the competency required? Test Automation!! It requires coding skill Domain knowledge is also important Knowledge in automation tools is also crucial for test automation also important As a tester testing skills is also needed Will I be able to manage all these skills?
5. Which is the right tool? Features 51 to 54 and 56 to 59 can be done by QTP Features 50 , 55, and 60 to 75 can be done by Selenium Feature s 1 to 40 can Should I buy all the tools? I want 100% coverage and also to be cost effective Feature s 1 to 40 can be done by Test Complete All the other features can use AutoIT How will I choose the best tool?
6. Test Automation in product line V1.1 V 3.0 - - - - Customer A Customer B Customer G Test automation to be done using test complete Test automation to be done using QTP V1.0 V 2.0 V1.2 V1.3 V 2.1 Customer B Customer B Customer C Customer C Customer D Customer E Customer F Customer H Oh my God!! How will I manage the test automation if a new version is added to the product line? What will be effort for doing test automation using test complete and QTP for v1.1
7. • Adapting into the additions in product • Huge Effort • Maintenanc e effort to • Lack of skill of resources • Difficulty in • Cost of multiple test automation The Lacunae in product lines e effort to incorporate the changes into the test automation scripts • Lack of competent resources • Difficulty in identificatio n of automation t0ols automation Tools
8. Easy to manage in product lines Easily adaptable to the customization done in the product Reduce the dependency to the programming skills needed by the tester Supports multiple test automation tool
10. Prepare the list of business components Design the Test cases System study The Workflow Test execution Implement tests using business components Implement business components Select the automation tool Domain Expert Automation Expert
11. BX-D Architecture Test Configuration Test Scenario Generation Test Scenario Collection Test Result Collection BC Collection BC Editor Framework Layer Test case Layer NeST Controlled System Under Test (SUT) XML Driven TC Model Tool 1 TC Collection Tool 2 TC Collection Tool 3 TC Collection Tool 4 TC Collection Tool N TC Collection Test Tool Layer …… .. Tool 1 Driver Library TC implementation Tool 2 Driver Library TC implementation Tool 3 Driver Library TC implementation Tool n Driver Library TC implementation
12. Framework Layer Framework Layer Domain Expert Test Configuration Test Scenario Generation Test Scenario Collection Test Result Collection BC Collection BC Editor Domain Expert Interacts with this layer Path settings, Network Authentication etc are configured Business components are defined Test scenarios are created using business components Handles execution sequence in selected tool Test reports are generated on the fly
13. Test case Layer Test Case Layer XML Driven TC Model Tool 1 TC Collection Tool 2 TC Collection Tool 3 TC Collection Tool 4 TC Collection Tool N TC Collection A middleware between Framework layer and Test tool layer XML representation of test cases are created automatically
14. Test Tool Layer Test Tool Layer …… .. Automation expert works at this layer Test script corresponding to each business components are created Tool 1 Driver Library TC implementation Tool 2 Driver Library TC implementation Tool 3 Driver Library TC implementation Tool n Driver Library TC implementation Library contains reusable functions Driver parses xml test cases and invokes corresponding business component script Driver saves intermediate results in xml files
15. Advantages Easy creation of test cases by drag and drop Easily adaptable framework Meets demands of multiple customers Easy creation of test cases by drag and drop business components Test cases can be managed easily in product lines
16. Advantages No more skill dependency Domain expert Automation expert Optimized usage of open source and licensed tools source and licensed tools Testing can be scheduled simultaneously for product lines
17. Case Study - Scenario Multiple product lines delivered to different customers I am having resources with manual testing skills but not test automation If I am using the resource available the effort for training and maintenance is huge Product New feature added
18. Case Study - Solution Develop a test framework which segregates the scripting part and test case design Define Business component Domain expert Test Execution Test case design Test Results Implement business components Automation expert Scripting Optimized tool license management Test Suite Maintenance Floating license of Test Complete used in other projects also
19. Case Study - ROI 80% of test cases automated using free tools and 20% using licensed tools Reduced the utilization cost of licensed tools by 80% Reduced the cost of training by 70% since dedicated team only needs to be trained Reduced the maintenance cost by 60% Reduced the tool investment cost from 200 Lakh INR to 40 L akh INR
20. Future Enhancement Cloud NeST Network