Model-Driven Testing for Large end-to-end Testing Programs: A Case Study

Presentation by Vikash R. Verma, Plenary Keynote Speaker @STC 2012.

Presentation Abstract

The Telecom industry has probably experienced the fastest and deepest evolution over the past 3-4 decades of all the other established industries: from government-owned companies to private businesses open to competition, with the introduction of mobile telephony, broadband, triple play, etc. For a provider of outsourcing services to this industry, it usually means being able to take up every day the toughest IT challenges.

Vikas Verma's talk will highlight the major changes he had to define and manage within his end-to-end testing organization in order to satisfy his main customers' demands. He will more particularly share his experience in the introduction of Model-Driven testing technology in an established large scale program.

Key Take Aways

This talk will highlight the particular challenges of large scale end-to-end testing programs of mature clients in the telecom industry, in agile context, with multi-vendors involvement and highly complex IT configurations; it will present a case study describing the improvements that had to me made in the organization (factory approach), the challenges of change management and the contribution of technological innovations such as model-driven technology.

About Viskash

Vikas has 18+ years of IT experience and managed, delivered & handled team for all functions of SDLC. He is heading the Test delivery unit of TechMahindra, transformed and innovated the Testing unit from day to day operation to more automated, factory way of operation to provide value for money and operational excellence to customer. Before joining TechMahindra; he was with Orange Business Service (FT) as head of Global Service Delivery.

Transcript

1. MODEL DRIVEN TESTING FOR A LARGE END TO END TESTING PROGRAM Vikas Verma, Head - E2E Testing, TechMahindra
2.  Dynamic Market with major innovations over the past 20 years (Broadband, Mobile phone, TV, Smartphones, etc)  Strong historical market positions (from the time when they had monopolistic position)  High maturity in the management of their vendors E2E Testing of a Tier1 UK-Based Telecom Operator
3. TechM Global Test Factory  Factory portal, Tools - RAPID, eConvergence  Workbook hierarchy  Clear escalation paths  Organisation stakeholder mapping  Real time reporting structure Customer Engagement  ADAPTIVE  INTELLIGENT  HIGH PERFORMANCE  LEAN TOOLS  SHARED SERVICES  ENGAGE RIGHT TALENT  STANDARDIZATION  KNOWLEDGE MANAGEMENT  BUSINESS ASSURANCE  STEP CHANGE IN COST  NEW REVENUE STREAMS  OPERATING MODEL IMPROVEMENT  E2E Test  User Acceptance Test  Operational Readiness Test  Security Testing  Non Functional Testing  Network Testing  Stub Development  System / Continuous Integration Testing  Customer Experience Factory Offerings Factory Principles
4. Life Cycle Automation Through Innovation
5. Why Model Based Testing ? Objectives & Benefits Triggers for MBT MBT Lack of defect detection in Early cycle Manual & Automation scripting in Silo Ever increasing Business Demands High Quality at Optimised cost High maintenance efforts Low Reusability Consistency
6. What is MBT and How it Works ?  Model-based testing is a test DESIGN solution  It combines a method (based on modeling) and a tool (automatic test case generation)  Formalization and graphical representation of the requirements and expected behavior  Automatic test case generation with the MBT tool  Automatic publication of the tests into the test repository (ex: QC) • Ready-to-execute manual test cases • Frame work for automation • Requirement traceability matrix  When requirements change, model is to be updated; then, impacts are computed and propagated automatically
7. Selection of MBT solution • Smartesting • Conformiq Evaluated solutions • Workshops • Proof of Concepts • Observed Benefits • Integration with existing environment (QC, SCM tools, QTP, Greybox ) Evaluation process • More adapted to End-to-End testing through the use of business-oriented modeling language (BPMN) • Test Optimization Techniques • Easy-to-adopt / easy-to-use / easy-to-integrate • Customizable publication output • Flexible licensing model Choice of Tool (Smartesting)
8. Implementation of MBT using Smartesting  Smartesting tool uses BPMN and UML modeling • BPMN is Business Process Model Notation which provides the E2E flow for a particular process • Behavior of the system , test data , pre requisites and post condition would be modeled using UML (Unified Modeling Language) • Business rules are modeled using decision tables (or a simple programing language called OCL)  Core team is responsible for developing the models required across all the products / teams
9. •Import & formalize business process models •Models verification and simulation •Models creation wizard •Test strategy definition - risks, requirements , coverage, pairwise Smart Asset™ •Coverage report with RBT user defined views •Systematic Requirement-to-tests traceability •Automated Impact analysis for any change requirements •Coverage+Tests Documentation creation for BAs Smart Coverage™ •Test cases & test scripts automated generation •Optimized number of tests for targeted objectives •Test generation dashboard with user defined views •Automated synchronization of tests and requirements in ALM solutions for maintenance Smart Engine™ Key takeaways of the Smartesting MBT solution
10. CFT – HL scenario CIT Model from E2E Flow Final E2E LLTC Business Demand Requirement E2E Design E2E model Component Model Component Design Continuous Integration Test Cases IVVT test cases Repository MBT as a Collaborative Tool
11.  Frist time efforts spent on the developing the models is high in MBT  Due to reusability of basic models , approximately 40% less efforts compared to manual test planning  Approximately 18 – 20 % effort saved in E2E Test Life Cycle Manual Vs MBT
12. Learning's Leaders to be Change Agent Detailed & defined implementation strategy Implementation in the area of high reusability Defined & Consistent Modeling Frame work Formation of Core team (SME’s) for Implementation and Trainings Build library of reusable Models CM of models
13. Thank You