Pairwise Framework in ERP Application

This presentation is among the Top 27 Best Papers/Practice/Tutorials selected, out of 460+ submissions received, to be presented @STC 2012.

Presentation Abstract

In ERP Applications, we find that there are a large number of parameters for each function and a plethora of integration points. The sheer volume of the number of combinations to test is phenomenal. In testing teams, given the constraints of time and resources, there is an inherent need to optimize the exhaustive set of test combinations. It is observed that most errors are a result of the interaction of two variables. One scientific way of ensuring that all the double mode faults (pairs) are covered is the Pairwise Algorithm. In this paper we elaborate on the application of Pairwise Algorithm in ERP Applications. It explains a methodology of optimizing tests at each phase of the SDLC using Pairwise algorithm.

About the Authors

ShriRam Paidi holds Master's Degree in Physics with distinction from Andhra University, Vishakhapatnam. Presently working for Oracle India Pvt. Ltd for the past 9 years with current position as 'Project Leader - Quality Assurance' in many quality initiatives and educating many QA teams on generating optimal number of test cases using pair combination framework test design technique in Oracle.

Kanaka Durga Bandaru holds Master's Degree in Technology from National Institute of Technology, Warangal. Presently working for Oracle India Pvt. Ltd for the past 6 years with current position as 'Project Leader - Quality Assurance' in many quality initiatives and coordinating ShriRam on educating many QA teams on generating optimal number of test cases using pair combination framework test design technique in Oracle.

Transcript

1. Pairwise Framework
   in ERP Application
   ~ ShriRam Paidi & Kanaka Durga Bandaru
   

   View Slide

2. Agenda
   • What is Pairwise Testing?
   • What is ERP?
   • Pairwise Implementation in ERP
   • Pairwise Implementation in ERP
   • Value and Benefit
   

   View Slide

3. Observation in ERP Application
   It is an observation in ERP Application that most faults are
   caused by interactions of two parameters
   caused by interactions of two parameters
   

   View Slide

4. Root Cause Analysis for Customer Bugs
   One Parameter
   Two Parameters
   Three Parameters
   Other
   

   View Slide

5. Pairwise Testing
   Pairwise Testing
   

   View Slide

6. Pairwise Testing
   Pairwise testing is an effective test case generation technique that
   ensures that every parameter pair is tested at least once
   ensures that every parameter pair is tested at least once
   

   View Slide

7. Example
   1 2 3 4
   Person Type (A) Employee (a1) Contingent Worker (a2)
   Termination (B) Retirement (b1) Voluntary (b2) Force (b3) Deceased (b4)
   Benefits (C) PF (c1) Gratuity (c2) Pension (c3)
   (2)
   (4)
   (3)
   Number of
   Parameter
   Values
   Exhaustive Pair Combinations 2 * 4 * 3 = 24
   Optimum Pair Combinations = max(X) * max [X \ max(X)]
   i.e. Optimum Pair Combinations = max(X) * next max (X)
   i.e. Optimum Pair Combinations = 4 * 3 = 12
   Note: X is a set which contains number of parameter values, i.e. X = {2, 4, 3}
   

   View Slide

8. Create Exhaustive Pair Combinations 2 * 4 * 3 = 24
   Create Empty Set for Optimum Pair Combinations 4 * 3 = 12
   Create Empty Set for Optimum Pair Combinations 4 * 3 = 12
   

   View Slide

9. Start Moving AB unused Pair Combination to Empty Set
   

   View Slide

10. Move all AB unused Pair Combination to Empty Set
    

    View Slide

11. Start Moving BC unused Pair Combination to Empty Set
    

    View Slide

12. Move all BC unused Pair Combination to Empty Set
    

    View Slide

13. Move all AC unused Pair Combination to Empty Set
    

    View Slide

14. Optimum Pair Combinations
    

    View Slide

15. Another Example
    1 2 3
    Person Type Employee (a1) Contingent Worker (a2) Applicant (a3)
    Gender Male (b1) Female (b2) Unknown (b3)
    Job Trainee (c1) Engineer (c2) Manager (c3)
    (3)
    (3)
    (3)
    Number of
    Parameter
    Values
    Exhaustive Pair Combinations 3 * 3 * 3 = 27
    Exhaustive Pair Combinations 3 * 3 * 3 = 27
    Optimum Number of Test Cases = max(X) * next max (X)
    i.e. Optimal Pair Combinations 3 * 3 = 9
    Note: X is a set which contains number of parameters, i.e. X = {3, 3, 3}
    

    View Slide

16. View Slide

17. ERP System
    

    View Slide

18. View Slide

19. View Slide

20. View Slide

21. View Slide

22. Pairwise in ERP Application
    

    View Slide

23. Unit Level
    System Level
    Integration Level
    

    View Slide

24. View Slide

25. Learning Management a ERP Application
    Learning Management is the product in ERP Application which
    automates Employee Trainings.
    Employee Employee Employee
    Employee
    Enrolls in a
    Training
    Employee
    Completes the
    Training
    Employee
    Skills are
    updated
    

    View Slide

26. Unit Level
    

    View Slide

27. Unit Level
    

    View Slide

28. Create a Training Course
    

    View Slide

29. Identify Field Level Data
    Create a New Training Course
    Parameter Data Values
    Course Offering Online, In a Class, Web Conference, Self Study (4)
    Course Type Normal Content, Assessment, Both (3)
    Learner Access Self, Approval, Mandatory (3)
    Enrollment Limit Limited, Unlimited (2)
    Waitlist Allowed, Not Allowed (2)
    Cost Free Course, Priced Course (2)
    Total Combinations (4 * 3 * 3 * 2 * 2 * 2 = 288)
    

    View Slide

30. Input Data to Pairwise Algorithm
    Enter Parameter and Parameter Values
    Note: We have written a Macro which Generates Test Cases using Pairwise Algorithm
    

    View Slide

31. Result – Unit Level - Using Pairwise
    

    View Slide

32. System Level
    

    View Slide

33. System Level
    

    View Slide

34. Identify System Level Transactions
    Parameter Data Values
    Feed Unit Level Test Cases 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 (14)
    Learner (Person) Applicant, Employee, Ex-Employee, Contingent Worker, Ex-Contingent
    Person Enrolls in a Training Course
    Learner (Person) Applicant, Employee, Ex-Employee, Contingent Worker, Ex-Contingent
    Worker (5)
    Enrollment Self, Manager, Mandatory, Admin Single, Admin Bulk, Approval (6)
    Enrollment Availability Available, Not Available, Waitlisted (3)
    Total Combinations (14 * 5 * 6 * 3 = 1260)
    

    View Slide

35. Result – System Level - Using Pairwise
    

    View Slide

36. Integration Level
    

    View Slide

37. Result – Unit Level - Using Pairwise
    

    View Slide

38. Identify Integration Points
    Parameter Data Values
    Employee Completes the Training and Skill are updated in Person Competency
    Integration Points – Learning Management Interface with Performance & Approvals
    Parameter Data Values
    Feed System Test Cases SLTC1, SLTC2, ……., SLTC84 (84)
    Assessment Score Card Passed in Course, Failed in Course, Not Applicable (3)
    Person Competency Attach, Do Not Attach (2)
    Approval Manager Approval, Auto Approve (2)
    Total Combinations (84 * 3 * 2 * 2 = 1008)
    

    View Slide

39. Result – Integration Level - Using Pairwise
    

    View Slide

40. Summary – without Pairwise
    Total Unit Level Combinations (3 * 4 * 3 * 2 * 2 * 2 = 288)
    Total System Level Combinations ( 288 * 5 * 6 * 3 = 25,920)
    Total System Level Combinations ( 288 * 5 * 6 * 3 = 25,920)
    Total Integration Level Combinations ( 25,920 * 4 * 2 = 2,07,360 )
    

    View Slide

41. Summary – with Pairwise
    Total Unit Level Combinations (3 * 4 * 3 * 2 * 2 * 2 = 288)
    Optimized Unit Level Combinations using Pairwise: 14
    Total System Level Combinations (14 * 5 * 6 * 3 = 1260)
    Optimized System Level Combinations using Pairwise: 84
    Total Integration Level Combinations (84 * 3 * 2 * 2 = 1008)
    Optimized Integration Combinations using Pairwise: 252
    

    View Slide

42. Summary
    Ensures that every parameter pair has been Tested at least once
    Optimizes Test Effort within the Project Time Constraints
    

    View Slide

43. Trend of Customer Bugs Reported
    150
    200
    250
    2009
    0
    50
    100
    150
    C1 C2 C3
    2009
    2010
    2011
    

    View Slide

44. Test Environment Configuration
    

    View Slide

45. Configurations for a ERP Application
    Browser - IE 8, 9, Firefox 3.5, 10, Safari 4, 5 (6)
    Client OS – Windows 7, XP, Vista, MAC, iOS (5)
    Server OS – Windows, Linux (2)
    Data Base - 10g, 11g (2)
    Receiving Pages from Difference Servers – Apache, IIS (2)
    Total Combinations (6 * 5 * 2 * 2 * 2 = 240)
    

    View Slide

46. Configurations for a ERP Application
    Creating Configuration is expensive
    Consumes lot of Time and Effort
    

    View Slide

47. Value and Benefits
    

    View Slide

48. Value and Benefits
    Optimal number of Test Cases for Maximum Test Yield
    Saves QA Effort in Documentation and Execution
    Cost to Quality in ERP Comes Down
    Cost to Quality in ERP Comes Down
    

    View Slide

49. Thank You
    

    View Slide