Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Test Advocacy Framework - A Structure for Powerful Argumentative Wins for Testers

Test Advocacy Framework - A Structure for Powerful Argumentative Wins for Testers

This presentation is among the Top 27 Best Papers/Practice/Tutorials selected, out of 460+ submissions received, to be presented @STC 2012.

Presentation Abstract

“Not a bug!” “Not in specifications!” “Not a priority!” These are familiar situations in the software development cycle that testers encounter every day. Most of these situations warrant a convincing argument to elucidate the quality perception. However there has not been much focus on this issue in QA literature today and there is no formal methodology that facilitates improvement of these skills.

Our paper describes a powerful advocacy framework which will help test professionals develop test advocacy skills. In the paper we examine scenarios where advocacy may be required and discuss each of these in detail. We elaborate on the techniques the testers could use to put forth the best prepared argument with the following steps in our framework:

1) Zoning
2) Framing
3) Probing
4) Clinching

We have also derived metrics such as Assertiveness Index of a Tester to benchmark and improve bug advocacy effectiveness.

About the Authors

Nikhila Premkumar, Senior Quality Assurance Engineer, Oracle India Private Limited. He holds a Bachelor of Engineering degree in Electrical and Electronics from Annamalai University, Chidambaram, India and a PGDBM in Systems and Marketing from XLRI, Jamshedpur. She has 5+years of experience working on Oracle Applications.

Anand Arumilli, Manager, Quality Assurance, Oracle India Private Limited, is a seasoned test practitioner, innovator and Quality evangelist. He has 10 years experience in Quality Assurance and 2 years in development. He currently works as a QA Manager in Oracle India and is part of the EBS Quality Assurance team in Oracle which tests Oracle Human Capital Management suite of products. He is an active participant in various QA forums in India and abroad.

More Decks by QAI Software Testing Conference

Other Decks in Technology

Transcript

  1. The Dark Side I don’t see that in the spec

    Downgrade this bug, Now! Design just changed, plea se test Dude, this is not a Bug That one? Not my Priority right now Now! Works fine on my machine
  2. The Test Advocacy Framework A structure for powerful argumentation wins

    for testers By Arumilli Anand & Nikhila Premkumar
  3. Tell Me Why The logic of why the tester did

    what he tester did what he had to
  4. Go/No Go Arguments The perception of a tester from a

    release perspective from a release perspective
  5. Background ‘GameDemon’ an online gaming company is planning to launch

    a new game ‘PlanetLuck’ in Texas. Texas. Tester ‘Greg’ has found an issue……
  6. Jack, I found an issue with the jackpot feature in

    PlanetLuck. Oh! What’s the issue? When more than 1000 players enroll in a When more than 1000 players enroll in a single game. The total amount collected is more than $10000. This is against the Texas state regulations for jackpots . I have never seen 1000 players enrolling in a single game in my entire career in Gaming.
  7. Don’t worry! This is one of those rarest of the

    rare cases. But.. This issue CAN happen, you know… Oh! I think this is a critical issue This can be deferred. I don’t see a reason to fix this now. Besides, I have other more critical issues to fix. Catch up later…
  8. Tester’s Challenges in Advocacy Different perceptions of Quality among Stakeholders

    Code complexity and deadlines effect Presenting problems to people under stress
  9. You are an Olympic Games contender. There’s a slim chance

    you’ll win a gold medal. But the chances are pretty good that you’ll go home with a silver or a bronze. Will you be happier with a silver medal or with a bronze medal?
  10. In my last conversation with Ron (Product manager), he expressed

    that we should not overlook any Jack, I feel this is a unique problem and requires your expertise. Do you feel, you could help. (Product manager), he expressed that we should not overlook any regulatory issues. Rocketgame (competitor) seems to have this check as it explicitly mentions it in its documentation
  11. Frame the narrative of the argument with a clear and

    concise core Framing with a clear and concise core
  12. Jack, though this is a rare case , the players

    may hit this scenario if the ticket prices increase. This would lead to damages As per government regulations we should not cross a limit of $10,000 for a jackpot. Currently our system does not have this check. may hit this scenario if the ticket prices increase. This would lead to damages in millions for non compliance. Gary(Market analyst) mentioned that a lot of new players may want to enroll after the new UI change. More jackpots can be expected.
  13. Questions that cause others to see why it makes sense

    to see or do Probing why it makes sense to see or do something your way
  14. What are the implications when this issue occurs Why cant

    we prioritize this issue for this release, given its impact. High Power Which are the varied situations that this issue can occur? Is this a critical issue? Low Power
  15. If this fix is not ready by the release date

    , maybe we can continue working on it and Jack, this is the first time we are launching our game in Texas. All eyes on us.. , maybe we can continue working on it and ship it as a hot-fix after release. I would request you to please relook at this issue and fix it as a priority.
  16. Implementation Orientation Incorporation 26.67 25.93 23.53 11.85 14.71 15 20

    25 30 8.33 11.85 0 5 10 %Contested Bugs %Converted Bugs
  17. Priority Conversion Ratio Upheld 58% 42% Total Number of bugs

    where severity was contested = 50 Waned 58% Contested bugs where final severity is higher = 21 Priority Conversion Ratio = 0.42
  18. Contested Validity Ratio Waned Upheld 38% 62% Total Number of

    Contested bugs = 34 Waned Upheld Contested Qualified bugs = 21 Contested Validity Ratio = 0.62
  19. Advocacy Index Priority Conversion Ratio = 0.42 Contested Validity Ratio

    = 0.62 Contested Validity Ratio = 0.62 Tester’s Advocacy Index = 0.52