Test Advocacy Framework - A Structure for Powerful Argumentative Wins for Testers

This presentation is among the Top 27 Best Papers/Practice/Tutorials selected, out of 460+ submissions received, to be presented @STC 2012.

Presentation Abstract

“Not a bug!” “Not in specifications!” “Not a priority!” These are familiar situations in the software development cycle that testers encounter every day. Most of these situations warrant a convincing argument to elucidate the quality perception. However there has not been much focus on this issue in QA literature today and there is no formal methodology that facilitates improvement of these skills.

Our paper describes a powerful advocacy framework which will help test professionals develop test advocacy skills. In the paper we examine scenarios where advocacy may be required and discuss each of these in detail. We elaborate on the techniques the testers could use to put forth the best prepared argument with the following steps in our framework:

1) Zoning
2) Framing
3) Probing
4) Clinching

We have also derived metrics such as Assertiveness Index of a Tester to benchmark and improve bug advocacy effectiveness.

About the Authors

Nikhila Premkumar, Senior Quality Assurance Engineer, Oracle India Private Limited. He holds a Bachelor of Engineering degree in Electrical and Electronics from Annamalai University, Chidambaram, India and a PGDBM in Systems and Marketing from XLRI, Jamshedpur. She has 5+years of experience working on Oracle Applications.

Anand Arumilli, Manager, Quality Assurance, Oracle India Private Limited, is a seasoned test practitioner, innovator and Quality evangelist. He has 10 years experience in Quality Assurance and 2 years in development. He currently works as a QA Manager in Oracle India and is part of the EBS Quality Assurance team in Oracle which tests Oracle Human Capital Management suite of products. He is an active participant in various QA forums in India and abroad.

Transcript

2. The Dark Side I don’t see that in the spec Downgrade this bug, Now! Design just changed, plea se test Dude, this is not a Bug That one? Not my Priority right now Now! Works fine on my machine
3. The Test Advocacy Framework A structure for powerful argumentation wins for testers By Arumilli Anand & Nikhila Premkumar
4. The Tester’s Arguments Advocacy Framework Three Sections Advocacy Framework Metrics for Test Advocacy
5. Classic Arguments The historic gulf of perception between a developer and tester
6. Classic Arguments Not a Bug
7. Tell Me Why The logic of why the tester did what he tester did what he had to
8. Tell Me Why
9. Go/No Go Arguments The perception of a tester from a release perspective from a release perspective
10. Go/No Go Arguments
11. Recap Classic Arguments Go/No Go Arguments Tell Me Why
12. A Tester’s Conversation Tester Developer Greg Jack
13. Background ‘GameDemon’ an online gaming company is planning to launch a new game ‘PlanetLuck’ in Texas. Texas. Tester ‘Greg’ has found an issue……
14. Jack, I found an issue with the jackpot feature in PlanetLuck. Oh! What’s the issue? When more than 1000 players enroll in a When more than 1000 players enroll in a single game. The total amount collected is more than $10000. This is against the Texas state regulations for jackpots . I have never seen 1000 players enrolling in a single game in my entire career in Gaming.
15. Don’t worry! This is one of those rarest of the rare cases. But.. This issue CAN happen, you know… Oh! I think this is a critical issue This can be deferred. I don’t see a reason to fix this now. Besides, I have other more critical issues to fix. Catch up later…
16. Tester’s Challenges in Advocacy Different perceptions of Quality among Stakeholders Code complexity and deadlines effect Presenting problems to people under stress
18. Fourfold Path Zoning Framing Probing Clinching
20. Creating an environment for maximum stakeholder receptivity Zoning maximum stakeholder receptivity
21. What’s in it for Me Who else says so Who else says so Compare
22. You are an Olympic Games contender. There’s a slim chance you’ll win a gold medal. But the chances are pretty good that you’ll go home with a silver or a bronze. Will you be happier with a silver medal or with a bronze medal?
23. In my last conversation with Ron (Product manager), he expressed that we should not overlook any Jack, I feel this is a unique problem and requires your expertise. Do you feel, you could help. (Product manager), he expressed that we should not overlook any regulatory issues. Rocketgame (competitor) seems to have this check as it explicitly mentions it in its documentation
26. Frame the narrative of the argument with a clear and concise core Framing with a clear and concise core
27. Twitter Test Redefine Amplify
28. Jack, though this is a rare case , the players may hit this scenario if the ticket prices increase. This would lead to damages As per government regulations we should not cross a limit of $10,000 for a jackpot. Currently our system does not have this check. may hit this scenario if the ticket prices increase. This would lead to damages in millions for non compliance. Gary(Market analyst) mentioned that a lot of new players may want to enroll after the new UI change. More jackpots can be expected.
30. “Thinking is not driven by answers but by questions.” Socrates
31. Questions that cause others to see why it makes sense to see or do Probing why it makes sense to see or do something your way
32. Powerful Questions Layering
33. What are the implications when this issue occurs Why cant we prioritize this issue for this release, given its impact. High Power Which are the varied situations that this issue can occur? Is this a critical issue? Low Power
35. Tap emotional needs and predispositions to clinch consent Clinching predispositions to clinch consent
36. Fleeting Opportunity BATNA Call for Action
37. If this fix is not ready by the release date , maybe we can continue working on it and Jack, this is the first time we are launching our game in Texas. All eyes on us.. , maybe we can continue working on it and ship it as a hot-fix after release. I would request you to please relook at this issue and fix it as a priority.
38. Recap Zoning Framing Probing Clinching
39. Implementation Orientation Incorporation 26.67 25.93 23.53 11.85 14.71 15 20 25 30 8.33 11.85 0 5 10 %Contested Bugs %Converted Bugs
40. Priority Conversion Ratio Tester’s Advocacy Index Contested Validity Ratio
41. Priority Conversion Ratio Upheld 58% 42% Total Number of bugs where severity was contested = 50 Waned 58% Contested bugs where final severity is higher = 21 Priority Conversion Ratio = 0.42
42. Contested Validity Ratio Waned Upheld 38% 62% Total Number of Contested bugs = 34 Waned Upheld Contested Qualified bugs = 21 Contested Validity Ratio = 0.62
43. Advocacy Index Priority Conversion Ratio = 0.42 Contested Validity Ratio = 0.62 Contested Validity Ratio = 0.62 Tester’s Advocacy Index = 0.52
44. Synergy between all Stakeholders Results Quicker Resolution of issues Quicker Resolution of issues Higher Quality releases
45. “May the Force be with you, Tester.”
46. Thank You