Non-regression tests with exploratory testing : h̶a̶r̶d̶e̶r̶, Easier, Better, Faster

Dans un contexte de responsabilisation de équipes de build autour de la qualité, les enjeux relatifs à la non-régression sont un levier stratégique.

Au travers d'une expérience menée chez Meetic je vous propose de découvrir les gains et enseignements d'une démarche où l'exploratory testing combiné au risk management a permis un abandon progressif des cahiers de test et un meilleur coverage lors des phases de non-régression.

Transcript

1. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER Hello
2. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER NON-regression testing
3. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER NON-regression testing Retest all =
4. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER Complete testing can be a dumb idea CONTEXT MAKES THE METHOD
5. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER FASTER BETTER, HARDER, EASIER NON-REGRESSION WITH EXPLORATORY TESTING
6. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER wHAT’s meetic? 3 PRODUCTS 8 BRANDS 15 COUNTRIES 4 PLATFORMS 15 MOVES TO PRODUCTION A DAY NEW VERSION OF NATIVES APPS EVERY 2 WEEKS 150 PEOPLE IN IT TEAMS / 7 QA
7. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER BACK IN TIME
8. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER BACK IN TIME 2013 ONE MAJOR RELEASE A WEEK ALL TESTS BY THE QA ONE MAJOR PRODUCT NO AUTOMATION QA IS APART
9. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER NON-REGRESSION? FULL NRT X2 12 HOURS TESTING TEST CASES REFERENTIALS
10. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER NON-REGRESSION? FULL NRT X2 12 HOURS TESTING TEST CASES REFERENTIALS TESTING SKILLS PRODUCT TECHNICAL
11. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER 2014 2015 2016 2017 2018
12. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER PARADIGM SHIFT 2018 FEATURE TEAMS ORGANISATION QUALITY IS EVERYONE DUTY EVERYBODY PERFORMS TESTS QA EMBED CONTINUOUS INTEGRATION
13. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER BuT … WE forgot something
14. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER Teams don’T perform nrt NOT MY JOB SKILLS ARE NEEDED – TIME IS RUNNING OUT TEST CASES REFERENTIALS? AUTOMATION? COVERAGE IS TOO LOW ARE YOU KIDDING?
15. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER QA PERFORMS A DAILY NRT (45 MIN) X 2 IN PROD FULL NRT ON 1 PLATEFORM = 8H TESTING LOW COVERAGE 10 DEFECTS FOUND IN THE FIRST SEMESTER OF 2018 CRITICAL CASES COMMON FLOWS EDGE CASES
16. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER EXPLORATORY TESTING ?
17. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER PLAN EXECUTE DESIGN REPORT
18. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER PLAN EXECUTE DESIGN REPORT LEARN interprete
19. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER EXPLORATORY TESTING LET’S DO A QUICK CHECK NO NEED FOR DOCUMENTATION CLICK AND TAP EVERY BUTTONS TEST RANDOMLY TO BREAK THE APP
20. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER LEARN ARE RISKS COVERED? HOW TO USPKILL? METHOD? HOW TO BEGIN? HOW TO PRIORITIZE?
21. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER sfdpo SAN FRANCISCO DEPOT
22. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER S F D P O STRUCTURE What the feature is? What are its components? FUNCTION What the feature does? What are its functions from both the business and user perspective? DATA What the feature processes? What input does it accept and what is the expected output? Is input sequence sensitive? PLATFORM What the feature depends on? What operating systems does it run on? third-party components? OPERATIONS How the feature will be used? Who will use it? Where and how will they use it?
23. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER
24. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER TESTING SKILLS PRODUCT TECHNICAL
25. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER 6 months later FROM THE FIRST DAY : DAILY NRT IS 1/3 SHORTER NRT BY TEAMS + 100 % MORE DEFECTS FOUND 40% OF NRT
26. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER Continuous improvement
27. EXPLORATORY TESTING: HARDER, EASIER, BETTER, FASTER NOW IT’s your turn