Quality Beacon: Exploring with Code

Transcript

1. Exploring with code When testing stops splitting in two Maaret Pyhäjärvi October 2026 © 2026 CGI Inc. 1
2. AI ENABLES TASK EXPANSION People can now do things by asking for it. © 2026 CGI Inc. 2
3. ASKING FOR IT Use playwright-cli to operate website https://www.workroomproductions.com/raster-reveal/ and create a script that systematically reveals the image that gets more accurate by mouse movement. Take regular screenshots and figure out for each if you know what is in the picture. Keep the browser open so that I can watch it unfold. © 2026 CGI Inc. 3
4. ABOUT TESTING Search for information that matters Test design Test cases 0:00 Test execution 0:46 animal-like shape ManualAis dark emerging on a light Confirmed: it's a quadtree that reveals finer detail in blocks my cursor actually crosses. Let me sweep the remaining bands top-tobottom. background, with Managing orange/gold accents. 3:12 Exploratory 13:26 That's unmistakably a black rubber duck with an orange beak and a gold Batman bat-symbol on its side/wing — a "Batman rubber duck" bath toy. Final identification Automated stands: a black rubber duck with an orange Doing beak and a gold Batman logo on its side. Avoid co nfidentl y incorr ect judg ement © 2026 CGI Inc. 4
5. SPLIT TING WORK WHILE MAINTAINING AGENCY The ways we split testing don’t make sense. MANAGING | DO ING MANUAL | AUTOMATED TEST CASES | EXPLORING Te s t i n g i s l e a r n i n g . L e a r n i n g i s l a y e r e d . © 2026 CGI Inc. 5
6. Testing works with invisible ink We target these… WHAT WHEN WHO HOW WHY …to find these... BUSINESS TIME USER Information of Relevance Test Ideas / Experiments …to tell if there’s more and how well we know. Coverage © 2026 CGI Inc. REQUIREMENTS RISKS (of problems) CODE ENVIRONMENTS RESULTS INPUTS 6
7. UNATTENDED TESTING AS A STRATEGIC CHOICE Test automation gives us more than regression Specification Feedback Regression Granularity Examples shared before implementation act as specification. We know when it works as we’re implementing the progress. We safeguard our past intent over lifetime of system ownership. We know when and what changed without extensive analysis. Documenting Extending reach Alerting to attend Guiding to detail Capture examples whenever we discuss them. Time. Environments. Data. Changes, both intended and unintended. Drives attention to details humans do repair for. © 2026 CGI Inc. 7
8. ATTENDED TESTING AS A STRATEGIC CHOICE Beyond automation, testing as a performance Guidance Understanding Models Serendipity Information is not pass or fail, but more like a degree into a direction. Seek disillusionment before the world shows its color at a time our hands are tied. Contribute to relearning and managing cognitive space. The more I practice, the luckier I get. It’s not that I am so smart, I just stay with the problems longer. © 2026 CGI Inc. 8
9. Most valuable form of coverage is results: empirical, actionable, and current information. © 2026 CGI Inc. 9
10. NEED TO DO BETTER Results gap 73 known issues Invis ible ink: proble ms are there ev en if no one re ported them. Practice at: https://exploratory-testing-academy.github.io/capture-the-bugs/ 72 known issues, 24 high impact AI alone: 5% | Traditional testers (n=57): 17% | Explorer: 65% | Explorer with AI: 100% © 2026 CGI Inc. 10
11. CONTEMPORARY EXPLORATORY TESTING IN AGE OF AI Interplay of attended and unattended © 2026 CGI Inc. 1 Existing automation allows for exploration reach 2 Creating automation forces exploration of details 3 Failing automation is an invitation to explore 4 AI is external imagination allowing task expansion 5 Take you there and continue attended 11
12. CO LLABO RATIVE ACTIVIT Y Extend from reporting issues into debugging, repairs, and shared team practices, including exploration at unit level. © 2026 CGI Inc. 12
13. AI-ENHANCED APPLICATION TESTING Smarter handoffs in value chain: task expansion in talent Secon d pair of eyes for qual ity pers pective through reviews Find issues f irst, te st cases follo w Repo rt bugs with a f ix for revie w 01 Start earlier Docu ment tes ts with t est auto mation Continue further Tasks of Testing From logistics of test targets to information, test cases and issue reports Build your ag entic to olkit to level up quality results in talen t © 2026 CGI Inc. 13
14. A majority of the production failures (77%) can be reproduced by a unit test. https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-yuan.pdf Through https://www.slideshare.net/Kevlin/the-error-of-our-ways © 2026 CGI Inc. 14
15. Solo – Pair - Ensemble © 2026 CGI Inc. Getting best out of everyone into the work we are doing 15
16. Short-term Break illusions in a timely fashion Test automation people can use when we’re gone Capture notes in test automation RESULTS Improve your skills Learn to pair & ensemble Generative AVAILABILITY TIMELINE Productive Improve skills of everyone Long-term © 2026 CGI Inc. 16
17. AGE OF AI Exploring intent, value, context, alignment and experience © 2026 CGI Inc. 17
18. AI-NATIVE: CONTEXT AND CODE LIVE TOGETHER AND GROW THROUGH EXPERIMENTS Delivery acceleration using AI Engineer Plan Implement Requirements, wireframes, skills Execute plan, refine and iterate Iterate and approve Iterate and approve Agent Plan.md © 2026 CGI Inc. Code Deploy 18
19. BREAKING ILLUSIONS WITH TESTING Skills that move you away from ’tester’ and ‘developer’ © 2026 CGI Inc. 19
20. AGE OF AI Actionable feedback that challenges well-maintained illusions has never been more important. © 2026 CGI Inc. 20
21. Insights you can act on Founded in 1976, CGI is among the largest IT and business consulting services firms in the world. We are insights-driven and outcomes-focused to help accelerate returns on your investments. Across hundreds of locations worldwide, we provide comprehensive, scalable and sustainable IT and business consulting services that are informed globally and delivered locally. cgi.com © 2026 CGI Inc. 21