Habit of AI in Quality Engineering

Four years of programmatic tests with GitHub Copilot. One year of AI-generated pull request reviews. Three years of applying chat-based generative AI. Those four years have lead to a habit of AI in the work I do, and I believe in 2025, we need that habit distributed across the organization.

In this talk we look at my use case examples:

Pair testing with Edge Copilot
Assisted programming with GitHub Copilot
Working with changes to code with help of generative AI

From these examples, we pull together a conceptual model of models, filters, flows, visible and invisible inputs, and UX integration. The tools we have are enough to build a habit, and the conceptual model helps us extend that habit as more features based on generative AI become available.

Let's build that habit of good use of the technology that serves us well as external imagination.

Transcript

1. © 2025 CGI Inc. 1 Habit of AI in Quality Engineering Maaret Pyhäjärvi
2. © 2025 CGI Inc. 2 #function to convert integer to English for values from 0 to 999 ApprovalTests 2021 edition
3. © 2025 CGI Inc. 3 https://gist.github.com/maaretp/cf9b1b0c6d385578a048258394697b5b
4. © 2025 CGI Inc. 4 We are accountable 1. Intent / Implementation 2. Domain for the Layman 3. Domain for the Expert 4. Reference Implementation 5. People Filtering 6. Interesting side effects
5. © 2025 CGI Inc. 5 Hello, I am an AI practitioner 👋 IVVES (2019) ITEA3 (EU) research project on AI in testing / testing AI GitHub Copilot (2021) Pair programming job interview with this lead to ‘Let’s Do a Thing and Call it Foo’ ChatGPT (2022) GenAI pair testing CodiumAI Qodo Merge Pro Code reviews to mostly ignore Microsoft Copilot Corporate constraints Models Integrations Integrations LLMs Contracts RAG CGI NAVI Generating artifacts TestZeus Hercules Gherkin to test results Open Source LLMs Open Source Agents
6. © 2025 CGI Inc. 6 “The secret of getting ahead is getting started. The secret of getting started is is breaking your complex overwhelming tasks into small manageable tasks, and starting the first one”. – Mark Twain 6 Photo by Aron Visuals on Unsplash
7. © 2025 CGI Inc. 7 Quality strategy for how quality is created, maintained and lost QUALITY ENGINEERING Test ideas Test assumptions DEV OPS D eploy D iscover Specify with examples Test changes Test system Test in production Test process Test through telemetry
8. © 2025 CGI Inc. 8 The tools are combinations of building blocks Generate › Summarize: more into less › Elaborate: less into more › Transform: format change › Large Language Models Remember › Memory: save keywords over time › Templates: output instructions › Domain: prompting reuse › Retrieval Augmented Generation See › Vision: classifying pictures › Classification Models Moderate › Input and output filters › Filters Orchestrate › High level to steps breakdown › Hyperautomation tools › Agents Integrate › Available where the work gets done
9. © 2025 CGI Inc. 9 No ‘one best tool’ – protect time while encouraging growing AI habit 9 Generate MODELS Hosted open- source models Filter GUARDRAILS Org memory CONTEXT Act AGENTS Decide AUTOMATION Cloud APIs to models AUTOMATE TestZeus Hercules DOCUMENT CGI NAVI CODE GitHub Copilot DOCUMENT Edge Copilot DOCUMENT ChatGPT CODE Cursor DOCUMENT CODE UIPath CODE OpenAI APIs DOCUMENT Ollama+ DOCUMENT CODE Chrome AI assistant Growth in AI habit
10. © 2025 CGI Inc. 10 Selected stories Manual agent orchestration Ask Claude. Ask ChatGPT. Ask Edge Copilot. Ask GitHub Copilot. Control their context. Build conceptual vocabulary New concepts in meetings. Ask genAI. Be curious about truth. Digital twinning Control your hidden input. Give samples of your content and style. Break rules and reflect Your team building problem solving by AI. Your training quiz by AI. Baselining homework GenAI from image above newbie median on a simple app of testing. Bug reports to bug fixes. Feature at time with GitHub Copilot Protect time with partner of choice. Features get copied. Pair work with genAI Routine of bringing tasks to habits. Notetaking Discipline to organize your texts into personal pipelines. Model from demos There’s so many tools, and so much smoke screen.
11. © 2025 CGI Inc. 11 Practice-level guardrails Expected values Pay attention to the old testing wisdom of oracles and how do we know. Our critical thinking, built on our learning through curiosity of the world is essential. 01 Anti-toolist worldwiew Realize that features in tools can be copied. Looking for the one best tool makes little sense. We need to protect our time to a partner of choice. 02 Taskwide learning Not lifelong learning or life wide learning, but it's task wide learning. Everything we do is learning activity. 03
12. © 2025 CGI Inc. 12 Time used on warning about test automation is time away from succeeding with it. 12 Photo by Filip Zrnzević on Unsplash
13. © 2025 CGI Inc. 13 13 https://knowingmachines.org/models-all-the-way
14. © 2025 CGI Inc. 14 Predictive text generation useful for recall not reasoning 14
15. © 2025 CGI Inc. 15 Shared Direction and Results that Show up in Scale AI-Driven Testing Measure and assess to baseline productivity and good practices Experiment together with customers to deliver promise of value in point, application and system solutions Capture in pipelines and methodology of sociotechnical guardrails Habitually apply and reflect to instill culture of learning Teach for co-creation, share and learn to avoid regional divide Scale with IP on improved service and licensed solutions Human-centric Enhancing Incremental Impactful at Scale
16. © 2025 CGI Inc. 16 Insights you can act on Founded in 1976, CGI is among the largest IT and business consulting services firms in the world. We are insights-driven and outcomes-based to help accelerate returns on your investments. Across hundreds of locations worldwide, we provide comprehensive, scalable and sustainable IT and business consulting services that are informed globally and delivered locally. cgi.com