Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Better together: Humans, AI and the next qualit...
Search
Maaret Pyhäjärvi
October 12, 2026
Programming
31
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Better together: Humans, AI and the next quality benchmark
Talk at PNSQC.
Maaret Pyhäjärvi
October 12, 2026
More Decks by Maaret Pyhäjärvi
See All by Maaret Pyhäjärvi
Increase personal AI coverage
maaretp
0
22
Testing Assembly: Working through uncertainty - collaborative learning
maaretp
0
59
Property-based WebUI Testing with Bombadil and Playwright
maaretp
0
100
PractiTest: Exploratory Testing Reimagined
maaretp
0
35
Quality Beacon: Exploring with Code
maaretp
0
41
Bitesizing Code Review in the Age of AI
maaretp
0
36
From observations to insights on test specialist careers with Okechukwu Egbete
maaretp
0
31
Token Optimization Folklore
maaretp
0
81
Exploratory testing reimagined
maaretp
0
120
Other Decks in Programming
See All in Programming
Vue.jsで作る空間解析アプリとCapacitorプラグインのData Bridge
hal_spidernight
1
170
S3 TablesのIcebergデータに、AIエージェントからアクセスしてみた / 20261016developersio2026joetsu-accessing-iceberg-in-s3tables-via-agent
kasacchiful
PRO
0
290
エージェントを使い続けるためのメタワーク / Meta-work for keeping using the agent
rkaga
6
2.2k
[DroidKnights 2026] Workflow AX: From Auto-Documented Policies to E2E Testing
raeminkang
0
540
Optunaを作ったときにやりたかったこと、その後 (2026/10/16, Optuna Meetup #3)
iwiwi
4
1.5k
SFCで実現する病院・医療システムの動的組み立て式UI
koukimiura
1
170
Crash Auto Pipelines
veronikapj
1
400
Railsバージョンを4倍にした話
optim
0
960
AIで開発が速くなった先で、開発チームは何を目指すか
shihochan
1
2k
AWS CDKを理解するために公式の定義を深掘って解説してみる / Understanding AWS CDK by Digging into Its Official Definition
akihisaikeda
3
290
선언형 햅틱 라이브러리 진동을 소개합니다
l2hyunwoo
1
300
AmebaブログはKMPから卒業します
kmars0605
0
340
Featured
See All Featured
How to Get Subject Matter Experts Bought In and Actively Contributing to SEO & PR Initiatives.
livdayseo
0
240
B2B Lead Gen: Tactics, Traps & Triumph
marketingsoph
1
300
Information Architects: The Missing Link in Design Systems
soysaucechin
1
1.3k
The SEO Collaboration Effect
kristinabergwall1
1
620
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
128
56k
Building an army of robots
kneath
307
47k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
230
23k
A brief & incomplete history of UX Design for the World Wide Web: 1989–2019
jct
2
560
The #1 spot is gone: here's how to win anyway
tamaranovitovic
4
1.2k
HTML-Aware ERB: The Path to Reactive Rendering @ RubyCon 2026, Rimini, Italy
marcoroth
5
910
How to build a perfect <img>
jonoalderson
1
6.2k
Context Engineering - Making Every Token Count
addyosmani
9
1.3k
Transcript
Better together Humans, AI and the next quality benchmark Maaret
Pyhäjärvi October 2026 © 2026 CGI Inc. 1
“Introduce yourself for 30 people” “Tell a joke” © 2026
CGI Inc. 2
“Make people get testing” © 2026 CGI Inc. 3
How Would You Test This? Raster Reveal by James Lyndsay
https://www.workroomproductions.com/raster-reveal/ © 2026 CGI Inc. 4
ABOUT TESTING Search for information that matters Test design Test
execution Test cases Exploratory Manual Automated Managing Doing Avoid co nfidentl y incorr ect judg ement © 2026 CGI Inc. 5
Find (some of) what other’s have missed! How Would You
Test This? Raster Reveal by James Lyndsay https://www.workroomproductions.com/raster-reveal/ Quality / Results Gap Balancing detailed reveal vs. jumping to conclusions Invest RIGHT – too much, too little, the wrong kind No one left alone with responsibilities too big for individuals © 2026 CGI Inc. 6
AI ENABLES TASK EXPANSION People can now do things by
asking for it. © 2026 CGI Inc. 7
ASKING FOR IT Use playwright-cli to operate website https://www.workroomproductions.com/raster-reveal/ and
create a script that systematically reveals the image that gets more accurate by mouse movement. Take regular screenshots and figure out for each if you know what is in the picture. Keep the browser open so that I can watch it unfold. © 2026 CGI Inc. 8
ABOUT TESTING Stopping criteria when automation is more accessible Test
design Test cases 0:00 Test execution 0:46 animal-like shape ManualAis dark emerging on a light Confirmed: it's a quadtree that reveals finer detail in blocks my cursor actually crosses. Let me sweep the remaining bands top-tobottom. background, with Managing orange/gold accents. 3:12 Exploratory 13:26 That's unmistakably a black rubber duck with an orange beak and a gold Batman bat-symbol on its side/wing — a "Batman rubber duck" bath toy. Final identification Automated stands: a black rubber duck with an orange Doing beak and a gold Batman logo on its side. Avoid co nfidentl y incorr ect judg ement © 2026 CGI Inc. 9
2018 RESEARCH A walker tool with 2 simple oracles beats
commodity testers at results. © 2026 CGI Inc. 10
20.11.2023 In this particular case, chatGPT did better than colleagues.
People are so afraid to say what they think and make decisions. © 2026 CGI Inc. 11
Practice at: https://exploratory-testing-academy.github.io/capture-the-bugs/ Scale d assess ment = self -asse
ssment © 2026 CGI Inc. 12
LATEST GROUP Asking AI for it: 7% People in pairs:
35% average 79% maximum © 2026 CGI Inc. 13
ESSENTIAL INSIG HTS ESSENTIAL ACTIO NS RESULTS AUTOMATIO N MISTAKES
THAT NEED EXPLAINING © 2026 CGI Inc. https://github.com/QE-at-CGI-FI/todoapp-solution 14
NEED TO DO BETTER Benchmark result: results gap Exploratory tester
with AI: 73 Exploratory tester: 45 AI with human: 40 Traditional testers (n=57): 13.5 Playwright agents: 12 Future talent ’25 (n=9): 11.5 AI without human: 4 100% 62% 55 % 18% 16% 16% 5% Invis ible ink: proble ms are there ev en if no one re ported them. © 2026 CGI Inc. 15
TEACHING DISCOVERY Finding empirical information about software is a teachable
skill. © 2026 CGI Inc. 16
https://qe-at-cgi-fi.github.io/atm/ © 2026 CGI Inc. 17
NEED TO DO BETTER Bringing in Properties 3/4 reasons for
failure previously unknown © 2026 CGI Inc. 18
Most valuable form of coverage is results: empirical, actionable, and
current information. © 2026 CGI Inc. 19
HIERARCHY OF IMPACT TO QUALITY Three layers of subsumed power
User experiences Tester searches the experience space · reports · decides Developer fixes · learns © 2026 CGI Inc. 20
INVISIBLE INK REVEAL We are *users* to this conference RESULTS
ABSENCE OF RESULTS © 2026 CGI Inc. 21
PROPERTIES AS ONE WAY TO TEST THIS PNSQC Conference Website,
three slice testing Effort: 2 min Duration: 30 min © 2026 CGI Inc. 22
TIME AN D EF FORT What do you do while
AI runs a job for you? © 2026 CGI Inc. 23
SPLIT TING WORK WHILE MAINTAINING AGENCY The ways we split
testing don’t make sense. MANAGING | DO ING MANUAL | AUTOMATED TEST CASES | EXPLORING Te s t i n g i s l e a r n i n g . L e a r n i n g i s l a y e r e d . © 2026 CGI Inc. 24
CO LLABO RATIVE ACTIVIT Y Extend from reporting issues into
debugging, repairs, and shared team practices, including exploration at unit level. © 2026 CGI Inc. 25
INTENTIONAL ABOUT HUMANS Human-in-the-loop, not Human-as-the-loop © 2026 CGI Inc.
26
Solo – Pair - Ensemble © 2026 CGI Inc. Getting
best out of everyone into the work we are doing 27
TESTING FOLDED INTO ENGINEERING Metaphorically speaking – testing not testers
15 $/kg Commodity testing © 2026 CGI Inc. 450 $/kg Contemporary exploratory testing 28
CONTEMPORARY EXPLORATORY TESTING IN AGE OF AI Things you should
ask for 1 Identify properties with skill, create programmatic tests and report visually. 2 What insights this week’s conversations add to what we try to remember from past agreements? 3 Could we talk over a cup of coffee? IDENTIF YING G APS Fully automated, deeply human © 2026 CGI Inc. 29
AGE OF AI Exploring intent, value, context, alignment and experience
© 2026 CGI Inc. 30
BREAKING ILLUSIONS WITH TESTING Skills that move you away from
’tester’ and ‘developer’ © 2026 CGI Inc. 31
Gym membership Access to tools isn’t the tools doing the
testing for you. © 2026 CGI Inc. 32
AGE OF AI Actionable feedback that challenges well-maintained illusions has
never been more important. © 2026 CGI Inc. 33
Insights you can act on Founded in 1976, CGI is
among the largest IT and business consulting services firms in the world. We are insights-driven and outcomes-focused to help accelerate returns on your investments. Across hundreds of locations worldwide, we provide comprehensive, scalable and sustainable IT and business consulting services that are informed globally and delivered locally. cgi.com © 2026 CGI Inc. 34