AI-Powered Software Test Design 3. Test Automation Circles 4. Case Study: Dealing with Operational Issues 5. Specialized Skills That Cannot Be Replaced by AI 6. Summary
- Challenges of software testing • Increase in ◦ testing man-hours ◦ the complexity of systems ◦ fast-paced development cycles • Limitations of test automation and expectations for AI “Why Mastering QA Skills Beyond AI is Needed” 9
• How AI can be used in test design ◦ Automated test case generation ◦ Automated test data generation ◦ Support for risk analysis in risk-based testing • Example: 10 1. Training Data AI 3. Test Case 2. Prompt • Requirement definitions • Functional specifications • ISTQB syllabus (test methodologies, etc.) Test Objectives: Test Scope: Test Type: Test Techniques: —---------------- Output Format: Test Code Language:
• Cautions for AI-powered test design ◦ AI hallucinations (generation of incorrect information) ◦ Biases in AI-generated test cases ◦ Black box of AI's reasoning 12
• Why (Core): Clarify the objective of the test ◦ AI's role Propose optimal strategies, risk analysis, and prioritization ◦ QA Engineer skills Evaluate and adjust strategies, clearly define the goal, and reach agreement with stakeholders • Examples of test goal: ◦ No bugs ◦ Fulfill the requirements ◦ Key functions are working 17
• How (Concept): Select test strategies and techniques ◦ AI's role Select appropriate test techniques, automatically generating test cases ◦ QA Engineer skills Review test cases, verify completeness and validity, identify risks and boundary conditions • Examples of test techniques: ◦ Unit Test : White box testing , coverage ◦ E2E : Use Case / scenario, Functional testing 18
• What (Architecture): Prepare tools and environment ◦ AI's role Select tools and frameworks needed for test automation, generate test scripts, and linking with CI/CD systems ◦ QA Engineer skills Maintain and improve test scripts, build and maintain test environment • Examples of tools and environments: ◦ Test automation tools ◦ CI/CD tools ◦ Test environment construction tool 19
• Scope of Automated Testing: Currently limited to End-to-End (E2E) testing that interacts with the UI layer. Unit tests and other types are not included • No Blame Assigned: This is not about blaming anyone 21 UI Service Unit No of tests
WATCHES] Conduct an automated test for regression checks to ensure basic functions are operating normally. • Goal ◦ Early detection of defects during development, market defects ◦ Reduction of rework, development cost, and testing man-hours • Target ◦ Regression testing in development and production environment (regression testing) for Production and Development App • Test Content Functional Test - 284 caes/suite Test Execution Frequency - Run twice a day (12AM / 12PM) 24
in Automated Test Operations • Issue: Test failures lead to lengthy investigation and resolution time ◦ Each case execution time takes 10-30 minutes → Rerun → Crashes midway... And crashes again during correction checks • Cause ◦ Automated screen-by-screen test cases • Countermeasure ◦ Redesign and split cases for automated testing. ◦ Increased from 48 → 284 cases (Final : 500 cases) • Outcome ◦ Significantly reduces investigation time for test failures ◦ Test are now stable 28 Too Long!
in Automated Test Operations • Issue: Tests fail due to minor screen changes ◦ Frequent modification of test scripts, leading to instability (There were always a lot of unstable parts) • Cause ◦ Long XPATH changes during scrolling ◦ Accuracy issues with OpenCV image matching • Countermeasure ◦ Add an ID (recognized by Appium) • Outcome ◦ Test are now stable ◦ Enhanced collaboration with the development team 29
of reviewing test designs generated by AI ◦ Correct AI errors and biases ◦ Identify risks that AI has not considered • Types of reviews: ◦ Test plan review ◦ Test design review ◦ Test case review ◦ Code review (Product & Test) 32
be a powerful tool for software testing • Human expertise is essential to effectively utilize AI 35 That is why Quality assurance through human-AI collaboration is important
tomasagi
nyankotaro
rfdnxbro
kintotechdev
joker1007
ymatsuwitter
htokoyo
miyatakoji
kojira
kawaji_scratch
shuta13
starfish719
nrslib
cromwellryan
smashingmag
aleyda
addyosmani
tamaranovitovic
honnibal
akademiya2063
afnizarnur
cassininazir
soysaucechin
