Chapter 5 – Deploying and Integrating Generative AI in Test Organisations (ISTQBR CT-GenAI v1.1). Slides

Format: Reading Materials (self-study or guided reading)
Estimated Duration: 80 minutes
Target Audience: Software Testers, Test Automation Engineers, Test Analysts, Test Managers, Software Developers and professionals who need a solid understanding of Generative AI (GenAI) in testing – project managers, quality managers, software development managers, business analysts, IT directors and consultants, professionals preparing for ISTQBR CT-GenAI certification

During this chapter, you will:
• Understand how organisations adopt and manage Generative AI in software testing
• Identify the risks, governance needs, and strategic considerations associated with GenAI usage
• Explore the skills and capabilities testers need to work effectively with AI-enabled testing tools
• Examine how testing roles, processes, and responsibilities evolve in AI-assisted test organisations

Join Software Testing Hub via Linkedin: https://www.linkedin.com/groups/16889021/
Join Software Testing Hub via Facebook: https://www.facebook.com/groups/746590458484807

Transcript

1. BUILD SOFTWARE TO TEST SOFTWARE exactpro.com ISTQBⓇ CT-GenAI Chapter 5. Deploying and Integrating Generative AI in Test Organisations Iuliia Emelianova, Dmitrii Degtiarenko TRAINING COURSE ISTQBⓇ CT-GenAI COURSE V1.1
2. 2 Learning Activity Overview…………………………………………………….…………………………………………………………….…………...… 4 Learning Objectives………………………………………………………………………………………………..……………………….…….………………… 5 5.1 Roadmap for the Adoption of Generative AI in Software Testing……….….……………….…... 6 5.1.1 Risks of Shadow AI………………………………………………………………………………………….……………….………………. 9 5.1.2 Key Aspects of a Generative AI Strategy in Software Testing…………………………….…….. 12 5.1.3 Selecting LLMs/SLMs for Software Test Tasks…………..…………………..………..……………………...… 16 5.1.4 Phases when Adopting Generative AI in Software Testing..…….………………………………….. 22 Key Takeaways – 5.1.………………………………………………………………………………………………………..…………………………… 27 Reflection – 5.1………………….…….…….…………………………………………………………………………………….………………………….. 28 5.2 Manage Change when Adopting Generative AI for Software Testing…...….……….……… 29 5.2.1 Essential Skills and Knowledge for Testing with Generative AI…………………..………….…. 31 5.2.2 Building Generative AI Capabilities in Test Teams………………………………………………………….… 36 5.2.3 Evolving Test Processes in AI-Enabled Test Organisations……………………………………….….. 40 Key Takeaways – 5.2.………………………………………………………………………………………………………..…………………………… 44 CONTENTS
3. 3 Reflection – 5.2………………….…….…….…………………………………………………………………………………….………………………….. 45 Key Takeaways and Summary…………………………………………………….……………………………………………………………………… 46 Reflection and Knowledge Check…………………………………………………………………..……………………….………………………… 47 References…………………………………………………….………………………………………………………………………………………………….…………… 48 Feedback and Evaluation…………………………………………………………………………………..……………………….………………………… 49 CONTENTS
4. 4 Chapter 5 – Deploying and Integrating Generative AI in Test Organisations (ISTQBⓇ CT-GenAI v1.1) Format Reading Materials (self-study or guided reading) Estimated Duration 80 minutes Target Audience Software Testers, Test Automation Engineers, Test Analysts, Test Managers, Software Developers and professionals who need a solid understanding of Generative AI (GenAI) in testing – project managers, quality managers, software development managers, business analysts, IT directors and consultants, professionals preparing for ISTQBⓇ CT-GenAI certification Programme Context This learning activity forms a part of the ISTQBⓇ CT-GenAI training programme and aligns with the syllabus version 1.1 Engagement During this chapter, you will: • Understand how organisations adopt and manage Generative AI in software testing • Identify the risks, governance needs, and strategic considerations associated with GenAI usage • Explore the skills and capabilities testers need to work effectively with AI-enabled testing tools • Examine how testing roles, processes, and responsibilities evolve in AI-assisted test organisations LEARNING ACTIVITY OVERVIEW
5. 5 By the end of this learning activity, participants will be able to: • Recall the risks of shadow AI • Explain the key aspects to consider when defining a Generative AI strategy for software testing • Summarise key criteria for selecting LLMs/SLMs for software test tasks in a given context • Recall key phases in the adoption of Generative AI in a test organisation • Explain the essential skills and knowledge areas required for testers to work effectively with generative AI in test processes • Recall strategies for cultivating AI skills within test teams to support the adoption of Generative AI in test activities • Recognise how test processes and responsibilities shift within a test organisation when adopting Generative AI LEARNING OBJECTIVES
6. 6 5.1 ROADMAP FOR THE ADOPTION OF GENERATIVE AI IN SOFTWARE TESTING
7. 7 Sec. 5.1 MAIN POINTS OF THE TEST STRATEGY THAT INCLUDES GENAI Specific test objectives you want to achieve Choice of appropriate LLMs or SLMs Quality and handling of input data used for prompting Need to comply with relevant AI standards and regulations A test strategy that includes Generative AI must thoughtfully address several connected concerns: the specific test objectives you want to achieve, the choice of appropriate LLMs or SLMs, the quality and handling of input data used for prompting, and the need to comply with relevant AI standards and regulations.
8. 8 Sec. 5.1 ROADMAP FOR THE ADOPTION OF GENAI Phased adoption steps Milestone checks for compliance and quality Mechanisms for feedback From this strategy, an organisation can build a realistic roadmap and monitor progress as GenAI functionality is integrated into test processes. The roadmap should include phased adoption steps, milestone checks for compliance and quality, and mechanisms for feedback so the strategy can be adjusted as real-world results and risks become clearer.
9. 9 5.1.1 RISKS OF SHADOW AI (K1)
10. 10 Sec. 5.1.1 Information-security and data-privacy weaknesses Compliance and regulatory issues Vague intellectual property ! ! ! RISKS OF SHADOW AI Shadow AI: The use of GenAI tools or systems within an organisation without formal approval or oversight Shadow AI, or the informal use of personal or unapproved AI tools by staff, introduces several significant risks. • Personal AI tools often lack enterprise-grade security and can expose sensitive information, creating information-security and data-privacy weaknesses. • Compliance and regulatory issues: When unapproved tools are used, organisations may fall afoul of compliance and regulatory requirements, which can lead to legal or audit consequences. • Vague intellectual property: Intellectual-property risks also arise when tooling has unclear licensing or when copyrighted data is processed without appropriate authorisation; this can expose the organisation to disputes or claims.
11. 11 Sec. 5.1.1 Information-security and data-privacy weaknesses Compliance and regulatory issues Vague intellectual property ! ! ! Clear GenAI strategy Controlled deployment steps Governance Approved tooling HOW TO AVOID HAZARDS RISKS OF SHADOW AI A clear GenAI strategy coupled with controlled deployment steps, governance and approved tooling helps test organisations avoid the hazards of shadow AI.
12. 12 5.1.2 KEY ASPECTS OF A GENERATIVE AI STRATEGY IN SOFTWARE TESTING (K2)
13. 13 X X X Sec. 5.1.2 WHAT INFLUENCES THE SELECTION OF LLMS/SLMS • Objectives of GenAI adoption in testing: ◦ raising test productivity ◦ compressing test cycles ◦ improving test quality • Integration with existing test infrastructure • Compliance with scalability requirements ✓ X Successful GenAI adoption in testing begins with well-defined, measurable objectives such as raising test productivity, compressing test cycles, or improving test quality. Selecting LLMs and SLMs should be guided by those objectives and by how well candidate models integrate with existing test infrastructure and meet scalability requirements.
14. 14 Sec. 5.1.2 DATA QUALITY Input data should be: • accurate • relevant • protected by robust security procedures Maintaining data quality is a prerequisite for trustworthy outputs. VERIFIED ✓ Data quality is central: the effectiveness of model-powered testing depends on accurate, relevant input data that is protected by robust security procedures; maintaining that data quality is therefore a prerequisite for trustworthy outputs.
15. 15 Sec. 5.1.2 TRAINING PROGRAMS • Testing teams acquire: ◦ technical skills ◦ ethical awareness • Organisations: ◦ define the metrics to measure GenAI effectiveness ◦ put in place process guidelines covering: ✓ the use of sensitive data ✓ transparency obligations ✓ quality gates that require review of generated testware before it is accepted Comprehensive training programs are essential so testing teams have both the technical skills and the ethical awareness required for responsible GenAI use. Alongside training, organisations should define the metrics to measure GenAI effectiveness (see Section 2.3.1), and put in place process guidelines covering the use of sensitive data, transparency obligations (for example, noting which artefacts were produced by GenAI), and quality gates that require review of generated testware before it is accepted. Together these measures form the operational backbone that turns a pilot GenAI capability into a repeatable, governed element of the test process.
16. 16 5.1.3 SELECTING LLMS/SLMS FOR SOFTWARE TEST TASKS (K2)
17. 17 Sec. 5.1.3 WHAT ARE THE DIFFERENCES BETWEEN LLMS AND SLMS • Functional capabilities: ◦ multimodal input ◦ reasoning capabilities • Technical features: ◦ context window size • Licensing types: ◦ commercial ◦ open source BENCHMARKING FOR TASKS ✓ Natural language processing ✓ Code generation ✓ Image analysis ✓ Software test tasks VS There is a wide range of LLMs/SLMs, each with different functional capabilities (multimodal input, reasoning capabilities), technical features (context window size), and licensing types (e.g., commercial vs. open source). While many benchmarks are available to evaluate LLMs/SLMs for tasks such as natural language processing, code generation, or image analysis, only a few are specifically focused on software test tasks (Wenhan 2024).
18. 18 Sec. 5.1.3 KEY CRITERIA FOR SELECTING LLMS/SLMS FOR TEST TASKS • Evaluating the model’s performance for the targeted test tasks against the organisation’s benchmarks using metrics: ◦ accuracy ◦ precision ◦ recall ◦ relevance and contextual fit ◦ diversity ◦ execution success rate ◦ time efficiency ◦ . . . 1. MODEL PERFORMANCE Therefore, selecting LLMs/SLMs for test tasks requires careful consideration of several key criteria: 1. Model performance. Evaluate the model’s performance for the targeted test tasks against the organisation’s benchmarks using metrics such as those presented in Section 2.3.1.
19. 19 Sec. 5.1.3 KEY CRITERIA FOR SELECTING LLMS/SLMS FOR TEST TASKS • Evaluating whether it is possible and useful to fine-tune the model with domain-specific data to improve performance 2. FINE-TUNING POTENTIAL How to make… 2. Fine-tuning potential. Evaluate whether it is possible and useful to fine-tune the language model (LLM or SLM) with domain-specific data to improve performance for a given use case, increasing accuracy and relevance in specialised contexts.
20. 20 Sec. 5.1.3 KEY CRITERIA FOR SELECTING LLMS/SLMS FOR TEST TASKS • Considering the recurring costs of using the model: ◦ licensing fees ◦ operational expenses • Ensuring fitting within the organisation’s budget for the targeted test tasks 3. RECURRING COST 3. Recurring cost. Consider the recurring costs of using the LLM/SLM, including licensing fees and operational expenses, to ensure that it fits within the organisation’s budget for the targeted test tasks.
21. 21 Sec. 5.1.3 KEY CRITERIA FOR SELECTING LLMS/SLMS FOR TEST TASKS 4. COMMUNITY AND SUPPORT • Choosing models with: ◦ active community support ◦ detailed documentation to aid in implementation and troubleshooting 4. Community and support. Choose models with active community support and detailed documentation to aid in implementation and troubleshooting. By carefully evaluating these criteria, test organisations can make informed decisions when selecting language models for specific test tasks.
22. 22 5.1.4 PHASES WHEN ADOPTING GENERATIVE AI IN SOFTWARE TESTING (K1) Adopting Generative AI in a test organisation is not a single decision or a one-time implementation. It is a gradual transition that typically unfolds in three overlapping phases.
23. 23 Sec. 5.1.4 1. DISCOVERY • The organisation is building basic awareness and confidence • Testers are: ◦ introduced to GenAI concepts ◦ given access to LLMs/SLMs ◦ encouraged to experiment with simple, low-risk use cases GOALS ✓ Learning by doing ✓ Understanding the strengths and limitations of GenAI ✓ Reducing uncertainty through hands-on exploration The first phase is discovery. At this stage, the organisation is building basic awareness and confidence. Testers are introduced to GenAI concepts, given access to LLMs or SLMs, and encouraged to experiment with simple, low-risk use cases. The goal here is not full-scale automation, but learning by doing, understanding the strengths and limitations of GenAI, and reducing uncertainty through hands-on exploration.
24. 24 Sec. 5.1.4 2. INITIATION AND USAGE DEFINITION • Shifting from experimentation to strategy • Identification, evaluation, and prioritisation of practical GenAI use cases in software testing • Conducting assessment of a suitable LLM-powered test infrastructure • Strengthening internal expertise • Ensuring alignment with the organisation's broader testing and quality goals GENAI USE CASES IN TESTING TEST INFRASTRUCTURE Once this initial familiarity is established, the organisation moves into the phase of initiation and usage definition. Here, the focus shifts from experimentation to strategy. Practical GenAI use cases in software testing are identified, evaluated, and prioritised. At the same time, suitable LLM-powered test infrastructure is assessed, internal expertise is strengthened, and alignment with the organisation’s broader testing and quality goals is ensured, in line with the guidance of the ISTQBⓇ CTFL syllabus.
25. 25 Sec. 5.1.4 3. UTILISATION AND ITERATION • GenAI is an integrated part of the test process • Usage of GenAI is: ◦ continuously monitored ◦ measured ◦ refined • Organisations: ◦ track whether GenAI actually delivers sustainable benefits ◦ adjust their approaches based on experience ◦ scale successful practices across teams and projects The final phase is utilisation and iteration. At this point, GenAI is no longer a novelty but an integrated part of the test process. Its usage is continuously monitored, measured, and refined. Organisations track whether GenAI actually delivers sustainable benefits, adjust their approaches based on experience, and scale successful practices across teams and projects.
26. 26 Sec. 5.1.4 DISCOVERY INITIATION AND USAGE DEFINITION UTILISATION AND ITERATION • Phases rarely occur in a strict sequence • Different use cases can mature at different speeds • Human factors PHASES WHEN ADOPTING GENAI IN PRACTICE ✓ Test report analysis ✓ Automated test generation In practice, these phases rarely occur in a strict sequence. Different use cases can mature at different speeds. For example, test report analysis may already be in the utilisation phase, while automated test generation is still in discovery. It is also essential to address human factors early, especially concerns about job displacement, as uncertainty and fear can significantly slow down adoption and reduce team engagement.
27. 27 • A successful Generative AI adoption roadmap requires clear objectives, phased implementation, governance, compliance monitoring, and continuous feedback mechanisms • Shadow AI creates significant security, compliance, and intellectual-property risks when employees use unapproved AI tools without organisational oversight • Effective GenAI strategies depend on high-quality and secure data, suitable LLM and SLM selection, staff training, and defined review processes for AI-generated testware • Selecting language models for software testing should consider model performance, fine-tuning potential, recurring cost, available community support and detailed documentation • Generative AI adoption in testing evolves through overlapping phases of discovery, strategic usage definition, and continuous utilisation and improvement, while also requiring attention to human and organisational change factors KEY TAKEAWAYS – 5.1
28. 28 1. What criteria would be most important when selecting an LLM or SLM for your testing environment? 2. Which phase of Generative AI adoption best reflects your project’s current maturity level, and what challenges might prevent progression to the next phase? REFLECTION – 5.1
29. 29 5.2 MANAGE CHANGE WHEN ADOPTING GENERATIVE AI FOR SOFTWARE TESTING
30. 30 Sec. 5.2 INTRODUCING GENAI INTO A TEST ORGANISATION How people work How roles evolve How quality is managed ! ! ! WHAT IS AFFECTED CHANGE MANAGEMENT ✓ Developing new skills ✓ Reshaping traditional testing roles ✓ Addressing both the technical and organisational sides of the transition Introducing Generative AI into a test organisation is not just a technical upgrade. It is a transformation that affects how people work, how roles evolve, and how quality is managed. For GenAI adoption to succeed, structured change management is essential. This includes developing new skills, reshaping traditional testing roles, and addressing both the technical and organisational sides of the transition. Without this structured approach, even the most powerful AI tools risk being underused, misused, or actively resisted.
31. 31 5.2.1 ESSENTIAL SKILLS AND KNOWLEDGE FOR TESTING WITH GENERATIVE AI (K2) To work effectively with GenAI, testers must acquire a new blend of skills that extends beyond traditional testing expertise.
32. 32 Sec. 5.2.1 SKILL SET FOR TESTING WITH GENAI ✓ The ability to formulate prompts: ◦ clear ◦ precise ◦ goal-oriented 1. PROMPT ENGINEERING ✓ Understanding: ◦ how model context windows work ◦ how the amount and structure of input influence output quality ✓ Ability to review and evaluate AI-generated testware: ◦ test cases ◦ defect reports ◦ synthetic test data At the core of this skill set is prompt engineering or the ability to formulate clear, precise, and goal-oriented prompts. Testers also need to understand how model context windows work and how the amount and structure of input influence output quality. Just as important is the ability to review and evaluate AI-generated testware, whether this involves test cases, defect reports, or synthetic test data.
33. 33 Sec. 5.2.1 SKILL SET FOR TESTING WITH GENAI ✓ Testers need to assess: ◦ what an LLM is capable of ◦ where LLM’s limits are ◦ how LLM’s output should be refined through iterative prompting ◦ how to evaluate AI-generated results DOMAIN KNOWLEDGE TESTING EXPERIENCE AI-SPECIFIC SKILLS 2. COMBINATION OF DOMAIN KNOWLEDGE, TESTING EXPERIENCE AND AI-SPECIFIC SKILLS In practice, testers must combine their domain knowledge and testing experience with AI-specific skills. They need to assess what an LLM is capable of, where its limits are, and how its output should be refined through iterative prompting. Understanding how to evaluate AI-generated results becomes just as important as designing tests manually once was.
34. 34 Sec. 5.2.1 SKILL SET FOR TESTING WITH GENAI ✓ Understanding the data security implications of sending test artifacts to LLMs 3. AWARENESS OF GENAI RISKS AND MITIGATION STRATEGIES ✓ Ability to apply proper data sanitisation by masking or removing sensitive, personal, or confidential information Privacy-preserving prompt engineering practices become part of everyday testing work. Equally critical is awareness of GenAI risks and mitigation strategies. Testers must understand the data security implications of sending test artifacts to LLMs and apply proper data sanitisation by masking or removing sensitive, personal, or confidential information. Privacy-preserving prompt engineering practices become part of everyday testing work.
35. 35 Sec. 5.2.1 SKILL SET FOR TESTING WITH GENAI ✓ Selecting models that are “right-sized” for the task 4. ENVIRONMENTAL AND COST CONSIDERATIONS ✓ X ✓ Optimising usage patterns to avoid unnecessary computation ✓ Balancing the productivity gains of GenAI against cost and energy consumption Environmental and cost considerations also enter the tester’s skill domain. This includes selecting models that are “right-sized” for the task, optimising usage patterns to avoid unnecessary computation, and balancing the productivity gains of GenAI against cost and energy consumption. In this new context, responsible testing means being not only technically competent but also economically and environmentally aware.
36. 36 5.2.2 BUILDING GENERATIVE AI CAPABILITIES IN TEST TEAMS (K1) Building real GenAI capability in a test team cannot be achieved through theory alone.
37. 37 Sec. 5.2.2 HANDS-ON APPROACH IN A TEST TEAM • Practical exposure to different LLMs and SLMs • Guided learning paths • Continuous opportunities to apply GenAI to real testing scenarios • Experimentation • Reflection • Shared experience A hands-on approach is essential. Testers need practical exposure to different LLMs and SLMs, guided learning paths, and continuous opportunities to apply GenAI to real testing scenarios. Capability grows gradually through experimentation, reflection, and shared experience.
38. 38 Sec. 5.2.2 PROGRESS FROM BASIC TO TEST-SPECIFIC PROMPTING Prompt patterns: • reusable prompt templates designed to produce consistent and reliable results • capture organisational knowledge for specific test tasks about: ◦ what works ◦ what does not work Testers typically progress from basic prompt creation to more focused, test-specific prompting techniques. Over time, they begin to use prompt patterns, which are reusable prompt templates designed to produce consistent and reliable results. These patterns capture organisational knowledge about what works and what does not when using GenAI for specific test tasks.
39. 39 Sec. 5.2.2 LEARNING PROCESS • Internal communities of practice • Regular knowledge-sharing sessions: ◦ exchanging successful GenAI use cases ◦ discussing failures and limitations ◦ refining approaches together • Strategic assets emerging from: ◦ shared prompt libraries ◦ documented lessons learned GenAI capability becomes a collective strength of the organisation Internal communities of practice play a central role in sustaining this learning process. Through regular knowledge-sharing sessions, teams exchange successful GenAI use cases, discuss failures and limitations, and refine their approaches together. Over time, shared prompt libraries and documented lessons learned emerge as strategic assets. This collaborative learning culture ensures that GenAI capability is not isolated with a few specialists but becomes a collective strength of the organisation.
40. 40 5.2.3 EVOLVING TEST PROCESSES IN AI-ENABLED TEST ORGANISATIONS (K1) As GenAI becomes embedded in test processes, both tester and test manager roles evolve in fundamental ways.
41. 41 Sec. 5.2.3 TESTER ROLE TEST DESIGNER, EXECUTOR AI-ASSISTED TEST SPECIALIST • Creating test cases • Executing tests • Guiding AI systems through prompts • Reviewing AI-generated results • Refining outputs through iterative prompting • Maintaining test-specific prompt libraries Testers shift from being primarily test designers and executors to becoming AI-assisted test specialists. Their responsibility no longer ends with creating test cases or executing tests. Instead, they guide AI systems through carefully crafted prompts, critically review AI-generated results, refine outputs through iterative prompting, and maintain test-specific prompt libraries. Human judgment becomes more important, not less, as testers must decide which AI-generated results can be trusted and which must be corrected or discarded.
42. 42 Sec. 5.2.3 TEST MANAGER ROLE LEADER OF HUMAN TESTERS LEADER OF HUMAN TESTERS AND COORDINATOR OF AI-ENABLED WORKFLOWS • Development of AI-based test strategies • AI-aware risk management • Monitoring and control of AI-supported test processes • Balancing human expertise with AI capabilities • Defining governance frameworks for the use of GenAI • Ensuring that quality remains under human accountability rather than being delegated blindly to automation Test managers experience an equally significant shift. Their responsibilities expand to include the development of AI-based test strategies, AI-aware risk management, and the monitoring and control of AI-supported test processes. They must balance human expertise with AI capabilities, define governance frameworks for the use of GenAI, and ensure that quality remains under human accountability rather than being delegated blindly to automation.
43. 43 Sec. 5.2.3 TEST MANAGER ROLE ✓ Leading human testers ✓ Coordinating AI-enabled workflows ✓ Ensuring compliance with policies and regulations ✓ Safeguarding the organisation against the strategic, ethical, and operational risks introduced by GenAI ORCHESTRATION ROLE: Aligning people, processes, and intelligent systems into a coherent, controlled testing strategy LEADER OF HUMAN TESTERS AND COORDINATOR OF AI-ENABLED WORKFLOWS In this new landscape, test managers do not only lead human testers. They also coordinate AI-enabled workflows, ensure compliance with policies and regulations, and safeguard the organisation against the strategic, ethical, and operational risks introduced by GenAI. Their role becomes one of orchestration – aligning people, processes, and intelligent systems into a coherent, controlled testing strategy.
44. 44 • Successful adoption of Generative AI in software testing requires structured change management that addresses both technical and organisational transformation • Testers need new AI-related skills, including prompt engineering, evaluation of AI-generated outputs, data sanitisation, and awareness of GenAI risks and limitations • Responsible AI-assisted testing also involves balancing productivity gains with cost, security, privacy, and environmental considerations • Building long-term GenAI capability depends on hands-on learning, reusable prompt patterns, shared knowledge, and collaborative communities of practice • As GenAI becomes integrated into testing, testers shift from being primarily test designers and executors to becoming AI-assisted test specialists, and the test manager’s role becomes one of integrating people, processes, and intelligent systems into a coherent, controlled testing strategy KEY TAKEAWAYS – 5.2
45. 45 1. What strategies could help teams develop consistent and reusable prompt patterns for common testing tasks? 2. How do you think the role of a tester will differ in an AI-enabled organisation compared with a traditional test environment? REFLECTION – 5.2
46. 46 • Organisations adopting Generative AI in software testing need a structured roadmap that includes clear objectives, governance, phased implementation, compliance monitoring, and continuous feedback • Organisations must actively manage the risks of shadow AI by using approved tools, protecting sensitive data, and addressing legal and intellectual-property concerns • Selecting suitable LLMs and SLMs requires evaluating model capabilities, scalability, cost, fine-tuning potential, and support for targeted testing activities • Generative AI adoption typically progresses through discovery, strategic implementation, and continuous utilisation and refinement within the test process • Successful change management requires testers and test managers to develop new AI-related skills, including prompt engineering, AI output evaluation, data sanitisation, and responsible AI usage practices • AI-enabled test organisations depend on collaborative learning, evolving tester and manager roles, and strong human oversight to ensure quality, accountability, and responsible AI usage KEY TAKEAWAYS AND SUMMARY
47. 47 Answer these questions after completing the reading: 1. Why is it important for organisations to develop a structured roadmap when adopting Generative AI in software testing? 2. What risks can arise from the use of shadow AI within a test organisation? 3. Describe the three phases typically involved in adopting Generative AI in software testing. 4. Why is Generative AI adoption considered a gradual transition rather than a one-time implementation? 5. How can organisations build long-term Generative AI capability within their test teams? 6. How can shared prompt libraries and communities of practice improve testing effectiveness? 7. In what ways do the roles of testers and test managers evolve in AI-enabled test organisations? (You should answer using examples from your own projects where possible.) REFLECTION AND KNOWLEDGE CHECK
48. 48 • ISTQB® Certified Tester Specialist Level Testing with Generative AI (CT-GenAI) Syllabus Version 1.1, 2026 • Wang, Wenhan, et al. “TESTEVAL: Benchmarking Large Language Models for Test Case Generation.” arXiv preprint arXiv:2406.04531 (2024) REFERENCES
49. 49 Learner feedback is collected to support continuous improvement of delivery and materials. Understanding is evaluated through: • Chapter quiz covering key concepts from this chapter • Q&A session to clarify questions arising from the activities and quiz FEEDBACK AND EVALUATION
50. 50 Thank You!