10x Speed With QA Agent Platform - How we scaled adoption from individual effort to organizational capability

As the use of AI agents continues to expand, individual proficiency alone is not enough to improve productivity across an entire organization. This is especially true in QA, where teams need to increase speed while also ensuring quality, safety, and reproducibility.

In this session, we will share practical insights from QA Agent Platform（QAAP）, an organization-wide platform that supports QA work with AI agents. Through this initiative, we explore how AI adoption can move beyond individual expertise and become part of an organization’s quality infrastructure.

The key is to build guardrails into the platform so that teams can use AI with confidence, while also designing operational practices that continuously share and improve how people work with AI. We will discuss the mechanisms used to balance safety and speed, the ways individual know-how was turned into organizational knowledge, and the process through which leaders and teams helped the platform take root through hands-on practice.

What does it take to turn “10x speed” from individual task efficiency into a sustainable organizational capability? This session offers reusable design and operational insights for anyone looking to introduce AI agents into their work, move beyond PoC, or scale individual AI practices across teams and organizations.

Transcript

1. 1 2026.06.29 LY Corporation Product QA Unit Yohei Ueda / Makoto Fukunaga 10x Speed With QA Agent Platform How we scaled adoption from individual effort to organizational capability
2. Agenda 1. About Us 2. What is QA Agent Platform ? 3. Two Barriers to Organizational AI Adoption 4. Overcoming Barriers - Systems 5. Overcoming Barriers - Initiative 6. Summary
3. Ueda Yohei Product Development CBU Product QA Unit Unit Lead
4. Who We Are
5. What is QA?  User Experience  Product Quality  Development Process Not just a pre-release bug gate ̶ We cover quality end-to-end.
6. QA Agent Platform Architecture
7. QA Agent Platform Architecture
8. QA Agent Platform Architecture
9. QA Agent Platform Architecture
10. QA Deliverables Count 1.54 x QA Task Time Reduction Excl. Test Execution 49.4% AI Substitution Rate 68.1% Results from QAAP
11. Fukunaga Makoto Product Development CBU Product QA Unit Product QA 1 Division Division Lead
12. Barriers & Responses
13. Accelerator “ Use it more! ” Brake “ Stay safe! ” Barrier 1: Accelerator vs. Brake
14. Barrier 2: From a Few to Everyone • Early adopters move fast and make it their own. • But their know-how often remains tacit. • And doing the same is rarely straightforward for others.
15. Tackle it with both  “Systems”  “Initiatives”
16. Tackle it with both  “Systems” Solving challenges through QAAP as a foundation.
17. Accelerator “ Use it more! ” Barrier 1: Accelerator Covered ̶ Now, the Brake  Systems ↓ QA-Preset Skills • Universal & org-wide ̶ today's main theme • Accelerator = QA-specific→ deep-dive another time
18. Solution to Barrier 1: Risk Control, Org-Wide New risks emerge every day. • Sensitive data exposure. • Prompt injection. • Supply-chain attacks. Every time we move forward, the brakes slam on.
19. Solution to Barrier 1: Risk Control, Org-Wide • And can we even tell who's doing it right? Not really. • Managing risk one person at a time doesn't scale.
20. Individual Vigilance to Shared Platform QAAP = Built-in guardrails • Fork per team • Centralized on Upstream • Sync to inherit 
21. Layered Defense If something slips through one layer, the next one catches it.
22. Example : Never store secrets in plaintext Secrets: API keys, tokens & more • Plaintext .env: high leak risk if device or repo is compromised • QAAP: keep them in 1Password / Keychain, pull only when needed
23. Enforced by the system, not by discipline Secrets blocked in depth at three layers Runtime Reading secrets from .env / env vars blocked at execution
24. Enforced by the system, not by discipline Secrets blocked in depth at three layers Prompt A secret in a prompt is stopped before it reaches the AI
25. Enforced by the system, not by discipline Secrets blocked in depth at three layers Pre-PR A static check catches secret before a PR is created
26. Guardrails that teach, not just block QAAP catches the risk and explains it in the same moment Guided remediation • On a catch: why it was blocked, how to fix it, link to the guide Dual outcome • Risk auto-prevented while understanding grows
27. Guardrails that teach, not just block QAAP catches the risk and explains it in the same moment Guided remediation • On a catch: why it was blocked, how to fix it, link to the guide Dual outcome • Risk auto-prevented while understanding grows "That alert is what made me understand why the risk matters ̶ and how to handle it."
28. More guardrails: supply chain + agent ops Supply chain (npm) • Internal registry only, no auto-run • install scripts ̶ on by default Agent ops • Destructive deletes blocked, routed to trash ̶ recoverable, not an instant erase • Even the worst case "the AI wiped everything, unrecoverable" designed to prevent
29. What you saw is only part of it ̶ at every step, the guardrails are built into the system. So why all these guardrails? This, more than the how, is what I most want to share today.
30. What we're really guarding against  Common Weak Spots 1. Executing tasks exactly as intended 2. Remembering important constraints 3. Understanding what's written accurately 4. Saying “I donʼt know” when uncertain 5. Looking up missing information proactively
31. What we're really guarding against  Common Weak Spots 1. Executing tasks exactly as intended 2. Remembering important constraints 3. Understanding what's written accurately 4. Saying “I donʼt know” when uncertain 5. Looking up missing information proactively
32. What we're really guarding against  Common Weak Spots 1. Executing tasks exactly as intended 2. Remembering important constraints 3. Understanding what's written accurately 4. Saying “I donʼt know” when uncertain 5. Looking up missing information proactively
33. What we're really guarding against  Common Weak Spots 1. Executing tasks exactly as intended 2. Remembering important constraints 3. Understanding what's written accurately 4. Saying “I donʼt know” when uncertain 5. Looking up missing information proactively
34. What we're really guarding against  Common Weak Spots 1. Executing tasks exactly as intended 2. Remembering important constraints 3. Understanding what's written accurately 4. Saying “I donʼt know” when uncertain 5. Looking up missing information proactively
35. What we're really guarding against  Common Weak Spots 1. Executing tasks exactly as intended 2. Remembering important constraints 3. Understanding what's written accurately 4. Saying “I donʼt know” when uncertain 5. Looking up missing information proactively
36. What we're really guarding against  Common Weak Spots 1. Executing tasks exactly as intended 2. Remembering important constraints 3. Understanding what's written accurately 4. Saying “I donʼt know” when uncertain 5. Looking up missing information proactively Not AI. “Humans” struggle with these.
37. What we're really guarding against AI is constantly changing. Humans, however, remain fundamentally the same. Guardrails go in before a slip turns fatal. Not by vigilance, but by the system.
38. Barrier 1：Summary • QAAP comes with built-in automatic brakes. • helps reduce risk, so you can hit the gas with confidence. But a system alone won't get everyone moving. Barrier 2 →
39. Barrier 2: From a Few to Everyone • Early adopters move fast and make it their own. • But their know-how often remains tacit. • And doing the same is rarely straightforward for others.
40. Tackle it with both  “Initiatives”
41. Solution to Barrier 2
42. Understand Current State
43. Understand Current State
44. Understand Current State
45. Understand Current State
46. Lv State Capability 1 Ad-hoc trial Chat AI for summarization & brainstorming 2 Partial QA support Business context + structured prompts for subtask support 3 Multi-context integration integrate multiple sources, AI-driven stable delibarables 4 Autonomous workflow Agent-based automation of end-to-end workflows 5 News standard Redesign & scale QA standard processes Understand Current State AI Skills Maturity Model DeNA Launches 'DeNA AI Readiness Score (DARS),' a Company-Wide Metric for Evaluating AI Skills
47. Lv State Capability 1 Ad-hoc trial Chat AI for summarization & brainstorming 2 Partial QA support Business context + structured prompts for subtask support 3 Multi-context integration integrate multiple sources, AI-driven stable delibarables 4 Autonomous workflow Agent-based automation of end-to-end workflows 5 News standard Redesign & scale QA standard processes Understand Current State AI Skills Maturity Model
48. Lv State Capability 1 Ad-hoc trial Chat AI for summarization & brainstorming 2 Partial QA support Business context + structured prompts for subtask support 3 Multi-context integration integrate multiple sources, AI-driven stable delibarables 4 Autonomous workflow Agent-based automation of end-to-end workflows 5 News standard Redesign & scale QA standard processes Understand Current State AI Skills Maturity Model
49. Understand Current State Nearly half at Lv1,2 Significant gap in adoption Lv1 Lv2 Lv3 Lv4 Lv5
50. Plan Actions
51. Plan Actions
52. Plan Actions
53. Plan Actions
54. The Improvement Loop
55. The Improvement Loop
56. The Improvement Loop
57. The Improvement Loop
58. The Improvement Loop
59. The Improvement Loop QAAP (Systems) specificity and reproducibility
60. So, What Happened? Number of Uses(Median) Usage %
61. So, What Happened? Usage % Number of Uses(Median) Product QA Unit 100% Adoption #1 Usage
62. QA Deliverables Count 1.54 x QA Task Time Reduction Excl. Test Execution 49.4% AI Substitution Rate 68.1% Results from QAAP
63. Key Insight Successful AI adoption is a systems problem, not a tools problem. Key Message AIʼs primary role in software development is that of an amplifier. It magnifies the strengths of high-performing organizations and the dysfunctions of struggling ones. https://dora.dev/research/2025/ DORA 2025 Report
64. Lead by Example Sense of Urgency 後で消す One Last Thing
65. Lead by Example Sense of Urgency Keep Going One Last Thing
66. Lead by Example Sense of Urgency Keep Going One Last Thing
67. Summary
68. Building Quality in Together join us #help_qaap
69. Appendix
70. Our concerns • Weʼre not security experts, so that makes us uneasy • Balancing safety with convenience is always a trade-off We need your help • We welcome feedback every day! • Check out our internal open repository • pull requests and comments are always appreciated! Turning “worries” into a strength we all share
71. QA Activity in SDLC
72. QA Activity in SDLC
73. QA Activity in SDLC
74. Plan Actions
75. The Improvement Loop