The Comprehensive Index of the Testing World

Justyn Trenner, QA-Vector 500

MOST Working Group Meeting
24 October 2019

About MOST
The Monitoring, On-boarding and Software Testing (MOST) Working Group was recently formed by the FIX Trading Community Members operating or working with global systemically important FMIs and looking to benefit from collaboration on finding innovative ways to approach quality assurance for FIX-based platforms.

Please contact the FIX Program Office via fix@fixtrading.org if you are interested in contributing to MOST.

Transcript

1. QA Vector® 500: The Comprehensive QA Hub QA Vector® Analytics: Proving the value of quality to support investment and demonstrate progress 24 October 2019
2. QA Vector® 500 The comprehensive QA hub • A unique procurement and vendor risk management tool • Complete details on specialist vendors of quality assurance services, consultants and systems integrators • Impartial, comprehensive and timely information about the vendors • Insights on the challenges and best practices as they emerge 24 October 2019
3. How do we demonstrate the value of quality? We all know examples of the hard cost of quality failures. And all firms measure many components – defects escaping to production, tests passed and failed, money spent on testing software and advisors. But how do we show the return on spend on QA? How do we know how we are improving in QA? How do we compare to peers in similar firms or across businesses and applications? How do demonstrate QA compliance to our regulator? How do we compare in-house approaches, open-source software, commercial software, consultant-led approaches and outsourced solutions? We need to measure the whole-of-quality-journey and investment 24 October 2019
4. Simple, comparable metrics We will begin with a Proof of Concept that exploits what we already measure: - Timeliness of application deliveries - Defects in development reaching QA - Defects in production - Cost / effort dedicated to QA specifically - Cost / effort dedicated to remediation of defects - Ratios of above to overall costs/efforts - Qualitative comments on experiences relative to these issues - Some measures will be more meaningful in a Waterfall environment, others for Agile We will sort these by line of business, mode of development, type of institution, criticality of application ….. 24 October 2019
5. The QA Vector® Analytics Proof of Concept We will provide a forum for discussing and refining measures We have already secured the participation of a number of global financial institutions We will be create a unique stream of output to permit reliable measurement of the value added by the quality stream Participation in this prototype project will not be chargeable, and development partners will be asked to participate in determining fair value for the ongoing service, receiving a discount on their first two years’ participation. 24 October 2019
6. For further information Justyn Trenner Director, QA Media Head of QA Vector® Research justyn.trenner@qa-financial.com 24 October 2019