The Comprehensive Index of the Testing World

Justyn Trenner, QA-Vector 500

MOST Working Group Meeting
24 October 2019

About MOST
The Monitoring, On-boarding and Software Testing (MOST) Working Group was recently formed by the FIX Trading Community Members operating or working with global systemically important FMIs and looking to benefit from collaboration on finding innovative ways to approach quality assurance for FIX-based platforms.

Please contact the FIX Program Office via fix@fixtrading.org if you are interested in contributing to MOST.

Transcript

1. QA Vector® 500: The Comprehensive QA Hub
   QA Vector® Analytics: Proving the value of quality
   to support investment and demonstrate progress
   24 October 2019
   

   View Slide

2. QA Vector® 500
   The comprehensive QA hub
   • A unique procurement and vendor risk management tool
   • Complete details on specialist vendors of quality assurance
   services, consultants and systems integrators
   • Impartial, comprehensive and timely information about the
   vendors
   • Insights on the challenges and best practices as they emerge
   24 October 2019
   

   View Slide

3. How do we demonstrate the value of quality?
   We all know examples of the hard cost of quality failures.
   And all firms measure many components – defects escaping to production, tests
   passed and failed, money spent on testing software and advisors.
   But how do we show the return on spend on QA?
   How do we know how we are improving in QA?
   How do we compare to peers in similar firms or across businesses and
   applications?
   How do demonstrate QA compliance to our regulator?
   How do we compare in-house approaches, open-source software,
   commercial software, consultant-led approaches and outsourced
   solutions?
   We need to measure the whole-of-quality-journey and investment
   24 October 2019
   

   View Slide

4. Simple, comparable metrics
   We will begin with a Proof of Concept that exploits what we
   already measure:
   - Timeliness of application deliveries
   - Defects in development reaching QA
   - Defects in production
   - Cost / effort dedicated to QA specifically
   - Cost / effort dedicated to remediation of defects
   - Ratios of above to overall costs/efforts
   - Qualitative comments on experiences relative to these issues
   - Some measures will be more meaningful in a Waterfall
   environment, others for Agile
   We will sort these by line of business, mode of development,
   type of institution, criticality of application …..
   24 October 2019
   

   View Slide

5. The QA Vector® Analytics Proof of Concept
   We will provide a forum for discussing and refining measures
   We have already secured the participation of a number of global financial institutions
   We will be create a unique stream of output to permit reliable measurement of the value
   added by the quality stream
   Participation in this prototype project will not be chargeable, and development partners
   will be asked to participate in determining fair value for the ongoing service, receiving a
   discount on their first two years’ participation.
   24 October 2019
   

   View Slide

6. For further information
   Justyn Trenner
   Director, QA Media
   Head of QA Vector® Research
   justyn.trenner@qa-financial.com
   24 October 2019
   

   View Slide