Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Delivering Large Post-Trade Initiatives: Quality Assurance and Key Challenges

Exactpro
September 14, 2014

Delivering Large Post-Trade Initiatives: Quality Assurance and Key Challenges

Difficulties in coping with functional complexity
experienced by the project team:
• Aligning tests correctly in batches that correspond to events in
the daily life cycle, including multiple days
• Accelerating the daily life cycle to speed up test execution and to cover
Settlement Failure processes in testing within a reasonable timeframe
• Accurately simulating and controlling test inputs from upstream systems
and outputs to downstream systems
• Management of complex Reference Data setup
• Involvement of complex components such as Risk Management
and Collateral Management systems
• Test cycle turnaround time, client certification and test automation

Keynote Speakers:
Alyona Lamash FRM Head of Risk Management Practice, Exactpro Systems
Mark Ryland Financial Markets Technology Consultant, Independent Consulting, United Kingdom

Exactpro

September 14, 2014
Tweet

More Decks by Exactpro

Other Decks in Technology

Transcript

  1. How Highly Effective Quality Assurance Techniques Can Make A Positive

    Difference In Delivering Large Post Trade Systems Initiatives Post Trade Forum, Vienna 11 September 2014
  2. Exactpro Company & Experience Exactpro is: 2 • A specialist

    firm focused on functional and non functional testing of wholesale financial products systems • An independent company incorporated in 2009 now employing over 240 specialists Our locations in Russia: • A US company with four QA & development centres in Russia and sales support in the UK • Mostly engaged in trading platforms for exchanges and brokers, but also significant post trade experience
  3. Introducing Your Speakers 3 Mark Ryland Strategic Adviser to Exactpro

    • Financial Consultancy and QA of complex Risk Management solutions, including calculation algorithms • BA for Collateral & Risk Management for investment banking; • QA Delivery Manager for large Post Trade projects Alyona Lamash Head of Exactpro Risk Management Practice • IT Manager/CTO • Banking & Market Infrastructure • Trading and Post Trade • Insight and experience of QA
  4. Talk Content PART A Challenges that Affect Large Projects -

    in General How Great QA Techniques Can Help – Generically PART B Challenges that Affect Post Trade Initiatives – Specifically QA Techniques that Help Post Trade Initiatives - Specifically 4
  5. Testing Making a Positive Contribution to the Whole Responding to

    Issues/ Unknowns Detailed Functional Understanding Achieving & then Maintaining Test Automation Speed of Test Execution Clarity & Accuracy of Test Results and Defects Reporting 5 Testing a Large System Poses Challenges PART A: Full Test Coverage
  6. How Can Effective QA Be Part of The Solution, Not

    Part of The Problem • GENERIC – Highly capable people with the right behaviours – Efficient and effective test process – Right use of technology – Independent & objective measurement of progress – RESULTS IN GREAT TEAM DYNAMICS ACROSS THE WHOLE PROJECT TEAM 6
  7. Make a Positive Contribution to the Overall Team 7 Conception

    Initiation Development Test Implement Project Sponsor Project Manager Business Analyst Architect/Designer Development Team QA Team Production Team
  8. 1. High Quality Staff The Elements of Highly Effective QA

    - 1 10 3. Energy, Enthusiasm & Proactivity 2. Deep Domain Knowledge 4. Creativity & Flexibility in Problem Solving Highly capable people with the right behaviours
  9. 8. Test Execution & Reporting 1. High Quality Staff 3.

    Energy, Enthusiasm & Proactivity 5. Planning & Delivering Ability 2. Deep Domain Knowledge 4. Creativity & Flexibility in Problem Solving The Elements of Highly Effective QA - 2 11 7. Defect Reporting & Evidence Highly capable people with the right behaviours Efficient and effective test process 6. Test Case Quality
  10. 6. Test Case Quality 8. Test Execution & Reporting 10.

    Range of Excellent Automation Tools 9. Detailed Management of Test Environment The Elements of Highly Effective QA - 3 12 7. Defect Reporting & Evidence Right use of technology 1. High Quality Staff 3. Energy, Enthusiasm & Proactivity 2. Deep Domain Knowledge 4. Creativity & Flexibility in Problem Solving Highly capable people with the right behaviours 5. Planning & Delivering Ability Efficient and effective test process
  11. Independence & Objectivity 14 1. Understanding of Requirements 1. Understanding

    of Requirements 2. Delivery of Solution 2. Delivery of Solution 3A. Independent Measurement by QA Team 3A. Independent Measurement by QA Team 3B. Independent Measurement by High Quality QA Team 3B. Independent Measurement by High Quality QA Team Developer Designer Business Analyst Business Manager End Users
  12. Team Dynamics When the QA Team is Excellent 15 DEVELOPERS

    DEVELOPERS Try harder to get it right in the first place Get rapid/timely feedback on what’s wrong Work harder to solve defects to keep the project on track Try harder to get it right in the first place Get rapid/timely feedback on what’s wrong Work harder to solve defects to keep the project on track ANALYSTS AND DESIGNERS ANALYSTS AND DESIGNERS Have clear problem statements to work from – with detailed supporting evidence Have clear problem statements to work from – with detailed supporting evidence SPONSORS & PROJECT MANAGEMENT SPONSORS & PROJECT MANAGEMENT Have (justified) confidence in the system Fewer problems are going to be found when Live Project will cost less Have (justified) confidence in the system Fewer problems are going to be found when Live Project will cost less
  13. 18 Collateral Upload Risk Run Margin Call Risk Run Trade

    Input Trade Input Settlement Run Trade Date Settlement Date Simulate Delivery Failure Trade Input Trade Input Day 1 Examples of Test Scenarios (Before We Get to Specific Problems) Penalties BUY IN Cash Settlement Risk Run Trade Input Day 2 Risk Run Corporate Action Trade Input Day 3 Risk Run Settlement Run Trade Input Trade Input Collateral Deficit Delivery Default Corporate Action Delivery Default Delivery Default
  14. 6 Resulting Challenges From a QA Perspective 1) Testing of

    complex multistep scenarios 2) Limited test availability of downstream and upstream systems 3) Most of endpoints are accessible via API or File exchange (no GUI) 4) Reference Data setup or migration 5) Complexity of Risk Calculation Algorithms 6) Regression Cycle for a substantial number of test scenarios 19
  15. Solving: 3) Most of endpoints are accessible via API or

    File exchange (no GUI) 25 REQUEST RESPONSE
  16. Solving: 4) Reference Data setup or migration 26 Reference Data

    required for Testing Data Management Tools Production Data Migration Tasks
  17. Solving: 6) Regression Cycle for a substantial number of test

    scenarios 28 Test Library (100K Test cases) Test Tools Automated Test Scripts Controlled Test Environment Automated Regression Run (“The Big Button”)