Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 2460, +1 415 830 38 49 www.exactpro.com

2 • Introduction • Regulators’ requirements • Test Harness for Algorithmic trading system • Minirobots • Algorithmic Test Agents • Technical Criteria • Business Criteria • Conclusion Agenda Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com

3 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Introduction Flash Crash in 2010 The Knight Capital runaway algorithm disaster in 2012

SEC ESMA Conducting testing: confirmation that core code components operate as intended and do not produce unintended consequences Clearly delineated development and testing methodologies Quality Assurance process should be separated from any development work Testing methodologies should include performance simulations / back-testing and non-live testing within a trading venue testing environment Periodically evaluating test controls Ensure that tests are commensurate with the risks that this strategy may pose to itself and to the fair and orderly functioning of the markets operated by the trading venue Data integrity, accuracy and workflow validation Periodically evaluating test controls Recording of all testing protocols and results Conducting all testing in a development environment that is segregated from production Investment firms should ensure that the production and testing environments are kept segregated at all times 4 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Regulators’ requirements

5 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system

6 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system

7 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system

8 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system

9 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system

10 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system

11 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system

12 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system

13 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Minirobots Item Description Testing Type Active Multi-Participants (applicable for testing at the confluence of functional and non-functional testing) Target SUT Trading Platforms and Market Data Delivery Systems SUT Interface Back-end (typically connected to message gateways / APIs, and DBs); GUI Testing Capabilities supported via plug-ins to other tools (e.g., Selenium) SUT Interaction Method Message injection and capture to emulate multiple participants’ activity in electronic markets (essential when there is a need to reproduce complex scenarios that can be created by trading algorithms) Protocols Extant plug-ins for Industry-standard (FIX and dialects, etc.) and proprietary protocols. New plug-ins for additional protocols developed by request Test Scripts Multi-threaded Java code specifying different liquidity profiles Test Management, Execution and Reporting Integrated (Web front-end), allows for multiple simultaneous heterogeneous connections, concurrent emulation of multiple participants, detailed test reports. Optional Big Button framework supported Platform requirements Low footprint cross-platform application, MySQL or other RDBMS

1. Arbitraging and Market Making Minirobots 2. Minirobots emulating ‘Slicing’ algorithms 3. Minirobots emulating ‘Synthetic’ algorithms 4. Exchange simulated orders 5. Aggressive buyer/seller (Market panic scenarios) 6. ‘Bandit’-algorithms • ‘Layering’ • ‘Quote-stuffing’ • ‘Painting the tape’ • ‘Abusive liquidity detection’ 14 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Algorithmic Test Agents

1. Functional criteria The smaller number of existing errors an algorithm contains, the lower the expectations of potential losses and fines are. 2. Non-Functional criteria • Performance • Scalability • Reliability • Efficiency • Recoverability 15 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Technical Criteria Shsha

1. Price • execution • price improvement • spread capture 2. Cost • explicit • market impact • adverse selection 3. Probability of execution 4. Implementation Shortfall (IS). 16 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Business Criteria

Thank you! 17 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Q&A