Upgrade to Pro — share decks privately, control downloads, hide ads and more …

TMPA-2015: Reference Test Harness for Algorithm...

Exactpro
November 30, 2015

TMPA-2015: Reference Test Harness for Algorithmic Trading Platforms

Reference Test Harness for Algorithmic Trading Platforms
Viktoria Leonchik, Alexey Sukhov, Eugene Ushakov, Iosif Itkin, Anna-Maria Lukina, Exactpro Systems

12 - 14 November 2015
Tools and Methods of Program Analysis in St. Petersburg

Exactpro

November 30, 2015
Tweet

More Decks by Exactpro

Other Decks in Science

Transcript

  1. Quality Assurance & Related Software Development for Innovative Trading Systems

    Tel: +7 495 640 2460, +1 415 830 38 49 www.exactpro.com
  2. 2 • Introduction • Regulators’ requirements • Test Harness for

    Algorithmic trading system • Minirobots • Algorithmic Test Agents • Technical Criteria • Business Criteria • Conclusion Agenda Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
  3. 3 Exactpro Systems, LLC Quality Assurance & Related Software Development

    for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Introduction Flash Crash in 2010 The Knight Capital runaway algorithm disaster in 2012
  4. SEC ESMA Conducting testing: confirmation that core code components operate

    as intended and do not produce unintended consequences Clearly delineated development and testing methodologies Quality Assurance process should be separated from any development work Testing methodologies should include performance simulations / back-testing and non-live testing within a trading venue testing environment Periodically evaluating test controls Ensure that tests are commensurate with the risks that this strategy may pose to itself and to the fair and orderly functioning of the markets operated by the trading venue Data integrity, accuracy and workflow validation Periodically evaluating test controls Recording of all testing protocols and results Conducting all testing in a development environment that is segregated from production Investment firms should ensure that the production and testing environments are kept segregated at all times 4 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Regulators’ requirements
  5. 5 Exactpro Systems, LLC Quality Assurance & Related Software Development

    for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system
  6. 6 Exactpro Systems, LLC Quality Assurance & Related Software Development

    for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system
  7. 7 Exactpro Systems, LLC Quality Assurance & Related Software Development

    for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system
  8. 8 Exactpro Systems, LLC Quality Assurance & Related Software Development

    for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system
  9. 9 Exactpro Systems, LLC Quality Assurance & Related Software Development

    for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system
  10. 10 Exactpro Systems, LLC Quality Assurance & Related Software Development

    for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system
  11. 11 Exactpro Systems, LLC Quality Assurance & Related Software Development

    for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system
  12. 12 Exactpro Systems, LLC Quality Assurance & Related Software Development

    for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Test Harness for Algorithmic trading system
  13. 13 Exactpro Systems, LLC Quality Assurance & Related Software Development

    for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Minirobots Item Description Testing Type Active Multi-Participants (applicable for testing at the confluence of functional and non-functional testing) Target SUT Trading Platforms and Market Data Delivery Systems SUT Interface Back-end (typically connected to message gateways / APIs, and DBs); GUI Testing Capabilities supported via plug-ins to other tools (e.g., Selenium) SUT Interaction Method Message injection and capture to emulate multiple participants’ activity in electronic markets (essential when there is a need to reproduce complex scenarios that can be created by trading algorithms) Protocols Extant plug-ins for Industry-standard (FIX and dialects, etc.) and proprietary protocols. New plug-ins for additional protocols developed by request Test Scripts Multi-threaded Java code specifying different liquidity profiles Test Management, Execution and Reporting Integrated (Web front-end), allows for multiple simultaneous heterogeneous connections, concurrent emulation of multiple participants, detailed test reports. Optional Big Button framework supported Platform requirements Low footprint cross-platform application, MySQL or other RDBMS
  14. 1. Arbitraging and Market Making Minirobots 2. Minirobots emulating ‘Slicing’

    algorithms 3. Minirobots emulating ‘Synthetic’ algorithms 4. Exchange simulated orders 5. Aggressive buyer/seller (Market panic scenarios) 6. ‘Bandit’-algorithms • ‘Layering’ • ‘Quote-stuffing’ • ‘Painting the tape’ • ‘Abusive liquidity detection’ 14 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Algorithmic Test Agents
  15. 1. Functional criteria The smaller number of existing errors an

    algorithm contains, the lower the expectations of potential losses and fines are. 2. Non-Functional criteria • Performance • Scalability • Reliability • Efficiency • Recoverability 15 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Technical Criteria Shsha
  16. 1. Price • execution • price improvement • spread capture

    2. Cost • explicit • market impact • adverse selection 3. Probability of execution 4. Implementation Shortfall (IS). 16 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Business Criteria
  17. Thank you! 17 Exactpro Systems, LLC Quality Assurance & Related

    Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Q&A