Exchange Simulators for SOR / Algo Testing: Advantages vs. Shortcomings

The Advantages of Exchange Simulators in Algo Trading Systems Testing
Alexey Zverev, Managing Director Exactpro Systems, Head of CFT QA

The Shortcomings of Exchange Simulators
Alyona Bulda, ClubQA Co-Founder, Kostroma

EXTENT Conference
October 29-30, 2011
Test Automation for Trading Systems

Transcript

1. Exchange Simulators for SOR / Algo Testing: Advantages vs. Shortcomings Alexey Zverev, Alyona Bulda Exactpro Systems LLC
2. SOR & Other Trading Algorithms  Introduction  Examples  Quoting and Market Making  Executing substantial orders automatically  Smart Order Routing  Arbitrage
3. SOR & Other Trading Algorithms Abstract Scheme
4. Testing of SOR & Trading Algorithms  SOR or Algo Trading systems:  Trade automatically  May or may not receive external instructions (parent orders)  Connects to one or multiple execution venues (markets, exchanges)  Connects to low latency Market Data Source to receive current prices  Makes a decision based on market data and statuses of the orders sent to the external markets.
5. Testing of SOR & Trading Algorithms  System Testing • Algo developers need an ability to verify the correctness of algorithm implementation within the trading platform • Testing of surrounding infrastructure (module testing)  Example order should be delivered correctly between algo and Market  Algo Performance Testing / Back testing • Analysts need an ability to test the algorithm profitability This is usually done via back testing (applying algo to historical data) or applying algo against some model simulating market behavior or combination of these approaches
6. Testing of Trading Systems Connected to Electronic Markets  All electronic markets provide test environments where clients are able to test their connectivity. These environments are supposed to be identical to production environment.  Alternative to testing against such test environment is using a Test Harness.
7. Testing of Trading Systems Connected to Electronic Markets  Trading system connectivity layer usually has a specific layer isolating other components from the specific Market API implementation  Exchange Emulator can either emulate a specific API or an intermediate component. This way we can reduce the amount of API development work for our Test Harness
8. Exchange Simulators: Their Role & Species  Exchange Simulators is something that simulates the market in order to test trading systems  There are two types Exchange Simulators:  API Emulators – they emulate exchange connectivity, provide scriptable responses and unsolicited events  Market Simulators – they simulate market/ multiple markets responses based on some model  The API emulator is used to emulate external exchange connection in a controlled manner  The Market Simulator is for testing algorithms via exchange simulations or historical market data replay (back testing)
9. SOR & Other Trading Algorithms vs. Exchange Simulators: Abstract Scheme
10. The Advantages of Using Exchange Simulators  Exchange Simulators:  Provide more control over testing  Enhance the ability to emulate various events to increase test coverage  Provide the ability to test the algo incorporated into the real production software (not in Excel or Matlab)  ... under the real load mimic the real production environments
11. The Shortcomings of Using Exchange Simulators, API Emulation  An Exchange is a complex proprietary black box system. Its nearly impossible to emulate its API behavior accurately  This black box constantly changes  Algo Trading Systems usually have to deal with multiple Exchanges  In a High Frequency Trading network, delays and hardware issues matter. This cannot be simulated  The certification by the Exchanges of the Gateways connecting to the Exchanges should happen in Exchange Test Connection, so API emulator development is duplication of effort
12. The Shortcomings of Using Exchange Simulators, API Emulation Real life examples:  The other side of the barricade is supporting client certification of a trading venue. The following issues could be discovered only via the certification process:  Missing Execution Reports recovery through resend request mechanism  XML instrument definition distribution  Exotic multi-leg trades  Runtime instrument creation  Implied liquidity dissemination
13. The Shortcomings of Using Exchange Simulators, Market Simulation  It is impossible to develop a Market Model, only some localized bits and pieces  The Market Model is a valuable intellectual property of buy side players. Information is not available for public research  Requires a lot of real trading / HFT experience to be able to come up with it  An Exchange Simulator must be really flexible in terms of its model  Dark/Hidden Liquidity Markets are impossible to analyze and predict  The Development of such a system requires a tremendous effort in terms of development and financial math resources and costs a lot; and anyway is far worse than the real market in terms of the development of real Algos
14. The Shortcomings of Using Exchange Simulators, Market Simulation Real life example: the Exactpro Exchange Simulator
15. The Shortcomings of Using Exchange Simulators, Market Simulation Exchange Simulator Design:  Order Book exactly replicating all exchange rules  This is already a challenge – real life example  Trading Systems connect to the simulator as if they are connecting to the real exchange (both for OE and MD)  Need to support adequate load generated by Trading Algos  Market Model is implemented in three instances:  The system is fed with historical orders recreating historical order books  A bunch of competing strategies representing the real market  Counter flow model constantly monitors the liquidity on the book and generates artificial events to simulate market impact of orders generated by the Trading System under test