Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Exchange Simulators for SOR / Algo Testing: Advantages vs. Shortcomings

Exactpro
November 13, 2011

Exchange Simulators for SOR / Algo Testing: Advantages vs. Shortcomings

The Advantages of Exchange Simulators in Algo Trading Systems Testing
Alexey Zverev, Managing Director Exactpro Systems, Head of CFT QA

The Shortcomings of Exchange Simulators
Alyona Bulda, ClubQA Co-Founder, Kostroma

EXTENT Conference
October 29-30, 2011
Test Automation for Trading Systems

Exactpro

November 13, 2011
Tweet

More Decks by Exactpro

Other Decks in Technology

Transcript

  1. Exchange Simulators for SOR / Algo
    Testing: Advantages vs. Shortcomings
    Alexey Zverev, Alyona Bulda
    Exactpro Systems LLC

    View full-size slide

  2. SOR & Other Trading Algorithms
     Introduction
     Examples
     Quoting and Market Making
     Executing substantial orders automatically
     Smart Order Routing
     Arbitrage

    View full-size slide

  3. SOR & Other Trading Algorithms
    Abstract Scheme

    View full-size slide

  4. Testing of SOR & Trading
    Algorithms
     SOR or Algo Trading systems:
     Trade automatically
     May or may not receive external instructions (parent orders)
     Connects to one or multiple execution venues (markets,
    exchanges)
     Connects to low latency Market Data Source to receive current
    prices
     Makes a decision based on market data and statuses of the
    orders sent to the external markets.

    View full-size slide

  5. Testing of SOR & Trading
    Algorithms
     System Testing
    • Algo developers need an ability to verify the correctness of algorithm
    implementation within the trading platform
    • Testing of surrounding infrastructure (module testing)
     Example order should be delivered correctly between algo and
    Market
     Algo Performance Testing / Back testing
    • Analysts need an ability to test the algorithm profitability
    This is usually done via back testing (applying algo to historical
    data) or applying algo against some model simulating
    market behavior or combination of these approaches

    View full-size slide

  6. Testing of Trading Systems
    Connected to Electronic Markets
     All electronic markets provide
    test environments where
    clients are able to test their
    connectivity. These
    environments are supposed
    to be identical to production
    environment.
     Alternative to testing against
    such test environment is
    using a Test Harness.

    View full-size slide

  7. Testing of Trading Systems
    Connected to Electronic Markets
     Trading system connectivity layer usually has a specific layer isolating other
    components from the specific Market API implementation
     Exchange Emulator can either emulate a specific API or an intermediate
    component. This way we can reduce the amount of API development work for
    our Test Harness

    View full-size slide

  8. Exchange Simulators: Their Role
    & Species
     Exchange Simulators is something that simulates the market in
    order to test trading systems
     There are two types Exchange Simulators:
     API Emulators – they emulate exchange connectivity, provide
    scriptable responses and unsolicited events
     Market Simulators – they simulate market/ multiple markets
    responses based on some model
     The API emulator is used to emulate external exchange
    connection in a controlled manner
     The Market Simulator is for testing algorithms via exchange
    simulations or historical market data replay (back testing)

    View full-size slide

  9. SOR & Other Trading Algorithms vs.
    Exchange Simulators: Abstract Scheme

    View full-size slide

  10. The Advantages of Using
    Exchange Simulators
     Exchange Simulators:
     Provide more control over testing
     Enhance the ability to emulate various events to increase test
    coverage
     Provide the ability to test the algo incorporated into the real
    production software (not in Excel or Matlab)
     ... under the real load mimic the real production environments

    View full-size slide

  11. The Shortcomings of Using
    Exchange Simulators, API Emulation
     An Exchange is a complex proprietary black box system. Its nearly
    impossible to emulate its API behavior accurately
     This black box constantly changes
     Algo Trading Systems usually have to deal with multiple
    Exchanges
     In a High Frequency Trading network, delays and hardware issues
    matter. This cannot be simulated
     The certification by the Exchanges of the Gateways connecting to
    the Exchanges should happen in Exchange Test Connection, so
    API emulator development is duplication of effort

    View full-size slide

  12. The Shortcomings of Using
    Exchange Simulators, API Emulation
    Real life examples:
     The other side of the barricade is supporting client certification
    of a trading venue. The following issues could be discovered
    only via the certification process:
     Missing Execution Reports recovery through resend
    request mechanism
     XML instrument definition distribution
     Exotic multi-leg trades
     Runtime instrument creation
     Implied liquidity dissemination

    View full-size slide

  13. The Shortcomings of Using Exchange
    Simulators, Market Simulation
     It is impossible to develop a Market Model, only some localized
    bits and pieces
     The Market Model is a valuable intellectual property of buy side
    players. Information is not available for public research
     Requires a lot of real trading / HFT experience to be able to come
    up with it
     An Exchange Simulator must be really flexible in terms of its model
     Dark/Hidden Liquidity Markets are impossible to analyze and
    predict
     The Development of such a system requires a tremendous effort in
    terms of development and financial math resources and costs a lot;
    and anyway is far worse than the real market in terms of the
    development of real Algos

    View full-size slide

  14. The Shortcomings of Using Exchange
    Simulators, Market Simulation
    Real life example: the Exactpro Exchange Simulator

    View full-size slide

  15. The Shortcomings of Using Exchange
    Simulators, Market Simulation
    Exchange Simulator Design:
     Order Book exactly replicating all exchange rules
     This is already a challenge – real life example
     Trading Systems connect to the simulator as if they are connecting to the real
    exchange (both for OE and MD)
     Need to support adequate load generated by Trading Algos
     Market Model is implemented in three instances:
     The system is fed with historical orders recreating historical order books
     A bunch of competing strategies representing the real market
     Counter flow model constantly monitors the liquidity on the book and
    generates artificial events to simulate market impact of orders generated
    by the Trading System under test

    View full-size slide