Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Technical Testing Introduction

Exactpro
PRO
September 27, 2011

Technical Testing Introduction

EXTENT February 2011
Alexey Zverev
Exactpro Systems LLC

Exactpro
PRO

September 27, 2011
Tweet

More Decks by Exactpro

Other Decks in Technology

Transcript

  1. Technical Testing
    Alexey Zverev
    EXTENT February 2011

    View Slide

  2. Contents
    • Technical Testing
    • Our world of trading systems
    • Approach – Performance Testing
    • Approach – Stability Testing
    • Approach – Operability Testing
    • Approach – Debugging
    • Tasks – Test Tools
    • Tasks – Test Preparation/Load Shape
    • Tasks – Test Execution
    • Tasks – Result analysis

    View Slide

  3. Technical Testing
    • Technical Testing or NFT
    – Validation of Non-functional Requirements
    • Performance
    • Stability
    • Operability
    • These requirements are usually applied to complex distributed server
    applications
    – Supports many interactive users
    – Processing high volume of data coming from and disseminated to several
    sources
    • Example: Web Sites or Web Services
    • Our world – Trading Systems, Exchanges, SOR, DMS, Brokerage, Trading
    Desks

    View Slide

  4. Our world of Trading Systems
    Trading
    System
    Broker/Trader
    Market Data Provider
    Downstream
    Systems
    ALGO
    Bad ALGO
    DMA/OMS
    SOR
    Transaction Storage
    Backoffice
    Post Trade
    Market Data
    Distributor

    View Slide

  5. The Approach to Performance
    Testing
    • Load Test
    – Emulate load
    – Measure KPIs (Latency, Throughput, Capacity)
    – Determine the load, under which the KPIs become worse than required
    • Stress Test
    – Emulate extreme load
    – Monitor system behaviour
    – Ensure that we sustain it
    • Soak Test (Long Running Test)
    – Emulate load for a long time
    – Check whether we have any deterioration

    View Slide

  6. Approach
    • Stability Testing
    – Apply Load
    • kill
    • unplug
    • do something crazy
    – Operability Testing
    • Manage the environment
    – Debugging
    • Help developers identify issues
    • Complex vs. Simple load

    View Slide

  7. Tasks Test Tools
    • Load injector
    • PCAP analyzer
    • These are some of the most essential pieces of any NFT project:
    – Correctly emulate the connections, implement protocol, use API
    – Make sure that KPI measurements are valid and accurate
    – Make sure that a tool does not experience performance issues itself and
    doesn’t influence KPI measurements
    – Developers whose application you are testing will blame your tools! You
    will require a proof of your figures based on valid sources (as the
    developers think)

    View Slide

  8. Tasks Test Preparation Load
    Shape
    • Live production system is the ultimate source of information

    View Slide

  9. Tasks Test Execution
    • Automated system restart and test execution is a good idea
    • Our system is a complex look at what’s going on
    • Be ready to run tests again and again

    View Slide

  10. Tasks Result Analysis
    • Monitoring hardware reconciles your latency and hardware results
    • Be ready to advocate test results
    • Perform in-depth analysis
    • Make sure results are repeatable – unicorns are non-existent
    – Most of your odd results are probably invalid test runs

    View Slide

  11. Questions & Answers
    hank you.

    View Slide