Exactpro Non-functional testing for trading systems

Trading Show Chicago
Non-functional testing for trading systems
19 May 2016
Iosif Itkin, Exactpro CEO & Co-Founder
London Stock Exchange Group

Transcript

1. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 1 Non-functional testing for trading systems 19 May 2016 Iosif Itkin, Exactpro CEO & Co-Founder London Stock Exchange Group Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 2460, +1 415 830 38 49 www.exactpro.com
2. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 2 London Stock Exchange Group Trading 23.3% Clearing 33.8% Information 36.5% Technology 5.7%
3. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 3 London Stock Exchange Group • A leading diversified international exchange and infrastructure group with assets across the entire exchange value chain • The Group can trace its history back to 1698 • Headquartered in the United Kingdom, with significant global operations, the Group employs approximately 5,500 people • Post trade and risk management services are a significant part of the Group’s business operations • Through FTSE Russell, the Group is a global leader in financial indexing, benchmarking and analytic services with approximately $10 trillion benchmarked to its indexes
4. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 4 Open Access Software Testing Exactpro is: • A specialist firm operating an open access model with focus on functional and operational testing of securities data distribution, trading systems, risk management, market surveillance and post- trade infrastructures. • A UK company with operations in the US and four QA & software development centres in Russia • Part of London Stock Exchange Group as of May 29, 2015 • Incorporated in 2009 with 10 people, our company has experienced significant growth as satisfied clients require more services; now employing over 350 specialists Clients Worldwide:
5. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 5 Trading Systems Types Proprietary Trading & HFT Brokerage Execution Venue
6. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 6 Performance Testing Cost Structure System Under Test Production Connectivity Test Tools Infrastructure: ≈ 10 - 100 servers infrastructure: thousands of servers infrastructure: 1 - 4 servers
7. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 7 The Idea behind Efficient Load Generation
8. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 8 The Idea behind Efficient Load Generation
9. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 9 Latency Measurement
10. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 10 Latency Distribution
11. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 11 Question Everything Do not Trust: • Test Tools • Average Metrics and Percentiles • Load Shape is Right • System Requirements • Anything
12. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 12 Keep Calm Have No Fear: • Annihilate the System Under Test • Repeat the Same Tests Many Times • Load Levels that Exceed Presumed Production • Make no Assumptions About Extrapolating Test Results into More Scalable Environments
13. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 13 Two Load Generator Types 1. 2.
14. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 14 Achieving Diverse and Relevant Load 1. Random load “Pace Maker” 2. Variety of Passive Liquidity 3. “Agressors” try to interact with the market to increase the number of test cases happening in the environment 4. Analyse the situation in retrospective. Make sure that what happened is correct. Tick checkboxes against tests that were actually executed Updated test library
15. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 15 Non-Functional Test Types
16. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 16 Use Passive Testing Tools to Collect Audit Trails
17. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 17 Questions & Answers https://ru.linkedin.com/in/iosifitkin http://extentconf.com