Using intelligent systems and structural analysis to assure orderly operations of the modern trading and exchange platforms Anna Gromova, Olga Moskaleva, Exactpro July 2016 Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 2460, +1 415 830 38 49 www.exactpro.com

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 2 London Stock Exchange Group • A leading diversified international exchange and infrastructure group with assets across the entire exchange value chain • The Group can trace its history back to 1698 • Headquartered in the United Kingdom, with significant global operations, the Group employs approximately 3,500 people • Post trade and risk management services are a significant part of the Group’s business operations • Through FTSE Russell, the Group is a global leader in financial indexing, benchmarking and analytic services with approximately $10 trillion benchmarked to its indexes

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 3 Software Quality Assurance experts for global financial markets enjoying steady growth thanks to expertise and unparalleled service Exactpro is: Clients Worldwide: • A specialist firm operating an open access model with focus on functional and operational testing of securities data distribution, trading systems, risk management, market surveillance and post-trade infrastructures. • A UK company with operations in the US and four QA & software development centres in Russia • Part of London Stock Exchange Group as of May 29, 2015 • Incorporated in 2009 with 10 people, our company has experienced significant growth as satisfied clients require more services; now employing over 370 specialists

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 4 • Fraud detection - prevent market manipulation, money laundering and other illegal activities. • Technical stability and search for possible software defects. Two dimensions of our work

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 5 Detection of abusive behavior via monitoring and analysis of all market events Surveillance systems

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 6 Main goals of surveillance systems: • detection and prevention from market manipulations: - insider trading; - wash sales; - layering the book; - marking the close. Surveillance systems

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 7 Defect management Technical stability

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 8 Areas of research in defect management: • automatic defect fixing • automatic defect detection • metrics and predictions of defect reports • quality of defect reports • triaging defect reports Defect Management

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 9 Modern tools for surveillance can: • Process a high frequency data flow. • Monitor, store and retrieve: - trade data; - order data; - market data. • Provide cross-market analysis. • Allow market replay. Surveillance systems

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 10 Surveillance systems

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 11 ● Fast growing markets ● Developing new tools and features ● Increasingly complex regulatory requirements Accumulating the huge volume of data Surveillance systems need to be more adaptive Surveillance systems

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 12 Methods of artificial intelligence in order to predict price movement: • deep neural networks for financial market prediction; • echo state networks in order to predict stock prices; • self-organized neuro-fuzzy system for stock market dynamics modeling and forecasting. Surveillance systems

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 13 Examples of metrics: • time to fix • which defects get reopened • which defects get rejected • which defects get fixed Defect Management

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 14 Defect clustering • Understand weaknesses of software • Improve testing strategy Defect Management

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 15 Attributes for cluster analysis: • Priority • status • resolution • time to resolve • count of comments • area of testing Defect Management

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 16 Defect Classification Defect Management

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 17 Reasons of defect reopening: • Bug report has insufficient information • Developers misunderstand the root causes of defect • Ambiguous requirements in specifications Defect Management

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 18 Using metric allows: • Define weaknesses in testing • Characterize actual quality of the bug fixing process • Define weaknesses in documentation Defect Management

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 19 Conclusions: 1. Intelligent procedures of the complex analysis need to be implemented in the surveillance systems 2. Clustering as way to understand and evaluate full array of bugs. Linguistic information of bug-reports is important for analysis. Future work: 1. Predict manipulations by methods of machine learning 2. Clustering of bug-reports and prediction of metrics “which defects get reopened”. Conclusions and future work

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 20 Thank you!