1 BUILD SOFTWARE TO TEST SOFTWARE Build Software to Test Software exactpro.com

2 BUILD SOFTWARE TO TEST SOFTWARE Introduction ● Started IT career 20+ years ago as a software developer ● Moved into performance and load testing in 2006 ● Co-founded Exactpro on 9th September 2009 ● Head of LOST (Laboratory of Software Testing) at Tomsk Polytechnic University ● Co-chair of FIX Trading Community MOST Working Group (Monitoring, Onboarding and Software Testing) ● TMPA Tools & Methods of Program Analysis Conference

3 BUILD SOFTWARE TO TEST SOFTWARE 12th anniversary of Exactpro

4 BUILD SOFTWARE TO TEST SOFTWARE 4 BUILD SOFTWARE TO TEST SOFTWARE About Exactpro A specialist firm focused on functional and non-functional testing services for financial market infrastructures. Exactpro projects include large-scale initiatives worldwide. Headquartered in the UK with operations in the US, Georgia, Russia, Lithuania and Sri-Lanka. Most of our clients are regulated market infrastructures. Exactpro provides independent software testing services for mission critical technology that underpins global financial markets. The firm is experienced with trading, clearing & settlement platforms, risk management systems, central data warehouses, accounting and treasury systems, regulatory reporting. Incorporated in 2009 with 10 people, Exactpro is now employing over 650 specialists. Part of the London Stock Exchange Group (LSEG) from May 2015 until January 2018, when the Exactpro management proceeded through the buyout of the company from LSEG

5 BUILD SOFTWARE TO TEST SOFTWARE 5 BUILD SOFTWARE TO TEST SOFTWARE Our clients include 10 of the world’s Top 20 exchange groups

6 BUILD SOFTWARE TO TEST SOFTWARE Build Software to Test Software exactpro.com Skytra selects Exactpro to test its new derivatives trading software Jeremy Norwood, CIO of Skytra: “Our timescales for launch are challenging, and we needed an experienced IT partner who could help us ensure that the delivered software works as per our business, operational and regulatory requirements. Exactpro will conduct a number of independent, unbiased functional and regression test cycles as part of our overall delivery programme. I am confident that Exactpro, with its proven track record of successful delivery of quality assurance projects for the financial services industry, will provide exceptional service for Skytra.” https://exactpro.com/news/exactpro/skytra-selects-exactpro-test-its-new-derivatives-trading-software

7 BUILD SOFTWARE TO TEST SOFTWARE 7 Build Software to Test Software exactpro.com Confidential exactpro.com Exactpro Supports Launch of MEMX into the U.S. Equity Market with Quality Assurance Testing “MEMX’s close collaboration with Exactpro provided additional operational resilience and contributed to the exchange’s seamless launch, rollout and, ultimately, live trading in all NMS symbols,” said MEMX Chief Technology Officer Dominick Paniscotti. “We greatly appreciate Exactpro’s partnership in working to ensure the exchange’s technology was ready to perform at full capacity from day one.” Read our latest Case Study: MEMX-Exactpro Collaboration on Exchange Quality Assurance https://exactpro.com/news/exactpro/exactpro-supports-launch-memx-us-equity-market-quality-assurance-testing

8 BUILD SOFTWARE TO TEST SOFTWARE

9 BUILD SOFTWARE TO TEST SOFTWARE ● Art ● Craft ● Research

10 BUILD SOFTWARE TO TEST SOFTWARE

11 BUILD SOFTWARE TO TEST SOFTWARE Software Testing - Art, Craft, Science, Engineering Continuum

12 BUILD SOFTWARE TO TEST SOFTWARE Software Testing - Art, Craft, Science, Engineering Continuum

13 BUILD SOFTWARE TO TEST SOFTWARE Software Testing is Relentless Learning Software Testing is an ● empirical ● technical ● investigation ● conducted to provide stakeholders ● with information ● about the quality ● of the product or service under test Software Testing is exploring software with the intent of finding bugs

14 BUILD SOFTWARE TO TEST SOFTWARE Software Testing and Automation in Testing https://fs.blog/2013/07/the-difference-between-science-and-engineering/

15 BUILD SOFTWARE TO TEST SOFTWARE Software Testing and Investigative Reporting Continuum

16 BUILD SOFTWARE TO TEST SOFTWARE Peer review process Peer review is the evaluation of work by one or more people with similar competencies as the producers of the work (peers). It functions as a form of self-regulation by qualified members of a profession within the relevant field. Peer review methods are used to maintain quality standards, improve performance, and provide credibility. In academia, scholarly peer review is often used to determine an academic paper’s suitability for publication Wiki:

17 BUILD SOFTWARE TO TEST SOFTWARE ● Achievements ● Challenges ● Dreams

18 BUILD SOFTWARE TO TEST SOFTWARE

19 BUILD SOFTWARE TO TEST SOFTWARE

20 BUILD SOFTWARE TO TEST SOFTWARE Test Oracles

21 BUILD SOFTWARE TO TEST SOFTWARE The Oracle Problem in Software Testing: A Survey

22 BUILD SOFTWARE TO TEST SOFTWARE Subscribe to Exactpro YouTube Channel

23 BUILD SOFTWARE TO TEST SOFTWARE Machine Learning Testing: Survey, Landscapes and Horizons

24 BUILD SOFTWARE TO TEST SOFTWARE Machine Learning Testing: Survey, Landscapes and Horizons https://www.slideshare.net/HeemengFoo/machine-learning-testing-survey-landscapes-and-horizons-the-cliff-notes https://speakerdeck.com/exactpro/tiestirovaniie-mashinnogho-obuchieniia-obzor-pieizazhi-i-ghorizonty https://www.eventbrite.com/o/ai-in-testing-and-testing-ai-meetup-group-29640157833

25 BUILD SOFTWARE TO TEST SOFTWARE https://testingconferences.org/ https://github.com/TestingConferences/testingconferences.github.io/

26 BUILD SOFTWARE TO TEST SOFTWARE 26 BUILD SOFTWARE TO TEST SOFTWARE Top Software Testing Conferences

27 BUILD SOFTWARE TO TEST SOFTWARE 27 BUILD SOFTWARE TO TEST SOFTWARE International Symposium on Software Testing and Analysis

28 BUILD SOFTWARE TO TEST SOFTWARE 28 BUILD SOFTWARE TO TEST SOFTWARE Artificial Intelligence Testing http://ieeetests.com/

29 BUILD SOFTWARE TO TEST SOFTWARE IEEE AV Testing Course http://av-test-challenge.org/

30 BUILD SOFTWARE TO TEST SOFTWARE Spring/Summer Young Researchers' Colloquium on Software Engineering http://syrcose.ispras.ru/

31 BUILD SOFTWARE TO TEST SOFTWARE tmpaconf.org

32 BUILD SOFTWARE TO TEST SOFTWARE Build Software to Test Software exactpro.com

33 BUILD SOFTWARE TO TEST SOFTWARE Build Software to Test Software exactpro.com FOLLOW US ON SOCIAL MEDIA: exactpro.com exactpro.com