Iosif Itkin CEO Exactpro, LSEG Machine Learning in Software Testing

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 2 Machine Learning in Software Testing Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 2460, +1 415 830 38 49 www.exactpro.com Iosif Itkin, Co-CEO and Co-Founder, Exactpro London Stock Exchange Group

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 3 Advances in Computer Science Cat on a Screen Bug in the Code

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 4 Technology Trends

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 5 Guess Who Relentlessly Tortures the Software Prevents from being Released into the Wild Breaks It and Criticises it Subject to Ruthless Soak and Negative Testing

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 6 Global Trends Job Loss? Death?

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 7 Contents 1) Software development life cycle 2) Key enablers for machine learning in testing 3) An approach to improve the test librarie 4) Challenges with real-time systems

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 8 Software Development Life Cycle Requirements

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 9 Software Development Life Cycle Requirements Design & Code

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 10 Software Development Life Cycle Requirements Software Bugs Design & Code

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 11 Software Development Life Cycle Requirements Design & Code Software Bugs Magic

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 12 Software Testing Factory Requiremen ts & Code

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 13 Big Data in Testing Identify requirements and areas most prone to defects in the next release of software Predict the probability of a defect to be re- opened Highlight the area of the code where the bug is most likely located

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 14 Dynamic Testing & Learning Non- Functional Testing Tools Passive Testing Capability Build-in Testability in the System Machine-Readable Specifications

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 15 Testing Algo Trading Systems

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 16 Machine Learning Defects Database Code Coverage Mutation Testing

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 17 Real-Time Systems Run All the Functional Tests Concurrently

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 18 Risk Controls Google DeepMind published a paper claiming that it is possible to implement "a button" to interrupt undesired behaviour

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 19 Risk Controls Will we have enough time to press the button once?

Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 20 Thank You Innovation Excellence Partnership Integrity