20 April 2019, Tbilisi, Georgia Large Scale Infrastructure and Fake Testing Eugene Litvinenko, QA Team Lead, Exactpro

20 April 2019 Tbilisi, Georgia 2 Introduction ● Eugene Litvinenko ● Started career at Exactpro as Junior QA Engineer 3 years ago; ● QA Team Lead since 2017; ● Involved in business analysis and project management since 2018.

20 April 2019 Tbilisi, Georgia Large Scale Infrastructures 3

20 April 2019 Tbilisi, Georgia We have a global software Quality Assurance client network 4

20 April 2019 Tbilisi, Georgia Software Testing That You Dislike FAKE TESTING 5

20 April 2019 Tbilisi, Georgia What is Fake Testing? It can happen anywhere around you, so how can we define it? 6

20 April 2019 Tbilisi, Georgia Software Testing Definition Providing objective, independent information about software quality Process, tools and methods 7

20 April 2019 Tbilisi, Georgia No Testing vs. Fake Testing Right Left 8

20 April 2019 Tbilisi, Georgia Software Testing is Relentless Learning 9

20 April 2019 Tbilisi, Georgia What if you are not interested in getting real feedback? 10

20 April 2019 Tbilisi, Georgia Have you ever seen a film where a software tester saves the world? 11

20 April 2019 Tbilisi, Georgia Fake Testing vs. Real Testing 12

20 April 2019 Tbilisi, Georgia Live Fire Test Burton strove for combat "realism" in tests and was focused on casualty prevention, urging Ballistic Research Laboratory (BRL) to conduct tests where the Bradley would stand, fully loaded and engine running, against overmatches- attacks with U.S. and Russian munitions that would clearly destroy the vehicle. BRL, on the other hand, wanted to test individual Bradley components in measured, incremental units using computerized vulnerability models. Burton also favored random angle ballistics testing, disagreeing with BRL’s method of testing against a computerized system and also with their rationale of conserving ammunition and "preventing extensive duplication" by using "preplanned shots to gather maximum information about selected vulnerabilities”. BradleyFightingVehicle-Scenario.pdf 13

20 April 2019 Tbilisi, Georgia Carefully Selected Ammunition ● Romanian rocket launcher with a smaller calibre was used instead of Soviet version ● It is forbidden to investigate the target for an hour after the test is completed ● Can it be that something is wrong with the test procedure? ● Why use a smaller missile when you do your testing? 14

20 April 2019 Tbilisi, Georgia Carefully Selected Ammunition Happens in every test injector products by the systems vendor 15

20 April 2019 Tbilisi, Georgia Preparing All Test Scenarios in Advance 16

20 April 2019 Tbilisi, Georgia What is Testability? 17

20 April 2019 Tbilisi, Georgia Real Testability 18

20 April 2019 Tbilisi, Georgia Testing in Large Infrastructures 19

20 April 2019 Tbilisi, Georgia Fake Testability in a Complex System 20

20 April 2019 Tbilisi, Georgia Testing Conditions 21

20 April 2019 Tbilisi, Georgia What is the fuel of the modern economy? 22

20 April 2019 Tbilisi, Georgia Fake Functional Testing ● New release – New Test Plan ● Fake Traceability 23

20 April 2019 Tbilisi, Georgia Fake Regression Testing ● Parallel Run – Replay Historical Data ● Garbage In – Garbage Out ● No idea about the actual test coverage ● Omission of rare events 24

20 April 2019 Tbilisi, Georgia Test Harness and The Law of Requisite Variety 25

20 April 2019 Tbilisi, Georgia Test Harness Team 26

20 April 2019 Tbilisi, Georgia Test Coverage – Exitus Acta Probat 27

20 April 2019 Tbilisi, Georgia People Acting Irrationally ● Xhosa cattle-killing movement ● Nongqawuse met three spirits ● 1854 – 1858 ● At least 40,000 people starved to death 28

20 April 2019 Tbilisi, Georgia Fake Testing will Turn into Sand 29

20 April 2019 Tbilisi, Georgia Build Software to Test Software 30

20 April 2019 Tbilisi, Georgia Thank You September 17 Leadenhall Building, London, 2019 Join us in discussing the newest fintech trends and solutions to the challenges in mission-critical trading and post trade systems! 31