Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Artificial Intelligence, Bias and Fake Software Testing

Exactpro
October 22, 2018

Artificial Intelligence, Bias and Fake Software Testing

Alexey Zverev, Sibos 2018 Sydney, Australia

Exactpro

October 22, 2018
Tweet

More Decks by Exactpro

Other Decks in Technology

Transcript

  1. Build Software to Test Software exactpro.com Artificial Intelligence, Bias and

    Fake Software Testing Alexey Zverev, Sibos 2018 Sydney, Australia
  2. 2 Build Software to Test Software exactpro.com Build Software to

    Test Software • A specialist firm focused on functional and non-functional testing of exchanges, clearing houses, depositories and other market infrastructures • Part of London Stock Exchange Group (LSEG) from May 2015 till January 2018. Exactpro management buyout from LSEG in January 2018. • We provide software testing services for mission critical technology that underpins global financial markets. Our clients are regulated by FCA, Bank of England and their counterparts from other countries. • Incorporated in 2009 with 10 people, our company has experienced significant growth as satisfied clients require more services; now employing 550 specialists.
  3. 5 Build Software to Test Software exactpro.com • It is

    very important to prepare all test cases upfront and have them signed off by the stakeholders before starting test execution • All tests must be traceable to requirements • We must be very careful and avoid raising invalid defects • The metric of our releases quality is test case pass/fail ratio • We rely on an industry-standard test management system to report out test execution Fake Testing Markers
  4. 6 Build Software to Test Software exactpro.com • Confirmation bias/Expectancy

    bias • Illusion of control • Law of the instrument • Ambiguity Effect/Ignorance of outliers • Anchoring • Anthropocentric Principle Example of biases driving Fake Testing
  5. 8 Build Software to Test Software exactpro.com • We are

    using a UI automation tool to test complex clearing systems • We have automated 85% of our tests • Performance, load and failover testing are considered to be a totally separate activity which is out of scope of our interests • Unit Tests from developers are our automated tests • We are relying on parallel run testing • We are removing “unrealistic scenarios” from our library • We are automating scenarios provided by “experts” Test Automation (Markers of Fake Testing)
  6. 11 Build Software to Test Software exactpro.com • Nostradamus •

    Smart Log Monitor • Smart Load Injector • The Big Button Results Analyzer • Cradle Test Database AI and Machine Learning in Exactpro