Testing the Intelligence of your AI

Iosif Itkin, CEO and co-founder
Elena Treshcheva, Business Development Manager and Researcher

QA Financial Forum
13 November 2019, New York

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/

Subscribe to Exactpro Vimeo channel https://vimeo.com/exactproSubscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

Transcript

1. Testing the Intelligence of your AI
   Iosif Itkin, CEO and co-founder
   Elena Treshcheva, Researcher
   Iosif Itkin, CEO and co-founder
   Elena Treshcheva, Business Development Manager and Researcher
   

   View Slide

2. Exactpro Overview
   ● A specialist firm focused on functional and non-functional testing of
   exchanges, clearing houses, depositories, trade repositories and other
   financial market infrastructures.
   ● Incorporated in 2009 with 10 people, our company has experienced
   significant growth and is now employing over 550 specialists.
   ● We were part of the London Stock Exchange Group (LSEG) from May 2015
   till January 2018. Exactpro management buyout from LSEG was successfully
   completed in January 2018. We are headquartered in the UK and have
   operations in the US, Georgia and Russia.
   

   View Slide

3. Exactpro Client Network
   

   View Slide

4. AI-based Systems in Finance
   Machine Learning in financial organizations:
   - already passed an initial development phase
   - the usage of live ML applications is about to
   dramatically increase over the next three years
   https://www.bankofengland.co.uk/-/media/boe/files/report/
   2019/machine-learning-in-uk-financial-services.pdf
   

   View Slide

5. AI-based Systems in Finance
   Machine Learning in financial organizations:
   - already passed an initial development phase
   - the usage of live ML applications is about to
   dramatically increase over the next three years
   ● Market Surveillance Systems
   ● Conversational Assistants
   ● Algo Trading Systems
   ● Pricing Calculators
   ● Machine Readable News
   ● Insurance Claims
   https://www.bankofengland.co.uk/-/media/boe/files/report/
   2019/machine-learning-in-uk-financial-services.pdf
   

   View Slide

6. AI-based Systems’ Quality Characteristics:
   - Ability to learn: The capacity of the system to learn from use for the
   system itself, or data and events it is exposed to.
   - Ability to generalize: The ability of the system to apply to different
   and previously unseen scenarios.
   - Trustworthiness: The degree to which the system is trusted by
   stakeholders, for example a health diagnostic
   A4Q AI and Software Testing
   Foundation
   Syllabus https://www.gasq.org/en/exam-modules/a4q-ai-and-software-testing.html
   Testing the
   Intelligence
   of your AI
   

   View Slide

7. Ability to Learn:
   https://www.deeplearning.ai/
   • Training set — Which you run your learning algorithm on.
   • Development set — Which you use to tune parameters, select
   features, and make other decisions regarding the learning algorithm.
   Sometimes also called the hold-out cross validation set.
   • Test set — which you use to evaluate the performance of the algorithm,
   but not to make any decisions regarding what learning algorithm or
   parameters to use.
   

   View Slide

8. Trustworthiness:
   https://innovation.defense.gov/ai/
   During the DIB’s quarterly public meeting on October 31, 2019, the DIB
   members voted to approve the proposed AI Principles.
   

   View Slide

9. Trustworthiness:
   https://www.mas.gov.sg/news/media-releases/2019/mas-partners-financial-industry
   -to-create-framework-for-responsible-use-of-ai
   

   View Slide

10. Ability to Generalize: Scope of End-to-End and Negative Testing
    

    View Slide

11. Congruence bias
    Confirmation
    bias
    Law of triviality
    Zero-risk bias
    Anthropocentric
    thinking
    Illusion of control
    Cognitive Biases Affecting Software Testing of AI-based Systems
    Automation bias
    

    View Slide

12. AI-based Systems: Machine-Readable News
    

    View Slide

13. Confirmation Bias
    

    View Slide

14. Salman, I. (2016). Cognitive biases in software quality and testing. Proceedings of
    the 38th International Conference on Software Engineering Companion - ICSE ’16.
    Pp. 823-826.
    

    View Slide

15. Mohanani, R., Salman, I., Turhan, B., Rodríguez, P., & Ralph, P. (2018).
    Cognitive Biases in Software Engineering: A Systematic Mapping Study.
    IEEE Transactions on Software Engineering
    

    View Slide

16. AI-based Systems: Conversational Assistants (Chatbots)
    Chatbot
    

    View Slide

17. Anthropocentric Bias
    We should not
    humanize computers.
    

    View Slide

18. Anthropocentric bias
    They dislike it a lot!
    

    View Slide

19. Anthropocentric Bias: Testing a Mine-Defusing Robot
    

    View Slide

20. Anthropocentric Bias: Why We Treat Robots Like Humans
    Darling, Kate and Nandy, Palash and Breazeal,
    Cynthia “Empathic Concern and the Effect of
    Stories in Human-Robot Interaction” (2015).
    Proceedings of the IEEE International Workshop on
    Robot and Human Communication (ROMAN),
    2015. 6 p.
    https://www.ted.com/talks/kate_darling_why_we_ha
    ve_an_emotional_connection_to_robots
    

    View Slide

21. Anthropocentric Bias: Testing Chatbots
    Anaphora / Context
    Human: I bought 500 Company X shares two years ago. The stocks’
    cost was 60,000 USD. What’s their today’s cost?
    Chatbot: What currency would you like to have for the rate? X
    Spelling / overall correctness
    Human: What is the setlement date of the tradeId XXX??
    Chatbot: ???
    

    View Slide

22. AI-based Systems: Algo Trading
    

    View Slide

23. Congruence Bias
    Direct
    Testing
    Indirect
    Testing
    Indirect
    Testing
    

    View Slide

24. Indirect Testing Methods
    Information
    extraction and
    Machine learning
    End-to-End
    Automated Test
    Library
    Whatever it
    takes!
    Test execution
    data and log
    analysis
    Passive Testing
    Whatever it
    takes!
    

    View Slide

25. Applications of the Proposed Approach
    https://unsplash.com/search/photos/san-francisco
    The First IEEE International Conference on Artificial
    Intelligence Testing (IEEE AITest 2019), April 4-9 2019, San
    Francisco East Bay, CA, USA
    User-Assisted Log Analysis for Quality
    Control of Distributed Fintech Systems
    Iosif Itkin, Anna Gromova, Anton Sitnikov, Rostislav Yavorskiy,
    Evgenii Tsymbalov, Andrey Novikov and Kirill Rudakov.
    

    View Slide

26. AI-based Systems: Pricing Calculator
    

    View Slide

27. Law of Triviality (the Bike-Shed Effect)
    

    View Slide

28. Automation Bias
    

    View Slide

29. AI-based Systems: Fraud Detection and Market Surveillance
    

    View Slide

30. Build Software to Test Software
    Click to know more about
    Exactpro Test Tools
    

    View Slide

31. AI-based Systems: Insurance Claims
    

    View Slide

32. Zero-Risk Bias
    

    View Slide

33. Non-deterministic Systems: Financial Market Infrastructures
    

    View Slide

34. The Illusion of Control and Happiness
    Sherman, G. D., Lee, J. J., Cuddy, A. J. C., Renshon, J., Oveis, C., Gross, J. J., & Lerner,
    J. S. (2012). Leadership is associated with lower levels of stress. Proceedings of
    the National Academy of Sciences, 109(44), 17903–17907.
    

    View Slide

35. Fenton-O’Creevy, M., Nicholson, N., Soane, E., &
    Willman, P. (2003). “Trading on illusions:
    Unrealistic perceptions of control and trading
    performance”. Journal of Occupational and
    Organizational Psychology, 76(1), 53–68.
    The Illusion of Control and Performance
    

    View Slide

36. Thank you
    

    View Slide