Test Design for AI Systems

Murad Mamedov
AI Researcher, Exactpro

“Machine Learning (ML) has achieved remarkable progress over the past decade and has been widely applied to many industry domains, including safety-critical ones. With the expansion of ML, the risks related to correctness and robustness are also evolving. Businesses and governments are mitigating the risks with regulatory activities. Since software testing is an important aspect of monitoring and control processes, its applications in ML-based systems (MLS) are also evolving.”

AI Testing Talks – Test Design for AI Systems. 13 May 2022

https://exactpro.com/events/external/ai-testing-talks-test-design-ai-systems?utm_source=speakerdeck&utm_medium=Refferer&utm_campaign=test-design

---

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro

Transcript

1. 1 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   BUILD SOFTWARE TO TEST SOFTWARE
   exactpro.com
   Test Design
   for AI Systems
   Murad Mamedov
   AI Researcher, Exactpro
   13 MAY | 3 PM SLST
   

   View Slide

2. 2 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   Table of Contents
   - neural network architecture
   - ML development process overview
   - current QA activities in industry
   - why test-design is important
   - black-box testing
   - white-box testing
   - data-box testing
   - conclusion
   

   View Slide

3. 3 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   Neural Net Architecture
   

   View Slide

4. 4 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   ML Development Process Overview
   

   View Slide

5. 5 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   Current Activities in Industry
   The emerging risk makes governments and businesses respond with quality
   assurance activities. The regulatory activities are also leveraging monitoring
   and control of AI development.
   - USA Data and Trust Alliance
   - EU AI Regulation Draft
   - ISO/IEC TR 29119-11:2020 Guidelines on the testing of AI-based systems
   

   View Slide

6. 6 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   Current Activities in Industry
   

   View Slide

7. 7 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   Why Test Design is Important
   

   View Slide

8. 8 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   Black Box Strategy: Mutational Approach
   It can be applied to:
   - to an algorithm itself
   - train data
   - test data
   Original
   Program
   Mutant
   Program
   Output
   Compare the results of
   both programs
   

   View Slide

9. 9 BUILD SOFTWARE TO TEST SOFTWARE
   AI Testing Talks
   Black Box Strategy: Combinatorial Approach
   The picture represents a decision map application to the
   Boldness and Discontinuity features, in order to
   generate use cases from high-level scenarios
   

   View Slide

10. 10 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Black Box Strategy: Business Logic Approach
    The approaches based on the
    business logic are closer to
    validation-level ones, since they are
    going directly to the question of
    whether a system meets the
    stakeholders’ expectations or not.
    An example of merging approaches:
    Model-based exploration of the
    frontier of behaviours for deep
    learning system testing
    Input
    Database
    

    View Slide

11. 11 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Black Box Strategy: Business Logic Approach
    The approaches based on the
    business logic are closer to
    validation-level ones, since they
    are going directly to the
    question of whether a system
    meets the stakeholders’
    expectations or not.
    An example of merging approaches:
    Model-based exploration of the
    frontier of behaviours for deep
    learning system testing
    Input
    Database
    Literature
    Features
    Libelled
    Inputs
    Open Coding
    Metric Identification
    Candidate
    Metrics
    Design
    Metrics
    Validation and
    Correlation
    Metrics
    Initial Labelling
    Consensus
    Meeting
    Final Labelling
    Feature name
    Score [5pt]
    Candidate
    Metrics
    

    View Slide

12. 12 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    White Box Strategy: Activation Testing
    What can be tested:
    - if a neuron is activated
    - which value it’s activated with
    - how the neurons interact
    - how the layers interact
    

    View Slide

13. 13 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    White Box Strategy: Tools
    

    View Slide

14. 14 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    White Box Strategy: What’s Missing?
    Activation testing focuses mostly on
    neurons/layers behaviour, and pays
    less attention to the predictions
    

    View Slide

15. 15 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Data Box Strategy
    Combinatorial EDA helps to represent the data from a
    use cases perspective and enhances the further ML
    testing activities such as oracle education, test
    generation, etc.
    

    View Slide

16. 16 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    Conclusion
    

    View Slide

17. 17 BUILD SOFTWARE TO TEST SOFTWARE
    AI Testing Talks
    AI Testing Talks
    Thank You!
    

    View Slide