Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AI for Testing

AI for Testing

Rostislav Yavorski
Head of Research, Exactpro

“In this lecture, we will provide an overview of using machine learning and AI tools in software testing. Particular applications include event clustering for system log analysis, recommendation systems for bug reporting, anomaly detection in non-functional testing, etc.”

QA Meetup. 25 April 2022

https://exactpro.com/events/external/qa-meetup-25-april?utm_source=speakerdeck&utm_medium=Refferer&utm_campaign=ai-for-testing

---

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro

Exactpro
PRO

April 25, 2022
Tweet

More Decks by Exactpro

Other Decks in Technology

Transcript

  1. 1 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup BUILD SOFTWARE

    TO TEST SOFTWARE exactpro.com AI for Testing Head of Research, Exactpro Rostislav Yavorski 25 APRIL 2022 ONLINE | HATCH WORKS, COLOMBO 1
  2. 2 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Machine learning

    areas 2
  3. 3 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Machine learning

    areas 3
  4. 4 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Unsupervised Learning

    A machine learning technique in which the users do not need to supervise the model. It is a type of algorithm that learns patterns from untagged data: https://towardsdatascience.com/supervised-vs-unsupervised-learning-in-2-minutes-72dad148f242 4
  5. 5 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Unsupervised Learning

    applications • Audience segmentation • Customer service personalization • Recommender Engines • Anomaly detection • Pattern recognition • Inventory management https://towardsdatascience.com/supervised-vs-unsupervised-learning-in-2-minutes-72dad148f242 5
  6. 6 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Supervised Learning

    The algorithm uses labeled datasets to train how to classify data or predict outcomes accurately. It is a function that maps an input to an output based on example input-output pairs. https://www.guru99.com/supervised-vs-unsupervised-learning.html 6
  7. 7 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup https://towardsdatascience.com/supervised-vs-unsupervised-learning-in-2-minutes-72dad148f242 7

  8. 8 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Supervised Learning

    applications Healthcare and medical diagnosis Predicting stock price Weather forecasting Text categorization Spam detection Face recognition https://www.guru99.com/supervised-vs-unsupervised-learning.html 8
  9. 9 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Reinforcement Learning

    Method based on rewarding desired behaviors and punishing undesired ones: https://vitalflux.com/reinforcement-learning-real-world-examples/ 9
  10. 10 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Reinforcement Learning

    applications Self-driving cars (autonomous vehicles) Industry automation and learning-based robots Natural language processing for chatbot dialogue Dynamic treatment regimes in chronic disease https://vitalflux.com/reinforcement-learning-real-world-examples/ 10
  11. 11 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup 11

  12. 12 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Simplest to

    apply 12
  13. 13 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Requires data

    labeling 13
  14. 14 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Most advanced

    14
  15. 15 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Applications to

    Software Testing 15
  16. 16 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup #1. System

    anomaly detection An anomaly is something that differs from a norm: • a deviation • an exception • a rare occurrence or event that doesn’t fit into the pattern • anything that seems suspicious 16 https://jhui.github.io/2017/01/15/Machine-learning-anomaly/
  17. 17 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup 17

  18. 18 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup 18

  19. 19 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Unsupervised learning

    for anomaly detection • A detection model is constructed using historical logs, which describe a variety of events of software systems. The model is used for: ◦ detecting various types of system behavior anomalies ◦ determining statistical load parameters • Another way is to extract semantic information of log events. The anomalies are detected from the contextual information in the log sequences based on the importance of different log events. 19
  20. 20 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup 20 System

    performance testing
  21. 21 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup #2. AI

    for bug reporting Report readability assessment Duplication detection Automatic fault localization Automated bug assignment Determining bug severity Bug fixing time prediction 21 https://www.softwaretestinghelp.com/
  22. 22 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup #3. AI

    for requirements text analysis Detect linguistic issues in requirements documents Evaluating the sufficiency of information and spec completeness Automated test cases generation from requirements Extracting modeling concepts and constructing UML models Transforming natural language format into formal specification 22 https://clockwise.software/blog/software-requirements-specification-document/
  23. 23 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup #4. Tester’s

    virtual assistant What new errors appeared today? Are there any traces of recent bugs we fixed? Especially, related to the issue we saw yesterday? When was the last major change in logs structure and characteristics? What applications logged an abnormal number of errors today? What kind of anomaly was detected in recent logs? 23 https://www.freepik.com/
  24. 24 BUILD SOFTWARE TO TEST SOFTWARE QA Meetup Thank You!

    24