AI Development at Exactpro

AI Development at Exactpro

Murad Mamedov, AI Researcher, Exactpro

Exactpro Webinar for Tbilisi State University | A Webinar on Software Testing and AI
16.06.2020

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/
Vkontakte https://vk.com/exactpro_llc

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

5206c19df417b8876825b5561344c1a0?s=128

Exactpro

June 16, 2020
Tweet

Transcript

  1. Build Software to Test Software exactpro.com AI Development at Exactpro

    Murad Mamedov, AI Researcher TSU Webinar, June 2020
  2. 2 Build Software to Test Software exactpro.com Plan - Machine

    How we research AI applications in Software Testing and how we build AI Tool on top of this - Platform Test Harness platform of new generation. Its overview from AI perspective - People How we prepare Software Testers for new set of tasks from Data Science and Machine Learning area
  3. 3 Build Software to Test Software exactpro.com Nostradamus Business Taks:

    To predict future states of the defects
  4. 4 Build Software to Test Software exactpro.com Example of a

    Bug
  5. 5 Build Software to Test Software exactpro.com AI Basics Fetch

    the Data Data Preprocessing Model Training Model Validation
  6. 6 Build Software to Test Software exactpro.com Nostradamus High-Level Details:

    Business Taks: To predict future states of the defects Fetch the Data Data Preprocessing Model Training From bug-tracking systems Empty values transformation, TTR calculation, text normalization, stemming TF-IDF Vectorizer, SMOTE, Select Percentile, SVC
  7. 7 Build Software to Test Software exactpro.com Example of Logs

  8. 8 Build Software to Test Software exactpro.com WhaleShark High-Level Details:

    Business Taks: Simplify log analysis and automate the process of relations recognition Fetch the Data Data Preprocessing Model Training From back-end of the systems under test IDs transformation, text normalization TF-IDF Vectorizer, k-means, SVD, word2vec
  9. 9 Build Software to Test Software exactpro.com Web Binary FIX

    Market Data Database How Sailfish Works
  10. 10 Build Software to Test Software exactpro.com Sailfish Report send

    NewOrderSingle (PASSED) [0.001s] - receive ExecutionReport (PASSED) [0.001s]: + Input Parameters - Verification: Message (PASSED): Field Expected Result Actual Result Status ... ... ... ... OrdType LIMIT LIMIT PASSED Status NEW NEW PASSED ... ... ... ... Passed Step of Test Case:
  11. 11 Build Software to Test Software exactpro.com send NewOrderSingle (PASSED)

    [0.001s] - receive ExecutionReport (FAILED) [0.421s]: + Input Parameters + Verification: Similar message [1]. Failed/Passed/Conditionally Passed/NA: 4/26/0/6 (FAILED) + Verification: Similar message [2]. Failed/Passed/Conditionally Passed/NA: 2/28/0/6 (FAILED) + Verification: Similar message [3]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED) - Verification: Similar message [4]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED): Field Expected Result Actual Result Status ... ... ... ... OrdType LIMIT LIMIT PASSED Status NEW REJECTED FAILED ... ... ... ... Failure Example Failed Step of Test Case:
  12. 12 Build Software to Test Software exactpro.com Decision Tree Example

    Trained Tree:
  13. 13 Build Software to Test Software exactpro.com Narrow Range Example:

    Prediction Example
  14. 14 Build Software to Test Software exactpro.com ML for Sailfish

    Business Taks: To make reports clear via discarding non-relevant messages High-Level Details: Business Taks: Automate the process of source and type recognition of the failures High-Level Details: Fetch the Data Data Preprocessing Model Training Marked up reports from test-library run in Sailfish Dimensionality reduction, Feature Importance, One-Hot Encoding Random Forest, SVN, Calibration Data Preprocessing Model Training Examples transformation, split by message type k-means, DB-scan, Agglomerative
  15. 15 Build Software to Test Software exactpro.com Clustering Task

  16. 16 Build Software to Test Software exactpro.com Organizational Workflow Computer

    Science Domain Expertise Math & Statistics Developers Testers DS/ML Engineers Data Analysis Model Training Validation & Verification Implementation
  17. 17 Build Software to Test Software exactpro.com Organizational Workflow Computer

    Science Domain Expertise Math & Statistics Developers Testers DS/ML Engineers Data Analysis Model Training Validation & Verification Implementation
  18. 18 Build Software to Test Software exactpro.com Organizational Workflow Computer

    Science Domain Expertise Math & Statistics Developers Testers DS/ML Engineers Data Analysis Model Training Validation & Verification Implementation
  19. 19 Build Software to Test Software exactpro.com Publications

  20. 20 Build Software to Test Software exactpro.com TH2 Schema

  21. 21 Build Software to Test Software exactpro.com Data Science University

    Different areas of ML & DS: NLP, Clustering, Regression Analysis, etc Practice Driven Approach: Tasks are connected to Software Testing activities Team work: major part of the task should be done with colleagues Presenting: Each task includes reporting of what was done
  22. 22 Build Software to Test Software exactpro.com Thank you! /m.m.ruslanovich

    /muradmamedov