AI Development at Exactpro

AI Development at Exactpro

Murad Mamedov, AI Researcher, Exactpro

Exactpro Webinar for Tbilisi State University | A Webinar on Software Testing and AI
16.06.2020

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/
Vkontakte https://vk.com/exactpro_llc

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

5206c19df417b8876825b5561344c1a0?s=128

Exactpro

June 16, 2020
Tweet

Transcript

  1. 1.

    Build Software to Test Software exactpro.com AI Development at Exactpro

    Murad Mamedov, AI Researcher TSU Webinar, June 2020
  2. 2.

    2 Build Software to Test Software exactpro.com Plan - Machine

    How we research AI applications in Software Testing and how we build AI Tool on top of this - Platform Test Harness platform of new generation. Its overview from AI perspective - People How we prepare Software Testers for new set of tasks from Data Science and Machine Learning area
  3. 5.

    5 Build Software to Test Software exactpro.com AI Basics Fetch

    the Data Data Preprocessing Model Training Model Validation
  4. 6.

    6 Build Software to Test Software exactpro.com Nostradamus High-Level Details:

    Business Taks: To predict future states of the defects Fetch the Data Data Preprocessing Model Training From bug-tracking systems Empty values transformation, TTR calculation, text normalization, stemming TF-IDF Vectorizer, SMOTE, Select Percentile, SVC
  5. 8.

    8 Build Software to Test Software exactpro.com WhaleShark High-Level Details:

    Business Taks: Simplify log analysis and automate the process of relations recognition Fetch the Data Data Preprocessing Model Training From back-end of the systems under test IDs transformation, text normalization TF-IDF Vectorizer, k-means, SVD, word2vec
  6. 9.

    9 Build Software to Test Software exactpro.com Web Binary FIX

    Market Data Database How Sailfish Works
  7. 10.

    10 Build Software to Test Software exactpro.com Sailfish Report send

    NewOrderSingle (PASSED) [0.001s] - receive ExecutionReport (PASSED) [0.001s]: + Input Parameters - Verification: Message (PASSED): Field Expected Result Actual Result Status ... ... ... ... OrdType LIMIT LIMIT PASSED Status NEW NEW PASSED ... ... ... ... Passed Step of Test Case:
  8. 11.

    11 Build Software to Test Software exactpro.com send NewOrderSingle (PASSED)

    [0.001s] - receive ExecutionReport (FAILED) [0.421s]: + Input Parameters + Verification: Similar message [1]. Failed/Passed/Conditionally Passed/NA: 4/26/0/6 (FAILED) + Verification: Similar message [2]. Failed/Passed/Conditionally Passed/NA: 2/28/0/6 (FAILED) + Verification: Similar message [3]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED) - Verification: Similar message [4]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED): Field Expected Result Actual Result Status ... ... ... ... OrdType LIMIT LIMIT PASSED Status NEW REJECTED FAILED ... ... ... ... Failure Example Failed Step of Test Case:
  9. 14.

    14 Build Software to Test Software exactpro.com ML for Sailfish

    Business Taks: To make reports clear via discarding non-relevant messages High-Level Details: Business Taks: Automate the process of source and type recognition of the failures High-Level Details: Fetch the Data Data Preprocessing Model Training Marked up reports from test-library run in Sailfish Dimensionality reduction, Feature Importance, One-Hot Encoding Random Forest, SVN, Calibration Data Preprocessing Model Training Examples transformation, split by message type k-means, DB-scan, Agglomerative
  10. 16.

    16 Build Software to Test Software exactpro.com Organizational Workflow Computer

    Science Domain Expertise Math & Statistics Developers Testers DS/ML Engineers Data Analysis Model Training Validation & Verification Implementation
  11. 17.

    17 Build Software to Test Software exactpro.com Organizational Workflow Computer

    Science Domain Expertise Math & Statistics Developers Testers DS/ML Engineers Data Analysis Model Training Validation & Verification Implementation
  12. 18.

    18 Build Software to Test Software exactpro.com Organizational Workflow Computer

    Science Domain Expertise Math & Statistics Developers Testers DS/ML Engineers Data Analysis Model Training Validation & Verification Implementation
  13. 21.

    21 Build Software to Test Software exactpro.com Data Science University

    Different areas of ML & DS: NLP, Clustering, Regression Analysis, etc Practice Driven Approach: Tasks are connected to Software Testing activities Team work: major part of the task should be done with colleagues Presenting: Each task includes reporting of what was done