Кластеризация фейлов: чему машина может научить человека

Мурад Мамедов
AI Researcher, Exactpro

14 мая 2020
Online Data Science Meetup

Мурад расскажет о том, как в рамках проекта Sailfish проходила кластеризация тестовых фейлов по типам и источникам, и к каким выводам удалось прийти в процессе.
Доклад будет вам полезен, если вы хотите начать применять машинное обучение в своей сфере.

Видео: https://youtu.be/zHDqVG4wPjU

---
Подписывайтесь на Exactpro в социальных сетях:

LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/

Подписывайтесь на YouTube канал Exactpro https://www.youtube.com/c/exactprosystems

Transcript

1. Build Software to Test Software
   exactpro.com
   Failure Clustering:
   What a Machine Can Teach Us
   Murad Mamedov, AI Researcher
   DS Meetup, May 2020
   

   View Slide

2. 2 Build Software to Test Software exactpro.com
   A specialist firm focused on functional
   and non-functional testing of
   exchanges, clearing houses,
   depositories, trade repositories and
   other financial market infrastructures.
   About Exactpro
   Headquartered in the UK with
   operations in the US and Eastern
   Europe. The clients are regulated by
   the FCA, Bank of England and their
   counterparts from other countries.
   Exactpro provides software testing services for mission critical technology that underpins global financial
   markets. The firm is experienced with trading and clearing and settlement platforms, market data systems,
   collateral management and risk management systems, central data warehouses, regulatory reporting, etc.
   Incorporated in 2009 with 10
   people, Exactpro is now employing
   over 580 specialists.
   Part of the London Stock Exchange
   Group (LSEG) from May 2015 until
   January 2018, when the Exactpro
   management proceeded through the
   buyout of the company from LSEG.
   

   View Slide

3. 3 Build Software to Test Software exactpro.com
   Introduction
   

   View Slide

4. 4 Build Software to Test Software exactpro.com
   Our AI Research
   Whaleshark
   Cradle
   

   View Slide

5. 5 Build Software to Test Software exactpro.com
   How it works
   exactpro.com/test-tools/sailfish
   github.com/Exactpro/sailfish-core
   

   View Slide

6. 6 Build Software to Test Software exactpro.com
   Web
   Binary
   FIX
   Market Data
   Database
   How Sailfish Works
   

   View Slide

7. 7 Build Software to Test Software exactpro.com
   ● Detailed Analytics and Recommendations
   ⬆ Understanding of each failure
   ⬆ Failures Source Detection
   ⬆ Relevant Message Recognition
   Task Decomposition
   

   View Slide

8. 8 Build Software to Test Software exactpro.com
   Relevant Message Recognition
   Automated Analysis of Testing
   Reports using Machine
   Learning Techniques
   Building a Classification
   System for Failed Test Runs:
   Industrial Report
   2019.secrus.org ieeeaitests.com
   

   View Slide

9. 9 Build Software to Test Software exactpro.com
   ● Detailed Analytics and Recommendations
   ⬆ Understanding of each failure
   ⬆ Failures Source Detection
   ⬆ Relevant Message Recognition
   Task Decomposition
   

   View Slide

10. 10 Build Software to Test Software exactpro.com
    Dataset Description
    

    View Slide

11. 11 Build Software to Test Software exactpro.com
    Cluster Analysis: Examples
    

    View Slide

12. 12 Build Software to Test Software exactpro.com
    Cluster Analysis: Examples
    

    View Slide

13. 13 Build Software to Test Software exactpro.com
    Cluster Analysis: Examples
    

    View Slide

14. 14 Build Software to Test Software exactpro.com
    Cluster Analysis: Examples
    

    View Slide

15. 15 Build Software to Test Software exactpro.com
    Failure Graph
    

    View Slide

16. 16 Build Software to Test Software exactpro.com
    Failure Graph
    

    View Slide

17. 17 Build Software to Test Software exactpro.com
    Failure Graph
    

    View Slide

18. 18 Build Software to Test Software exactpro.com
    Two Approaches to Use ML
    Default way
    - Train an algorithm
    - Release it into production
    Supportive way
    - Train algorithm and extract certain determined rules
    - Then adopt the rules in production
    

    View Slide

19. 19 Build Software to Test Software exactpro.com
    3:44 7:21 5:37
    watch on Youtube watch on Youtube
    watch on Youtube
    

    View Slide

20. 20 Build Software to Test Software exactpro.com
    Thank you!
    

    View Slide