Upgrade to Pro — share decks privately, control downloads, hide ads and more …

EXTENT-2019: Dancing with Whales. Adaptive Log Classification System

5206c19df417b8876825b5561344c1a0?s=47 Exactpro
PRO
September 17, 2019

EXTENT-2019: Dancing with Whales. Adaptive Log Classification System

Dancing with Whales. Adaptive Log Classification System
Elena Treshcheva
Business Development Manager & Researcher, Exactpro

EXTENT-2019
17 September 2019, London

Video: https://youtu.be/_VSztM9m5fw

To learn more about Exactpro, visit our website https://exactpro.com/
EXTENT Conference website: https://extent.exactpro.com/
Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

5206c19df417b8876825b5561344c1a0?s=128

Exactpro
PRO

September 17, 2019
Tweet

Transcript

  1. Dancing with Whales: Adaptive Log Classification System Elena Treshcheva, Business

    Development Manager, Researcher, Exactpro
  2. Problem vs. Catastrophe

  3. What turns a problem into a disaster?

  4. Mission-Critical Software - Lots of log messages daily

  5. Huge volumes of data vs. error class clusters

  6. WhaleShark WhaleShark The First IEEE International Conference on Artificial Intelligence

    Testing (IEEE AITest 2019), April 4-9 2019, San Francisco East Bay, CA, USA User-Assisted Log Analysis for Quality Control of Distributed Fintech Systems Iosif Itkin, Anna Gromova, Anton Sitnikov, Rostislav Yavorskiy, Evgenii Tsymbalov, Andrey Novikov and Kirill Rudakov.
  7. One error class - different text strings text text text

    text 0100101010001010000 ERROR text text text text text text text text text text text text text text text text FATAL…………………………………… text text text text text text text text It’s so cold and dark... I’m scared...
  8. Step 1: Normalization ERROR 07 Aug 2018 13:04:39 - Unable

    to load messages from file '/UAT/collectors/mq1' ERROR 25 Sep 2018 18:24:55 - Unable to load messages from file '/prod/collectors/mq3' ERROR 16 Oct 2018 15:15:10 - Unable to load messages from file '/prod/collectors/mq5'
  9. Step 1: Normalization ERROR 07 Aug 2018 13:04:39 - Unable

    to load messages from file '/UAT/collectors/mq1' ERROR 25 Sep 2018 18:24:55 - Unable to load messages from file '/prod/collectors/mq3' ERROR ** *** **** **:**:** - Unable to load messages from file '/*/*/*' ERROR 16 Oct 2018 15:15:10 - Unable to load messages from file '/prod/collectors/mq5'
  10. Step 2: Clustering

  11. Final step: User Notification

  12. Result: System state visualization

  13. More Control over system outputs

  14. Thank you! 14