EXTENT-2019: Dancing with Whales. Adaptive Log Classification System

Dancing with Whales. Adaptive Log Classification System
Elena Treshcheva
Business Development Manager & Researcher, Exactpro

EXTENT-2019
17 September 2019, London

Video: https://youtu.be/_VSztM9m5fw

To learn more about Exactpro, visit our website https://exactpro.com/
EXTENT Conference website: https://extent.exactpro.com/
Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

Transcript

1. Dancing with Whales: Adaptive Log Classification System Elena Treshcheva, Business Development Manager, Researcher, Exactpro
2. Problem vs. Catastrophe
3. What turns a problem into a disaster?
4. Mission-Critical Software - Lots of log messages daily
5. Huge volumes of data vs. error class clusters
6. WhaleShark WhaleShark The First IEEE International Conference on Artificial Intelligence Testing (IEEE AITest 2019), April 4-9 2019, San Francisco East Bay, CA, USA User-Assisted Log Analysis for Quality Control of Distributed Fintech Systems Iosif Itkin, Anna Gromova, Anton Sitnikov, Rostislav Yavorskiy, Evgenii Tsymbalov, Andrey Novikov and Kirill Rudakov.
7. One error class - different text strings text text text text 0100101010001010000 ERROR text text text text text text text text text text text text text text text text FATAL…………………………………… text text text text text text text text It’s so cold and dark... I’m scared...
8. Step 1: Normalization ERROR 07 Aug 2018 13:04:39 - Unable to load messages from file '/UAT/collectors/mq1' ERROR 25 Sep 2018 18:24:55 - Unable to load messages from file '/prod/collectors/mq3' ERROR 16 Oct 2018 15:15:10 - Unable to load messages from file '/prod/collectors/mq5'
9. Step 1: Normalization ERROR 07 Aug 2018 13:04:39 - Unable to load messages from file '/UAT/collectors/mq1' ERROR 25 Sep 2018 18:24:55 - Unable to load messages from file '/prod/collectors/mq3' ERROR ** *** **** **:**:** - Unable to load messages from file '/*/*/*' ERROR 16 Oct 2018 15:15:10 - Unable to load messages from file '/prod/collectors/mq5'
10. Step 2: Clustering
11. Final step: User Notification
12. Result: System state visualization
13. More Control over system outputs
14. Thank you! 14