Dancing with Whales: Adaptive Log Classification System Elena Treshcheva, Business Development Manager, Researcher, Exactpro

Problem vs. Catastrophe

What turns a problem into a disaster?

Mission-Critical Software - Lots of log messages daily

Huge volumes of data vs. error class clusters

WhaleShark WhaleShark The First IEEE International Conference on Artificial Intelligence Testing (IEEE AITest 2019), April 4-9 2019, San Francisco East Bay, CA, USA User-Assisted Log Analysis for Quality Control of Distributed Fintech Systems Iosif Itkin, Anna Gromova, Anton Sitnikov, Rostislav Yavorskiy, Evgenii Tsymbalov, Andrey Novikov and Kirill Rudakov.

One error class - different text strings text text text text 0100101010001010000 ERROR text text text text text text text text text text text text text text text text FATAL…………………………………… text text text text text text text text It’s so cold and dark... I’m scared...

Step 1: Normalization ERROR 07 Aug 2018 13:04:39 - Unable to load messages from file '/UAT/collectors/mq1' ERROR 25 Sep 2018 18:24:55 - Unable to load messages from file '/prod/collectors/mq3' ERROR 16 Oct 2018 15:15:10 - Unable to load messages from file '/prod/collectors/mq5'

Step 1: Normalization ERROR 07 Aug 2018 13:04:39 - Unable to load messages from file '/UAT/collectors/mq1' ERROR 25 Sep 2018 18:24:55 - Unable to load messages from file '/prod/collectors/mq3' ERROR ** *** **** **:**:** - Unable to load messages from file '/*/*/*' ERROR 16 Oct 2018 15:15:10 - Unable to load messages from file '/prod/collectors/mq5'

Step 2: Clustering

Final step: User Notification

Result: System state visualization

More Control over system outputs

Thank you! 14