TMPA-2021: Process Mining Algorithm for Online Intrusion Detection System

Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa

Process Mining Algorithm for Online Intrusion Detection System

TMPA is an annual International Conference on Software Testing, Machine Learning and Complex Process Analysis. The conference will focus on the application of modern methods of data science to the analysis of software quality.

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro

Transcript

1. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Process Mining Algorithm for Online Intrusion Detection System Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Department of Computer Science, The University of Liverpool International Conference on Software Testing, Machine Learning and Complex Process Analysis 25-27 November 2021 Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
2. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Outline Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
3. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Process mining Process discovering Analyse Aim We propose process mining inspired technique to be used at the pre-processing stage to generate a behaviour model, which subsequently be classified as attack/no attack or normal/malicious behaviour by trained machine learning models. Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
4. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Process mining 1 a, b, c, d, e, g 2 a, b, c, d, f, g 3 a, c, d, b, f, g 4 a, b, d, c, e, g 5 a, d, c, b, f, g Figure: Process model mined with fuzzy miner. Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
5. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Network packet data Question How to generate the event log from network traffic data? Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
6. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Network packet data Remark We only consider the non-numerical values. e.g. events and transitions. Definition The packets observed on the wire is the sequence P = (pi )n i=1 . Definition The observed packets is also equivalent to a set of TCP flows T = {ti }m i=1 , where T is the event log and t is cases in process mining. Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
7. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Network packet data pn Timestamp Src IP Dst IP Scr port Dst port Flags 1 2017/07/04 14:00:35.190179000 192.168.10.5 68.67.178.110 52892 80 000.SYN. 2 2017/07/04 14:00:35.222535000 68.67.178.110 192.168.10.5 80 52892 000.ACK.SYN. 3 2017/07/04 14:00:35.222586000 192.168.10.5 68.67.178.110 52892 80 000.ACK. 4 2017/07/04 14:00:35.237412000 192.168.10.5 68.67.178.110 52892 80 000.ACK.PSH. 5 2017/07/04 14:00:35.270301000 68.67.178.110 192.168.10.5 80 52892 000.ACK. 6 2017/07/04 14:00:35.270305000 192.168.10.16 23.194.141.47 51226 443 000.SYN. 7 2017/07/05 17:03:45.498235000 68.67.178.110 192.168.10.5 80 52892 000.ACK. 8 2017/07/05 17:03:45.521284000 23.194.141.47 192.168.10.16 443 51226 000.ACK.SYN. 9 2017/07/05 17:03:45.521360000 192.168.10.16 23.194.141.47 51226 443 000.ACK. 10 2017/07/05 17:03:45.521561000 192.168.10.16 23.194.141.47 51226 443 000.ACK.PSH. 11 2017/07/05 17:03:45.544708000 23.194.141.47 192.168.10.16 443 51226 000.ACK. 12 2017/07/05 17:03:45.545025000 23.194.141.47 192.168.10.16 443 51226 000.ACK.PSH. 13 2017/07/05 17:03:45.545073000 192.168.10.16 23.194.141.47 51226 443 000.ACK. Table: Network packet data. Rebuild traces needed. Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
8. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Network packet data Attributes: flags and sides. P = (p1, · · · , p13). T = {t1, t2}. t1 = (p1, p2, p3, p4, p5, p7). t2 = (p6, p8, p9, p10, p11, p12, p13). Case ID (tm) pn Flags Side 1 1 000.SYN. C 1 2 000.ACK.SYN. S 1 3 000.ACK. C 1 4 000.ACK.PSH. C 1 5 000.ACK. S 2 6 000.SYN. C 1 7 000.ACK. S 2 8 000.ACK.SYN. S 2 9 000.ACK. C 2 10 000.ACK.PSH. C 2 11 000.ACK. S 2 12 000.ACK.PSH. S 2 13 000.ACK. C Table: Event log for network data. Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
9. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Network packet data A large model mined from a large event log. Aim A process mining algorithm that is designed for online intrusion detection. Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
10. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Process mining in an online fashion 1 Sliding window with size l. 2 Mine a small model (a snapshot) A based on consecutive l packets. e.g. (pi )l i=1 . 3 Shift to the next packet and repeat. Figure: Packets P500 and Pn belong to traces that are marked as attacks, therefore, A1 and An−500+1 are also labelled as attacks. Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
11. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Process mining in an online fashion A series of snapshots A: (Ai )n−l+1 i=1 Table: 26 event classes used. 000.SYN.|C 000.ACK.SYN.|S 000.ACK.|C 000.ACK.PSH.|C 000.ACK.PSH.|S 000.ACK.FIN.|C 000.ACK.|S 000.ACK.FIN.|S 000.ACK.RST.|C 000.ACK.RST.|S 000.RST.|S 000.ACK.PSH.FIN.|S 000.RST.|C 000.CWR.ECN.SYN.|C 000.ECN.ACK.SYN.|S 000.NS.ACK.FIN.|S 000.ACK.PSH.FIN.|C 000.CWR.ACK.PSH.|C 000.CWR.ACK.|C 000.CWR.ACK.|S 000.CWR.ACK.PSH.|S 000.CWR.ACK.RST.|S 000.CWR.ACK.RST.|C START END OTHERS Ai is 26 by 26 adjacency matrices where weights of edges are the numbers of transitions. Figure: Packets P500 and Pn belong to traces that are marked as attacks, therefore, A1 and An−500+1 are also labelled as attacks. Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
12. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Process mining in an online fashion We use a ”state table” like approach for keeping the last transition and improving the performance. Data are labelled for training. Figure: Packets P500 and Pn belong to traces that are marked as attacks, therefore, A1 and An−500+1 are also labelled as attacks. Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
13. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Results 0 50000 100000 150000 200000 250000 300000 A i 0.0 0.1 0.2 0.3 0.4 0.5 0.6 Frequency START-000.CWR.ECN.SYN.|C 000.CWR.ECN.SYN.|C-000.ACK.SYN.|S 000.ACK.SYN.|S-000.ACK.|C 000.ACK.|C-000.ACK.PSH.|C 000.ACK.PSH.|C-000.ACK.|S 000.ACK.|S-000.ACK.PSH.|S 000.ACK.PSH.|S-000.ACK.|C 000.ACK.|S-000.ACK.|S 000.ACK.|S-000.ACK.|C 000.ACK.|C-000.ACK.PSH.|S 0 50000 100000 150000 200000 250000 300000 A i 0.0 0.1 0.2 0.3 0.4 Frequency 000.ACK.PSH.|S-000.ACK.PSH.|C 000.ACK.PSH.|C-000.ACK.PSH.|S 000.ACK.PSH.|S-000.ACK.PSH.|S 000.ACK.PSH.|C-000.ACK.PSH.|C 000.ACK.PSH.|C-000.ACK.FIN.|C 000.ACK.PSH.|S-000.ACK.|S 000.ACK.|C-000.ACK.|S 000.CWR.ECN.SYN.|C-000.ECN.ACK.SYN.|S 000.ECN.ACK.SYN.|S-000.ACK.|C START-000.SYN.|C Figure: The chart show frequency fluctuation. 20 out of 676 possible relations are given as the example. Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
14. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Results Table: comparison between preprocessors. Attack Type Our Preprocessor CICFlowMeter FTP-BruteForce 0.9990 0.98 SSH-Bruteforce 0.9764 0.96 DoS-GoldenEye 0.9434 0.47 DoS-Slowloris 0.9948 0.66 DoS-SlowHTTP 0.9984 1 DoS-Hulk 0.7314 1 DDoS-LOIC-HTTP 0.8406 1 DDOS-HOIC 0.7559 1 BruteForce-Web 0.9741 0.3 BruteForce-XSS 0.9827 0.65 Botnet 0.8623 1 Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
15. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Results Figure: The receiver operating characteristic (ROC) for anomaly-based intrusion detection setup. The first chart shows the performance of our preprocessor, where the second chart shows the performance of CICFlowMeter. Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
16. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Conclusion Promising performance and online processing. Extensible pre-processing algorithm that allows techniques such as abstraction to be applied. Classifiers can be easily applied. Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System
17. Outline Process mining Network packet data Process mining in an online fashion Results Conclusion Thank you! Yinzheng Zhong, Yannis Goulermas and Alexei Lisitsa Process Mining Algorithm for Online Intrusion Detection System