Applying Parallelization Technique to Speedup ClamAV Anti-Virus

Scanning large file with multithread on ClamAV

Transcript

1. Chatsiri Rattana Putchong Uthayopas High Performance Computing and Networking Center (HPCNC)
2.  Introduction  Related Works ◦ .Aho, A. V. and M. J. Corasick., Efficient string matching: an aid to bibliographic search, Commun. ACM 18(6), 1975, 333-340 ◦ Aho-Corasick String Matching on Shared and Distributed-Memory Parallel Architectures , Tumeo, A., O. Villa, et al., ◦ Accelerating Real-Time String Searching with Multicore Processors, 2008  Backgrounds ◦ Aho-Corasick Algorithm ◦ Thread and Multithread  Computational Details ◦ Normal Operation of the programs ◦ Multithread operation of the programs  Results and Discussions  References
3.  String matching concept. ◦ Aho-Coraisk, Knuth–Morris–Pratt , BoyerMoore, Bloom Filter  Thread and Multithread programming. ◦ POSIX, Thread Win32  Multithread scanning with Clamav. ◦ Serial and Multithread scanning.
4. ◦ Efficient string matching: an aid to bibliographic search ◦ Aho-Corasick String Matching on Shared and Distributed- Memory Parallel Architectures ◦ Accelerating Real-Time String Searching with Multicore Processors
5. • Aho, A. V. and M. J. Corasick., Efficient string matching: an aid to bibliographic search, Commun. ACM 18(6), 1975 [1] • Step for implement algorithm  Send string to node of Trie string (Tree Structure).  Build go-to and failure functions from node on Tire.  Search string from Trie structure of string. Picture No. 1 : Picture presents insert string to Tire
6. • Aho-Corasick String Matching on Shared and Distributed-Memory Parallel Architectures , Tumeo, A., O. Villa, et al., [4] • Method for research • Minimize the number of memory references. • Reduce memory contention by STT reduced before compares with string. • Optimization Aho-Corasick Techniques  Alphabet Shuffling.  State Replication. ◦ Test with Mulithread, MPI and MPI with CUDA.
7. • Accelerating Real-Time String Searching with Multicore Processors, 2008 [3] • Method for research • Developing programs on Cell Boardband Engine of Full-Text Search, Network Content Monitoring, Instruction Detection and Virus Scanning. • Optimization Aho-Corasick Techniques  State Caching for enhance memory bandwidth.  State shuffling improved largest state space search.
8.  ClamAV ◦ ClamAV have 2 kinds o f viruses scan algorithm : Aho- Corasick(AC)[1] and Boyer-Moore[2](BM). ◦ ClamAV compare signature and binary file with bytes by bytes. ◦ String searching and Polymorphic scan virus.  POSIX ◦ POSIX threads definition of thread can control the program independent. Parallel threads creates work in a shared memory multiprocessor architecture POSIX Thread or Pthread[5].
9.  ClamAV traditional scan steps. 1. Binary file inserted to memory. 2. Selected types of signature.  MD5, SHA-1of file.  Hex signature inside file. 3. Binary file compare with signature by serial scanning.
10.  ClamAV improved scan steps applied parallelization. 1. File types are sort sizes of file and send to memory. 2. Compare file with signature per N-Thread on system.
11.  Aho-Corasick Algorithm ◦ Create NFA and convert to DFA.  DFA time to created was O(n+m+z) where n set of string compare, m is value of string compare with set T[1…m] and Z is order resent to T. Picture No. 2 : State insert to Tire.
12.  Aho-Corasick Algorithm ◦ Go-To function (g)  G function created to connects and compare with string.  Big-O of the creation is O(n). ◦ Fail function (f)  F function role connect state to state and consider change status from normal to fail state.
13.  Computation procedure as follow step. ◦ Program keeps virus signature into extension .hdb file in order to compare file buffer want to detected. Picture No. 4: File N send to Program threads scan virus.
14. Picture No. 3 : File buffer on memory send to N Threads  Computation procedure as follow step.(#2) ◦ Aho-Corasick in order to comparison with to byte by byte of four kind file extension. ◦ Memory files send buffer of files to N Threads.
15.  Computation procedure as follow step.(#3) ◦ Virus signature feed into tree structure. ◦ Compare four file extension for the existing virus signatures.
16.  Result scan EXE file with multithreads. 0 5 10 15 20 100 500 1000 2000 Threads 4 Threads 8 Threads 16 Speedup (Y) Number of Files (X)
17.  Result scan UPX file with multithreads. 0 5 10 15 20 25 100 500 1000 2000 Threads 4 Threads 8 Threads 16 Speedup (Y) Number of Files (X)
18.  Result scan CAB file with multithreads. 0 2 4 6 8 10 100 500 1000 2000 Threads 4 Threads 8 Threads 16 Speedup (Y) Number of Files (X)
19.  Result scan PDF file with multithreads. 0 2 4 6 8 10 12 14 100 500 1000 2000 Threads 4 Threads 8 Threads 16 Speedup (Y) Number of Files (X)
20.  The result software achieves a substantial speedup over the sequential one.  Multithread supported ClamAV.  Correctness scanning when scan test case extensions are EXE, PE, CAB and PDF files.
22. 1. Aho, A. V. and M. J. Corasick., Efficient string matching: an aid to bibliographic search, Commun. ACM 18(6), 1975, 333-340. 2. Boyer, R. S. and J. S. Moore (1977). A fast string searching algorithm, Commun. ACM 20(10): 762-772. 3. Villa, O., D. P. Scarpazza, et al. (2008). Accelerating Real-Time String Searching with Multicore Processors. Computer 41(4): 42-50. 4. Tumeo, A., O. Villa, et al. (2012). Aho-Corasick String Matching on Shared and Distributed-Memory Parallel Architectures, Parallel and Distributed Systems, IEEE Transactions on 23(3): 436-443. 5. Herath, D., C. Lakmali, et al. (2012). Accelerating string matching for bio-computing applications on multi-core CPUs. Industrial and Information Systems (ICIIS), 2012 7th IEEE International Conference on.