1 25-27 NOVEMBER, TOMSK, RUSSIA SOFTWARE TESTING, MACHINE LEARNING AND COMPLEX PROCESS ANALYSIS DETECTION OF FLYING OBJECTS USING THE YOLOV4 CONVOLUTIONAL NEURAL NETWORK Semen Tkachev and Nikolay Markov National Research Tomsk Polytechnic University

2 Overview ● Relevance of the topic ● Architecture of the YOLOv4 CNN ● Preparation of datasets ● Training of the YOLOv4 CNN ● Researching of the efficiency of the YOLOv4 CNN ● Discussion of the results

3 Relevance of the topic ● Task of localization and classification of flying objects (object detection) ● Using the YOLOv4 CNN as a solution to the problem of object detection drone 0.99

4 Purpose and objectives Purpose: ● To evaluate the efficiency of the YOLOv4 CNN Objectives: ● Preparation of datasets ● Researching of the YOLOv4 CNN with various parameters ● Analysis of results

5 Architecture of the YOLOv4 CNN The YOLOv4 CNN refers to one-stage detectors and consists of blocks: ● Input: source images ● Backbone: this block uses CSPDarknet53 ● Neck: this block uses SPP for additional layers and PAN for path aggregation ● Dense Prediction: this block uses the YOLOv3 CNN

6 Preparation of datasets Helicopter-type unmanned aerial vehicles and gliders datasets: ● About 5000 images of each class: ■ One half of dataset is optical images ■ Other half of dataset is images obtained in the infrared wavelength range ● Preliminary data augmentation: ■ Crop operation ■ Flip operation ■ Rotate operation ● For training – 80% and for testing/researching – 20%

7 Software implementation of the YOLOv4 CNN

8 Parameters for training and researching of the YOLOv4 CNN The following parameters were changed: ● Input image size: 416x416, 512x512 and 608x608 pixels ● Mini-batch size: 4 and 8 ● Activation function: Leaky ReLU and Mish The following parameters were unchanged: ● Number of epochs: 200 ● Learning rate: 0.001 ● Optimizer: Adam algorithm

9 Examples of detection of flying objects in images a) drone b) glider

10 Results of experiment No. 1 The accuracy of detection of flying objects in optical images for different sizes of mini- batches and of input images. Mini-batch size Image size Using BoF/BoS mAP, % 4 416x416 – 61,3 8 416x416 – 62,4 4 512x512 + 64,4 8 512x512 + 64,5 mAP – mean Average Precision

11 Results of experiment No. 2 The accuracy of detection of flying objects in optical images for different activation functions and sizes of input images. Mini-batch size Image size Using BoF/BoS mAP, % 4 416x416 – 61,3 8 416x416 – 62,4 4 512x512 + 64,4 8 512x512 + 64,5 Activation function Image size mAP, % Leaky ReLU 416x416 62,4 Mish 416x416 62,8 Leaky ReLU 512x512 64,2 Mish 512x512 64,5 Leaky ReLU 608x608 65,2 Mish 608x608 65,7 mAP – mean Average Precision

12 Results of experiment No. 3 The rate of detection of flying objects in optical images for different sizes of input images. Mini-batch size Image size Using BoF/BoS mAP, % 4 416x416 – 61,3 8 416x416 – 62,4 4 512x512 + 64,4 8 512x512 + 64,5 Image size Frames per second (FPS) Average time of detection, ms 416x416 46 21,7 512x512 37 27,0 608x608 30 33,3

13 Results of experiment No. 4 The accuracy of detection of flying objects in images obtained in the infrared wavelength range for different sizes of mini-batches and of input images. Mini-batch size Image size Using BoF/BoS mAP, % 4 416x416 – 55,4 8 416x416 – 56,7 4 512x512 + 58,9 8 512x512 + 59,1 mAP – mean Average Precision

14 Results of experiment No. 5 The accuracy of detection of flying objects in images obtained in the infrared wavelength range for different activation functions and sizes of input images. Mini-batch size Image size Using BoF/BoS mAP, % 4 416x416 – 61,3 8 416x416 – 62,4 4 512x512 + 64,4 8 512x512 + 64,5 Activation function Image size mAP, % Leaky ReLU 416x416 56,7 Mish 416x416 57,1 Leaky ReLU 512x512 58,6 Mish 512x512 59,1 Leaky ReLU 608x608 60,2 Mish 608x608 60,7 mAP – mean Average Precision

15 Results of experiment No. 6 The rate of detection of flying objects in images obtained in the infrared wavelength range for different sizes of input images. Mini-batch size Image size Using BoF/BoS mAP, % 4 416x416 – 61,3 8 416x416 – 62,4 4 512x512 + 64,4 8 512x512 + 64,5 Image size FPS Average time of detection, ms 416x416 42 23,8 512x512 36 27,7 608x608 27 37,0

16 Analysis of results ● The accuracy of detection of flying objects in images using the YOLOv4 CNN changes insignificantly when using BoF/BoS. ● The accuracy of detection of flying objects in images using the YOLOv4 CNN is higher when using the Mish activation function for different sizes of input images. ● The best rate of detection of flying objects in images using the YOLOv4 CNN can be obtained with the input image size of 416x416 pixels. ● The accuracy of detection of flying objects using the YOLOv4 CNN in optical images is 5-6% (according to the mAP metric) higher than in images obtained in the infrared wavelength range.

17 Conclusion ● Researching of the efficiency of the YOLOv4 CNN were performed in detection of two classes of flying objects in images: helicopter-type unmanned aerial vehicles and gliders. ● It was found that the accuracy of detection of flying objects using the YOLOv4 CNN increases with increasing of the size of the input images and the best results were obtained using the Mish activation function. ● It is shown that the accuracy of detection of flying objects using the YOLOv4 CNN in optical images is higher than in images obtained in the infrared wavelength range. ● The obtained estimates of the rate of detection of flying objects in images indicate the possibility of creating a computer vision system based on the YOLOv4 CNN that detects flying objects in real time.

18 Thank You! Follow TMPA on Facebook TMPA-2021 Conference