Face recognition self-study project

Transcript

1. Python Face Recognition project Ada Chang This project is for self-study purposes. Some sensitive information and images have been blurred to protect intellectual properties
2. pp plugin_Face Recognition-Overview Left view Middle view Right view Picture of Person Detected Information of person detected Original image or video Start to recognize people in the 3 views Adjust the 3 views ( , zoom) Save image in the view Video conrol bar Zoom out, zoom in
3. App plugin_Face Recognition-Tools/modules UI QtDesigner, pyuic5 Fisheye images Manipulation SDK Image processing , opencv-python Data Manipulation pandas, numpy Facial Analysis deepface, keras-vggface, face_alignment (face_recognition)
4. App plugin_Face Recognition-Machenism Database Each folder contains the person's images Use vgg-face model to extract the 4096-dim feature from every image Crop, align, resize Pickle containing all features
5. Input Crop, align, resize extract the 4096-dim feature from every face Calculate the feature distance between input features and those in database Pickle containing all database features Verify if the person is in our database moilApp plugin_Face Recognition-Machenism (cont.)
6. Analyze the distance of paired faces 1 (2p_10_10) • Purpose: To determine whether the distances between feature vectors can be utilized for face recognition/verification. • If the feature distances between pairs of the same individual's face are smaller than those between face pairs of different individuals, then the distances can be used as an indicator for implementing face recognition.
7. Steps 1. Collect images & create datasets 2. Extract features 3. Calculate distances of paired features 4. Check if pairwise distances vary between the same person and different individuals
8. Collect images & create datasets • Tools: Image Downloader, face_alignment • We download the president Tsai's and singer Deng's images to do the experiment. • The dataset consists of 10 Tsai's and 10 Deng's frontal image (manually selected) • We use the alignment tool to crop and align the face, then resize all the face images to (224, 224, 3)
9. Collect images & create datasets Tsai Deng
10. Extract features • We use VGGFace model (vgg16, 2015) implemented by Keras to extract the feature of each face • Target layer: 'fc7' • The dimension of feature is 4096 • We extract all 10 features from both Tsai's and Deng's face and normalize them into L2 format, then save them to pickle file
11. Calculate distances of paired features • We have 2 distance calculation method 1. Euclidean distance 2. Cosine distance • There are 3 kinds of conditions to calculate 1. Distance between all Tsai's image (45 pairs) 2. Distance between all Deng's image (45 pairs) 3. Distance between Deng's face and Tsai's face (100 pairs)
12. Result Euclidean distance Cosine distance
13. Analyze the distance of paired faces 2 (2p_57_46) • Purpose: To check if the distance of feature vectors can be used for face recognition/verification • The dataset consists of 57 Tsai's and 46 Deng's image (frontal and profile) • We use the alignment tool to crop and align the face, then resize all the face images to (224, 224, 3)
14. Collect images & create datasets Tsai Deng
15. Calculate distances of paired features • We have 2 distance calculation method 1. Euclidean distance 2. Cosine distance • There are 3 kinds of conditions to calculate 1. Distance between all Tsai's image (1596 pairs) 2. Distance between all Deng's image (1035 pairs) 3. Distance between Deng's face and Tsai's face (2622 pairs)
16. Result Euclidean distance Cosine distance
17. Observations and Hypotheses 1. The resolution of Deng's images is relatively low comparing to that of Tsai's, it may cause the features extracted vary. → Use PIL.ImageFilter.UnsharpMask() on Deng's images and see if there's any improvement. 2. Maybe each person need different threshold for verification, or we should try other classification tools such as SVM.
18. Analyze the distance of paired faces 3 (2p_57_46) • Purpose: To check if the distance of feature vectors can be used for face recognition/verification • The dataset consists of 57 Tsai's and 46 Deng's image (frontal and profile) • All Deng's images are sharpened using PIL.ImageFilter.UnsharpMask() UnsharpMask(radius=2, percent=150, threshold=3) (Default) • We use the alignment tool to crop and align the face, then resize all the face images to (224, 224, 3)
19. Sharpen the images Sharpen
20. Result of experiment_euclidean distance Before sharpen After sharpen Std slightly decreased
21. Result of experiment_cosine distance Before sharpen After sharpen Std slightly decreased