Face Detection and Recognition for Fun and Profit

31f1cd560a4a4910c62eb3ebcfe34c6a?s=47 Karl Monaghan
September 10, 2013

Face Detection and Recognition for Fun and Profit

Using OpenCV (http://opencv.org/) for face recognition along with native iOS face detection.

31f1cd560a4a4910c62eb3ebcfe34c6a?s=128

Karl Monaghan

September 10, 2013
Tweet

Transcript

  1. 2.

    • And you are...? • OpenCV Face Detection With OpenCV

    Recognition • Native Face Detection With OpenCV Face Recognition • Other 3rd Party Alternatives • Questions 2 Monday 9 September 13
  2. 3.

    • Software developer for the last 14 years • Currently

    working for SkillPages • Have a few app side projects (Broadsheet.ie, TaxCalc.ie, ) And You Are...? 3 Monday 9 September 13
  3. 4.

    Prosopagnosia • a disorder of face perception where the ability

    to recognise faces is impaired, while other aspects of visual processing (e.g., object discrimination) and intellectual functioning (e.g., decision making) remain intact • No cure/lasting therapy • https://en.wikipedia.org/wiki/Prosopagnosia 4 Monday 9 September 13
  4. 6.

    OpenCV • An open source computer vision and machine learning

    library • BSD Licensed • C/C++/Python/Java interfaces • Windows/Linux/Mac/iOS/Android libraries • http://opencv.org/ 6 Monday 9 September 13
  5. 7.

    OpenCV • 2D and 3D feature toolkits • Egomotion estimation

    • Gesture recognition • Human–computer interaction (HCI) • Mobile robotics • Motion understanding • Object identification • Segmentation and Recognition • Stereopsis Stereo vision: depth perception from 2 cameras • Structure from motion (SFM) • Motion tracking • Augmented reality 7 Monday 9 September 13
  6. 8.

    OpenCV • Object Identification • Segmentation and Recognition We’re only

    interested in two abilities today 8 Monday 9 September 13
  7. 9.

    Object Identification • Need to create a Haar Classifier •

    A few default classifications available (different face features, full face, bodies etc.) • Make your own: http://coding-robin.de/ 2013/07/22/train-your-own-opencv-haar- classifier.html 9 Monday 9 September 13
  8. 10.

    Face Recognition Algorithms • Eigenfaces • Fisherfaces • Local Binary

    Patterns Histogram http://docs.opencv.org/modules/contrib/doc/facerec/ facerec_tutorial.html 10 Monday 9 September 13
  9. 12.

    Identifying A Face • Pull image from somewhere (video stream

    or just a still image) • Check for a face 12 Monday 9 September 13
  10. 13.

    Identifying A Face • Normalise the face to 100x100 greyscale

    image • Feed to face recogniser model • Hopefully get a match back (with a confidence level) 13 Monday 9 September 13
  11. 14.

    (Simplified) Code Sample cv::Mat cvImage; UIImageToMat(uiimage, cvImage, NO); cv::CascadeClassifier faceCascade;

    faceCascade.load(<a Haar classifier>); std::vector<cv::Rect> faces; faceCascade.detectMultiScale(image, faces, 1.1, 2, CV_HAAR_FIND_BIGGEST_OBJECT | CV_HAAR_DO_ROUGH_SEARCH, cv::Size(60, 60)); cv::Ptr<cv::FaceRecognizer> _model = cv::createEigenFaceRecognizer(); _model->train(images, labels); for each face { int predictedLabel = -1; double confidence = 0.0; cv::Mat onlyTheFace; cv::cvtColor(cvImage(face), onlyTheFace, CV_RGB2GRAY); cv::resize(onlyTheFace, onlyTheFace, cv::Size(100, 100), 0, 0); _model->predict(onlyTheFace, predictedLabel, confidence); } 14 Monday 9 September 13
  12. 17.

    Native iOS Methods • CIDetector - Face detection in still

    images available since iOS 5 • AVCaptureMetaDataOutput - added in iOS 6 for real-time detection 17 Monday 9 September 13
  13. 18.

    AVCaptureMetadataOutput • Optimised to use GPU • Produces an AVMetadataFaceObject

    • Face bounds • Yaw • Roll • Tracking faces over time • 4s/iPad 2 and up 18 Monday 9 September 13
  14. 19.

    CIDetector • Can use images from anywhere • Face bounds

    • Left eye • Right eye • Mouth position • Track faces between frames 19 Monday 9 September 13
  15. 20.

    Face Recognition • Deliberately no available native face recognition so

    still need OpenCV (the face tracking ability implies to me some sort of native recognition) • Exact same process as just using OpenCV 20 Monday 9 September 13
  16. 21.

    (Simplified) Code Sample NSDictionary *detectorOptions = @{CIDetectorAccuracy : CIDetectorAccuracyLow, CIDetectorTracking

    : @YES}; ! self.faceDetector = [CIDetector detectorOfType:CIDetectorTypeFace context:nil options:detectorOptions]; NSDictionary *imageOptions = @{CIDetectorImageOrientation: imageOrientation}; NSArray *features = [self.faceDetector featuresInImage:ciImage options:imageOptions]; cv::Ptr<cv::FaceRecognizer> _model = cv::createEigenFaceRecognizer(); _model->train(images, labels); for each face { int predictedLabel = -1; double confidence = 0.0; cv::Mat onlyTheFace; cv::cvtColor(cvImage(face.bounds), onlyTheFace, CV_RGB2GRAY); cv::resize(onlyTheFace, onlyTheFace, cv::Size(100, 100), 0, 0); _model->predict(onlyTheFace, predictedLabel, confidence); } 21 Monday 9 September 13
  17. 23.

    Issues • Need a selection of photos of the same

    face • Needs to load all known faces before attempting to recognise • Small sample size increases false positives 23 Monday 9 September 13
  18. 24.

    Issues • Front camera produces poor results • Lighting conditions

    can have big impact 24 Monday 9 September 13
  19. 25.

    Alternatives to on device detection • http://www.rekognition.com/ • https://www.bioid.com/solutions/solutions- by-application/bioid-for-facedotcom.html

    • http://api.animetrics.com • http://www.identitykit.it/ • http://www.skybiometry.com/ 25 Monday 9 September 13
  20. 26.

    Pros/Cons • Can often do age/gender/glasses/smile detection as well •

    All have a cost associated • If realtime not an issue can produce better results 26 Monday 9 September 13