Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Face Detection and Recognition for Fun and Profit

Karl Monaghan
September 10, 2013

Face Detection and Recognition for Fun and Profit

Using OpenCV (http://opencv.org/) for face recognition along with native iOS face detection.

Karl Monaghan

September 10, 2013
Tweet

More Decks by Karl Monaghan

Other Decks in Technology

Transcript

  1. Face Detection and Recognition for Fun and Profit 1 Monday

    9 September 13
  2. • And you are...? • OpenCV Face Detection With OpenCV

    Recognition • Native Face Detection With OpenCV Face Recognition • Other 3rd Party Alternatives • Questions 2 Monday 9 September 13
  3. • Software developer for the last 14 years • Currently

    working for SkillPages • Have a few app side projects (Broadsheet.ie, TaxCalc.ie, ) And You Are...? 3 Monday 9 September 13
  4. Prosopagnosia • a disorder of face perception where the ability

    to recognise faces is impaired, while other aspects of visual processing (e.g., object discrimination) and intellectual functioning (e.g., decision making) remain intact • No cure/lasting therapy • https://en.wikipedia.org/wiki/Prosopagnosia 4 Monday 9 September 13
  5. OpenCV Face Detection With OpenCV Recognition 5 Monday 9 September

    13
  6. OpenCV • An open source computer vision and machine learning

    library • BSD Licensed • C/C++/Python/Java interfaces • Windows/Linux/Mac/iOS/Android libraries • http://opencv.org/ 6 Monday 9 September 13
  7. OpenCV • 2D and 3D feature toolkits • Egomotion estimation

    • Gesture recognition • Human–computer interaction (HCI) • Mobile robotics • Motion understanding • Object identification • Segmentation and Recognition • Stereopsis Stereo vision: depth perception from 2 cameras • Structure from motion (SFM) • Motion tracking • Augmented reality 7 Monday 9 September 13
  8. OpenCV • Object Identification • Segmentation and Recognition We’re only

    interested in two abilities today 8 Monday 9 September 13
  9. Object Identification • Need to create a Haar Classifier •

    A few default classifications available (different face features, full face, bodies etc.) • Make your own: http://coding-robin.de/ 2013/07/22/train-your-own-opencv-haar- classifier.html 9 Monday 9 September 13
  10. Face Recognition Algorithms • Eigenfaces • Fisherfaces • Local Binary

    Patterns Histogram http://docs.opencv.org/modules/contrib/doc/facerec/ facerec_tutorial.html 10 Monday 9 September 13
  11. Eignfaces vs Fisherfaces http://docs.opencv.org/modules/contrib/doc/facerec/ facerec_tutorial.html#local-binary-patterns-histograms 11 Monday 9 September 13

  12. Identifying A Face • Pull image from somewhere (video stream

    or just a still image) • Check for a face 12 Monday 9 September 13
  13. Identifying A Face • Normalise the face to 100x100 greyscale

    image • Feed to face recogniser model • Hopefully get a match back (with a confidence level) 13 Monday 9 September 13
  14. (Simplified) Code Sample cv::Mat cvImage; UIImageToMat(uiimage, cvImage, NO); cv::CascadeClassifier faceCascade;

    faceCascade.load(<a Haar classifier>); std::vector<cv::Rect> faces; faceCascade.detectMultiScale(image, faces, 1.1, 2, CV_HAAR_FIND_BIGGEST_OBJECT | CV_HAAR_DO_ROUGH_SEARCH, cv::Size(60, 60)); cv::Ptr<cv::FaceRecognizer> _model = cv::createEigenFaceRecognizer(); _model->train(images, labels); for each face { int predictedLabel = -1; double confidence = 0.0; cv::Mat onlyTheFace; cv::cvtColor(cvImage(face), onlyTheFace, CV_RGB2GRAY); cv::resize(onlyTheFace, onlyTheFace, cv::Size(100, 100), 0, 0); _model->predict(onlyTheFace, predictedLabel, confidence); } 14 Monday 9 September 13
  15. Demo 15 Monday 9 September 13

  16. Native Face Detection With OpenCV Face Recognition 16 Monday 9

    September 13
  17. Native iOS Methods • CIDetector - Face detection in still

    images available since iOS 5 • AVCaptureMetaDataOutput - added in iOS 6 for real-time detection 17 Monday 9 September 13
  18. AVCaptureMetadataOutput • Optimised to use GPU • Produces an AVMetadataFaceObject

    • Face bounds • Yaw • Roll • Tracking faces over time • 4s/iPad 2 and up 18 Monday 9 September 13
  19. CIDetector • Can use images from anywhere • Face bounds

    • Left eye • Right eye • Mouth position • Track faces between frames 19 Monday 9 September 13
  20. Face Recognition • Deliberately no available native face recognition so

    still need OpenCV (the face tracking ability implies to me some sort of native recognition) • Exact same process as just using OpenCV 20 Monday 9 September 13
  21. (Simplified) Code Sample NSDictionary *detectorOptions = @{CIDetectorAccuracy : CIDetectorAccuracyLow, CIDetectorTracking

    : @YES}; ! self.faceDetector = [CIDetector detectorOfType:CIDetectorTypeFace context:nil options:detectorOptions]; NSDictionary *imageOptions = @{CIDetectorImageOrientation: imageOrientation}; NSArray *features = [self.faceDetector featuresInImage:ciImage options:imageOptions]; cv::Ptr<cv::FaceRecognizer> _model = cv::createEigenFaceRecognizer(); _model->train(images, labels); for each face { int predictedLabel = -1; double confidence = 0.0; cv::Mat onlyTheFace; cv::cvtColor(cvImage(face.bounds), onlyTheFace, CV_RGB2GRAY); cv::resize(onlyTheFace, onlyTheFace, cv::Size(100, 100), 0, 0); _model->predict(onlyTheFace, predictedLabel, confidence); } 21 Monday 9 September 13
  22. Demo 22 Monday 9 September 13

  23. Issues • Need a selection of photos of the same

    face • Needs to load all known faces before attempting to recognise • Small sample size increases false positives 23 Monday 9 September 13
  24. Issues • Front camera produces poor results • Lighting conditions

    can have big impact 24 Monday 9 September 13
  25. Alternatives to on device detection • http://www.rekognition.com/ • https://www.bioid.com/solutions/solutions- by-application/bioid-for-facedotcom.html

    • http://api.animetrics.com • http://www.identitykit.it/ • http://www.skybiometry.com/ 25 Monday 9 September 13
  26. Pros/Cons • Can often do age/gender/glasses/smile detection as well •

    All have a cost associated • If realtime not an issue can produce better results 26 Monday 9 September 13
  27. Sample Code https://github.com/kmonaghan/FaceRecognition git submodule update --init --recursive 27 Monday

    9 September 13
  28. Questions? 28 Monday 9 September 13

  29. @karlmonaghan http://karlmonaghan.com http://github.com/kmonaghan 29 Monday 9 September 13