Face Detection and Recognition for Fun and Profit

Using OpenCV (http://opencv.org/) for face recognition along with native iOS face detection.

Transcript

1. Face Detection and
   Recognition for Fun
   and Profit
   1
   Monday 9 September 13
   

   View Slide

2. • And you are...?
   • OpenCV Face Detection With OpenCV
   Recognition
   • Native Face Detection With OpenCV Face
   Recognition
   • Other 3rd Party Alternatives
   • Questions
   2
   Monday 9 September 13
   

   View Slide

3. • Software developer for the last 14 years
   • Currently working for SkillPages
   • Have a few app side projects
   (Broadsheet.ie, TaxCalc.ie, )
   And You Are...?
   3
   Monday 9 September 13
   

   View Slide

4. Prosopagnosia
   • a disorder of face perception where the
   ability to recognise faces is impaired, while
   other aspects of visual processing (e.g.,
   object discrimination) and intellectual
   functioning (e.g., decision making) remain
   intact
   • No cure/lasting therapy
   • https://en.wikipedia.org/wiki/Prosopagnosia
   4
   Monday 9 September 13
   

   View Slide

5. OpenCV Face
   Detection With
   OpenCV Recognition
   5
   Monday 9 September 13
   

   View Slide

6. OpenCV
   • An open source computer vision and
   machine learning library
   • BSD Licensed
   • C/C++/Python/Java interfaces
   • Windows/Linux/Mac/iOS/Android libraries
   • http://opencv.org/
   6
   Monday 9 September 13
   

   View Slide

7. OpenCV
   • 2D and 3D feature toolkits
   • Egomotion estimation
   • Gesture recognition
   • Human–computer interaction
   (HCI)
   • Mobile robotics
   • Motion understanding
   • Object identification
   • Segmentation and Recognition
   • Stereopsis Stereo vision: depth
   perception from 2 cameras
   • Structure from motion (SFM)
   • Motion tracking
   • Augmented reality
   7
   Monday 9 September 13
   

   View Slide

8. OpenCV
   • Object Identification
   • Segmentation and Recognition
   We’re only interested in two abilities today
   8
   Monday 9 September 13
   

   View Slide

9. Object Identification
   • Need to create a Haar Classifier
   • A few default classifications available
   (different face features, full face, bodies etc.)
   • Make your own: http://coding-robin.de/
   2013/07/22/train-your-own-opencv-haar-
   classifier.html
   9
   Monday 9 September 13
   

   View Slide

10. Face Recognition
    Algorithms
    • Eigenfaces
    • Fisherfaces
    • Local Binary Patterns Histogram
    http://docs.opencv.org/modules/contrib/doc/facerec/
    facerec_tutorial.html
    10
    Monday 9 September 13
    

    View Slide

11. Eignfaces vs Fisherfaces
    http://docs.opencv.org/modules/contrib/doc/facerec/
    facerec_tutorial.html#local-binary-patterns-histograms
    11
    Monday 9 September 13
    

    View Slide

12. Identifying A Face
    • Pull image from
    somewhere (video
    stream or just a still
    image)
    • Check for a face
    12
    Monday 9 September 13
    

    View Slide

13. Identifying A Face
    • Normalise the
    face to 100x100
    greyscale image
    • Feed to face
    recogniser model
    • Hopefully get a
    match back (with a
    confidence level)
    13
    Monday 9 September 13
    

    View Slide

14. (Simplified) Code Sample
    cv::Mat cvImage;
    UIImageToMat(uiimage, cvImage, NO);
    cv::CascadeClassifier faceCascade;
    faceCascade.load();
    std::vector faces;
    faceCascade.detectMultiScale(image, faces, 1.1, 2,
    CV_HAAR_FIND_BIGGEST_OBJECT | CV_HAAR_DO_ROUGH_SEARCH, cv::Size(60, 60));
    cv::Ptr _model = cv::createEigenFaceRecognizer();
    _model->train(images, labels);
    for each face {
    int predictedLabel = -1;
    double confidence = 0.0;
    cv::Mat onlyTheFace;
    cv::cvtColor(cvImage(face), onlyTheFace, CV_RGB2GRAY);
    cv::resize(onlyTheFace, onlyTheFace, cv::Size(100, 100), 0, 0);
    _model->predict(onlyTheFace, predictedLabel, confidence);
    }
    14
    Monday 9 September 13
    

    View Slide

15. Demo
    15
    Monday 9 September 13
    

    View Slide

16. Native Face Detection
    With OpenCV Face
    Recognition
    16
    Monday 9 September 13
    

    View Slide

17. Native iOS Methods
    • CIDetector - Face detection in still images
    available since iOS 5
    • AVCaptureMetaDataOutput - added in iOS
    6 for real-time detection
    17
    Monday 9 September 13
    

    View Slide

18. AVCaptureMetadataOutput
    • Optimised to use GPU
    • Produces an AVMetadataFaceObject
    • Face bounds
    • Yaw
    • Roll
    • Tracking faces over time
    • 4s/iPad 2 and up
    18
    Monday 9 September 13
    

    View Slide

19. CIDetector
    • Can use images from anywhere
    • Face bounds
    • Left eye
    • Right eye
    • Mouth position
    • Track faces between frames
    19
    Monday 9 September 13
    

    View Slide

20. Face Recognition
    • Deliberately no available native face
    recognition so still need OpenCV (the face
    tracking ability implies to me some sort of
    native recognition)
    • Exact same process as just using OpenCV
    20
    Monday 9 September 13
    

    View Slide

21. (Simplified) Code Sample
    NSDictionary *detectorOptions = @{CIDetectorAccuracy : CIDetectorAccuracyLow,
    CIDetectorTracking : @YES};
    !
    self.faceDetector = [CIDetector detectorOfType:CIDetectorTypeFace
    context:nil
    options:detectorOptions];
    NSDictionary *imageOptions = @{CIDetectorImageOrientation: imageOrientation};
    NSArray *features = [self.faceDetector featuresInImage:ciImage
    options:imageOptions];
    cv::Ptr _model = cv::createEigenFaceRecognizer();
    _model->train(images, labels);
    for each face {
    int predictedLabel = -1;
    double confidence = 0.0;
    cv::Mat onlyTheFace;
    cv::cvtColor(cvImage(face.bounds), onlyTheFace, CV_RGB2GRAY);
    cv::resize(onlyTheFace, onlyTheFace, cv::Size(100, 100), 0, 0);
    _model->predict(onlyTheFace, predictedLabel, confidence);
    }
    21
    Monday 9 September 13
    

    View Slide

22. Demo
    22
    Monday 9 September 13
    

    View Slide

23. Issues
    • Need a selection of photos of the same
    face
    • Needs to load all known faces before
    attempting to recognise
    • Small sample size increases false positives
    23
    Monday 9 September 13
    

    View Slide

24. Issues
    • Front camera produces poor results
    • Lighting conditions can have big impact
    24
    Monday 9 September 13
    

    View Slide

25. Alternatives to on
    device detection
    • http://www.rekognition.com/
    • https://www.bioid.com/solutions/solutions-
    by-application/bioid-for-facedotcom.html
    • http://api.animetrics.com
    • http://www.identitykit.it/
    • http://www.skybiometry.com/
    25
    Monday 9 September 13
    

    View Slide

26. Pros/Cons
    • Can often do age/gender/glasses/smile
    detection as well
    • All have a cost associated
    • If realtime not an issue can produce better
    results
    26
    Monday 9 September 13
    

    View Slide

27. Sample Code
    https://github.com/kmonaghan/FaceRecognition
    git submodule update --init --recursive
    27
    Monday 9 September 13
    

    View Slide

28. Questions?
    28
    Monday 9 September 13
    

    View Slide

29. @karlmonaghan
    http://karlmonaghan.com
    http://github.com/kmonaghan
    29
    Monday 9 September 13
    

    View Slide