Face Detection and Recognition for Fun and Profit 1 Monday 9 September 13

• And you are...? • OpenCV Face Detection With OpenCV Recognition • Native Face Detection With OpenCV Face Recognition • Other 3rd Party Alternatives • Questions 2 Monday 9 September 13

• Software developer for the last 14 years • Currently working for SkillPages • Have a few app side projects (Broadsheet.ie, TaxCalc.ie, ) And You Are...? 3 Monday 9 September 13

Prosopagnosia • a disorder of face perception where the ability to recognise faces is impaired, while other aspects of visual processing (e.g., object discrimination) and intellectual functioning (e.g., decision making) remain intact • No cure/lasting therapy • https://en.wikipedia.org/wiki/Prosopagnosia 4 Monday 9 September 13

OpenCV Face Detection With OpenCV Recognition 5 Monday 9 September 13

OpenCV • An open source computer vision and machine learning library • BSD Licensed • C/C++/Python/Java interfaces • Windows/Linux/Mac/iOS/Android libraries • http://opencv.org/ 6 Monday 9 September 13

OpenCV • 2D and 3D feature toolkits • Egomotion estimation • Gesture recognition • Human–computer interaction (HCI) • Mobile robotics • Motion understanding • Object identification • Segmentation and Recognition • Stereopsis Stereo vision: depth perception from 2 cameras • Structure from motion (SFM) • Motion tracking • Augmented reality 7 Monday 9 September 13

OpenCV • Object Identification • Segmentation and Recognition We’re only interested in two abilities today 8 Monday 9 September 13

Object Identification • Need to create a Haar Classifier • A few default classifications available (different face features, full face, bodies etc.) • Make your own: http://coding-robin.de/ 2013/07/22/train-your-own-opencv-haar- classifier.html 9 Monday 9 September 13

Face Recognition Algorithms • Eigenfaces • Fisherfaces • Local Binary Patterns Histogram http://docs.opencv.org/modules/contrib/doc/facerec/ facerec_tutorial.html 10 Monday 9 September 13

Eignfaces vs Fisherfaces http://docs.opencv.org/modules/contrib/doc/facerec/ facerec_tutorial.html#local-binary-patterns-histograms 11 Monday 9 September 13

Identifying A Face • Pull image from somewhere (video stream or just a still image) • Check for a face 12 Monday 9 September 13

Identifying A Face • Normalise the face to 100x100 greyscale image • Feed to face recogniser model • Hopefully get a match back (with a confidence level) 13 Monday 9 September 13

(Simplified) Code Sample cv::Mat cvImage; UIImageToMat(uiimage, cvImage, NO); cv::CascadeClassifier faceCascade; faceCascade.load(); std::vector faces; faceCascade.detectMultiScale(image, faces, 1.1, 2, CV_HAAR_FIND_BIGGEST_OBJECT | CV_HAAR_DO_ROUGH_SEARCH, cv::Size(60, 60)); cv::Ptr _model = cv::createEigenFaceRecognizer(); _model->train(images, labels); for each face { int predictedLabel = -1; double confidence = 0.0; cv::Mat onlyTheFace; cv::cvtColor(cvImage(face), onlyTheFace, CV_RGB2GRAY); cv::resize(onlyTheFace, onlyTheFace, cv::Size(100, 100), 0, 0); _model->predict(onlyTheFace, predictedLabel, confidence); } 14 Monday 9 September 13

Demo 15 Monday 9 September 13

Native Face Detection With OpenCV Face Recognition 16 Monday 9 September 13

Native iOS Methods • CIDetector - Face detection in still images available since iOS 5 • AVCaptureMetaDataOutput - added in iOS 6 for real-time detection 17 Monday 9 September 13

AVCaptureMetadataOutput • Optimised to use GPU • Produces an AVMetadataFaceObject • Face bounds • Yaw • Roll • Tracking faces over time • 4s/iPad 2 and up 18 Monday 9 September 13

CIDetector • Can use images from anywhere • Face bounds • Left eye • Right eye • Mouth position • Track faces between frames 19 Monday 9 September 13

Face Recognition • Deliberately no available native face recognition so still need OpenCV (the face tracking ability implies to me some sort of native recognition) • Exact same process as just using OpenCV 20 Monday 9 September 13

(Simplified) Code Sample NSDictionary *detectorOptions = @{CIDetectorAccuracy : CIDetectorAccuracyLow, CIDetectorTracking : @YES}; ! self.faceDetector = [CIDetector detectorOfType:CIDetectorTypeFace context:nil options:detectorOptions]; NSDictionary *imageOptions = @{CIDetectorImageOrientation: imageOrientation}; NSArray *features = [self.faceDetector featuresInImage:ciImage options:imageOptions]; cv::Ptr _model = cv::createEigenFaceRecognizer(); _model->train(images, labels); for each face { int predictedLabel = -1; double confidence = 0.0; cv::Mat onlyTheFace; cv::cvtColor(cvImage(face.bounds), onlyTheFace, CV_RGB2GRAY); cv::resize(onlyTheFace, onlyTheFace, cv::Size(100, 100), 0, 0); _model->predict(onlyTheFace, predictedLabel, confidence); } 21 Monday 9 September 13

Demo 22 Monday 9 September 13

Issues • Need a selection of photos of the same face • Needs to load all known faces before attempting to recognise • Small sample size increases false positives 23 Monday 9 September 13

Issues • Front camera produces poor results • Lighting conditions can have big impact 24 Monday 9 September 13

Alternatives to on device detection • http://www.rekognition.com/ • https://www.bioid.com/solutions/solutions- by-application/bioid-for-facedotcom.html • http://api.animetrics.com • http://www.identitykit.it/ • http://www.skybiometry.com/ 25 Monday 9 September 13

Pros/Cons • Can often do age/gender/glasses/smile detection as well • All have a cost associated • If realtime not an issue can produce better results 26 Monday 9 September 13

Sample Code https://github.com/kmonaghan/FaceRecognition git submodule update --init --recursive 27 Monday 9 September 13

Questions? 28 Monday 9 September 13

@karlmonaghan http://karlmonaghan.com http://github.com/kmonaghan 29 Monday 9 September 13