Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8

The ability to detect, track, and analyze faces opens up a wide range of interesting use cases, ranging from interactive smart applications and real-time video processing, all the way to biometric security and augmented reality.

This talk will showcase the available tools built by the Python community and their corresponding pros & cons, limitations, and complexity. While discussing the possible scenarios and what is actually required to DIY with Python, I will compare such handmade solutions with Cloud-based products and APIs.

Transcript

1. clda.co/pycon8-­‐facial-­‐analysis Facial  Analysis  Techniques  for  Pythonista (and  beyond!) 4/9/2017 PYCON  OTTO   @  Florence
2. About  Me @alex_casalboni clda.co/pycon8-­‐facial-­‐analysis Computer  Science  Background Master  in  Sound  &  Music  Engineering Sr.  SoDware  Engineer  &  Web  Developer Cloud  Evangelist  @  Cloud  Academy
3. Agenda What  is  a  Face? Problem  decomposiLon AlternaLves  to  DIY clda.co/pycon8-­‐facial-­‐analysis
4. What  is  a  Face? You  thought  you  knew,  but  you  didn’t…  ? clda.co/pycon8-­‐facial-­‐analysis
5. About  “Normal”  Faces clda.co/pycon8-­‐facial-­‐analysis
6. What  about  rotaCon  invariance? clda.co/pycon8-­‐facial-­‐analysis
7. What  about  animals? clda.co/pycon8-­‐facial-­‐analysis
8. What  about  painCngs? clda.co/pycon8-­‐facial-­‐analysis
9. What  about  masks? clda.co/pycon8-­‐facial-­‐analysis
10. What  about  smilies? clda.co/pycon8-­‐facial-­‐analysis
11. Problem  decomposiLon What  are  the  main  building  blocks  for  facial  analysis? clda.co/pycon8-­‐facial-­‐analysis
12. 1.  Face  DetecLon clda.co/pycon8-­‐facial-­‐analysis
13. Face  DetecCon clda.co/pycon8-­‐facial-­‐analysis Input:  1  image  &  unknown  context Goal:  finding  faces  (how  many?) Output:  lists  of  coordinates Difficulty:  preUy  easy
14. Face  DetecCon  Results clda.co/pycon8-­‐facial-­‐analysis
15. Face  DetecCon  Techniques clda.co/pycon8-­‐facial-­‐analysis Algorithmical  techniques Not  too  much  “intelligence” Real-­‐Lme  tracking
16. Face  DetecCon  Techniques  -­‐  HOG clda.co/pycon8-­‐facial-­‐analysis Histogram  of  Oriented  Gradients
17. HOG  w/  OpenCV  and  dlib clda.co/pycon8-­‐facial-­‐analysis *  Vectors  allow  for  more  advanced  analysis  (see   hUp:/ /www.paulvangent.com/2016/08/05/ emoLon-­‐recogniLon-­‐using-­‐facial-­‐landmarks/) *  That  .dat  file  is  100+MB
18. Face  DetecCon  Techniques  -­‐  Haar  Cascades clda.co/pycon8-­‐facial-­‐analysis Haar  Feature-­‐based  Cascade  Classifiers
19. Haar  Cascades  w/  OpenCV clda.co/pycon8-­‐facial-­‐analysis
20. clda.co/pycon8-­‐facial-­‐analysis
21. 2.  Face  RecogniLon clda.co/pycon8-­‐facial-­‐analysis
22. Face  RecogniCon clda.co/pycon8-­‐facial-­‐analysis Input:  1  reference  and  1  target  image Goal:  finding  facial  matches Output:  lists  of  (potenLal)  matches Difficulty:  medium
23. Facial  Encoding clda.co/pycon8-­‐facial-­‐analysis Vector  RepresentaLon  (128D)  * *  could  be  learned  with  DL
24. Facial  Distance clda.co/pycon8-­‐facial-­‐analysis A  and  B  are  the  same  person   if  distance(A,  B)  <  tolerance
25. Face  Matching  w/  face_recogni2on clda.co/pycon8-­‐facial-­‐analysis
26. 3.  Facial  Analysis clda.co/pycon8-­‐facial-­‐analysis
27. Facial  Analysis clda.co/pycon8-­‐facial-­‐analysis Input:  1  detected  face Goal:  extracLng  high-­‐level  informaLon Output:  gender,  age,  emoLons,  headwear,  etc. Difficulty:  preUy  hard
28. Facial  Analysis clda.co/pycon8-­‐facial-­‐analysis ML  Model  (gender) ML  Model  (emoLons) ML  Model  (….) ML  Model  (age) ML  Model  (headwear)
29. Facial  Analysis clda.co/pycon8-­‐facial-­‐analysis How  many  training  sets? Parallel  features  extracLon  &  predicLon Accuracy  is  more  subjecLve  (source/target  audience) Real-­‐Lme  is  not  guaranteed
30. AlternaLves  to  DIY How  about  Facial  Analysis  services? clda.co/pycon8-­‐facial-­‐analysis
31. Facial  Analysis  Services Amazon  RekogniLon Google  Cloud  Vision Azure  Face  API Face++   Kairos EmoVu clda.co/pycon8-­‐facial-­‐analysis
32. Amazon  RekogniCon  &  Python clda.co/pycon8-­‐facial-­‐analysis
33. Google  Cloud  Vision  &  Python clda.co/pycon8-­‐facial-­‐analysis
34. Azure  Face  API  &  Python clda.co/pycon8-­‐facial-­‐analysis
35. clda.co/pycon8-­‐facial-­‐analysis
36. clda.co/pycon8-­‐facial-­‐analysis Cloud  Services  Pros Language  agnosLc  (RESTful  APIs) Models  are  updated  under  the  hood No  infrastructure  to  manage PAYG  model  (w/  free  Ler) Great  for  embedded  systems Granted  accuracy  (globally)
37. clda.co/pycon8-­‐facial-­‐analysis Cloud  Services  Cons Hardly  real-­‐Lme  (HTTPs  calls) ConnecLvity  is  always  needed Training  set  is  never  customizable ML  Models  are  a  black  box
38. Thank  you  :) jobs.cloudacademy.com P.S.  we  are  hiring! PYCON  OTTO   @  Florence