Save 37% off PRO during our Black Friday Sale! »

Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8

Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8

The ability to detect, track, and analyze faces opens up a wide range of interesting use cases, ranging from interactive smart applications and real-time video processing, all the way to biometric security and augmented reality.

This talk will showcase the available tools built by the Python community and their corresponding pros & cons, limitations, and complexity. While discussing the possible scenarios and what is actually required to DIY with Python, I will compare such handmade solutions with Cloud-based products and APIs.


Alex Casalboni

April 09, 2017


  1.­‐facial-­‐analysis Facial  Analysis  Techniques  for  Pythonista (and  beyond!) 4/9/2017 PYCON

     OTTO   @  Florence
  2. About  Me @alex_casalboni­‐facial-­‐analysis Computer  Science  Background Master  in  Sound

     &  Music  Engineering Sr.  SoDware  Engineer  &  Web  Developer Cloud  Evangelist  @  Cloud  Academy
  3. Agenda What  is  a  Face? Problem  decomposiLon AlternaLves  to  DIY­‐facial-­‐analysis
  4. What  is  a  Face? You  thought  you  knew,  but  you

     didn’t…  ?­‐facial-­‐analysis
  5. About  “Normal”  Faces­‐facial-­‐analysis

  6. What  about  rotaCon  invariance?­‐facial-­‐analysis

  7. What  about  animals?­‐facial-­‐analysis

  8. What  about  painCngs?­‐facial-­‐analysis

  9. What  about  masks?­‐facial-­‐analysis

  10. What  about  smilies?­‐facial-­‐analysis

  11. Problem  decomposiLon What  are  the  main  building  blocks  for  facial

  12. 1.  Face  DetecLon­‐facial-­‐analysis

  13. Face  DetecCon­‐facial-­‐analysis Input:  1  image  &  unknown  context Goal:

     finding  faces  (how  many?) Output:  lists  of  coordinates Difficulty:  preUy  easy
  14. Face  DetecCon  Results­‐facial-­‐analysis

  15. Face  DetecCon  Techniques­‐facial-­‐analysis Algorithmical  techniques Not  too  much  “intelligence”

    Real-­‐Lme  tracking
  16. Face  DetecCon  Techniques  -­‐  HOG­‐facial-­‐analysis Histogram  of  Oriented  Gradients

  17. HOG  w/  OpenCV  and  dlib­‐facial-­‐analysis *  Vectors  allow  for

     more  advanced  analysis  (see   hUp:/ / emoLon-­‐recogniLon-­‐using-­‐facial-­‐landmarks/) *  That  .dat  file  is  100+MB
  18. Face  DetecCon  Techniques  -­‐  Haar  Cascades­‐facial-­‐analysis Haar  Feature-­‐based  Cascade

  19. Haar  Cascades  w/  OpenCV­‐facial-­‐analysis


  21. 2.  Face  RecogniLon­‐facial-­‐analysis

  22. Face  RecogniCon­‐facial-­‐analysis Input:  1  reference  and  1  target  image

    Goal:  finding  facial  matches Output:  lists  of  (potenLal)  matches Difficulty:  medium
  23. Facial  Encoding­‐facial-­‐analysis Vector  RepresentaLon  (128D)  * *  could  be

     learned  with  DL
  24. Facial  Distance­‐facial-­‐analysis A  and  B  are  the  same  person

      if  distance(A,  B)  <  tolerance
  25. Face  Matching  w/  face_recogni2on­‐facial-­‐analysis

  26. 3.  Facial  Analysis­‐facial-­‐analysis

  27. Facial  Analysis­‐facial-­‐analysis Input:  1  detected  face Goal:  extracLng  high-­‐level

     informaLon Output:  gender,  age,  emoLons,  headwear,  etc. Difficulty:  preUy  hard
  28. Facial  Analysis­‐facial-­‐analysis ML  Model  (gender) ML  Model  (emoLons) ML

     Model  (….) ML  Model  (age) ML  Model  (headwear)
  29. Facial  Analysis­‐facial-­‐analysis How  many  training  sets? Parallel  features  extracLon

     &  predicLon Accuracy  is  more  subjecLve  (source/target  audience) Real-­‐Lme  is  not  guaranteed
  30. AlternaLves  to  DIY How  about  Facial  Analysis  services?­‐facial-­‐analysis

  31. Facial  Analysis  Services Amazon  RekogniLon Google  Cloud  Vision Azure  Face

     API Face++   Kairos EmoVu­‐facial-­‐analysis
  32. Amazon  RekogniCon  &  Python­‐facial-­‐analysis

  33. Google  Cloud  Vision  &  Python­‐facial-­‐analysis

  34. Azure  Face  API  &  Python­‐facial-­‐analysis


  36.­‐facial-­‐analysis Cloud  Services  Pros Language  agnosLc  (RESTful  APIs) Models  are

     updated  under  the  hood No  infrastructure  to  manage PAYG  model  (w/  free  Ler) Great  for  embedded  systems Granted  accuracy  (globally)
  37.­‐facial-­‐analysis Cloud  Services  Cons Hardly  real-­‐Lme  (HTTPs  calls) ConnecLvity  is

     always  needed Training  set  is  never  customizable ML  Models  are  a  black  box
  38. Thank  you  :) P.S.  we  are  hiring! PYCON  OTTO

      @  Florence