Save 37% off PRO during our Black Friday Sale! »

Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8

Facial Analysis Techniques for Pythonista (and beyond!) - PyCon8

The ability to detect, track, and analyze faces opens up a wide range of interesting use cases, ranging from interactive smart applications and real-time video processing, all the way to biometric security and augmented reality.

This talk will showcase the available tools built by the Python community and their corresponding pros & cons, limitations, and complexity. While discussing the possible scenarios and what is actually required to DIY with Python, I will compare such handmade solutions with Cloud-based products and APIs.

36ca8ae7e19067021d0e39c0b72acc2a?s=128

Alex Casalboni

April 09, 2017
Tweet

Transcript

  1. clda.co/pycon8-­‐facial-­‐analysis Facial  Analysis  Techniques  for  Pythonista (and  beyond!) 4/9/2017 PYCON

     OTTO   @  Florence
  2. About  Me @alex_casalboni clda.co/pycon8-­‐facial-­‐analysis Computer  Science  Background Master  in  Sound

     &  Music  Engineering Sr.  SoDware  Engineer  &  Web  Developer Cloud  Evangelist  @  Cloud  Academy
  3. Agenda What  is  a  Face? Problem  decomposiLon AlternaLves  to  DIY

    clda.co/pycon8-­‐facial-­‐analysis
  4. What  is  a  Face? You  thought  you  knew,  but  you

     didn’t…  ? clda.co/pycon8-­‐facial-­‐analysis
  5. About  “Normal”  Faces clda.co/pycon8-­‐facial-­‐analysis

  6. What  about  rotaCon  invariance? clda.co/pycon8-­‐facial-­‐analysis

  7. What  about  animals? clda.co/pycon8-­‐facial-­‐analysis

  8. What  about  painCngs? clda.co/pycon8-­‐facial-­‐analysis

  9. What  about  masks? clda.co/pycon8-­‐facial-­‐analysis

  10. What  about  smilies? clda.co/pycon8-­‐facial-­‐analysis

  11. Problem  decomposiLon What  are  the  main  building  blocks  for  facial

     analysis? clda.co/pycon8-­‐facial-­‐analysis
  12. 1.  Face  DetecLon clda.co/pycon8-­‐facial-­‐analysis

  13. Face  DetecCon clda.co/pycon8-­‐facial-­‐analysis Input:  1  image  &  unknown  context Goal:

     finding  faces  (how  many?) Output:  lists  of  coordinates Difficulty:  preUy  easy
  14. Face  DetecCon  Results clda.co/pycon8-­‐facial-­‐analysis

  15. Face  DetecCon  Techniques clda.co/pycon8-­‐facial-­‐analysis Algorithmical  techniques Not  too  much  “intelligence”

    Real-­‐Lme  tracking
  16. Face  DetecCon  Techniques  -­‐  HOG clda.co/pycon8-­‐facial-­‐analysis Histogram  of  Oriented  Gradients

  17. HOG  w/  OpenCV  and  dlib clda.co/pycon8-­‐facial-­‐analysis *  Vectors  allow  for

     more  advanced  analysis  (see   hUp:/ /www.paulvangent.com/2016/08/05/ emoLon-­‐recogniLon-­‐using-­‐facial-­‐landmarks/) *  That  .dat  file  is  100+MB
  18. Face  DetecCon  Techniques  -­‐  Haar  Cascades clda.co/pycon8-­‐facial-­‐analysis Haar  Feature-­‐based  Cascade

     Classifiers
  19. Haar  Cascades  w/  OpenCV clda.co/pycon8-­‐facial-­‐analysis

  20. clda.co/pycon8-­‐facial-­‐analysis

  21. 2.  Face  RecogniLon clda.co/pycon8-­‐facial-­‐analysis

  22. Face  RecogniCon clda.co/pycon8-­‐facial-­‐analysis Input:  1  reference  and  1  target  image

    Goal:  finding  facial  matches Output:  lists  of  (potenLal)  matches Difficulty:  medium
  23. Facial  Encoding clda.co/pycon8-­‐facial-­‐analysis Vector  RepresentaLon  (128D)  * *  could  be

     learned  with  DL
  24. Facial  Distance clda.co/pycon8-­‐facial-­‐analysis A  and  B  are  the  same  person

      if  distance(A,  B)  <  tolerance
  25. Face  Matching  w/  face_recogni2on clda.co/pycon8-­‐facial-­‐analysis

  26. 3.  Facial  Analysis clda.co/pycon8-­‐facial-­‐analysis

  27. Facial  Analysis clda.co/pycon8-­‐facial-­‐analysis Input:  1  detected  face Goal:  extracLng  high-­‐level

     informaLon Output:  gender,  age,  emoLons,  headwear,  etc. Difficulty:  preUy  hard
  28. Facial  Analysis clda.co/pycon8-­‐facial-­‐analysis ML  Model  (gender) ML  Model  (emoLons) ML

     Model  (….) ML  Model  (age) ML  Model  (headwear)
  29. Facial  Analysis clda.co/pycon8-­‐facial-­‐analysis How  many  training  sets? Parallel  features  extracLon

     &  predicLon Accuracy  is  more  subjecLve  (source/target  audience) Real-­‐Lme  is  not  guaranteed
  30. AlternaLves  to  DIY How  about  Facial  Analysis  services? clda.co/pycon8-­‐facial-­‐analysis

  31. Facial  Analysis  Services Amazon  RekogniLon Google  Cloud  Vision Azure  Face

     API Face++   Kairos EmoVu clda.co/pycon8-­‐facial-­‐analysis
  32. Amazon  RekogniCon  &  Python clda.co/pycon8-­‐facial-­‐analysis

  33. Google  Cloud  Vision  &  Python clda.co/pycon8-­‐facial-­‐analysis

  34. Azure  Face  API  &  Python clda.co/pycon8-­‐facial-­‐analysis

  35. clda.co/pycon8-­‐facial-­‐analysis

  36. clda.co/pycon8-­‐facial-­‐analysis Cloud  Services  Pros Language  agnosLc  (RESTful  APIs) Models  are

     updated  under  the  hood No  infrastructure  to  manage PAYG  model  (w/  free  Ler) Great  for  embedded  systems Granted  accuracy  (globally)
  37. clda.co/pycon8-­‐facial-­‐analysis Cloud  Services  Cons Hardly  real-­‐Lme  (HTTPs  calls) ConnecLvity  is

     always  needed Training  set  is  never  customizable ML  Models  are  a  black  box
  38. Thank  you  :) jobs.cloudacademy.com P.S.  we  are  hiring! PYCON  OTTO

      @  Florence