Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Understanding Faces

Be1c8a24b76f8b2b23f53eb22d401810?s=47 Imperial ACM
November 29, 2013

Understanding Faces

Powered by ever more sophisticated sensors, the machines of the future will interact with humans far more naturally than is possible today. Most of us find face-to-face interaction the most comfortable and for good reason - the brain does an outstanding job of interpreting the identity and emotional state of another human - all from just looking at their face.

With this in mind we seek to develop algorithmic approaches to understanding the human face. I’ll explain how powerful generative models of the face can be constructed, and what we can learn from them. I’ll then demonstrate the usefulness of such models in identity and emotion recognition, and highlight how our collaboration with Great Ormond Street Hospital is helping advance craniofacial surgery techniques. Finally, I’ll talk about where we can take statistical facial modelling in the future, and discuss some of the challenges that we most overcome in order to advance.

Be1c8a24b76f8b2b23f53eb22d401810?s=128

Imperial ACM

November 29, 2013
Tweet

Transcript

  1. Understanding Faces James Booth

  2. >>> ./facedetector done. running..

  3. >>> ./facedetector done. running..

  4. None
  5. >>> ./emotiondetection runnning…

  6. >>> ./emotiondetection runnning… ERROR: EmotionError - No hard edges found.

    Is the image of sufficient resolution? ah! we need to enhance obviously.
  7. >>> enhance —csi=‘miami’ :-) arbitrarily enhancing… done.

  8. >>> ./emotiondetection detecting emotion… done. FACE IS HAPPY :-)

  9. Understanding Faces • What does iBUG do? • What key

    techniques do we use? • What are the applications of our work? • What challenges lie in the future?
  10. About me • 2008-11 BSc Physics, Imperial • 2011-12 MSc

    Computing Science, Imperial • 2012-13 RA, iBUG, Imperial • 2013-present PhD candidiate under Stefanos Zafeiriou, iBUG, Imperial
  11. iBUG • Intelligent Behaviour Understanding Group • anything to do

    with machines understanding humans • machine learning • Group roughly split in two • behavioural • computer vision (faces)
  12. iBUG - Computer Vision • vision problems fall into two

    categories REAL TIME OFFLINE
  13. Morphable Models video OFFLINE

  14. 3D Morphable Models • Built from a statistical model of

    the face • Learning a detailed 3D face from a 2D face. • Generally not real time, quality is key, not speed • Used in games, CGI effects in movies • Local optimisation method - needs to be pre aligned (probably by a tracker) • My main focus! OFFLINE
  15. Facial Tracker demo REAL TIME

  16. Facial tracker • Built with a statistical model of the

    face • Expected to be real time • Robust to occlusions (stuff getting in the way) • May be able to deal with multiple persons • Often required to bootstrap of more vision algorithms • Consumer applications already here (Kinect) REAL TIME
  17. PCA PCA 3D scans images Morphable Model Shape Model Lucas!

    Kanade Cascade of! regressors … … Morphable Model Face Tracker
  18. Principal Component Analysis • The bedrock of the CV community

    • Basic approach extremely simple! • Many exotic derivatives for incremental improvements. • Going to explain the basics
  19. Principal Components Analysis x y 100 80

  20. Principal Components Analysis x y μ 100 80

  21. Principal Components Analysis x y μ 100 80

  22. Principal Components Analysis x y μ PC1 100 80 variance

    maximised this is the First " Principal Component
  23. Principal Components Analysis x y μ PC1 PC2 100 80

    variance maximised this is the First " Principal Component
  24. Principal Component Analysis • Technique to find linear trends in

    data • Principal Components are orthogonal to each other and are ordered in terms of the variance they capture • Can be applied in N dimensions
  25. Principal Components as a basis x y 100 80

  26. Principal Components as a basis x y 100 80 =

    71x + 62y 71 62
  27. Principal Components as a basis x y μ PC1 PC2

    100 80 = 71x + 62y 71 62
  28. Principal Components as a basis x y μ PC1 PC2

    100 80 = 71x + 62y = μ + 0.8PC1 - 0.9PC2 0.8 -0.9
  29. Principal Components as a basis x y μ PC1 PC2

    100 80 = 71x + 62y = μ + 0.8PC1 - 0.9PC2 = 91x + 32y = μ + 0.8PC1 - 3.0PC2
  30. Principal Components as a basis x y μ PC1 PC2

    100 80 = 71x + 62y = μ + 0.8PC1 - 0.9PC2 = 91x + 32y ! = μ + 0.8PC1 - 3.0PC2 > 1.0 == outlier
  31. Shape Model demo

  32. What is shape? • What remains after affects of scale,

    rotation and translation have been removed
  33. What is shape? • What remains after affects of scale,

    rotation and translation have been removed
  34. What is shape? • What remains after affects of scale,

    rotation and translation have been removed
  35. What is shape? • What remains after affects of scale,

    rotation and translation have been removed
  36. Vectorizing shapes • Just assign each coordinate to a new

    vector axis • As long as we are consistent in the mapping, can always go from vector back to shape • This is why landmarks were numbered • Can be done for arbitrary data Face landmarks (68 points of 2D) 136D vector space landmark 1:x axis 1 landmark 1:y axis 2 landmark 2:x axis 3 … … landmark 68:y axis 136 Take PCA on this!
  37. Statistical Shape Models • Summary - Each component captures key

    traits in face • Gives an ideal for how ‘normal’ a face is from the component weightings • Works well because faces have a very similar structure! • Same idea applied to pixels yields texture models
  38. Generating novel faces • By creating linear combinations of components

    we generate novel faces • Very powerful. Normally first 30 components are sufficient to generate a close approximation to anyones face • We just need an algorithm to find the weightings..
  39. Fitting algorithms • Complex beasts! Outside the scope of today

    • Generally find component weightings that minimise an error function • Lots of active research in this area
  40. Great Ormond Street • Collaboration with craniofacial surgeons there •

    Current surgical approach is an ‘art’ • Want to see if our models can help improve surgery techniques • Requires answers to difficult questions - what is ‘normal’? What is ‘beautiful’?
  41. My PhD goal • Emotive Morphable Models • Current techniques

    can show full range of emotion • Needs lots of data and new techniques • iBUG solving data problem..
  42. iBUG4D • 4D (3D at 60fps) capture setup • Capturing

    data from a range of subjects right now! • Small financial reward, only takes 20mins • Get a cool 3D scan of yourself • Email Teresa if you fancy it t.ng@imperial.ac.uk
  43. Thoughts on PhD • Founded a team project in iBUG

    - PyBug - a Python toolkit for statistical modelling and analysis • Working in a team transforms your PhD - seek collaboration where you can! • Important to raise engineering standards in Computing research, keen to play a role • Imperial Python Users Group…?
  44. ? t.ng@imperial.ac.uk For capture experiment: