VSS 2018

Hm...

Transcript

1. 1 A learned generative model of faces for experiments on human identity jordan suchow, josh peterson, & tom griffiths
2. Goal: Create a universal (and usable) generative model of human identity and appearance. Applications in: - Psychology & neuroscience - Computer vision & technology - Art & design
3. current models
4. A. The so-called “face space” and B. The renderer and C. The training data learned generative models of identity & appearance
5. Photorealistic images Not uncanny High resolution Quick to render Existing in a low-dimensional, smooth face space Diverse, representative…universal? desiderata
6. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
7. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
8. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
9. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
10. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
11. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
12. humanae
13. humanae
14. held constant Photographer Camera model Lens model Lens focal length Focal point Aperture Exposure time Distance to subject Camera placement Light design Facial expression Pose Background texture Ambient lighting Level of comfort / arousal Image resolution Digital format Color grading White balance Gamma correction Compression level & quality Watermarking (i.e., none) Digital alteration & touchups
15. varies Age Skin tone Place of origin Gender identity Hair style Profession Economic status Disability Injury & illness Body modification & cosmetic surgery Makeup Hair dying Tattooing
16. alignment
17. (Kazemi & Sullivan, CVPR 2014) alignment
18. (Kazemi & Sullivan, CVPR 2014) alignment
19. (Kazemi & Sullivan, CVPR 2014) alignment
20. (Kazemi & Sullivan, CVPR 2014) alignment
21. cropping
23. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
24. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
25. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
26. generative model of identity - Probability distribution over images - Deep neural networks as function approximators - Many different network architectures
27. variational autoencoder, &c. . Image Encoder Latent Variables Decoder PixelCNN layers OR Upsampler
28. variational autoencoder, &c. . Image Encoder Latent Variables Decoder PixelCNN layers OR Upsampler
29. variational autoencoder, &c. . Image Encoder Latent Variables Decoder PixelCNN layers OR Upsampler
30. variational autoencoder, &c. . Image Encoder Latent Variables Decoder PixelCNN layers OR Upsampler
31. variational autoencoder, &c. . Image Encoder Latent Variables Decoder PixelCNN layers OR Upsampler
32. variational autoencoder, &c. . Image Encoder Latent Variables Decoder PixelCNN layers OR Upsampler
33. variational autoencoder, &c. . Image Encoder Latent Variables Decoder PixelCNN layers OR Upsampler
34. variational autoencoder, &c. . Image Encoder Latent Variables Decoder PixelCNN layers OR Upsampler
35. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
36. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
37. 37 samples
38. test #1
39. test #2
40. test #3
41. test #4 A visual Turing test
42. humanae
43. current approach
44. dfc-vae
45. wpgan-gp
46. test #4
47. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
48. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
49. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
50. 1 2 3 4 5 6 7 8 9 10 Round Seed Samples “A five-year-old girl with long blonde hair” “A forty-year- woman with sho ⋮
51. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
52. 1. Find the perfect image set. 2. Train a model w/ an architecture well suited to human identity. 3. Explore model’s latent face space & generated images. 4. Test suitability with challenging experimental paradigms. plan of attack
53. A. The so-called “face space”: isotropic Gaussian and B. The renderer: variational autoencoder’s decoder and C. The training data: Humanæ conclusion
54. humanae.tumblr.com/ thanks to angélica dass
55. jordan suchow josh peterson tom griffiths @suchow @joshuacpeterson cocosci.berkeley.edu/tom thanks to: humanae.tumblr.com/