Leveraging Data Science to Advance the Scientific Study of Smiling

Brownbag (Chalk) Talk Slides from Jeffrey M. Girard, PhD (Carnegie Mellon University)

Transcript

1. Leveraging Data Science to Advance the Scientific Study of Smiling bit.ly/girard2020b
2. Data Science in Psychology • • • • • • • • • •
3. Data Science in Psychology Solutions (Hammers) • • • • • • • • • • • • • Challenges (Nails) • • • • • • • • • • • • •
4. Presentation Roadmap
5. Study 1: Web-Scraped Celebrity Smiles
6. Web-Scraped Celebrity Smiles: Design • • • • • •
7. Web-Scraped Celebrity Smiles: Collect/Process http://bit.ly/celebs_by_nation
8. Web-Scraped Celebrity Smiles: Collect/Process
9. Web-Scraped Celebrity Smiles: Process
10. Web-Scraped Celebrity Smiles: Validate Measure Algorithm Positive Smile Positive Rating 0.79 Smile Rating 0.78 0.94 Expert FACS 0.87 0.97 0.94 Measure ICC(A,5) 95% CI Positive Rating 0.90 [0.88, 0.92] Smile Rating 0.90 [0.88, 0.92]
11. Web-Scraped Celebrity Smiles: Model • • • • • • • • Zero-Inflated Beta Regression ZI Beta = 0 ∈ (0, 1)
12. Web-Scraped Celebrity Smiles: Model/Visualize
13. Web-Scraped Celebrity Smiles: Visualize
14. Study 2: Spontaneous Emotion Database
15. Spontaneous Emotion Database: Design • • • •
16. Spontaneous Emotion Database: Design • • • •
17. Spontaneous Emotion Database: Design/Collect Emotion Rating Scale Afraid/Scared 0 1 2 3 4 5 Angry/Upset 0 1 2 3 4 5 Disgusted 0 1 2 3 4 5 … … Other (write in) 0 1 2 3 4 5
18. Spontaneous Emotion Database: Collect/Visualize
19. Spontaneous Emotion Database: Validate/Visualize
20. Spontaneous Emotion Database: Collect AU 1 + 10 + 12 + 15 + 17 + 64 Action Unit Occurrence Did the action unit happen? {Absent = 0, 1 = Present} Action Unit Intensity How strong was the action unit? {Trace = 1, 2, 3, 4, 5 = Maximum}
21. Spontaneous Emotion Database: Collect/Process AU12 AU12 AU06 2 2 3 4 4 4 2 4 2 2 2 2 2 2 3 3 3 3 3 0 0 1 1 0 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 Action Unit Occurrence: {0=Absent, 1=Present} Action Unit Intensity: {1=Trace, 2=Slight, 3=Pronounced, 4=Extreme, 5=Maximum}
22. Spontaneous Emotion Database: Collect
23. Spontaneous Emotion Database: Validate agreement github.com/jmgirard/agreement mreliability.jmgirard.com 0 1 1 1 0 0 1 1 0 1 0 1
24. Study 3: Algorithmic Smile Interpretation
25. Algorithmic Smile Interpretation: Design • • • •
26. Algorithmic Smile Interpretation: Process Example Smile Event Video Clips Fr AU12 AU6 1 3 0 2 3 3 3 4 2 4 0 0 5 2 0 6 2 0 Frs AU12i AU6o AU6i 1 to 3 4 1 3 5 to 6 2 0 0
27. Algorithmic Smile Interpretation: Model Smile Appearance Reported Emotion Algorithm
28. Algorithmic Smile Interpretation: Model Support Vector Machine Random Forest Multilayer Perceptron
29. Algorithmic Smile Interpretation: Validate
30. Algorithmic Smile Interpretation: Visualize
31. Discussion Questions • • •