Leveraging Data Science to Advance the Scientific Study of Smiling

Transcript

1. Leveraging Data Science to Advance the Scientific Study of Smiling

   bit.ly/girard2020b

2. Data Science in Psychology • • • • • •

   • • • •

3. Data Science in Psychology Solutions (Hammers) • • • •

   • • • • • • • • • Challenges (Nails) • • • • • • • • • • • • •

4. Presentation Roadmap

5. Study 1: Web-Scraped Celebrity Smiles

6. Web-Scraped Celebrity Smiles: Design • • • • • •

7. Web-Scraped Celebrity Smiles: Collect/Process http://bit.ly/celebs_by_nation

8. Web-Scraped Celebrity Smiles: Collect/Process

9. Web-Scraped Celebrity Smiles: Process

10. Web-Scraped Celebrity Smiles: Validate Measure Algorithm Positive Smile Positive Rating

    0.79 Smile Rating 0.78 0.94 Expert FACS 0.87 0.97 0.94 Measure ICC(A,5) 95% CI Positive Rating 0.90 [0.88, 0.92] Smile Rating 0.90 [0.88, 0.92]

11. Web-Scraped Celebrity Smiles: Model • • • • • •

    • • Zero-Inflated Beta Regression ZI Beta = 0 ∈ (0, 1)

12. Web-Scraped Celebrity Smiles: Model/Visualize

13. Web-Scraped Celebrity Smiles: Visualize

14. Study 2: Spontaneous Emotion Database

15. Spontaneous Emotion Database: Design • • • •

16. Spontaneous Emotion Database: Design • • • •

17. Spontaneous Emotion Database: Design/Collect Emotion Rating Scale Afraid/Scared 0 1

    2 3 4 5 Angry/Upset 0 1 2 3 4 5 Disgusted 0 1 2 3 4 5 … … Other (write in) 0 1 2 3 4 5

18. Spontaneous Emotion Database: Collect/Visualize

19. Spontaneous Emotion Database: Validate/Visualize

20. Spontaneous Emotion Database: Collect AU 1 + 10 + 12

    + 15 + 17 + 64 Action Unit Occurrence Did the action unit happen? {Absent = 0, 1 = Present} Action Unit Intensity How strong was the action unit? {Trace = 1, 2, 3, 4, 5 = Maximum}

21. Spontaneous Emotion Database: Collect/Process AU12 AU12 AU06 2 2 3

    4 4 4 2 4 2 2 2 2 2 2 3 3 3 3 3 0 0 1 1 0 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 Action Unit Occurrence: {0=Absent, 1=Present} Action Unit Intensity: {1=Trace, 2=Slight, 3=Pronounced, 4=Extreme, 5=Maximum}

22. Spontaneous Emotion Database: Collect

23. Spontaneous Emotion Database: Validate agreement github.com/jmgirard/agreement mreliability.jmgirard.com 0 1 1

    1 0 0 1 1 0 1 0 1

24. Study 3: Algorithmic Smile Interpretation

25. Algorithmic Smile Interpretation: Design • • • •

26. Algorithmic Smile Interpretation: Process Example Smile Event Video Clips Fr

    AU12 AU6 1 3 0 2 3 3 3 4 2 4 0 0 5 2 0 6 2 0 Frs AU12i AU6o AU6i 1 to 3 4 1 3 5 to 6 2 0 0

27. Algorithmic Smile Interpretation: Model Smile Appearance Reported Emotion Algorithm

28. Algorithmic Smile Interpretation: Model Support Vector Machine Random Forest Multilayer

    Perceptron

29. Algorithmic Smile Interpretation: Validate

30. Algorithmic Smile Interpretation: Visualize

31. Discussion Questions • • •