Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Leveraging Data Science to Advance the Scientific Study of Smiling
Search
Jeffrey M Girard
January 28, 2020
Science
1
55
Leveraging Data Science to Advance the Scientific Study of Smiling
Brownbag (Chalk) Talk Slides from Jeffrey M. Girard, PhD (Carnegie Mellon University)
Jeffrey M Girard
January 28, 2020
Tweet
Share
More Decks by Jeffrey M Girard
See All by Jeffrey M Girard
Large-scale observational research
jmgirard
2
62
Interdisciplinary Frontiers in Affective Health & Communication
jmgirard
1
36
Data Visualization Principles and Practice
jmgirard
1
260
Other Decks in Science
See All in Science
AI(人工知能)の過去・現在・未来 —AIは人間を超えるのか—
tagtag
0
250
AI(人工知能)の過去・現在・未来 —AIは人間を超えるのか—
tagtag
0
110
マルチモーダルモデルと自動運転 車載モデルのコスト・スループット・レイテンシ / LLM in Production Meetup #2 20231023
yuyamaguchi
1
1k
Spark_Task_Optimization_Journey_How_I_Increased_10x_Speed_by_Performance_Tuning
tlyu0419
0
190
Machine Learning for Materials (Lecture 6)
aronwalsh
0
410
論文を批判的に読むときのチェックリスト
koro485
1
21k
【論文紹介】DocTr_ Document Transformer for Structured Information Extraction in Documents / iccv2023-doctr
yuya4
3
530
(neuro)science with AI: Machine learning as scientific modeling
gaelvaroquaux
0
680
HAS Dark Site Orientation
astronomyhouston
0
4.8k
iRIC v4 Solvers
nkmr_rl
0
3k
History towards Universal Neural Network Potential for Material Discovery
matlantis
0
120
早わかり W3C Community Group
takanorip
0
250
Featured
See All Featured
Web development in the modern age
philhawksworth
201
10k
Rails Girls Zürich Keynote
gr2m
91
13k
Creatively Recalculating Your Daily Design Routine
revolveconf
209
11k
Designing the Hi-DPI Web
ddemaree
275
33k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
24
2.2k
JazzCon 2018 Closing Keynote - Leadership for the Reluctant Leader
reverentgeek
178
11k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
501
140k
What’s in a name? Adding method to the madness
productmarketing
PRO
14
2.6k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
18
6.8k
Done Done
chrislema
178
15k
Become a Pro
speakerdeck
PRO
8
4.4k
Transcript
Leveraging Data Science to Advance the Scientific Study of Smiling
bit.ly/girard2020b
Data Science in Psychology • • • • • •
• • • •
Data Science in Psychology Solutions (Hammers) • • • •
• • • • • • • • • Challenges (Nails) • • • • • • • • • • • • •
Presentation Roadmap
Study 1: Web-Scraped Celebrity Smiles
Web-Scraped Celebrity Smiles: Design • • • • • •
Web-Scraped Celebrity Smiles: Collect/Process http://bit.ly/celebs_by_nation
Web-Scraped Celebrity Smiles: Collect/Process
Web-Scraped Celebrity Smiles: Process
Web-Scraped Celebrity Smiles: Validate Measure Algorithm Positive Smile Positive Rating
0.79 Smile Rating 0.78 0.94 Expert FACS 0.87 0.97 0.94 Measure ICC(A,5) 95% CI Positive Rating 0.90 [0.88, 0.92] Smile Rating 0.90 [0.88, 0.92]
Web-Scraped Celebrity Smiles: Model • • • • • •
• • Zero-Inflated Beta Regression ZI Beta = 0 ∈ (0, 1)
Web-Scraped Celebrity Smiles: Model/Visualize
Web-Scraped Celebrity Smiles: Visualize
Study 2: Spontaneous Emotion Database
Spontaneous Emotion Database: Design • • • •
Spontaneous Emotion Database: Design • • • •
Spontaneous Emotion Database: Design/Collect Emotion Rating Scale Afraid/Scared 0 1
2 3 4 5 Angry/Upset 0 1 2 3 4 5 Disgusted 0 1 2 3 4 5 … … Other (write in) 0 1 2 3 4 5
Spontaneous Emotion Database: Collect/Visualize
Spontaneous Emotion Database: Validate/Visualize
Spontaneous Emotion Database: Collect AU 1 + 10 + 12
+ 15 + 17 + 64 Action Unit Occurrence Did the action unit happen? {Absent = 0, 1 = Present} Action Unit Intensity How strong was the action unit? {Trace = 1, 2, 3, 4, 5 = Maximum}
Spontaneous Emotion Database: Collect/Process AU12 AU12 AU06 2 2 3
4 4 4 2 4 2 2 2 2 2 2 3 3 3 3 3 0 0 1 1 0 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 Action Unit Occurrence: {0=Absent, 1=Present} Action Unit Intensity: {1=Trace, 2=Slight, 3=Pronounced, 4=Extreme, 5=Maximum}
Spontaneous Emotion Database: Collect
Spontaneous Emotion Database: Validate agreement github.com/jmgirard/agreement mreliability.jmgirard.com 0 1 1
1 0 0 1 1 0 1 0 1
Study 3: Algorithmic Smile Interpretation
Algorithmic Smile Interpretation: Design • • • •
Algorithmic Smile Interpretation: Process Example Smile Event Video Clips Fr
AU12 AU6 1 3 0 2 3 3 3 4 2 4 0 0 5 2 0 6 2 0 Frs AU12i AU6o AU6i 1 to 3 4 1 3 5 to 6 2 0 0
Algorithmic Smile Interpretation: Model Smile Appearance Reported Emotion Algorithm
Algorithmic Smile Interpretation: Model Support Vector Machine Random Forest Multilayer
Perceptron
Algorithmic Smile Interpretation: Validate
Algorithmic Smile Interpretation: Visualize
Discussion Questions • • •