Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
OpenTalks.AI - Ольга Перепелкина, Human-centere...
Search
OpenTalks.AI
February 21, 2020
Science
2.2k
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
OpenTalks.AI - Ольга Перепелкина, Human-centered AI: распознавание эмоций и физиологических сигналов по видео
OpenTalks.AI
February 21, 2020
Transcript
42 slides
!1 HUMAN-CENTERED AI 2020 Olga Perepelkina, COO & Scientific Director
!2 ONE BILLION $ QUESTIONS What do people really think
and feel? How do people make decisions, for example, about purchases? What do people want?
!3
!4 IMAGES ID Sex Age Emotions?
«READING» THE EMOTIONS
SOFTWARE • Affectiva: Affdex • CrowdEmotion: FaceVideo • Emotient: Facet
• Microsoft: Cognitive Services • MorphCast: EmotionalTracking • Neurodatalab: EmotionRecognition (facial NN) • VicarVision: FaceReader • VisageTechnologies: FaceAnalysis DATABASES • BU-4DFE: 468 videos, posed • UT-Dallas: 470 videos, spontaneous https://psyarxiv.com/kzhds/
Classifiers VisageTechnologies (FaceAnalysis) VicarVision (FaceReader) Neurodatalab (EmotionRecognition) MorphCast (EmotionalTracking) Microsoft
(Cognitive Services) Emotient (Facet) CrowdEmotion (Face Video) Affectiva (Affdex) Human Observers Accuracy, % 0 25 50 75
Classifiers VisageTechnologies (FaceAnalysis) VicarVision (FaceReader) Neurodatalab (EmotionRecognition) MorphCast (EmotionalTracking) Microsoft
(Cognitive Services) Emotient (Facet) CrowdEmotion (Face Video) Affectiva (Affdex) Human Observers Accuracy, % 0 25 50 75
«READING» THE EMOTIONS ?
!10 WIN OR LOSE? Aviezer, H., Trope, Y. and Todorov,
A. 2012
!11 LOSE WIN OR LOSE? LOSE WIN LOSE WIN WIN
Aviezer, H., Trope, Y. and Todorov, A. 2012
!12 WIN OR LOSE? Aviezer, H., Trope, Y. and Todorov,
A. 2012
!13 WIN OR LOSE? Aviezer, H., Trope, Y. and Todorov,
A. 2012 LOSE WIN
!14 EMOTION != FACIAL EXPRESSION OTHER MODALITIES NEED TO BE
CONSIDERED
!15 VIDEOS ID Sex Age Emotions
MULTIMODAL APPROACH !16
!17 A: audio, V: video, T: text, B: body 7
classes: Happy, Sad, Angry, Surprised, Anxious, Disgusted, Neutral
!18 VIDEOS ID Sex Age Emotions What else?
BEYOND HUMAN PERCEPTION… !19
Lorem ipsum dolor sit amet consectetur !20 1
Lorem ipsum dolor sit amet consectetur !21 2
!22 VIDEOS ID Sex Age Emotions Heart rate
!23
!24 Video frames Diff frames 3D Attention CNN Heart Rate
!25
!26 Video frames Diff frames 3D Attention CNN Heart Rate
Pretrain on synthetic data
!27 MAE: 6.71 RMSE: 10.63 MAE: 4.93 RMSE: 9.41
!28 Wearable MAE (Device) MAE (NDL) Honor Band 4 8.69
0.97 Amazfit Bip 3.51 3.92 Xiaomi Mi Band 3 5.07 3.71 Apple Watch 2 3.81 2.61 Garmin 2.03 2.46 Samsung Gear S3 2.03 3.28 Mean ± SD 3.6 ± 4.9 2.4 ± 4.5 Device - heart rate detected by wearables NDL - heart rate detected from video by Neurodata Lab
!29 VIDEOS ID Sex Age Emotions Heart rate What else?
!30
Lorem ipsum dolor sit amet consectetur !31
!32 MAE: 0.7 ECG-based (contact) Video-based (contactless) Breathing Rate
!33 1 2
!34 VIDEOS ID Sex Age Emotions Heart rate Breathing rate
Eye Tracking
!35
!36 APPLICATIONS
!37 Male 33 years
!38 Male 33 years VALUES Offline targeting Higher conversion +5-125%
Test faster & find out what works best Better optimization
None
None
!41 https://api.neurodatalab.dev/
!42 THANK YOU 2020 Olga Perepelkina o.perepelkina@neurodatala
More Decks by OpenTalks.AI
See All by OpenTalks.AI
OpenTalks.AI - Виктор Лемпицкий, Моделирование 3Д сцен: новые подходы в 2020 году
opentalks
0
510
OpenTalks.AI - Алексей Чернявский, Нейросетевые алгоритмы для повышения качества медицинских изображений
opentalks
0
460
OpenTalks.AI - Александр Громов, Устойчивость нейросетевых моделей при анализе КТ/НДКТ-исследований
opentalks
0
390
OpenTalks.AI - Денис Тимонин, Megatron-LM: Обучение мультимиллиардных LMs при помощи техники Model Parallelism
opentalks
0
550
OpenTalks.AI - Егор Филимонов, Возможности платформы Huawei Atlas и эффективный гетерогенный инференс.
opentalks
0
200
OpenTalks.AI - Александр Прозоров, Референсная архитектура робота сервисного центра в отраслях с изменчивыми бизнес-процессами
opentalks
0
410
OpenTalks.AI - Наталья Лукашевич, Анализ тональности по отношению к компании — с чем не справился BERT
opentalks
0
350
OpenTalks.AI - Константин Воронцов, Фейковые новости и другие типы потенциально опасного дискурса: типология, подходы, датасеты, соревнования
opentalks
0
470
OpenTalks.AI - Дмитрий Ветров, Фрактальность функции потерь, эффект двойного спуска и степенные законы в глубинном обучении - фрагменты одной мозаики
opentalks
0
510
Other Decks in Science
See All in Science
CVPR2026_VGGTとその仲間たち
mickey_0226
0
1.3k
How a camera trap data standard enabled an ecosystem of interoperable tools
peterdesmet
0
160
SAT ソルバーの仕組みと制約ソルバーへの応用
tsoh
3
790
白金鉱業Meetup Vol.25 【初学者向け発表枠】「で、この施策って効いてるの」に答える効果検証の基礎 ~ATE / ATT / CATE / LATEを現場の問いに翻訳する~
brainpadpr
1
460
機械学習 - SVM
trycycle
PRO
2
1.3k
自社のレビュー履歴からAIコードレビュアーをつくる方法
estie
5
2k
サンプル対応のない複数遺伝子発現プロファイルに対するテンソル分解型統合解析の要約
tagtag
PRO
0
290
機械学習 - K近傍法 & 機械学習のお作法
trycycle
PRO
1
1.7k
勾配流から乖離した学習ダイナミクス
hanbao
1
1.2k
プレイがつなぐ研究と社会:市⺠‧クリエイター‧SF作家による実践を通じて
hayataka88
1
140
Controller Design for Symbolic Input-Output Systems Using Feedforward Neural Networks
konakalab
0
160
データベース15: ビッグデータ時代のデータベース
trycycle
PRO
1
620
Featured
See All Featured
Bootstrapping a Software Product
garrettdimon
PRO
306
120k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
250
1.3M
The Pragmatic Product Professional
lauravandoore
37
7.5k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
6.2k
How to Get Subject Matter Experts Bought In and Actively Contributing to SEO & PR Initiatives.
livdayseo
1
240
Fireside Chat
paigeccino
43
4.1k
How To Speak Unicorn (iThemes Webinar)
marktimemedia
1
640
The State of eCommerce SEO: How to Win in Today's Products SERPs - #SEOweek
aleyda
2
12k
A designer walks into a library…
pauljervisheath
211
25k
Unlocking the hidden potential of vector embeddings in international SEO
frankvandijk
0
1k
StorybookのUI Testing Handbookを読んだ
zakiyama
31
7.1k
Are puppies a ranking factor?
jonoalderson
2
4.2k