Pro Yearly is on sale from $80 to $50! »

OpenTalks.AI - Ольга Перепелкина, Human-centered AI: распознавание эмоций и физиологических сигналов по видео

Ad8ae7af280edaecb09bd73a551b5e5f?s=47 OpenTalks.AI
February 21, 2020

OpenTalks.AI - Ольга Перепелкина, Human-centered AI: распознавание эмоций и физиологических сигналов по видео

Ad8ae7af280edaecb09bd73a551b5e5f?s=128

OpenTalks.AI

February 21, 2020
Tweet

Transcript

  1. !1 HUMAN-CENTERED AI 2020 Olga Perepelkina, COO & Scientific Director

  2. !2 ONE BILLION $ QUESTIONS What do people really think

    and feel? How do people make decisions, for example, about purchases? What do people want?
  3. !3

  4. !4 IMAGES ID Sex Age Emotions?

  5. «READING» THE EMOTIONS

  6. SOFTWARE • Affectiva: Affdex • CrowdEmotion: FaceVideo • Emotient: Facet

    • Microsoft: Cognitive Services • MorphCast: EmotionalTracking • Neurodatalab: EmotionRecognition (facial NN) • VicarVision: FaceReader • VisageTechnologies: FaceAnalysis DATABASES • BU-4DFE: 468 videos, posed • UT-Dallas: 470 videos, spontaneous https://psyarxiv.com/kzhds/
  7. Classifiers VisageTechnologies (FaceAnalysis) VicarVision (FaceReader) Neurodatalab (EmotionRecognition) MorphCast (EmotionalTracking) Microsoft

    (Cognitive Services) Emotient (Facet) CrowdEmotion (Face Video) Affectiva (Affdex) Human Observers Accuracy, % 0 25 50 75
  8. Classifiers VisageTechnologies (FaceAnalysis) VicarVision (FaceReader) Neurodatalab (EmotionRecognition) MorphCast (EmotionalTracking) Microsoft

    (Cognitive Services) Emotient (Facet) CrowdEmotion (Face Video) Affectiva (Affdex) Human Observers Accuracy, % 0 25 50 75
  9. «READING» THE EMOTIONS ?

  10. !10 WIN OR LOSE? Aviezer, H., Trope, Y. and Todorov,

    A. 2012
  11. !11 LOSE WIN OR LOSE? LOSE WIN LOSE WIN WIN

    Aviezer, H., Trope, Y. and Todorov, A. 2012
  12. !12 WIN OR LOSE? Aviezer, H., Trope, Y. and Todorov,

    A. 2012
  13. !13 WIN OR LOSE? Aviezer, H., Trope, Y. and Todorov,

    A. 2012 LOSE WIN
  14. !14 EMOTION != FACIAL EXPRESSION
 OTHER MODALITIES NEED TO BE

    CONSIDERED
  15. !15 VIDEOS ID Sex Age Emotions

  16. MULTIMODAL APPROACH !16

  17. !17 A: audio, V: video, T: text, B: body 7

    classes: Happy, Sad, Angry, Surprised, Anxious, Disgusted, Neutral
  18. !18 VIDEOS ID Sex Age Emotions What else?

  19. BEYOND 
 HUMAN PERCEPTION… !19

  20. Lorem ipsum dolor sit amet consectetur !20 1

  21. Lorem ipsum dolor sit amet consectetur !21 2

  22. !22 VIDEOS ID Sex Age Emotions Heart rate

  23. !23

  24. !24 Video frames Diff frames 3D Attention CNN Heart Rate

  25. !25

  26. !26 Video frames Diff frames 3D Attention CNN Heart Rate

    Pretrain on synthetic data
  27. !27 MAE: 6.71 RMSE: 10.63 MAE: 4.93 RMSE: 9.41

  28. !28 Wearable MAE (Device) MAE (NDL) Honor Band 4 8.69

    0.97 Amazfit Bip 3.51 3.92 Xiaomi Mi Band 3 5.07 3.71 Apple Watch 2 3.81 2.61 Garmin 2.03 2.46 Samsung Gear S3 2.03 3.28 Mean ± SD 3.6 ± 4.9 2.4 ± 4.5 Device - heart rate detected by wearables NDL - heart rate detected from video by Neurodata Lab
  29. !29 VIDEOS ID Sex Age Emotions Heart rate What else?

  30. !30

  31. Lorem ipsum dolor sit amet consectetur !31

  32. !32 MAE: 0.7 ECG-based (contact) Video-based (contactless) Breathing Rate

  33. !33 1 2

  34. !34 VIDEOS ID Sex Age Emotions Heart rate Breathing rate

    Eye Tracking
  35. !35

  36. !36 APPLICATIONS

  37. !37 Male 33 years

  38. !38 Male 33 years VALUES Offline targeting Higher conversion +5-125%

    Test faster & find out what works best Better optimization
  39. None
  40. None
  41. !41 https://api.neurodatalab.dev/

  42. !42 THANK YOU 2020 Olga Perepelkina o.perepelkina@neurodatala