The Humane Machine: Building AI systems with Emotional Intelligence

AI based systems have become more and more a part of our everyday lives in the last few years. We now interact routinely with AI based systems like Siri and Alexa. However most such systems today are designed to be functional with little attention paid to the emotional context of these interactions. This is the space of the emerging science of Affective Computing, sometimes referred to as "Emotional AI".

In this session we'll look at the state of the Artificial Emotional Intelligence space as well as the components of modern AEI systems especially in the domain of digital conversational agents. We'll also look at how humans react to such systems and conclude with the risks and rewards of building systems that are capable of understanding and simulating emotions.

Transcript

1. CONFIDENTIAL The Humane Machine Building AI systems with emotional intelligence Arpit Mathur
 Principal Engineer, Comcast Labs

2. https://en.wikipedia.org/wiki/File:NASA_Mars_Rover.jpg
3. “The fact Mars rovers last message before it went silent is “my battery is low and its getting dark” literally has me in tears like it probably died out there in mars alone and afraid hoping nasa would save it” Last message was, "my battery is low and its getting dark out”. Why tf am i crying in the club over a robot Farewell, tiny Martian robot car. One day you’ll be found by our descendants, a monument to our primitive 21st century technology. Y’all I’m tearing up over a robot help me. That “my battery is low and its getting dark” got me. You did great, sweetie. You did so great. Brb crying over a space robot @NASA #Oppy #MarsRover "My battery is low and it's getting dark" is perhaps the saddest thing I can imagine a little robot saying. And, thanks to my dumb human pack bonding instincts, I can think of robots like little pets so I'm crying. Goodnight, #Oppy
4. We are wired for emotional connection
5. Scenario • User is frustrated with a lag in the application/game he is playing. Do you • Ignore user’s state? • Let the user vent? • Acknowledge the user’s frustration?
6. Scenario • User is frustrated with a lag in the application/game he is playing. Do you • Ignore user’s state? • Let the user vent? • Acknowledge the user’s frustration? Source: 
 This computer responds to user frustration 
 J. Klein ; Y. Moon ; R.W. Picard
7. Affective Computing The study and development of systems and devices that can recognize, process, and simulate human affects
8. Sentiment Emotion Emotion vs Sentiment
9. Recognition
10. Modalities Text Voice Facial Expressions EEG Heart Rate Skin Conductance
11. Linguistic Inquiry and Word Count (LIWC) • Text Analysis module + Internal Dictionaries • Main dictionary composed of almost 6,400 words, word stems, and selected emoticons • Each dictionary entry maps to multiple categories • Categories are arranged hierarchically • 90 different output dimensions “I love this restaurant. Best sushi in town. And a BYOB for extra win! ” Source: http://liwc.wpengine.com
12. Face Emotion Recognition • DIY • Source publicly available image datasets ( Microsoft FER+ is a great fit ) • EmoPy or OpenCV or just Github search • APIs • Microsoft Azure, Amazon Recognition, Google Source: Microsoft FER+
13. Processing
14. Circumplex Model of Emotion
15. Plutchik’s wheel of Emotions
16. MFOEM Ontology An Emotion Ontology to describe affective phenomena, as a branch of the broader Mental Functioning ontology effort developed by The Swiss Centre for Affective Sciences, in collaboration with the University at Buffalo https://github.com/jannahastings/emotion-ontology
17. Emotion Understanding as a Service IBM Tone Analyzer (text) Microsoft (vision) Amazon Rekognition Emoshape EPU Voicery generation Anger TRUE TRUE TRUE TRUE TRUE Fear TRUE TRUE Joy / Happy TRUE TRUE TRUE TRUE TRUE Sadness TRUE TRUE TRUE TRUE TRUE Analytical TRUE Confident TRUE TRUE Tentative TRUE Surprized TRUE TRUE Calm TRUE TRUE Conversational Flirty Flustered Scared TRUE TRUE Disgust TRUE Indifference TRUE Regret TRUE Anticipation TRUE Trust TRUE Desire TRUE Contempt TRUE
18. Simulation
19. Affective Text Cake Chat (Replica.ai) E.L.S.A (MIT Media Lab)
20. Microsoft Xioice Source: 
 The Design and Implementation of XiaoIce, an Empathetic Social Chatbot 
 Li Zhou, Jianfeng Gao, Di Li, Heung-Yeung Shum
21. Affective Speech • Gender • Location (Accents) • Prosody • Wavenet • Tacotron • DeepVoice • Personality • Voicery Source: Google AI Blog
 Expressive Speech Synthesis with Tacotron
22. Parting thoughts:
 Designing Affective Experiences Responsibly
23. References • Affective Computing Group @ MIT Media Labs • Swiss Centre for Affective Sciences • https://en.wikipedia.org/wiki/Affective_computing • Replika.ai (Research Github) • Why we are Wired to Connect (Scientific American) • EmoPy: A machine learning toolkit for emotion expression