Synthetic Intimacy

Transcript

1. UX Australia 2019 (AUUXAU2908A) Main Room, Day 1 – 29th August, 2019 TRIP O'DELL: Hello. Are we live here? Hi, I am Trip. Thank you, Steve, for having me. That was a terrible snowstorm, my kids were out of school for a month. As painful as it was for Steve to get to the airport, it was worse for us. I have had a pretty unlikely career. I have done everything from being a teacher, ditch digger, door-to-door meat salesman, congressional staffer in the US. But for the last 15 years, I have worked on leading design on large-scale global highly complex products at Adobe, Microsoft, Amazon and now I have my independent consultancy Dark Matter where we work on similar stuff, solving complex human problems using tech and AI. Intimacy. Synthetic intimacy. It's a bit of sex sells. But the intimacy we're talking about today is the thing that makes us most human, our ability to connect with people, and how that is complicated. You say that term, it's a bit uncomfortable, it is a bit of a nudge and a wink, but it's about how we connect with people and our ability to form the bonds that make us human. So, I'm going to ask a few questions. How many of you might say, "it's complicated," when I ask you what your relationship is with your phone? How many of you worry about the time your children spend on screens and how that is shaping their ability to shape relationships? How many of you are uneasy about Alexa and Siri, and how they might intrude on the most private parts of our lives? Those are uncomfortable questions for many of us, because if you design human-centred experiences, you must accept that part of your job is to exploit the essential desires and cognitive biases in order to help people with the technology to make it easier. So today we're going to talk about the ways that humans think, how that shapes our behaviour, why both voice and AI are powerful and also potentially dangerous, and why it is easy for computers to become our special invisible friends. And how we can confront the ethical considerations that we must accept as designers. So, we make it easy for our invisible friends to trick us because we evolved that way. As a technologist and somebody that has spent a lot of time working with children, when I was working on Alexa I led about 50% of what Alexa does in the first year of release. And I pay attention because children are often the canaries in the coal mine for how technology is going to evolve. I found it both endearing and it really brought it home how much they were treating this person as a special friend. And for me, it reminded me of when my oldest child was about 18 months, the first time she really started to notice television. I think my wife was watching – these are great, aren't they – Oprah Winfrey or something, and then she is looking at the screen and then she goes around behind the TV to see where those people are. Right? They are not born to think about these things in the same way that we are. They think about how media and technology should be modelled in the same way they experience the world.
2. UX Australia 2019 (AUUXAU2908A) Main Room, Day 1 – 29th August, 2019 Page 2 of 10 Just a moment. So we do not see things as they are; we see things as we are. We are wired to create relationships, as social animals. And we really can only take our observations of the world as we have it inside our box that sits on top of our neck, and extrapolate what that means for other people – how they encounter the world. And the more human something acts, the more human it seems. Those are evolved cognitive traits. We are primates. We are part of tribes, troops and bands, and we have rules that go with that. We have the need for acceptance and status, reciprocity, intimacy, and intimacy forms the basis of trust. One of the reasons that we are involved is because of the way our brains work with stories. We learn through stories. And the cognitive scientists would call that episodic memory. It's the part of the brain where we record things like a movie clip, and we go back and review that. We have a reminder of something like fire is hot, we have that flash memory, and we're constantly revisiting and revising what are called schemas and heuristics around what it means, how it is or how it works. That is informed by our episodic memory. And stories serve as knowledge compression. And media serves as a transmission device. And it allows people to timeshift knowledge, like expand it and to distribute it. Things like metaphor serve as patterns for other ways we can understand the world. And we evaluate the world through that narrative lens. Culture and means, culture gets encapsulated in references to media that compress it further and transmit it further. Stories are the basis of empathy. How we connect. Way So let's talk a little bit about how we form those stories. Right? I found this illustration online, credits are at the end, but this is a good illustration of a metaphor for how that part of the brain works. We have different types of memory, short-term memory, we have the episodic buffer, we have the schemas and frameworks for how we understand the world – we constantly revise it. For long-term memory, we revise what we see in short-term memory, we pattern match against what we know, and we constantly revise that. This is a perfect example of how we use story and metaphor to understand the world. Here is a more nerdy version of it. This is memory and cognition. You have two parts of memory, long- term and short-term. Those are broken up into what we call declarative memory, words and language and ideas, and procedural memory, which is like how to ride a bike. You have both audio, the phonological store and the visual-spatial sketchpad which allows us to have memory around images and voice, and we have the episodic buffer where many people have heard of flow where we get the experience and it is mixed in together. On top of that, we have the area called the central executive. It is the most recently evolved part of our brain, evolving about 50,000 years ago, and it is the part of the brain focused on task management and complex thought. They call it limited capacity, there is a limited capacity of short-term memory and as soon as it translates from short-term memory to the central
3. UX Australia 2019 (AUUXAU2908A) Main Room, Day 1 – 29th August, 2019 Page 3 of 10 executive, like when you get distracted, your cortisol and your stressors shoot up and you become overwhelmed. That's why it is hard to learn, it is hard to be at school, it is hard to try new things. That was one of the things, this is Daniel Kahneman, he is a Nobel laureate a behavioural economist, he and his partner Tibor pioneered the area of behavioural economics. These people were thought as weird and interesting and then they started running crazy experiments to see how people reacted and the big insight was they came out with this idea that we use two systems in our thinking. System one thinking is really that everyone is firing, I remember what I'm doing, I know my schedule, I don't have to think about it, I am in my routine and I know what my route is to work. That is system one thinking, everything is firing. That's the mode we prefer to be in. We prefer to be in the moment, not having to think too much. And some of the cognitives, I can't remember who used the term, but we are cognitive misers because of that constrain short-term memory – we don't like to use more than we have to, so we chunk it down. These are the guys who invented the term heuristic, there is a framework, a pattern to apply to an idea to see if it is true or false and it informs the system. The central executive is more complex thought. It is awesome, but it is also when you have to slow down. This is the stuff that kills people when you're checking your phone behind the wheel. Your short-term memory gets overwhelmed. The central executive gets in and the papers in this illustration, that's when I start getting dropped. You lose stuff. I love this picture. These are not Homo sapiens. But this picture reminds me of my brothers and I when I was younger. (Laughs) Short-term memory, slow thinking can get you killed. In the state of nature, if something is moving, it can kill you. I react to that, I might startle, I might run. Fight or flight, the amygdala, the part of the brain that deals with fight or flight, goes off. But slow thinking can also kill you. Not anticipating things like, wow, hold my beer. That's the type of thing that can also get you killed. So you need both systems. And being able to anticipate outcomes is second-order thinking. As I said, we prefer type I thinking, system one thinking. That relies on instinct, subconscious biases and heuristics. We use stories, schemas and preferences for behaviours we have already established. It avoids pain and seeks pleasure, as well social connection and status. And it requires far less energy. When you are stressed out and stress eating, avoiding pain and seeking pleasure, that is system one. That was one of Daniel Kahneman's experiments. System two is hard. It is very slow and energy intensive. They looked at kids at school who don't eat first, and performance goes down because learning is calorie intensive. It makes the modern world possible, and it is essential for learning new skills. Quick example. I am deathly afraid of snakes. But every time I go to the zoo, I visit the snake
4. UX Australia 2019 (AUUXAU2908A) Main Room, Day 1 – 29th August, 2019 Page 4 of 10 house. I don't know why. A friend of mine lives in Sydney, who lives in the Blue Mountains, took me to the Sydney zoo yesterday. Of course, everything kills you here, so we went to the snake house because I wanted to see the inland taipan. I saw that. This is system one thinking. In the US, we have snakes. There is one called a scarlet corn snake and a coral snake, the coral snake being the deadliest snake in the US. It is almost on par with Australia. But not quite. (Laughs) You guys are like a more intense America. No. (Laughs) (Laughter) TRIP O'DELL: No, especially not now. But the funny thing about my visit to the zoo is I have got an Apple watch, and I was there and there was a young Indian family there. The kids were up there next to the glass and my heart rate went up and I got a ping on here. And I scared everybody in the snake house. And the father was like, "Get it together, man!" (Laughter) TRIP O'DELL: That is system one thinking, fight or flight. Example 2, language, story and executive function. We have a rhyme in the US, red touch is black, safe for Jack. Red touch is yellow, kills a fellow. Heuristic, but you have to slow down your response enough to remember that rhyme. I would be a block away already. You will see that snakes are a recurring function in here. But we use stories, the world is like we are until it proves otherwise. We are cognitive misers. And we are foundationally social. Those are some of the ways we are hardwired to learn. Let's talk about the social. Kahneman says familiarity breeds liking. We know what it is, we are comfortable, we don't have to think about it. We like it. Let's have some Jesus toast. That familiarity, we look for heuristics for where we can recognise things as being human. We constantly look for the thing as being human and recognising that pattern. We do that all the time. We do not see things in the way they are; we see things in the way that we are. Sometimes ridiculously. It goes to this idea of social circles. You see this a lot in the social sciences where they talk about the circles of familiarity with the self, family, close friends, and it goes further and further out. Layers of relationships.
5. UX Australia 2019 (AUUXAU2908A) Main Room, Day 1 – 29th August, 2019 Page 5 of 10 But people can transit these relationships. You have the personal and the professional life, but how many people met their spouse or significant other at work? That is transiting a circle. But that is not the first time. That is not the first time something that is not human has happened like this. About 20,000-40,000 years ago, our instincts lead to domestication. I love that dog, that hat. That says it all. We're seeing a little person there. Animal domestication is way more impactful than AI – at least right now, but I hope so. The question is, why dogs? Why not snakes? Snakes are horrible. (Laughter) TRIP O'DELL: Dogs are warm blooded, they have legs, hair, expressive faces, they are social, they can learn, they have expressive behaviour – familiarity breeds liking. We are coevolved for mutual benefit although that is what the evolutionary scientists tell us. Dog might not agree. He doesn't look that happy right now. Microsoft is talking about scaling human potential with AI in the cloud. Other evolutionary scientists talking about who actually benefits, dogs scale human potential too but who benefited most from that relationship? In the US last year we spent $46 million on dogs alone, on dog care. We have a homelessness problem. Man, that dog is cute. Those manipulative bastards. They didn't design it that way, but they evolved that way, recent findings about the evolution of canines is dogs have muscles on parts of their eyes that wolves don't have and that allows for puppy dog eyes. They have less cartilage, labradors have soft floppy ears, less cartilage than wolves, and there was an experiment in the Soviet Union back in I think the '60s where he was rapidly breeding Arctic foxes or grey foxes and in a couple of generations, the cartilage would soften and they would become social, more docile, more cute and more likely to please us. They were becoming rapidly domesticated. UC smaller dogs, lapdogs, they say more and more like human babies. Heuristically, we are matching that with something that needs to be cared for. What about calculated social responses? Now we are in a world where we are designing things to be cute, designing things to act in ways we would expect a natural part of the world to act. What happens when we intentionally elicit those responses? Are we creating technology that would help a parasite? This is Clifford Nass, a researcher at Stanford, he started out studying computers and using usability to see how people interact with computers. What he and one of his collaborators, Byron Reeves, found in their book The Media Equation, it goes across media, computers and objects, the more human something seems, the more media seems like real life, media being
6. UX Australia 2019 (AUUXAU2908A) Main Room, Day 1 – 29th August, 2019 Page 6 of 10 broadly technology. A compacted version or a reflection of life, the more real it seems, the more we have real life. That effect is characters and the more those things conform to physical norms. Who is this, shout it out. That looks like Harrison Ford. How many of you believe Han shot first? Why do you care? It's a revision of a history of a person who does not exist. Friends that don't love us back, when media equals real life, there is a notion of a para-social relationship, one directional relationship. If you think about celebrity culture, that's a para-social relationship. Steve Jobs or Apple, you have a reaction to that persona. The one-way, the person that feels that sense of connection has real feelings of love, loyalty, longing and engagement. It negates a chance for rejection but welcomes invasive curiosity like stalking. How many people have opinions about these people? Maybe a few. How many of you have met them? How about these people? These are not people, they are characters. Carrie Fisher is Princess Leia. Objects and animals, we name boats and aeroplanes. Some people names snakes, I wouldn't. The more we take it into media we will anthropomorphise and give them human characteristics to tell stories. We will even create them from whole cloth. Now it will get weird. This is a girlfriend simulator. This is something that started, it's a chatbot, the point of the game is to try and convince this person to date you or be your girlfriend or to have some sort of relationship with you. First cursory search on Google play, 250 hits of all sorts. We all want to connect. Technology is pushing it further. There has been lots of news about realistic lifelike synthetic dolls replacing human to human sex. And it's a very dystopian view for where things could go. But it gets back to the heart of the problem of why are we wired to create those relationships? We are wired to love things that cannot love us back because familiarity breeds liking. The more human something is, the more human we treat it. So, when I was working on Alexa, in 2017 Alexa got more than 1 million marriage proposals. It would respond with, "I'm attached..." To the wall. But it would also say things like, "I love you, thank you, that's nice of you to say." Flash back to college. (Laughs) That was intentional. I had friends on the personality team at Alexa and we were trying to be ethical about how we create these elections. We use a persona which was Peggy from 'Madmen', we wanted her to be professional, friendly, cheerful and not too personal. It is a computer, not a person, it only speaks when spoken to. Those were immutable principles. As much as we would like it to be polite or friendly, when you have difficulty treating humans, when you treat humans like objects regularly in society, treating objects like humans becomes problematic as envisioned in shows like 'West World'.
7. UX Australia 2019 (AUUXAU2908A) Main Room, Day 1 – 29th August, 2019 Page 7 of 10 So meet Cortana. She started in a video game, a character in 'Halo', a disembodied avatar who would talk you through the game and help you learn it. She became Siri from Microsoft. We talk about her as 'she' but she has transited, she has a back story and history, and there is this myth that now she is in our pocket. The avatar did not come into the system but what we do have is a motion library of emotions. Cortana can express ourselves with a body in the way that we would but it can have happy and sad and confused emotions that will help understand the user having trouble. It's complicated, applies to a lot of things. Snakes are way down there, that's not complicated. Behavioural biases, we are biased in the way we experience the world and relate to the world, and how we engage with the world. That is the power of voice. I say 'voice' here because AI, Siri and Alexa and other systems are what we would think about with voice. If we talk about a person who is deaf and uses sign language, deaf culture has a very rich culture and has its own expressiveness, its own means and jokes, you have to learn the language to engage with that culture. And culture and identity are closely related to pattern. Language does not equal sound. Most of us use our because we are evolved for it and we match voice to sound to language. We are hardwired for it. This is the developmental timeline of a baby starting to hear and respond orienting to others' voices at 25 weeks in the womb. Shortly after birth you have milestones that are quickly being hit. The core of the operating system of the brain as it comes online and you become a social creature. You can distinguish voices, recognise words. Those are important parts of becoming a human. But not all communication is words. We have body language and so forth but there are things in voice AI, phatic expressions, how many I have heard the Google demo where they are ordering at a restaurant? They are using ums and ahs, social cues – wait a moment, I'm thinking of something, give me a pause. Episodic buffer and short-term memory saying I need a minute. We have to chunk it down. Those cues also create intimacy. It's that look across the table with a raised eyebrow at the end of the night to your significant other, that is emphatic communication, intimacy. Marshall McLuhan, how many of you have heard of him, the medium is the message? Famous for a couple of big ideas, one of them is stories change through media. You read the book and watch the movie, they are different stories. There is also a gradient of media, what he called hot and cold. It also maps to what Daniel Kahneman talked about with system one and system two thinking. Voice is hot, media creates intimacy. National public radio in the US and one of the designers would talk about driveway moments, a story so compelling you had to sit in the car waiting for the story to end. I worked at Audible for years, I have over 2000 audiobooks in my account, they can't turn off my account. It is because of that voice in your head, the voice that is so familiar, the narrator can
8. UX Australia 2019 (AUUXAU2908A) Main Room, Day 1 – 29th August, 2019 Page 8 of 10 change the entire tone of the story, things that are passive, linear, story based require less processing in the system one, are warm and emotional. System two is interactive, computer systems. There is a reason we have a job. You have to think about it. Video games are multimodal. Those are examples of hot and cold and cool media. And the reason that works that way is that if you think about voice being hot, one person is saying something and you are making eye contact and you can transmit an idea, you can watch what the other person is doing. Conversation and when we talk about voice AI in conversation that we give strong guidance, we try to avoid multi turn interaction, follow-up questions. There is so much context that can be missed that breaks the illusion if the system get it's wrong. You have this back and forth. Humans are really good at various layers of verbal communication. Everything from the tempo, the speed, the energy I'm using, the way I am projecting my voice, whether I'm speaking in a monotone or increasing my energy to make a point. Voice system cannot do that, they are very bad at so far. It will take a lot of work but it will happen fast. The hunter gatherer brain, we use stories to understand the world, we are social creatures, we seek echoes about ourself in the world. We are born listeners, language is complex and mediated experiences are fundamentally social and natural and model the natural world. That brings us to us. We create illusions, whether in UI or voice or motion or interaction, UI copy content, all of these things are all story, creating a human voice and layer on top of that. All designers are liars, including me. We are creating an illusion. We live visually to create relationships and model human voice or human intimacy. People can hate obscuritism all they like, but the use of shadows, lighting direction, and physicality, makes those systems easier to use and you don't have to think as much. It attempts to mimic real life. Cortana, again, it is mimicking the emotion and the body language without the body. Amazon Echo follows natural and social rules both in the hardware and the sound design, the thunk sound it makes. The sound designer, he was trying to mimic it, the echo looks like a log, so if you hit it, it makes a thunk sound. It tries to mimic that. The little lights moving around and orienting on you, "I am listening." It's not actually sending anything to the internet. Trust is important in Amazon. You do not violate customer trust. Believe it or not, Amazon is the most ethical company I have ever worked with – don't throw anything – and it is ethical because it has principles that apply to what we do. Customer obsession, ownership, bias for action. Those are things that we evaluate for what we do. Intuitive = don't make me think = don't make me use system two thinking. That is what good user experience is. So it's really easy for us to confuse computers and people. We do it all the time. I have a hard time calling Alexa or Cortana 'it'. But we are very, very different. We have complex distance, those frameworks I showed you, our brains are not trying or shape.
9. UX Australia 2019 (AUUXAU2908A) Main Room, Day 1 – 29th August, 2019 Page 9 of 10 They are matrices, there are layers, bilateral processing across hemispheres, and we are creating illusions for how we think things should work rather than how they actually work. And the para-social relationships we are creating are one-way. Those users experience flow when we can make the system hot, when we can actually give them a sense of autonomy, and they know exactly how the system is working, they don't have to think about it. But you're a wizard. So we craft illusions to make complex technology seem easy. There are good wizards and bad wizards. Voice interactions feel like magic when they work. There were two quotes early on in Alexa, "Alexa is like my best friend." My children, the responses you were seeing from them, they were trying to make a friend. A were trying to learn about somebody they thought was inside that can. That can be used for good or it can be used for evil. So using your talents responsibly as a designer goes beyond user need. You are also accountable for the ethics of that illusion. The problems of being a wizard, an axe can be used to heat a home or to commit murder. Human- centred means humans first always. We don't have the privilege of being unintentional. Use your powers and AI for good. Because we are the wizards behind the curtain, whether we want that responsibility or not, if you work in this job, that's who you are. Because it is so easy for us to get the user's trust and then to exploit it. We talk about dark patterns, AI is already being weaponised. There is the unintended consequences of what you have wrought in the world. And that is really hard when you're living is tied to it. It's not easy to walk away from a job or to call your boss out when they are doing the wrong thing and using systems one thinking. They want to get to the goals they have been keyed to for the end of the year. Being that moral voice is not easy. Good designers are accountable for what we make. So, defence against the dark arts. How does the customer benefit from the voice interaction, the AI, what you are building? At Amazon, we would ask, how is this customer obsessed? What is in it for the customer? Who is the person you are creating? If you're going to build a para-social interaction, if you're going to write a friendly UI, UI copy, or create a flow that reduces a lot of choices, what is the personality of that brand of that person you're creating? What are your team's ethics? How are those ethics are afflicted in what you are shipping? And in user intent, the relationship is always asymmetrical. Technology is always in service to the people. Break the illusion to let customers know when they are making important decisions, remove the minor decision so they can stay focused on the important one like driving. And think outside the device in the moment. Fellow wizards, we cannot... I'm going to leave you with these challenges. We cannot afford moral cowards in design. The stakes are too high. And it can be really hard. Luckily we are privileged, we are in a high-demand industry, we are well-compensated, it is easy to find another job. But that privilege comes with responsibilities to build good things. To
10. UX Australia 2019 (AUUXAU2908A) Main Room, Day 1 – 29th August, 2019 Page 10 of 10 benefit society. None of us want to be Robert Oppenheimer that shipped the friendly AI that made the wrong decision. Thank you. (Applause)