Tweet
Tweet

Slide 1

Slide 1 text

© 2022, Amazon Web Services, Inc. or its affiliates. © 2022, Amazon Web Services, Inc. or its affiliates. How AI is changing User Experience design Gillian Armstrong Senior Specialist Builder Solutions Architect AWS

Slide 2

Slide 2 text

© 2022, Amazon Web Services, Inc. or its affiliates.

Slide 3

Slide 3 text

© 2022, Amazon Web Services, Inc. or its affiliates.

Slide 4

Slide 4 text

© 2022, Amazon Web Services, Inc. or its affiliates.

Slide 5

Slide 5 text

© 2022, Amazon Web Services, Inc. or its affiliates.

Slide 6

Slide 6 text

© 2022, Amazon Web Services, Inc. or its affiliates.

Slide 7

Slide 7 text

© 2022, Amazon Web Services, Inc. or its affiliates.

Slide 8

Slide 8 text

© 2022, Amazon Web Services, Inc. or its affiliates.

Slide 9

Slide 9 text

© 2022, Amazon Web Services, Inc. or its affiliates.

Slide 10

Slide 10 text

© 2022, Amazon Web Services, Inc. or its affiliates. Amazon Lex Alexa Conversational AI on Amazon / AWS

Slide 11

Slide 11 text

© 2022, Amazon Web Services, Inc. or its affiliates. How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Fulfilment CUX/VUX TTS Machine Learning

Slide 12

Slide 12 text

© 2022, Amazon Web Services, Inc. or its affiliates. Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Mapping of Utterance to Intent (“what the user meant”) Extraction of Slots (variables like time, place, number, etc) Fulfilment Intent maps to Fulfillment (“what we need to do”) CUX/VUX TTS Machine Learning

Slide 13

Slide 13 text

© 2022, Amazon Web Services, Inc. or its affiliates. Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Mapping of Utterance to Intent (“what the user meant”) Extraction of Slots (variables like time, place, number, etc) Fulfilment Intent maps to Fulfillment (“what we need to do”) CUX/VUX TTS Machine Learning

Slide 14

Slide 14 text

© 2022, Amazon Web Services, Inc. or its affiliates. Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Mapping of Utterance to Intent (“what the user meant”) Extraction of Slots (variables like time, place, number, etc) Fulfilment Intent maps to Fulfillment (“what we need to do”) CUX/VUX Conversational / Voice User Experience (“how it sounds and the words it says”) TTS Text to Speech Machine Learning

Slide 15

Slide 15 text

© 2022, Amazon Web Services, Inc. or its affiliates. The new user experience 1. Mental Models 2. Explainability and Trust 3. Feedback and Control

Slide 16

Slide 16 text

© 2022, Amazon Web Services, Inc. or its affiliates. The new user experience 1. Mental Models

Slide 17

Slide 17 text

© 2022, Amazon Web Services, Inc. or its affiliates. Traditional web UX

Slide 18

Slide 18 text

© 2022, Amazon Web Services, Inc. or its affiliates. What about a chatbot?

Slide 19

Slide 19 text

© 2022, Amazon Web Services, Inc. or its affiliates. Conversational UX For information about your orders press 1, to learn about Prime benefits press 2, to add a new address press 3………..

Slide 20

Slide 20 text

© 2022, Amazon Web Services, Inc. or its affiliates. Conversational UX For information about your orders say orders, to learn about Prime benefits say prime, to add a new address say address……

Slide 21

Slide 21 text

© 2022, Amazon Web Services, Inc. or its affiliates. Conversational UX For information about your orders say orders, to learn about Prime benefits say prime, to add a new address say address……

Slide 22

Slide 22 text

© 2022, Amazon Web Services, Inc. or its affiliates. Model your design on a human, not a menus, forms or websites People are the original conversational interface The new paradigm in UX

Slide 23

Slide 23 text

© 2022, Amazon Web Services, Inc. or its affiliates. The Web is about Telling the user what to do Conversational UI is about Listening to what the user wants to do The new paradigm in UX

Slide 24

Slide 24 text

© 2022, Amazon Web Services, Inc. or its affiliates. Conversational UX I need to add a new address

Slide 25

Slide 25 text

© 2022, Amazon Web Services, Inc. or its affiliates. Conversational UX I’ve moved house

Slide 26

Slide 26 text

© 2022, Amazon Web Services, Inc. or its affiliates. Conversational UX Can I get packages delivered to my office too?

Slide 27

Slide 27 text

© 2022, Amazon Web Services, Inc. or its affiliates. Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Mapping of Utterance to Intent (“what the user meant”) Extraction of Slots (variables like time, place, number, etc) Fulfilment Intent maps to Fulfillment (“what we need to do”) CUX/VUX Conversational / Voice User Experience (“how it sounds and the words it says”) TTS Text to Speech Machine Learning

Slide 28

Slide 28 text

© 2022, Amazon Web Services, Inc. or its affiliates. How Lex Interaction Model Works INTENT 1 • Example utterance • Example utterance • Example utterance INTENT 2 • Example utterance • Example utterance • Example utterance

Slide 29

Slide 29 text

© 2022, Amazon Web Services, Inc. or its affiliates. How Lex Interaction Model Works INTENT 1 • Example utterance • Example utterance • Example utterance INTENT 2 • Example utterance • Example utterance • Example utterance YourOrders • Where is my order? • When will my order be delivered? • Has my order been dispatched? YourAddresses • I need to add a new address • I’ve moved house • Can I get packages delivered to my office too?

Slide 30

Slide 30 text

© 2022, Amazon Web Services, Inc. or its affiliates. How Lex Interaction Model Works Input Expected Output Where is my order? YourOrders When will my order be delivered? YourOrders Has my order been dispatched? YourOrders I need to add a new address YourAddresses I’ve moved house YourAddresses Can I get packages delivered to my office too? YourAddresses Utterance (“what the user said”) Intent (“what the user meant”)

Slide 31

Slide 31 text

© 2022, Amazon Web Services, Inc. or its affiliates. How Lex Interaction Model Works Input Expected Output Where is my order number {orderId}? YourOrders When will my order be delivered? YourOrders Has my order {orderId} been dispatched? YourOrders I need to add a new address YourAddresses I’ve moved house YourAddresses Can I get packages delivered to my office too? YourAddresses Utterance (“what the user said”) Intent (“what the user meant”) Slot (variables like time, place, number, etc.) orderId ABC1234 HDNF112 A234ASD Slot Values (here we would likely just define this as AlphaNumberic or a regex)

Slide 32

Slide 32 text

© 2022, Amazon Web Services, Inc. or its affiliates. The new user experience 1. Mental Models 2. Explainability and Trust

Slide 33

Slide 33 text

© 2022, Amazon Web Services, Inc. or its affiliates. Web UX

Slide 34

Slide 34 text

© 2022, Amazon Web Services, Inc. or its affiliates. Confidence Score Action based on click location Prediction i.e. guess - (although hopefully a good one!) Confidence: 100% Confidence: ??%

Slide 35

Slide 35 text

© 2022, Amazon Web Services, Inc. or its affiliates. Machine learning Confidence Score A number between 0 and 1 that represents the likelihood that the output of a particular prediction from a ML model is correct Precision For a given test set what percentage of the items predicted as a particular answer were correct Recall For a given test set what percentage of the items that should have been a particular answer were correctly identified

Slide 36

Slide 36 text

© 2022, Amazon Web Services, Inc. or its affiliates. Precision vs Recall Phrase for add address Intent Phrase for something else Predicted as add address Intent Good precision Good recall

Slide 37

Slide 37 text

© 2022, Amazon Web Services, Inc. or its affiliates. Precision vs Recall e.g. • When to launch my rocket • Recommendations • Email spam filtering e.g. • Alert you of potential fraud on your card • Critical medical test • High risk content moderation Good precision Good recall When you really don’t want to get it wrong When you really don’t want to miss it

Slide 38

Slide 38 text

© 2022, Amazon Web Services, Inc. or its affiliates. Using Confidence Score e.g. • When to launch my rocket • Recommendations • Email spam filtering e.g. • Alert you of potential fraud on your card • Critical medical test • High risk content moderation Only use high confidence answers Use lower confidence answers When you really don’t want to get it wrong When you really don’t want to miss it

Slide 39

Slide 39 text

© 2022, Amazon Web Services, Inc. or its affiliates. Using Confidence Score Only use high confidence answers Ok to use lower confidence answers Cancelling your order Our lines are open 9am to 5pm every day Difficult to reverse actions should be confirmed first I’ll be angry if you get this wrong… The wrong answer is a less annoying than have to repeat if it was correct… Change the address on my last order What times can I call?

Slide 40

Slide 40 text

© 2022, Amazon Web Services, Inc. or its affiliates. Web UX

Slide 41

Slide 41 text

© 2022, Amazon Web Services, Inc. or its affiliates. Affordance – a title and breadcrumbs show me what page I am on I can see all the fields I need to fill out I have some prompts on how to fill the fields in

Slide 42

Slide 42 text

© 2022, Amazon Web Services, Inc. or its affiliates. What about a chatbot? What’s your phone number? I need to add a new address

Slide 43

Slide 43 text

© 2022, Amazon Web Services, Inc. or its affiliates. What about a chatbot? I need to add a new address Did it understand? What’s your phone number?

Slide 44

Slide 44 text

© 2022, Amazon Web Services, Inc. or its affiliates. What about a chatbot? To add a new address I’ll first need to get your phone number I need to add a new address

Slide 45

Slide 45 text

© 2022, Amazon Web Services, Inc. or its affiliates.

Slide 46

Slide 46 text

© 2022, Amazon Web Services, Inc. or its affiliates. You did this wrong!!!!

Slide 47

Slide 47 text

© 2022, Amazon Web Services, Inc. or its affiliates. Conversational UX oh, umm, let me check which one I’ll put on this What’s your phone number?

Slide 48

Slide 48 text

© 2022, Amazon Web Services, Inc. or its affiliates. Conversational UX oh, umm, let me check which one I’ll put on this That’s an incorrect phone number

Slide 49

Slide 49 text

© 2022, Amazon Web Services, Inc. or its affiliates. Conversational UX The customer has never said the wrong thing – the chatbot just hasn’t understood

Slide 50

Slide 50 text

© 2022, Amazon Web Services, Inc. or its affiliates. Conversational UX oh, umm, let me check which one I’ll put on this No problem. This will be the number we would contact you on if there was an issue with the delivery to that address.

Slide 51

Slide 51 text

© 2022, Amazon Web Services, Inc. or its affiliates. Conversational UX oh, umm, let me check which one I’ll put on this I’m sorry - I didn’t get the phone number. Can you repeat the phone number for the new address?

Slide 52

Slide 52 text

© 2022, Amazon Web Services, Inc. or its affiliates. The new user experience 1. Mental Models 2. Explainability and Trust 3. Feedback and Control

Slide 53

Slide 53 text

© 2022, Amazon Web Services, Inc. or its affiliates. Always keep your user in control Is your customer clear they are talking to a bot?

Slide 54

Slide 54 text

© 2022, Amazon Web Services, Inc. or its affiliates. Always keep your user in control I’d really prefer to talk to a human Does your bot understand how to give a customer other options? Is your customer clear they are talking to a bot?

Slide 55

Slide 55 text

© 2022, Amazon Web Services, Inc. or its affiliates. Always make sure you know what is happening too… Do you know how well your bot is working? Do you know what your customers are asking? Could you explain why your bot answered a certain way? Getting feedback both directly and through monitoring is critical to good ML system

Slide 56

Slide 56 text

© 2022, Amazon Web Services, Inc. or its affiliates. The new user experience 1. Mental Models 2. Explainability and Trust 3. Feedback and Control

Slide 57

Slide 57 text

© 2022, Amazon Web Services, Inc. or its affiliates. Take-aways • We need to break away from our traditional patterns – start thinking about how to make the computer understand the human rather than the other way around • An understanding on Machine Learning is required to build great User Experiences that use AI • The future is exciting!

Slide 58

Slide 58 text

© 2022, Amazon Web Services, Inc. or its affiliates. Thank you! © 2022, Amazon Web Services, Inc. or its affiliates. Gillian Armstrong [email protected] @virtualgill