AI is making new ways of interacting with technology possible, but understanding the new paradigms of human-computer interaction and user experience is critical to take full advantage of them.
2022, Amazon Web Services, Inc. or its affiliates. How AI is changing User Experience design Gillian Armstrong Senior Specialist Builder Solutions Architect AWS
(“what the user said”) Automatic Speech Recognition (Speech to Text) How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Mapping of Utterance to Intent (“what the user meant”) Extraction of Slots (variables like time, place, number, etc) Fulfilment Intent maps to Fulfillment (“what we need to do”) CUX/VUX TTS Machine Learning
(“what the user said”) Automatic Speech Recognition (Speech to Text) How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Mapping of Utterance to Intent (“what the user meant”) Extraction of Slots (variables like time, place, number, etc) Fulfilment Intent maps to Fulfillment (“what we need to do”) CUX/VUX TTS Machine Learning
(“what the user said”) Automatic Speech Recognition (Speech to Text) How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Mapping of Utterance to Intent (“what the user meant”) Extraction of Slots (variables like time, place, number, etc) Fulfilment Intent maps to Fulfillment (“what we need to do”) CUX/VUX Conversational / Voice User Experience (“how it sounds and the words it says”) TTS Text to Speech Machine Learning
(“what the user said”) Automatic Speech Recognition (Speech to Text) How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Mapping of Utterance to Intent (“what the user meant”) Extraction of Slots (variables like time, place, number, etc) Fulfilment Intent maps to Fulfillment (“what we need to do”) CUX/VUX Conversational / Voice User Experience (“how it sounds and the words it says”) TTS Text to Speech Machine Learning
Lex Interaction Model Works INTENT 1 • Example utterance • Example utterance • Example utterance INTENT 2 • Example utterance • Example utterance • Example utterance
Lex Interaction Model Works INTENT 1 • Example utterance • Example utterance • Example utterance INTENT 2 • Example utterance • Example utterance • Example utterance YourOrders • Where is my order? • When will my order be delivered? • Has my order been dispatched? YourAddresses • I need to add a new address • I’ve moved house • Can I get packages delivered to my office too?
Lex Interaction Model Works Input Expected Output Where is my order? YourOrders When will my order be delivered? YourOrders Has my order been dispatched? YourOrders I need to add a new address YourAddresses I’ve moved house YourAddresses Can I get packages delivered to my office too? YourAddresses Utterance (“what the user said”) Intent (“what the user meant”)
Lex Interaction Model Works Input Expected Output Where is my order number {orderId}? YourOrders When will my order be delivered? YourOrders Has my order {orderId} been dispatched? YourOrders I need to add a new address YourAddresses I’ve moved house YourAddresses Can I get packages delivered to my office too? YourAddresses Utterance (“what the user said”) Intent (“what the user meant”) Slot (variables like time, place, number, etc.) orderId ABC1234 HDNF112 A234ASD Slot Values (here we would likely just define this as AlphaNumberic or a regex)
learning Confidence Score A number between 0 and 1 that represents the likelihood that the output of a particular prediction from a ML model is correct Precision For a given test set what percentage of the items predicted as a particular answer were correct Recall For a given test set what percentage of the items that should have been a particular answer were correctly identified
vs Recall e.g. • When to launch my rocket • Recommendations • Email spam filtering e.g. • Alert you of potential fraud on your card • Critical medical test • High risk content moderation Good precision Good recall When you really don’t want to get it wrong When you really don’t want to miss it
Confidence Score e.g. • When to launch my rocket • Recommendations • Email spam filtering e.g. • Alert you of potential fraud on your card • Critical medical test • High risk content moderation Only use high confidence answers Use lower confidence answers When you really don’t want to get it wrong When you really don’t want to miss it
Confidence Score Only use high confidence answers Ok to use lower confidence answers Cancelling your order Our lines are open 9am to 5pm every day Difficult to reverse actions should be confirmed first I’ll be angry if you get this wrong… The wrong answer is a less annoying than have to repeat if it was correct… Change the address on my last order What times can I call?
UX oh, umm, let me check which one I’ll put on this No problem. This will be the number we would contact you on if there was an issue with the delivery to that address.
keep your user in control I’d really prefer to talk to a human Does your bot understand how to give a customer other options? Is your customer clear they are talking to a bot?
make sure you know what is happening too… Do you know how well your bot is working? Do you know what your customers are asking? Could you explain why your bot answered a certain way? Getting feedback both directly and through monitoring is critical to good ML system
• We need to break away from our traditional patterns – start thinking about how to make the computer understand the human rather than the other way around • An understanding on Machine Learning is required to build great User Experiences that use AI • The future is exciting!
virtualgill
ken5scal
yoshidashingo
shigashiyama
hiyanger
nttcom
sonatard
negi111111
naospon
hirosatogamo
uhyo
salaboy
hinono
bcantrill
geeforr
matthewcrist
maltzj
skipperchong
jmmastey
mza
cassininazir
wjessup
eitanlees
productmarketing
jrom
