$30 off During Our Annual Pro Sale. View Details »

How AI is changing User Experience design

How AI is changing User Experience design

AI is making new ways of interacting with technology possible, but understanding the new paradigms of human-computer interaction and user experience is critical to take full advantage of them.

Gillian Armstrong

August 25, 2022
Tweet

More Decks by Gillian Armstrong

Other Decks in Technology

Transcript

  1. © 2022, Amazon Web Services, Inc. or its affiliates. ©

    2022, Amazon Web Services, Inc. or its affiliates. How AI is changing User Experience design Gillian Armstrong Senior Specialist Builder Solutions Architect AWS
  2. © 2022, Amazon Web Services, Inc. or its affiliates.

  3. © 2022, Amazon Web Services, Inc. or its affiliates.

  4. © 2022, Amazon Web Services, Inc. or its affiliates.

  5. © 2022, Amazon Web Services, Inc. or its affiliates.

  6. © 2022, Amazon Web Services, Inc. or its affiliates.

  7. © 2022, Amazon Web Services, Inc. or its affiliates.

  8. © 2022, Amazon Web Services, Inc. or its affiliates.

  9. © 2022, Amazon Web Services, Inc. or its affiliates.

  10. © 2022, Amazon Web Services, Inc. or its affiliates. Amazon

    Lex Alexa Conversational AI on Amazon / AWS
  11. © 2022, Amazon Web Services, Inc. or its affiliates. How

    conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Fulfilment CUX/VUX TTS Machine Learning
  12. © 2022, Amazon Web Services, Inc. or its affiliates. Utterance

    (“what the user said”) Automatic Speech Recognition (Speech to Text) How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Mapping of Utterance to Intent (“what the user meant”) Extraction of Slots (variables like time, place, number, etc) Fulfilment Intent maps to Fulfillment (“what we need to do”) CUX/VUX TTS Machine Learning
  13. © 2022, Amazon Web Services, Inc. or its affiliates. Utterance

    (“what the user said”) Automatic Speech Recognition (Speech to Text) How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Mapping of Utterance to Intent (“what the user meant”) Extraction of Slots (variables like time, place, number, etc) Fulfilment Intent maps to Fulfillment (“what we need to do”) CUX/VUX TTS Machine Learning
  14. © 2022, Amazon Web Services, Inc. or its affiliates. Utterance

    (“what the user said”) Automatic Speech Recognition (Speech to Text) How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Mapping of Utterance to Intent (“what the user meant”) Extraction of Slots (variables like time, place, number, etc) Fulfilment Intent maps to Fulfillment (“what we need to do”) CUX/VUX Conversational / Voice User Experience (“how it sounds and the words it says”) TTS Text to Speech Machine Learning
  15. © 2022, Amazon Web Services, Inc. or its affiliates. The

    new user experience 1. Mental Models 2. Explainability and Trust 3. Feedback and Control
  16. © 2022, Amazon Web Services, Inc. or its affiliates. The

    new user experience 1. Mental Models
  17. © 2022, Amazon Web Services, Inc. or its affiliates. Traditional

    web UX
  18. © 2022, Amazon Web Services, Inc. or its affiliates. What

    about a chatbot?
  19. © 2022, Amazon Web Services, Inc. or its affiliates. Conversational

    UX For information about your orders press 1, to learn about Prime benefits press 2, to add a new address press 3………..
  20. © 2022, Amazon Web Services, Inc. or its affiliates. Conversational

    UX For information about your orders say orders, to learn about Prime benefits say prime, to add a new address say address……
  21. © 2022, Amazon Web Services, Inc. or its affiliates. Conversational

    UX For information about your orders say orders, to learn about Prime benefits say prime, to add a new address say address……
  22. © 2022, Amazon Web Services, Inc. or its affiliates. Model

    your design on a human, not a menus, forms or websites People are the original conversational interface The new paradigm in UX
  23. © 2022, Amazon Web Services, Inc. or its affiliates. The

    Web is about Telling the user what to do Conversational UI is about Listening to what the user wants to do The new paradigm in UX
  24. © 2022, Amazon Web Services, Inc. or its affiliates. Conversational

    UX I need to add a new address
  25. © 2022, Amazon Web Services, Inc. or its affiliates. Conversational

    UX I’ve moved house
  26. © 2022, Amazon Web Services, Inc. or its affiliates. Conversational

    UX Can I get packages delivered to my office too?
  27. © 2022, Amazon Web Services, Inc. or its affiliates. Utterance

    (“what the user said”) Automatic Speech Recognition (Speech to Text) How conversational AI works NLU NLP INTENTS I n t e r a c t i o n M o d e l SLOTS ASR Mapping of Utterance to Intent (“what the user meant”) Extraction of Slots (variables like time, place, number, etc) Fulfilment Intent maps to Fulfillment (“what we need to do”) CUX/VUX Conversational / Voice User Experience (“how it sounds and the words it says”) TTS Text to Speech Machine Learning
  28. © 2022, Amazon Web Services, Inc. or its affiliates. How

    Lex Interaction Model Works INTENT 1 • Example utterance • Example utterance • Example utterance INTENT 2 • Example utterance • Example utterance • Example utterance
  29. © 2022, Amazon Web Services, Inc. or its affiliates. How

    Lex Interaction Model Works INTENT 1 • Example utterance • Example utterance • Example utterance INTENT 2 • Example utterance • Example utterance • Example utterance YourOrders • Where is my order? • When will my order be delivered? • Has my order been dispatched? YourAddresses • I need to add a new address • I’ve moved house • Can I get packages delivered to my office too?
  30. © 2022, Amazon Web Services, Inc. or its affiliates. How

    Lex Interaction Model Works Input Expected Output Where is my order? YourOrders When will my order be delivered? YourOrders Has my order been dispatched? YourOrders I need to add a new address YourAddresses I’ve moved house YourAddresses Can I get packages delivered to my office too? YourAddresses Utterance (“what the user said”) Intent (“what the user meant”)
  31. © 2022, Amazon Web Services, Inc. or its affiliates. How

    Lex Interaction Model Works Input Expected Output Where is my order number {orderId}? YourOrders When will my order be delivered? YourOrders Has my order {orderId} been dispatched? YourOrders I need to add a new address YourAddresses I’ve moved house YourAddresses Can I get packages delivered to my office too? YourAddresses Utterance (“what the user said”) Intent (“what the user meant”) Slot (variables like time, place, number, etc.) orderId ABC1234 HDNF112 A234ASD Slot Values (here we would likely just define this as AlphaNumberic or a regex)
  32. © 2022, Amazon Web Services, Inc. or its affiliates. The

    new user experience 1. Mental Models 2. Explainability and Trust
  33. © 2022, Amazon Web Services, Inc. or its affiliates. Web

    UX
  34. © 2022, Amazon Web Services, Inc. or its affiliates. Confidence

    Score Action based on click location Prediction i.e. guess - (although hopefully a good one!) Confidence: 100% Confidence: ??%
  35. © 2022, Amazon Web Services, Inc. or its affiliates. Machine

    learning Confidence Score A number between 0 and 1 that represents the likelihood that the output of a particular prediction from a ML model is correct Precision For a given test set what percentage of the items predicted as a particular answer were correct Recall For a given test set what percentage of the items that should have been a particular answer were correctly identified
  36. © 2022, Amazon Web Services, Inc. or its affiliates. Precision

    vs Recall Phrase for add address Intent Phrase for something else Predicted as add address Intent Good precision Good recall
  37. © 2022, Amazon Web Services, Inc. or its affiliates. Precision

    vs Recall e.g. • When to launch my rocket • Recommendations • Email spam filtering e.g. • Alert you of potential fraud on your card • Critical medical test • High risk content moderation Good precision Good recall When you really don’t want to get it wrong When you really don’t want to miss it
  38. © 2022, Amazon Web Services, Inc. or its affiliates. Using

    Confidence Score e.g. • When to launch my rocket • Recommendations • Email spam filtering e.g. • Alert you of potential fraud on your card • Critical medical test • High risk content moderation Only use high confidence answers Use lower confidence answers When you really don’t want to get it wrong When you really don’t want to miss it
  39. © 2022, Amazon Web Services, Inc. or its affiliates. Using

    Confidence Score Only use high confidence answers Ok to use lower confidence answers Cancelling your order Our lines are open 9am to 5pm every day Difficult to reverse actions should be confirmed first I’ll be angry if you get this wrong… The wrong answer is a less annoying than have to repeat if it was correct… Change the address on my last order What times can I call?
  40. © 2022, Amazon Web Services, Inc. or its affiliates. Web

    UX
  41. © 2022, Amazon Web Services, Inc. or its affiliates. Affordance

    – a title and breadcrumbs show me what page I am on I can see all the fields I need to fill out I have some prompts on how to fill the fields in
  42. © 2022, Amazon Web Services, Inc. or its affiliates. What

    about a chatbot? What’s your phone number? I need to add a new address
  43. © 2022, Amazon Web Services, Inc. or its affiliates. What

    about a chatbot? I need to add a new address Did it understand? What’s your phone number?
  44. © 2022, Amazon Web Services, Inc. or its affiliates. What

    about a chatbot? To add a new address I’ll first need to get your phone number I need to add a new address
  45. © 2022, Amazon Web Services, Inc. or its affiliates.

  46. © 2022, Amazon Web Services, Inc. or its affiliates. You

    did this wrong!!!!
  47. © 2022, Amazon Web Services, Inc. or its affiliates. Conversational

    UX oh, umm, let me check which one I’ll put on this What’s your phone number?
  48. © 2022, Amazon Web Services, Inc. or its affiliates. Conversational

    UX oh, umm, let me check which one I’ll put on this That’s an incorrect phone number
  49. © 2022, Amazon Web Services, Inc. or its affiliates. Conversational

    UX The customer has never said the wrong thing – the chatbot just hasn’t understood
  50. © 2022, Amazon Web Services, Inc. or its affiliates. Conversational

    UX oh, umm, let me check which one I’ll put on this No problem. This will be the number we would contact you on if there was an issue with the delivery to that address.
  51. © 2022, Amazon Web Services, Inc. or its affiliates. Conversational

    UX oh, umm, let me check which one I’ll put on this I’m sorry - I didn’t get the phone number. Can you repeat the phone number for the new address?
  52. © 2022, Amazon Web Services, Inc. or its affiliates. The

    new user experience 1. Mental Models 2. Explainability and Trust 3. Feedback and Control
  53. © 2022, Amazon Web Services, Inc. or its affiliates. Always

    keep your user in control Is your customer clear they are talking to a bot?
  54. © 2022, Amazon Web Services, Inc. or its affiliates. Always

    keep your user in control I’d really prefer to talk to a human Does your bot understand how to give a customer other options? Is your customer clear they are talking to a bot?
  55. © 2022, Amazon Web Services, Inc. or its affiliates. Always

    make sure you know what is happening too… Do you know how well your bot is working? Do you know what your customers are asking? Could you explain why your bot answered a certain way? Getting feedback both directly and through monitoring is critical to good ML system
  56. © 2022, Amazon Web Services, Inc. or its affiliates. The

    new user experience 1. Mental Models 2. Explainability and Trust 3. Feedback and Control
  57. © 2022, Amazon Web Services, Inc. or its affiliates. Take-aways

    • We need to break away from our traditional patterns – start thinking about how to make the computer understand the human rather than the other way around • An understanding on Machine Learning is required to build great User Experiences that use AI • The future is exciting!
  58. © 2022, Amazon Web Services, Inc. or its affiliates. Thank

    you! © 2022, Amazon Web Services, Inc. or its affiliates. Gillian Armstrong GillianQ@amazon.co.uk @virtualgill