Design workshop - Voice Augmented eXperiences for Apps

Transcript

1. Voice Augmented eXperiences for Apps
2. For the workshop later...
3. Warm Up What does voice augmented UX mean to you?
4. Let’s address the first in the room...
5. How does voice work? Trees for the forest. NLP ASR Speech to text Text to Intention TTS
6. 史蒂夫 The chinese room problem *The Chinese is gibberish How does my computer understand me? 史蒂夫 何人 你是我 帅哥 有是不 你我们 你我们
7. 史蒂夫 The chinese ML room problem *The Chinese is gibberish How does my computer understand me? 史蒂夫 何人 你是我 帅哥 有是不 你我们 你我们 0.25 0.75 0.25
8. Book Flight How does my computer understand me? Here some flights 0.25 0.75 Flight List Only Morning Filter Book Flight The English ML room problem
9. All that’s cool, do I need to worry about this? Slightly, but more on that later...
10. Why do you want to voicify your app? Accessibility for the disabled Enable users get things done faster Lack of screen real estate Accessibility for the elderly Agentive Design Increase throughput through familiarity Understand true intent. Accessibility across languages
11. The Bottom Line Voice follows the Visual? The Visual follows the Voice? The Visual works in Tandem with Voice? Accessibility Avataars GUI elements control
12. The Bottom Line Design to get the best of both Voice + Touch Voice: Can I order a Margherita Pizza. Touch: Scroll through the various option find Margarita and click it I’d like to just repeat my order. Search for repeat order button and click Can you tell me all the pizza’s available Look through the list and browse
13. Demo Time
14. I am sold Now what do I have to do to make this work?
15. The Basics
16. Intent Entities Prompts Utterances
17. Intent The action the user wants to perform. Book a ticket List the flights Filter for the morning ones. Cancel my order Go Back How many bags are allowed? Repeat that Maps to an executable function
18. Entities Parameters to be collected if the function is to be fulfilled. Book a ticket from Bangalore to Delhi for the 21st Entity: Source Entity Type: City Entity: Destination Entity Type: City Entity: Dep Date Entity Type: Date
19. Utterances Examples that help distinguish between Intents given the spoken sentence Book a ticket Can I book a ticket Are there any tickets available from Bangalore to Delhi I need a ticket from Bangalore to Delhi I want to book a ticket from Bangalore to Delhi Any reservations from Bangalore to Delhi Tickets for two from Bangalore to Delhi
20. Prompts Help collect required entities, by asking the user for them. Date: For which day should I book the onward journey? Return: Is it one way or return? Dest: Where do you want to fly to? Source: Where are you flying from? Number: How many tickets do you want to book?
21. Confirmations Feedback to the user. Let them know all is well, or not. Can be a prompt: Sure, would you like to continue? Can be a positive statement: Your ticket for the 21st from Mumbai to Bangalore for one person has been booked successfully. Can be a negative statement: I am sorry there are no flights available that day from Mumbai to Bangalore.
22. Exercise Time But first, any Questions?
23. The Basics ^ not so
24. Conversations String Dialogues Together Make them human like I’d like to book a flight Alright, for when? Umm, the 21st. Sure, from where? From Mumbai to Bangalore Alright, here are some flights... (Scrolling a bit) Umm, show me only the non stop flights. Alright, filtered out the non stop flights. (Scrolling a bit more) Or you know what scratch that just show me all of them. Sure, removing the number of stops filter.
25. Conversations Okay, show me flights to Chennai You want to see flights from Mumbai to Chennai for the 21st? No No No, for the 22nd. Alright, showing flights from Mumbai to Chennai for the 22nd. (Scrolls through and touches a flight listing) Okay, I selected my flight, I want to checkout. Sure, selecting the Slang Airways flight S9 2910, from Mumbai to Chennai for one person for the 21st. Would you like to pre book some of our award winning in flight refreshments? String Dialogues Together Make them human like
26. Context The glue between conversation dialogues Okay, show me flights to Chennai You want to see flights from Mumbai to Chennai for the 21st? Intent: Book Flight Entities: { Source_city: Mumbai Dest_city: Chennai Date: 21-12-2018 Return: False }
27. Multi- modality Alright, showing flights from Mumbai to Chennai for the 22nd. (Scrolls through and touches a flight listing) Okay, I selected my flight, I want to checkout. Switch between Touch and Speech Seamlessly Select the 2nd one and go to checkout.
28. Error Handling
29. Error Handling No speech detected Speech Detected, not recognized Speech recognized incorrectly Recognized but not handled Not handled at application business layer Escalating Errors Novice vs Experienced Users
30. Signifiers How do the users know what is speech enabled what is not? GUI elements Vocal Elements Icons Help Screens On Boarding Screens Beepers Vocal Messages Vocal On Boarding
31. Some Brain Candy
32. Design Patterns Reuse - Reduce - Recycle Most VUI patterns will repeat themselves. Lists Dialogues will repeat Filters Help Conversation Markers GUI Flow Control Stringing Intents *Many more ...
33. Intelligence Small Talk General Knowledge Coreference resolution Hey Slang, who made you? Can you me show some auburn T shirts My food hasn’t arrived yet, is it on the way? Understanding Slang Yeah bro, some deep inception stuff
34. Personality *Stolen Used without permission from the New Yorker
35. Personalization Facts: Hi Ved, how are you Learned Profile: Book me the usual. Personalized Vocabulary: Add some aloo and pyaaz.
36. Let’s address the other in the room
37. Privacy
38. Let’s Recap?
39. Now, a for
40. Workshop Time And as usual, any Questions? If we have time left over (or interest)
41. Thank You 42@slanglabs.in