Building Conversational Experiences for Google Assistant '18

Meet your Google Assistant. Ask it questions. Tell it to do things. It’s your own personal Google, always ready to help. This session will go through Google Assistant features, how to use, and how can we develop for it too. We'll have real demo on some of Google devices that supports Assistant, and will explore both easy and advanced ways to develop real world voice commands and actions for it. You're free to use these slides in your talks, I'd appreciate giving credits though - https://goo.gl/5swDc4

Transcript

1. Building Conversational Experiences for Google Assistant ‘18 Abdelrahman Omran | Rinvex | @Omranic | me@omranic.com
2. Be excellent to each other. Treat everyone with respect. Participate while acknowledging that everyone deserves to be here — and each of us has the right to enjoy our experience without fear of harassment, discrimination, or condescension, whether blatant or via micro-aggressions. Jokes shouldn’t demean others. Consider what you are saying and how it would feel if it were said to or about you. Practice saying "Yes and" to each other. It’s a theatre improv technique to build on each other’s ideas. We all benefit when we create together. Speak up if you see or hear something. Harassment is not tolerated, and you are empowered to politely engage when you or others are disrespected. The person making you feel uncomfortable may not be aware of what they are doing, and politely bringing their behavior to their attention is encouraged. In case you see or hear something or feel uncomfortable, reach out to dev-mena@google.com Code of Conduct
4. Ready to help across 500M+ devices
5. A growing audience: devices
6. Japanese German FR French CA French Korean UK English AU English CA English US English Italian Spanish BR Portuguese LATAM Spanish Hindi & IN English A growing audience: languages Swedish Danish Norwegian Indonesian Russian Thai Dutch
7. Google Home/Mobile device — The surface to interact with the Assistant. The Google Assistant — A conversation between you and Google that helps you get things done in your world. Actions on Google — How developers can extend the assistant (via Assistant apps)
9. The Mobile Voice Study 55% of teens and 41% of adults Use voice search more than once a day http://googleblog.blogspot.com/2014/10/omg-mobile-voice-survey-reveals-teens.html
10. Teens Adults http://googleblog.blogspot.com/2014/10/omg-mobile-voice-survey-reveals-teens.html
11. Actions On Google Direct and Conversation Actions
12. Turn on the living room lights Sure thing Play my dinner party playlist on Spotify Ok, here you go. Hi, how can I help?
13. I need an Uber Sure! I’ll connect you to Uber Hi Scott, where are you headed? I’m going to the Ferry Building Ok, would you like an UberX again? No, we’ll need an UberXL this time Ok, Ahmed will arrive in 3 minutes in a black Toyota Sequoia Hi, how can I help?
16. How does it work? The life cycle of voice action
18. Assistant app { conversation api request } { conversation api response } user input app response
19. How to get started? 1. Design 2. Develop 3. Discover
20. 01 Create Persona 02 Think out of Actions 03 Context 04 Speech Errors 05 Think Big Design
21. 01 Create a Dialogflow agent 02 Create intents & entities 03 Define User says phrases 04 Define Actions & Responses 05 Test it! Develop
22. 01 Create a new cloud project 02 Register your Conversation Action 03 Deploy your actions 04 Confirm your deployment 05 Interact with your Conversation Action Deploy
23. Conversational Design Intro Top 3 Design Tips
24. [1] Create a Persona
25. A Persona is conveyed through: • Tone • Word and phrase choices • Functional design • Style • Technique • Voice And it is based on: • Your user population • Their needs • The imagery & qualities associated with your brand
27. Cowabunga, dude! What flavors do you crave? The bus is approaching your stop now. Your trip will take 45 minutes.
28. Buy a pizza Getting there soon Purchase a ticket Arriving soon Look at your order Review your schedule
29. [2] Think outside the box. Literally!
30. code code code code code code code code code code code “ dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code code code STRUCTURE CODE
31. code code code code code code code code code code code “ dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code code code STRUCTURE CODE
32. DIALOG STRUCTURE
33. [3] In a conversation, there are no “errors”
34. real life user conditions {
35. VUI Tips & Tricks for better voice user experience
36. GUI : VUI What’s the difference?
37. “I don’t understand you.”
38. When a so-called “error” occurs in a conversation, it should be treated simply as a new turn in the dialog, only with different conditions.
39. Users have intent
40. Conversation repair for timeouts USER Ok Google, let me talk to Number Genie. GOOGLE ASSISTANT Sure, Here's Number Genie. NUMBER GENIE Welcome to Number Genie! I'm thinking of a number from 0 to 100. What's your first guess? USER (says nothing) NUMBER GENIE I didn't hear a number. USER (silent or muffled) NUMBER GENIE If you're still there, what's your guess? Sample Dialogs What to include: 1. Canonical “Happy path” 2. First time experience 3. Tapered experience (Return user) 4. Repair
41. Quit game. User gives up and ends the game USER Hey Google, let me talk to Number Genie. GOOGLE ASSISTANT Sure, Here's Number Genie. NUMBER GENIE Welcome to Number Genie! I'm thinking of a number from 0 to 100. What's your first guess? USER 21- NUMBER GENIE It’s higher than 21. What’s your next guess? USER I give up. [LOGIC: catch ending intent] NUMBER GENIE Sure, I’ll tell you the number anyway. It was 90. What to include: 1. Canonical “Happy path” 2. First time experience 3. Tapered experience (Return user) 4. Repair Sample Dialogs
42. Give users credit They know how to talk Avoid force feeding commands. (such as “to reply to a message, say ‘reply’” or “to hear that again, say ‘repeat’”.) People hardly ever say nonsense. They just weren’t parsed correctly. There was a misalignment in meaning. They know what they want There’s no such thing as a query with no intent. The user wanted to do something, even if they didn’t say so. They want to help Don’t create dead ends when something doesn’t work. If you ask for them to say something again, they will, but rarely exactly the same way. Honor that they’re invested in outcome of a repair attempt.
43. Use the context Environmental Where is the user? What are they doing? What type of device are they using? Situational What’s their intent? Where is the user’s frame of mind? Temporal What happened right before? What’s happening next? How is the experience influenced over time? Behavioral What do users know going in? What data can you use to enrich the experience predictively or to cut out unnecessary steps?
44. Rapid reprompt “What was that?” “Say that again?” Reframe the question “What time is this for?” → “Sorry, what time?” “For when?” → “What time would you like to book this for?” Ask another way “If it helps, we can do this one piece a time.” Be ready for questions about the question “I have your name and email from your account, so now all I need is your phone number.” “You can give me the day, the time, or both.” Be proactive “I could put you down for 6 for now, does that work?” “Do you want to finish this later?” Example strategies to have ready
45. Implementation Details Let’s get technical, and do a quick demo
47. Well, it’s kind of cold outside, so I'd like something to warm me up, like a hot soup, and I want it fast. I have some chicken, and also canned tomatoes.
48. Ok Google, talk to Personal Chef Well, it’s kind of cold outside, so I’d like... Sure, here’s Personal Chef What are you in the mood for? What protein would you like to use? Speech to Text NLP Knowledge Graph ML Ranking User Profile Text to Speech Speech to Text Text to Speech ... Invoke Personal Chef action Parse query and generate response
49. What about smart displays Can we get the same amazing experience of Google Assistant on smart displays?!
51. Create and publish your app Get started by building actions for your app
52. Use existing templates goo.gl/ggpCr1 goo.gl/h1jaaS goo.gl/5rVBTH
53. Build a custom app goo.gl/bhToDa goo.gl/TZsupv goo.gl/C4AXBr
54. What’s next? To the infinity and beyond™
56. g.co/dev/ActionsDesign
57. developers.google.com/actions
58. codelabs.developers.google.com
59. goo.gl/PMdr7N
60. Additional Resources Community Program: developers.google.com/actions G+ Developer community: g.co/actionsdev Videos: bit.ly/aog-tips Dialogflow: dialogflow.com/google-assistant Twitter: actionsongoogle@ Our mailing list: assistant.google.com/developer
62. Thank you! Abdelrahman Omran | @Omranic | me@omranic.com