Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Conversational Experiences for Google Assistant '18

Building Conversational Experiences for Google Assistant '18

Meet your Google Assistant. Ask it questions. Tell it to do things. It’s your own personal Google, always ready to help. This session will go through Google Assistant features, how to use, and how can we develop for it too. We'll have real demo on some of Google devices that supports Assistant, and will explore both easy and advanced ways to develop real world voice commands and actions for it. You're free to use these slides in your talks, I'd appreciate giving credits though - https://goo.gl/5swDc4

D05cd3860dfbf8c68cba35bd6a7ae0ee?s=128

Abdelrahman Omran

May 15, 2018
Tweet

Transcript

  1. Building Conversational Experiences for Google Assistant ‘18 Abdelrahman Omran |

    Rinvex | @Omranic | me@omranic.com
  2. Be excellent to each other. Treat everyone with respect. Participate

    while acknowledging that everyone deserves to be here — and each of us has the right to enjoy our experience without fear of harassment, discrimination, or condescension, whether blatant or via micro-aggressions. Jokes shouldn’t demean others. Consider what you are saying and how it would feel if it were said to or about you. Practice saying "Yes and" to each other. It’s a theatre improv technique to build on each other’s ideas. We all benefit when we create together. Speak up if you see or hear something. Harassment is not tolerated, and you are empowered to politely engage when you or others are disrespected. The person making you feel uncomfortable may not be aware of what they are doing, and politely bringing their behavior to their attention is encouraged. In case you see or hear something or feel uncomfortable, reach out to dev-mena@google.com Code of Conduct
  3. None
  4. Ready to help across 500M+ devices

  5. A growing audience: devices

  6. Japanese German FR French CA French Korean UK English AU

    English CA English US English Italian Spanish BR Portuguese LATAM Spanish Hindi & IN English A growing audience: languages Swedish Danish Norwegian Indonesian Russian Thai Dutch
  7. Google Home/Mobile device — The surface to interact with the

    Assistant. The Google Assistant — A conversation between you and Google that helps you get things done in your world. Actions on Google — How developers can extend the assistant (via Assistant apps)
  8. None
  9. The Mobile Voice Study 55% of teens and 41% of

    adults Use voice search more than once a day http://googleblog.blogspot.com/2014/10/omg-mobile-voice-survey-reveals-teens.html
  10. Teens Adults http://googleblog.blogspot.com/2014/10/omg-mobile-voice-survey-reveals-teens.html

  11. Actions On Google Direct and Conversation Actions

  12. Turn on the living room lights Sure thing Play my

    dinner party playlist on Spotify Ok, here you go. Hi, how can I help?
  13. I need an Uber Sure! I’ll connect you to Uber

    Hi Scott, where are you headed? I’m going to the Ferry Building Ok, would you like an UberX again? No, we’ll need an UberXL this time Ok, Ahmed will arrive in 3 minutes in a black Toyota Sequoia Hi, how can I help?
  14. None
  15. None
  16. How does it work? The life cycle of voice action

  17. None
  18. Assistant app { conversation api request } { conversation api

    response } user input app response
  19. How to get started? 1. Design 2. Develop 3. Discover

  20. 01 Create Persona 02 Think out of Actions 03 Context

    04 Speech Errors 05 Think Big Design
  21. 01 Create a Dialogflow agent 02 Create intents & entities

    03 Define User says phrases 04 Define Actions & Responses 05 Test it! Develop
  22. 01 Create a new cloud project 02 Register your Conversation

    Action 03 Deploy your actions 04 Confirm your deployment 05 Interact with your Conversation Action Deploy
  23. Conversational Design Intro Top 3 Design Tips

  24. [1] Create a Persona

  25. A Persona is conveyed through: • Tone • Word and

    phrase choices • Functional design • Style • Technique • Voice And it is based on: • Your user population • Their needs • The imagery & qualities associated with your brand
  26. None
  27. Cowabunga, dude! What flavors do you crave? The bus is

    approaching your stop now. Your trip will take 45 minutes.
  28. Buy a pizza Getting there soon Purchase a ticket Arriving

    soon Look at your order Review your schedule
  29. [2] Think outside the box. Literally!

  30. code code code code code code code code code code

    code “ dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code code code STRUCTURE CODE
  31. code code code code code code code code code code

    code “ dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code code code STRUCTURE CODE
  32. DIALOG STRUCTURE

  33. [3] In a conversation, there are no “errors”

  34. real life user conditions {

  35. VUI Tips & Tricks for better voice user experience

  36. GUI : VUI What’s the difference?

  37. “I don’t understand you.”

  38. When a so-called “error” occurs in a conversation, it should

    be treated simply as a new turn in the dialog, only with different conditions.
  39. Users have intent

  40. Conversation repair for timeouts USER Ok Google, let me talk

    to Number Genie. GOOGLE ASSISTANT Sure, Here's Number Genie. NUMBER GENIE Welcome to Number Genie! I'm thinking of a number from 0 to 100. What's your first guess? USER (says nothing) NUMBER GENIE I didn't hear a number. USER (silent or muffled) NUMBER GENIE If you're still there, what's your guess? Sample Dialogs What to include: 1. Canonical “Happy path” 2. First time experience 3. Tapered experience (Return user) 4. Repair
  41. Quit game. User gives up and ends the game USER

    Hey Google, let me talk to Number Genie. GOOGLE ASSISTANT Sure, Here's Number Genie. NUMBER GENIE Welcome to Number Genie! I'm thinking of a number from 0 to 100. What's your first guess? USER 21- NUMBER GENIE It’s higher than 21. What’s your next guess? USER I give up. [LOGIC: catch ending intent] NUMBER GENIE Sure, I’ll tell you the number anyway. It was 90. What to include: 1. Canonical “Happy path” 2. First time experience 3. Tapered experience (Return user) 4. Repair Sample Dialogs
  42. Give users credit They know how to talk Avoid force

    feeding commands. (such as “to reply to a message, say ‘reply’” or “to hear that again, say ‘repeat’”.) People hardly ever say nonsense. They just weren’t parsed correctly. There was a misalignment in meaning. They know what they want There’s no such thing as a query with no intent. The user wanted to do something, even if they didn’t say so. They want to help Don’t create dead ends when something doesn’t work. If you ask for them to say something again, they will, but rarely exactly the same way. Honor that they’re invested in outcome of a repair attempt.
  43. Use the context Environmental Where is the user? What are

    they doing? What type of device are they using? Situational What’s their intent? Where is the user’s frame of mind? Temporal What happened right before? What’s happening next? How is the experience influenced over time? Behavioral What do users know going in? What data can you use to enrich the experience predictively or to cut out unnecessary steps?
  44. Rapid reprompt “What was that?” “Say that again?” Reframe the

    question “What time is this for?” → “Sorry, what time?” “For when?” → “What time would you like to book this for?” Ask another way “If it helps, we can do this one piece a time.” Be ready for questions about the question “I have your name and email from your account, so now all I need is your phone number.” “You can give me the day, the time, or both.” Be proactive “I could put you down for 6 for now, does that work?” “Do you want to finish this later?” Example strategies to have ready
  45. Implementation Details Let’s get technical, and do a quick demo

  46. None
  47. Well, it’s kind of cold outside, so I'd like something

    to warm me up, like a hot soup, and I want it fast. I have some chicken, and also canned tomatoes.
  48. Ok Google, talk to Personal Chef Well, it’s kind of

    cold outside, so I’d like... Sure, here’s Personal Chef What are you in the mood for? What protein would you like to use? Speech to Text NLP Knowledge Graph ML Ranking User Profile Text to Speech Speech to Text Text to Speech ... Invoke Personal Chef action Parse query and generate response
  49. What about smart displays Can we get the same amazing

    experience of Google Assistant on smart displays?!
  50. None
  51. Create and publish your app Get started by building actions

    for your app
  52. Use existing templates goo.gl/ggpCr1 goo.gl/h1jaaS goo.gl/5rVBTH

  53. Build a custom app goo.gl/bhToDa goo.gl/TZsupv goo.gl/C4AXBr

  54. What’s next? To the infinity and beyond™

  55. None
  56. g.co/dev/ActionsDesign

  57. developers.google.com/actions

  58. codelabs.developers.google.com

  59. goo.gl/PMdr7N

  60. Additional Resources Community Program: developers.google.com/actions G+ Developer community: g.co/actionsdev Videos:

    bit.ly/aog-tips Dialogflow: dialogflow.com/google-assistant Twitter: actionsongoogle@ Our mailing list: assistant.google.com/developer
  61. None
  62. Thank you! Abdelrahman Omran | @Omranic | me@omranic.com