Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Conversational Experiences for Google Assistant '18

Building Conversational Experiences for Google Assistant '18

Meet your Google Assistant. Ask it questions. Tell it to do things. It’s your own personal Google, always ready to help. This session will go through Google Assistant features, how to use, and how can we develop for it too. We'll have real demo on some of Google devices that supports Assistant, and will explore both easy and advanced ways to develop real world voice commands and actions for it. You're free to use these slides in your talks, I'd appreciate giving credits though - https://goo.gl/5swDc4

Abdelrahman Omran

May 15, 2018
Tweet

More Decks by Abdelrahman Omran

Other Decks in Technology

Transcript

  1. Building Conversational Experiences
    for Google Assistant ‘18
    Abdelrahman Omran | Rinvex | @Omranic | [email protected]

    View full-size slide

  2. Be excellent to each other.
    Treat everyone with respect. Participate while
    acknowledging that everyone deserves to be here — and
    each of us has the right to enjoy our experience without fear
    of harassment, discrimination, or condescension, whether
    blatant or via micro-aggressions. Jokes shouldn’t demean
    others. Consider what you are saying and how it would feel
    if it were said to or about you.
    Practice saying "Yes and" to each other.
    It’s a theatre improv technique to build on each other’s
    ideas. We all benefit when we create together.
    Speak up if you see or hear something.
    Harassment is not tolerated, and you are empowered
    to politely engage when you or others are
    disrespected. The person making you feel
    uncomfortable may not be aware of what they are
    doing, and politely bringing their behavior to their
    attention is encouraged.
    In case you see or hear something or feel
    uncomfortable, reach out to [email protected]
    Code of Conduct

    View full-size slide

  3. Ready to
    help across
    500M+ devices

    View full-size slide

  4. A growing audience:
    devices

    View full-size slide

  5. Japanese
    German
    FR French
    CA French
    Korean
    UK English
    AU English
    CA English
    US English
    Italian
    Spanish BR Portuguese
    LATAM Spanish
    Hindi & IN English
    A growing audience:
    languages
    Swedish
    Danish
    Norwegian Indonesian
    Russian
    Thai
    Dutch

    View full-size slide

  6. Google Home/Mobile device — The surface to interact with
    the Assistant.
    The Google Assistant — A conversation between you and
    Google that helps you get things done in your world.
    Actions on Google — How developers can extend the
    assistant (via Assistant apps)

    View full-size slide

  7. The Mobile Voice Study
    55% of teens and 41% of adults
    Use voice search more than once a day
    http://googleblog.blogspot.com/2014/10/omg-mobile-voice-survey-reveals-teens.html

    View full-size slide

  8. Teens
    Adults
    http://googleblog.blogspot.com/2014/10/omg-mobile-voice-survey-reveals-teens.html

    View full-size slide

  9. Actions On Google
    Direct and Conversation Actions

    View full-size slide

  10. Turn on the living room lights
    Sure thing
    Play my dinner party playlist on Spotify
    Ok, here you go.
    Hi, how can I help?

    View full-size slide

  11. I need an Uber
    Sure! I’ll connect you to Uber
    Hi Scott, where are you headed?
    I’m going to the Ferry Building
    Ok, would you like an UberX again?
    No, we’ll need an UberXL this time
    Ok, Ahmed will arrive in 3 minutes
    in a black Toyota Sequoia
    Hi, how can I help?

    View full-size slide

  12. How does it work?
    The life cycle of voice action

    View full-size slide

  13. Assistant app
    { conversation
    api request }
    { conversation
    api response }
    user input
    app
    response

    View full-size slide

  14. How to get started?
    1. Design 2. Develop 3. Discover

    View full-size slide

  15. 01 Create Persona
    02 Think out of Actions
    03 Context
    04 Speech Errors
    05 Think Big
    Design

    View full-size slide

  16. 01 Create a Dialogflow agent
    02 Create intents & entities
    03 Define User says phrases
    04 Define Actions & Responses
    05 Test it!
    Develop

    View full-size slide

  17. 01 Create a new cloud project
    02 Register your Conversation Action
    03 Deploy your actions
    04 Confirm your deployment
    05 Interact with your Conversation Action
    Deploy

    View full-size slide

  18. Conversational Design Intro
    Top 3 Design Tips

    View full-size slide

  19. [1] Create a Persona

    View full-size slide

  20. A Persona is conveyed through:
    • Tone
    • Word and phrase choices
    • Functional design
    • Style
    • Technique
    • Voice
    And it is based on:
    • Your user population
    • Their needs
    • The imagery &
    qualities associated
    with your brand

    View full-size slide

  21. Cowabunga, dude!
    What flavors do you crave?
    The bus is approaching your stop now.
    Your trip will take 45 minutes.

    View full-size slide

  22. Buy a pizza
    Getting there soon
    Purchase a ticket
    Arriving soon
    Look at your order Review your schedule

    View full-size slide

  23. [2] Think outside the box.
    Literally!

    View full-size slide

  24. code code code code code
    code code code code
    code code “ dialog string” code
    code code code
    code code code code code
    code “dialog string” code code
    code code code code code code code
    code code code
    code code
    code
    code code
    code
    code
    code code code
    code “dialog string” code code code
    code code
    code code code
    code code code code code code code code
    code code code code code code code code
    code
    STRUCTURE CODE

    View full-size slide

  25. code code code code code
    code code code code
    code code “ dialog string” code
    code code code
    code code code code code
    code “dialog string” code code
    code code code code code code code
    code code code
    code code
    code
    code code
    code
    code
    code code code
    code “dialog string” code code code
    code code
    code code code
    code code code code code code code code
    code code code code code code code code
    code
    STRUCTURE CODE

    View full-size slide

  26. DIALOG STRUCTURE

    View full-size slide

  27. [3] In a conversation,
    there are no “errors”

    View full-size slide

  28. real life
    user
    conditions
    {

    View full-size slide

  29. VUI Tips & Tricks
    for better voice user experience

    View full-size slide

  30. GUI : VUI
    What’s the difference?

    View full-size slide

  31. “I don’t understand you.”

    View full-size slide

  32. When a so-called “error” occurs
    in a conversation, it should be treated
    simply as a new turn in the dialog,
    only with different conditions.

    View full-size slide

  33. Users have intent

    View full-size slide

  34. Conversation repair for timeouts
    USER Ok Google, let me talk to Number Genie.
    GOOGLE ASSISTANT Sure, Here's Number Genie.
    NUMBER GENIE Welcome to Number Genie! I'm thinking of a number from 0
    to 100. What's your first guess?
    USER (says nothing)
    NUMBER GENIE I didn't hear a number.
    USER (silent or muffled)
    NUMBER GENIE If you're still there, what's your guess?
    Sample Dialogs
    What to include:
    1. Canonical “Happy path”
    2. First time experience
    3. Tapered experience
    (Return user)
    4. Repair

    View full-size slide

  35. Quit game. User gives up and ends the game
    USER Hey Google, let me talk to Number Genie.
    GOOGLE ASSISTANT Sure, Here's Number Genie.
    NUMBER GENIE Welcome to Number Genie! I'm thinking of a number from 0
    to 100. What's your first guess?
    USER 21-
    NUMBER GENIE It’s higher than 21. What’s your next guess?
    USER I give up. [LOGIC: catch ending intent]
    NUMBER GENIE Sure, I’ll tell you the number anyway. It was 90.
    What to include:
    1. Canonical “Happy path”
    2. First time experience
    3. Tapered experience
    (Return user)
    4. Repair
    Sample Dialogs

    View full-size slide

  36. Give users credit
    They know how to talk
    Avoid force feeding commands.
    (such as “to reply to a message,
    say ‘reply’” or “to hear that
    again, say ‘repeat’”.)
    People hardly ever say
    nonsense. They just weren’t
    parsed correctly. There was a
    misalignment in meaning.
    They know what they want
    There’s no such thing as a query with
    no intent. The user wanted to do
    something, even if they didn’t say so.
    They want to help
    Don’t create dead ends when
    something doesn’t work.
    If you ask for them to say
    something again, they will, but
    rarely exactly the same way.
    Honor that they’re invested in
    outcome of a repair attempt.

    View full-size slide

  37. Use the context
    Environmental
    Where is the user?
    What are they doing?
    What type of device
    are they using?
    Situational
    What’s their intent?
    Where is the user’s
    frame of mind?
    Temporal
    What happened right before?
    What’s happening next?
    How is the experience
    influenced over time?
    Behavioral
    What do users know
    going in?
    What data can you use to
    enrich the experience
    predictively or to cut out
    unnecessary steps?

    View full-size slide

  38. Rapid reprompt “What was that?”
    “Say that again?”
    Reframe the question “What time is this for?” → “Sorry, what time?”
    “For when?” → “What time would you like to book this for?”
    Ask another way “If it helps, we can do this one piece a time.”
    Be ready for questions
    about the question
    “I have your name and email from your account, so now all I need is
    your phone number.”
    “You can give me the day, the time, or both.”
    Be proactive “I could put you down for 6 for now, does that work?”
    “Do you want to finish this later?”
    Example strategies to have ready

    View full-size slide

  39. Implementation Details
    Let’s get technical, and do a quick demo

    View full-size slide

  40. Well, it’s kind of cold outside, so I'd like
    something to warm me up, like a hot
    soup, and I want it fast.
    I have some chicken, and also canned
    tomatoes.

    View full-size slide

  41. Ok Google,
    talk to
    Personal Chef
    Well, it’s kind of
    cold outside, so
    I’d like...
    Sure, here’s
    Personal Chef
    What are you
    in the mood
    for?
    What protein
    would you like
    to use?
    Speech to Text
    NLP
    Knowledge Graph
    ML Ranking
    User Profile
    Text to Speech
    Speech to Text
    Text to Speech
    ...
    Invoke Personal
    Chef action
    Parse query and
    generate
    response

    View full-size slide

  42. What about smart displays
    Can we get the same amazing experience of Google Assistant on smart displays?!

    View full-size slide

  43. Create and publish your app
    Get started by building actions for your app

    View full-size slide

  44. Use existing templates
    goo.gl/ggpCr1 goo.gl/h1jaaS
    goo.gl/5rVBTH

    View full-size slide

  45. Build a custom app
    goo.gl/bhToDa goo.gl/TZsupv
    goo.gl/C4AXBr

    View full-size slide

  46. What’s next?
    To the infinity and beyond™

    View full-size slide

  47. g.co/dev/ActionsDesign

    View full-size slide

  48. developers.google.com/actions

    View full-size slide

  49. codelabs.developers.google.com

    View full-size slide

  50. goo.gl/PMdr7N

    View full-size slide

  51. Additional Resources
    Community Program: developers.google.com/actions
    G+ Developer community: g.co/actionsdev
    Videos: bit.ly/aog-tips
    Dialogflow: dialogflow.com/google-assistant
    Twitter: actionsongoogle@
    Our mailing list: assistant.google.com/developer

    View full-size slide

  52. Thank you!
    Abdelrahman Omran | @Omranic | [email protected]

    View full-size slide