Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to build & measure natural conversations for the Google Assistant

Lee Boonstra
February 06, 2019

How to build & measure natural conversations for the Google Assistant

How to build & measure natural conversations for the Google Assistant

Lee Boonstra

February 06, 2019
Tweet

More Decks by Lee Boonstra

Other Decks in Technology

Transcript

  1. 1
    How to build & measure
    natural conversations for
    the Google Assistant
    Lee Boonstra
    Customer Engineer, Google
    Twitter: @ladysign

    View full-size slide

  2. 2
    Lee Boonstra
    @ladysign

    View full-size slide

  3. 3
    A conversation between
    you and Google
    that helps you get more
    done in your world
    Meet the Google Assistant

    View full-size slide

  4. 4
    Terminology
    Google Assistant — The virtual assistant of
    Google.
    Out of the box on Android 6+. For iOS available as
    app.
    Action — A third party app, running on top of the
    Google Assistant.
    Google Home — Voice-activated speaker powered
    by Voice.
    Smart Display — Voice-activated speaker with
    screen powered by Voice and Touch.

    View full-size slide

  5. 5
    ● Google Assistant is now in 80 countries
    with support for 30 languages.
    ● Google Assistant will soon be on 1
    billion devices thanks to Android
    phones

    View full-size slide

  6. 7
    Assistance to improve your customer
    experience
    Callbots in callcenters
    Chatbots in websites, apps
    or social media platforms.
    Voice Activated Speakers &
    smart assistants.

    View full-size slide

  7. 8
    50% of enterprises spend more time on
    creating bots than on mobile app
    development by 2021
    —Gartner

    View full-size slide

  8. 9
    Chatbots are expected to trim business costs
    by more than $8 billion per year by 2022.
    - Juniper Research

    View full-size slide

  9. Confidential + Proprietary
    Confidential + Proprietary
    Chatbots exists since the 90’s…
    So why are they popular now?

    View full-size slide

  10. 11
    1994, the first chatbot was born
    2016, Focus on AI First

    View full-size slide

  11. 12
    Machine Learning
    How did you learn your first
    language?
    or

    View full-size slide

  12. 13
    It’s easier to make computers
    learn than to build smarter computers
    Machine Learning
    Process of making
    computers learn
    Artificial Intelligence
    Process of building
    smarter computers

    View full-size slide

  13. 14
    It’s all about Machine Learning
    ● Natural Language Understanding
    ● Text to Speech
    ● Speech Models (Wavenet)

    View full-size slide

  14. 15
    70% of
    requests are in
    natural
    language
    Is it gonna rain tomorrow?
    Tomorrow it will be sunny
    In Amsterdam!

    View full-size slide

  15. Confidential + Proprietary
    Confidential + Proprietary
    You will need to design your conversation

    View full-size slide

  16. 18
    Website
    Filter on account name
    or account number
    Lot’s of results on a screen.

    View full-size slide

  17. 19
    Website with Natural Language...
    Natural way of asking!

    View full-size slide

  18. 20
    How much have I spent on
    taxis last month?
    It looks like, you spent about
    20 euros on taxis last month.
    You took the TCA taxi twice.
    Voice channels
    There’s no screen.
    Focus on the conversation.

    View full-size slide

  19. 21
    It looks like, you spent about 20 euros
    on taxis last month. You took the TCA
    taxi twice.
    Here’s an overview:
    Voice channels with screens
    How much have I spent on
    taxis last month?
    Focus on the conversation.
    But also display stuff.

    View full-size slide

  20. 22
    Assistance is not just about voice

    View full-size slide

  21. Confidential + Proprietary
    Confidential + Proprietary
    What happens under the hood?

    View full-size slide

  22. Google | Proprietary &
    Confidential
    24
    GOOGLE ASSISTANT
    USER
    Hey Google..
    Idle

    View full-size slide

  23. Google | Proprietary &
    Confidential
    25
    GOOGLE ASSISTANT
    USER
    Hey Google..
    ..will it rain
    today?
    Listening

    View full-size slide

  24. Google | Proprietary &
    Confidential
    26
    GOOGLE ASSISTANT
    USER WEB SERVER
    Hey Google..
    ..will it rain
    today?
    GET
    www.weather.com/info
    city: Amsterdam
    Date: 2019-02-06
    Recognizing

    View full-size slide

  25. Google | Proprietary &
    Confidential
    27
    GOOGLE ASSISTANT
    USER WEB SERVER
    Hey Google..
    ..will it rain
    today?
    GET
    www.weather.com/info
    city: Amsterdam
    Date: 2019-02-06
    {
    location: “amsterdam”
    weather: “rain”,
    temperature: 8
    }
    Thinking

    View full-size slide

  26. Google | Proprietary &
    Confidential
    28
    GOOGLE ASSISTANT
    USER WEB SERVER
    Hey Google..
    ..will it rain
    today?
    GET
    www.weather.com/info
    city: Amsterdam
    Date: 2019-02-06
    {
    location: “amsterdam”
    weather: “rain”,
    temperature: 8
    }
    Yes, it will rain
    in Amsterdam
    all day today.
    Speaking

    View full-size slide

  27. Confidential + Proprietary
    Confidential + Proprietary
    How can you build your own action
    on top of the Google Assistant?

    View full-size slide

  28. 30
    3rd party integration
    Extend the Google
    Assistant
    with your own custom
    actions.
    Hey Google, let me talk to BookAMeetingRoom
    Welcome, how can I help you?
    I want to book a meeting room
    for 3 persons.
    Let’s get BookAMeetingRoom
    Sure, for when?
    Tomorrow, from 2pm to 3pm.

    View full-size slide

  29. 31
    Ok Google, talk
    to __________.
    Ok Google, connect
    me with __________.
    Ok Google, get
    __________.
    Start a 3rd party action
    There is an app directory!
    (appstore).
    And the Google Assistant can
    give app suggestions.

    View full-size slide

  30. 32
    Many brands are available on
    Google Assistant
    ● Bol.com
    ● Albert Heijn
    ● Post.nl
    ● Rabobank
    ● Centraal Beheer
    ● ...

    View full-size slide

  31. 33
    ● Write the conversation - Dialogflow (Enterprise)
    ● Deploy on GA+ UX components - Actions on Google
    Optional:
    ● Back-end integration - Fulfillment app (dialogflow/aog SDK)
    ● Communication to back-ends - Your own APIs
    What do I need to build
    my own action?

    View full-size slide

  32. 34
    ● Previously known as API.AI
    ○ (Sept 2016, acquired by Google)
    ● Powered by Machine Learning:
    ○ Natural Language Understanding (NLU)
    ○ Intent Matching
    ○ Conversation Training
    ● Cross platform
    ● Build faster with the Web UI
    ● Scalable: separate your conversation text from
    code
    ● Speech / Voice Integration
    ● Multi-lingual bot support (20+ languages)
    ● Part of Google Cloud (60+ cloud services)
    Development
    suite for building
    Conversational
    UIs.

    View full-size slide

  33. 35
    ● Bring your agents to smart speakers (Google
    Home) or phones (Android, iOS app)
    ● Actions on Google includes:
    ○ UI toolkit,
    ○ Audio toolkit
    ○ Account Linking API
    ○ SDKs
    ○ tutorial guides
    ● UI components such as:
    ○ Buttons, Images
    ○ Cards, Carousels,
    ○ Lists
    ○ Tables
    Program for developers of
    Actions (“apps”) that run via
    Google Assistant
    Actions on Google

    View full-size slide

  34. Confidential + Proprietary
    Demo

    View full-size slide

  35. 37
    Dialogflow has been used by over 600,000+
    developers

    View full-size slide

  36. 39
    Intent Responses

    View full-size slide

  37. 41
    ● Google Assistant
    ● Amazon Alexa
    ● MS Cortana
    ● Web
    ● iOS
    ● Android
    Integrations
    ● Facebook Messenger
    ● Twitter
    ● Slack
    ● Telegram
    ● Skype
    ● Cisco Spark
    ● Dialogflow Phone
    ● Twilio
    ● Viber
    ● Kik
    ● Line

    View full-size slide

  38. 42
    Fulfillment

    View full-size slide

  39. Confidential + Proprietary
    Confidential + Proprietary
    How to measure your app usage?

    View full-size slide

  40. Confidential + Proprietary
    Which customers are unhappy and why?

    View full-size slide

  41. Confidential + Proprietary
    How can I improve the user experience?

    View full-size slide

  42. 46
    Sure!
    I’d like to transfer 100 euro
    to my wife her bank
    account.”
    A customer communicates with
    the Google Assistant

    View full-size slide

  43. 47
    What happens under the hood...

    View full-size slide

  44. 48
    Collect real-time chats
    from Dialogflow SDK

    View full-size slide

  45. 49
    Mask sensitive
    Information
    with DLP API

    View full-size slide

  46. 50
    Understand the text
    with NLP API

    View full-size slide

  47. 51
    Store all data in
    a data-warehouse

    View full-size slide

  48. 52
    Optimize your agent

    View full-size slide

  49. Confidential + Proprietary
    Demo

    View full-size slide

  50. Confidential + Proprietary
    Confidential + Proprietary
    Virtual assistance is an important channel.
    Your customers expect to find you there.

    View full-size slide

  51. Confidential + Proprietary
    Confidential + Proprietary
    Virtual assistance complements
    the other channels.
    (social media, website, contact center...)

    View full-size slide

  52. BigQuery
    Text to
    S Speech
    Google
    Assistant
    Website
    Social media
    Channel
    Agent
    Assist
    Call Center
    Agent
    Your System

    View full-size slide

  53. Confidential + Proprietary
    Confidential + Proprietary
    But if even if you just add one new AI channel.
    You can improve your customer experience
    and trim business costs.

    View full-size slide

  54. Like smart assistants...
    Dialogflow
    Enterprise
    Google
    Assistant
    Call Center
    Agent

    View full-size slide

  55. Confidential + Proprietary
    Confidential + Proprietary
    Thank you!
    Lee Boonstra
    @ladysign

    View full-size slide