Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Digital Wednesday

Lee Boonstra
November 29, 2018

Digital Wednesday

Lee Boonstra

November 29, 2018
Tweet

More Decks by Lee Boonstra

Other Decks in Technology

Transcript

  1. 1
    Creating Conversations
    For Smart Assistants
    Lee Boonstra
    Customer Engineer, Google
    Twitter: @ladysign

    View full-size slide

  2. 2
    Lee Boonstra
    @ladysign

    View full-size slide

  3. Use cases in the contact center and beyond
    ● Customer service
    ● E-commerce
    ● Home entertainment
    ● Auto
    ● Organizational knowledge
    ● Onboarding & Helpdesk

    View full-size slide

  4. Consumers have high expectations
    Based on data collected in Fall 2017 from sample of 1,051 U.S. adults, ages 18-64.

    View full-size slide

  5. 5
    50% of enterprises spend more time on
    creating bots than on mobile app
    development by 2021
    —Gartner

    View full-size slide

  6. 6
    Chatbots are expected to trim business costs
    by more than $8 billion per year by 2022.
    - Juniper Research

    View full-size slide

  7. Confidential + Proprietary
    Confidential + Proprietary
    Chatbots exists since the 90’s…
    So why are they popular now?

    View full-size slide

  8. 8
    1994, the first chatbot was born
    2016, Focus on AI First

    View full-size slide

  9. 9
    Machine Learning
    How did you learn your first
    language?
    or

    View full-size slide

  10. 10
    It’s easier to make computers
    learn than to build smarter computers
    Machine Learning
    Process of making
    computers learn
    Artificial Intelligence
    Process of building
    smarter computers

    View full-size slide

  11. Dialogflow users benefit from
    world-class AI assets & capabilities,
    including:
    Google is an
    AI company
    Goal: Make AI easy, fast,
    and useful for enterprises
    and developers
    Machine
    learning
    leadership
    Natural language
    understanding
    Knowledge
    graph
    Search across
    unstructured
    data
    Speech
    recognition &
    synthesis
    TPUs and
    unrivaled
    compute infra
    for ML

    View full-size slide

  12. Cloud AI
    Less ML
    expertise
    More ML
    expertise
    Cloud AI solutions
    Cloud
    Job Discovery
    Contact
    Center
    Document
    Understanding
    Cloud AI building blocks
    Cloud AI platform
    Cloud ML
    Engine
    Cloud Video
    Intelligence
    ML professionals & service partners
    ASL
    Professional Services
    Organization
    Cloud
    AutoML Vision
    Vision
    Cloud
    Vision
    Language Conversation
    Cloud Natural
    Language
    Cloud
    AutoML NL
    Dialogflow
    Enterprise
    Cloud
    Translation
    Cloud
    Speech-to-Text
    Cloud
    Text-to-Speech
    Cloud AutoML
    Translation
    New
    New
    New
    Cloud
    GPU
    Cloud
    TPU
    Cloud
    Dataflow
    Cloud
    Dataproc
    Machine & Deep
    Learning
    ML accelerators ML libraries
    Tensorflow Kubeflow
    Kaggle/datasets
    Datasets

    View full-size slide

  13. 13
    Storage Compute

    View full-size slide

  14. 14
    Bots to improve your customer experience
    Callbots in callcenters
    Chatbots in websites, apps
    or social media platforms.
    Voice Activated Speakers &
    smart assistants.

    View full-size slide

  15. 15
    70% of
    requests are in
    natural
    language
    Is it gonna rain tomorrow?
    Tomorrow it will be sunny
    In Amsterdam!

    View full-size slide

  16. 16
    A conversation between
    you and Google
    that helps you get more
    done in your world
    Meet the Google Assistant

    View full-size slide

  17. 17
    Terminology
    Google Assistant — The virtual assistant of
    Google.
    Out of the box on Android 6+. For iOS available as
    app.
    Action — A third party app, running on top of the
    Google Assistant.
    Google Home — Voice-activated speaker powered
    by Voice.
    Smart Display — Voice-activated speaker with
    screen powered by Voice and Touch.

    View full-size slide

  18. 18
    ● Manage tasks
    Ok Google, what’s my confirmation number for my flight?
    ● Plan the day
    Ok Google, how is my commute to the office?
    ● Get answers:
    Ok Google, who is the King of the Netherlands?
    ● Make memories
    Ok Google, remember I left my keys on the Kitchen table.
    ● Control your home
    Ok Google, turn off the bedroom lights.
    ● Enjoy Entertainment
    Ok Google, play the Pixies on Spotify.
    Things you can ask Google...

    View full-size slide

  19. Google | Proprietary &
    Confidential
    19
    Idle Listening Recognizing Speaking
    Thinking

    View full-size slide

  20. Google | Proprietary &
    Confidential
    20
    recognitionResult
    listenEnd
    recognitionResult
    voicestart
    listenStart listenEnd
    voiceEnd
    eventName

    View full-size slide

  21. 21
    3rd party integration
    Extend the Google
    Assistant
    with your own custom
    actions.
    Hey Google, let me talk to BookAMeetingRoom
    Welcome, how can I help you?
    I want to book a meeting room
    for 3 persons.
    Let’s get BookAMeetingRoom
    Sure, for when?
    Tomorrow, from 2pm to 3pm.

    View full-size slide

  22. Confidential + Proprietary
    Confidential + Proprietary
    How can you build your own action
    on top of the Google Assistant?

    View full-size slide

  23. 23
    Sketch (+ Google Assistant UI Toolkit)
    Prototyping tools
    Twine

    View full-size slide

  24. 24
    ● Write the conversation - Dialogflow (Enterprise)
    ● Deploy on GA+ UX components - Actions on Google
    Optional:
    ● Back-end integration - Fulfillment app (dialogflow/aog SDK)
    ● Communication to back-ends - Your own APIs
    Tooling

    View full-size slide

  25. 25
    Testing tools
    Ngrok secure internet tunnels to localhost
    ● Actions on Google Simulator
    ● Device Testing
    ● Dialogflow Simulator
    ● Dialogflow Diagnostics
    ● GCP Stackdriver
    ● Cloud Functions Emulator
    ● Ngrok Secure Tunneling
    ● Dialogflow Training

    View full-size slide

  26. Confidential + Proprietary
    Dialogflow

    View full-size slide

  27. 27
    ● Previously known as API.AI
    ○ (Sept 2016, acquired by Google)
    ● Powered by Machine Learning:
    ○ Natural Language Understanding (NLU)
    ○ Intent Matching
    ○ Conversation Training
    ● Cross platform
    ● Build faster with the Web UI
    ● Scalable: separate your conversation text from
    code
    ● Speech / Voice Integration
    ● Multi-lingual bot support (20+ languages)
    ● Part of Google Cloud (60+ cloud services)
    Development
    suite for building
    Conversational
    UIs.

    View full-size slide

  28. 28
    Dialogflow has been used by over 600,000+
    developers

    View full-size slide

  29. 30
    Intent Responses

    View full-size slide

  30. 32
    ● Google Assistant
    ● Amazon Alexa
    ● MS Cortana
    ● Web
    ● iOS
    ● Android
    Integrations
    ● Facebook Messenger
    ● Twitter
    ● Slack
    ● Telegram
    ● Skype
    ● Cisco Spark
    ● Dialogflow Phone
    ● Twilio
    ● Viber
    ● Kik
    ● Line

    View full-size slide

  31. 33
    Fulfillment

    View full-size slide

  32. 34
    ● Your application back-end code.
    ○ Cloud Function / App Engine / GKE ...?
    ● Router to catch all intents, that have webhooks enabled.
    ● Application code
    ● Response messages
    ● Calls to external / on-premise APIs
    Fulfillment App

    View full-size slide

  33. Confidential + Proprietary
    Actions on Google

    View full-size slide

  34. 36
    ● Bring your agents to smart speakers (Google
    Home) or phones (Android, iOS app)
    ● Actions on Google includes:
    ○ UI toolkit,
    ○ Audio toolkit
    ○ Account Linking API
    ○ SDKs
    ○ tutorial guides
    ● UI components such as:
    ○ Buttons, Images
    ○ Cards, Carousels,
    ○ Lists
    ○ Tables
    Program for developers of
    Actions (“apps”) that run via
    Google Assistant
    Actions on Google

    View full-size slide

  35. 37
    SSML
    Speech Synthesis
    Markup Language (SSML)
    is an XML based markup
    language for defining
    speech.
    ● By using SSML, you can make your agent's
    responses seem more life-like.
    ● Tags for defining:
    ○ Breaks
    ○ Audio snippets
    ○ Type of text constructs (date, time,
    alphabet..)
    ○ Pitch, Speaking rates
    ○ Emphasis

    View full-size slide

  36. 38
    ● Formatting Numbers - ($100.000 - > 100000) http://numeraljs.com/
    ● Format JS internationalization (dates etc) - https://formatjs.io/
    ● Moment JS (dates, times) - https://momentjs.com/
    ● Pluralize - https://www.npmjs.com/package/pluralize
    ● Lightweight Translation Module - (Multi languages in fulfillment apps) -
    https://github.com/mashpie/i18n-node
    Handy JS libraries

    View full-size slide

  37. 39
    Assistance
    is not just about
    voice

    View full-size slide

  38. 40
    Your experience
    can run across
    multiple devices.

    View full-size slide

  39. 41
    Suggestion Chips

    View full-size slide

  40. 42
    Suggestion Chips

    View full-size slide

  41. 43
    Basic Cards

    View full-size slide

  42. 44
    Basic Cards

    View full-size slide

  43. 49
    Media Responses

    View full-size slide

  44. 50
    Media Responses

    View full-size slide

  45. 51
    Table Responses

    View full-size slide

  46. 52
    Table Responses

    View full-size slide

  47. 53
    Action Directory
    Submit your
    action to
    production.

    View full-size slide

  48. Confidential + Proprietary
    Confidential + Proprietary
    How to measure your app usage?

    View full-size slide

  49. Confidential + Proprietary
    Which customers are unhappy and why?

    View full-size slide

  50. Confidential + Proprietary
    How can I improve the user experience?

    View full-size slide

  51. 57
    Sure!
    I’d like to transfer 100 euro
    to my wife her bank
    account.”
    A customer communicates with
    the Google Assistant

    View full-size slide

  52. 58
    What happens under the hood...

    View full-size slide

  53. 59
    Collect real-time chats
    from Dialogflow SDK

    View full-size slide

  54. 60
    Mask sensitive
    Information
    with DLP API

    View full-size slide

  55. 61
    Understand the text
    with NLP API

    View full-size slide

  56. 62
    Store all data in
    a data-warehouse

    View full-size slide

  57. 63
    Optimize your agent

    View full-size slide

  58. Confidential + Proprietary
    Confidential + Proprietary
    Demo!

    View full-size slide

  59. Confidential + Proprietary
    Advanced Chatflow with machine learning bot analytics
    User types to custom UI
    or channel
    Chatbot replies
    Dialogflow
    Enterprise
    Customer Client
    JS Angular 5 web front-end
    Kubernetes Engine
    Chat Server
    Dialogflow SDK / socket.io
    Kubernetes Engine
    Back-end CRM
    Python / Django
    Kubernetes Engine
    Container
    Registry
    Containers images can be
    stored in the Container Registry
    Messaging Publisher
    Pub/Sub
    Webhook
    Router
    Cloud Function
    Webhook
    Container
    Builder
    Building Dev
    Pipelines

    View full-size slide

  60. Confidential + Proprietary
    Advanced Chatflow with machine learning and bot analytics
    User types to custom UI
    or channel
    Chatbot replies
    Dialogflow
    Enterprise
    Customer Client
    JS Angular 5 web front-end
    Kubernetes Engine
    Chat Server
    Dialogflow SDK / socket.io
    Kubernetes Engine
    Back-end CRM
    Python / Django
    Kubernetes Engine
    Subscription
    Cloud Function
    Sensitivity
    Filter
    DLP API
    Sentiment
    Detector
    NLP API
    Data
    Warehouse
    BigQuery
    Messaging Publisher
    Pub/Sub
    Webhook
    Router
    Cloud Function
    Webhook

    View full-size slide

  61. Confidential + Proprietary
    Confidential + Proprietary
    Thank you!

    View full-size slide

  62. Confidential & Proprietary
    Demo: A TV Guide
    Do it yourself:
    https://goo.gl/pH86aT
    Ok Google, let me talk to the TV Guide
    Welcome, I’m the Dutch TV
    Guide. You can ask me the
    following: What’s on a channel?
    What’s on RTL 4?
    Sure! Here’s The TV Guide
    Enter Earcon
    On RTL 4:
    At 7am: Nieuws
    At 7:30am: Nieuws
    ...
    Exit Earcon
    . . .

    View full-size slide

  63. Confidential + Proprietary
    ElasticSearch live search results
    Architecture: Dialogflow with live search results
    Router
    Cloud Functions
    Webhook
    Workflow
    Management
    Calls to logics layers
    Dialogflow
    Enterprise
    Elastic
    Search
    User types in chat UI
    Chatbot or User replies
    On premise
    banking
    systems

    View full-size slide