Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Digital Wednesday

Digital Wednesday

8a235da15adae86851fa3216834198ed?s=128

Lee Boonstra

November 29, 2018
Tweet

Transcript

  1. 1 Creating Conversations For Smart Assistants Lee Boonstra Customer Engineer,

    Google Twitter: @ladysign
  2. 2 Lee Boonstra @ladysign

  3. Use cases in the contact center and beyond • Customer

    service • E-commerce • Home entertainment • Auto • Organizational knowledge • Onboarding & Helpdesk
  4. Consumers have high expectations Based on data collected in Fall

    2017 from sample of 1,051 U.S. adults, ages 18-64.
  5. 5 50% of enterprises spend more time on creating bots

    than on mobile app development by 2021 —Gartner
  6. 6 Chatbots are expected to trim business costs by more

    than $8 billion per year by 2022. - Juniper Research
  7. Confidential + Proprietary Confidential + Proprietary Chatbots exists since the

    90’s… So why are they popular now?
  8. 8 1994, the first chatbot was born 2016, Focus on

    AI First
  9. 9 Machine Learning How did you learn your first language?

    or
  10. 10 It’s easier to make computers learn than to build

    smarter computers Machine Learning Process of making computers learn Artificial Intelligence Process of building smarter computers
  11. Dialogflow users benefit from world-class AI assets & capabilities, including:

    Google is an AI company Goal: Make AI easy, fast, and useful for enterprises and developers Machine learning leadership Natural language understanding Knowledge graph Search across unstructured data Speech recognition & synthesis TPUs and unrivaled compute infra for ML
  12. Cloud AI Less ML expertise More ML expertise Cloud AI

    solutions Cloud Job Discovery Contact Center Document Understanding Cloud AI building blocks Cloud AI platform Cloud ML Engine Cloud Video Intelligence ML professionals & service partners ASL Professional Services Organization Cloud AutoML Vision Vision Cloud Vision Language Conversation Cloud Natural Language Cloud AutoML NL Dialogflow Enterprise Cloud Translation Cloud Speech-to-Text Cloud Text-to-Speech Cloud AutoML Translation New New New Cloud GPU Cloud TPU Cloud Dataflow Cloud Dataproc Machine & Deep Learning ML accelerators ML libraries Tensorflow Kubeflow Kaggle/datasets Datasets
  13. 13 Storage Compute

  14. 14 Bots to improve your customer experience Callbots in callcenters

    Chatbots in websites, apps or social media platforms. Voice Activated Speakers & smart assistants.
  15. 15 70% of requests are in natural language Is it

    gonna rain tomorrow? Tomorrow it will be sunny In Amsterdam!
  16. 16 A conversation between you and Google that helps you

    get more done in your world Meet the Google Assistant
  17. 17 Terminology Google Assistant — The virtual assistant of Google.

    Out of the box on Android 6+. For iOS available as app. Action — A third party app, running on top of the Google Assistant. Google Home — Voice-activated speaker powered by Voice. Smart Display — Voice-activated speaker with screen powered by Voice and Touch.
  18. 18 • Manage tasks Ok Google, what’s my confirmation number

    for my flight? • Plan the day Ok Google, how is my commute to the office? • Get answers: Ok Google, who is the King of the Netherlands? • Make memories Ok Google, remember I left my keys on the Kitchen table. • Control your home Ok Google, turn off the bedroom lights. • Enjoy Entertainment Ok Google, play the Pixies on Spotify. Things you can ask Google...
  19. Google | Proprietary & Confidential 19 Idle Listening Recognizing Speaking

    Thinking
  20. Google | Proprietary & Confidential 20 recognitionResult listenEnd recognitionResult voicestart

    listenStart listenEnd voiceEnd eventName
  21. 21 3rd party integration Extend the Google Assistant with your

    own custom actions. Hey Google, let me talk to BookAMeetingRoom Welcome, how can I help you? I want to book a meeting room for 3 persons. Let’s get BookAMeetingRoom Sure, for when? Tomorrow, from 2pm to 3pm.
  22. Confidential + Proprietary Confidential + Proprietary How can you build

    your own action on top of the Google Assistant?
  23. 23 Sketch (+ Google Assistant UI Toolkit) Prototyping tools Twine

  24. 24 • Write the conversation - Dialogflow (Enterprise) • Deploy

    on GA+ UX components - Actions on Google Optional: • Back-end integration - Fulfillment app (dialogflow/aog SDK) • Communication to back-ends - Your own APIs Tooling
  25. 25 Testing tools Ngrok secure internet tunnels to localhost •

    Actions on Google Simulator • Device Testing • Dialogflow Simulator • Dialogflow Diagnostics • GCP Stackdriver • Cloud Functions Emulator • Ngrok Secure Tunneling • Dialogflow Training
  26. Confidential + Proprietary Dialogflow

  27. 27 • Previously known as API.AI ◦ (Sept 2016, acquired

    by Google) • Powered by Machine Learning: ◦ Natural Language Understanding (NLU) ◦ Intent Matching ◦ Conversation Training • Cross platform • Build faster with the Web UI • Scalable: separate your conversation text from code • Speech / Voice Integration • Multi-lingual bot support (20+ languages) • Part of Google Cloud (60+ cloud services) Development suite for building Conversational UIs.
  28. 28 Dialogflow has been used by over 600,000+ developers

  29. 29 Intents

  30. 30 Intent Responses

  31. 31 Entities

  32. 32 • Google Assistant • Amazon Alexa • MS Cortana

    • Web • iOS • Android Integrations • Facebook Messenger • Twitter • Slack • Telegram • Skype • Cisco Spark • Dialogflow Phone • Twilio • Viber • Kik • Line
  33. 33 Fulfillment

  34. 34 • Your application back-end code. ◦ Cloud Function /

    App Engine / GKE ...? • Router to catch all intents, that have webhooks enabled. • Application code • Response messages • Calls to external / on-premise APIs Fulfillment App
  35. Confidential + Proprietary Actions on Google

  36. 36 • Bring your agents to smart speakers (Google Home)

    or phones (Android, iOS app) • Actions on Google includes: ◦ UI toolkit, ◦ Audio toolkit ◦ Account Linking API ◦ SDKs ◦ tutorial guides • UI components such as: ◦ Buttons, Images ◦ Cards, Carousels, ◦ Lists ◦ Tables Program for developers of Actions (“apps”) that run via Google Assistant Actions on Google
  37. 37 SSML Speech Synthesis Markup Language (SSML) is an XML

    based markup language for defining speech. • By using SSML, you can make your agent's responses seem more life-like. • Tags for defining: ◦ Breaks ◦ Audio snippets ◦ Type of text constructs (date, time, alphabet..) ◦ Pitch, Speaking rates ◦ Emphasis
  38. 38 • Formatting Numbers - ($100.000 - > 100000) http://numeraljs.com/

    • Format JS internationalization (dates etc) - https://formatjs.io/ • Moment JS (dates, times) - https://momentjs.com/ • Pluralize - https://www.npmjs.com/package/pluralize • Lightweight Translation Module - (Multi languages in fulfillment apps) - https://github.com/mashpie/i18n-node Handy JS libraries
  39. 39 Assistance is not just about voice

  40. 40 Your experience can run across multiple devices.

  41. 41 Suggestion Chips

  42. 42 Suggestion Chips

  43. 43 Basic Cards

  44. 44 Basic Cards

  45. 45 Lists

  46. 46 Lists

  47. 47 Carousels

  48. 48 Carousels

  49. 49 Media Responses

  50. 50 Media Responses

  51. 51 Table Responses

  52. 52 Table Responses

  53. 53 Action Directory Submit your action to production.

  54. Confidential + Proprietary Confidential + Proprietary How to measure your

    app usage?
  55. Confidential + Proprietary Which customers are unhappy and why?

  56. Confidential + Proprietary How can I improve the user experience?

  57. 57 Sure! I’d like to transfer 100 euro to my

    wife her bank account.” A customer communicates with the Google Assistant
  58. 58 What happens under the hood...

  59. 59 Collect real-time chats from Dialogflow SDK

  60. 60 Mask sensitive Information with DLP API

  61. 61 Understand the text with NLP API

  62. 62 Store all data in a data-warehouse

  63. 63 Optimize your agent

  64. Confidential + Proprietary Confidential + Proprietary Demo!

  65. Confidential + Proprietary Advanced Chatflow with machine learning bot analytics

    User types to custom UI or channel Chatbot replies Dialogflow Enterprise Customer Client JS Angular 5 web front-end Kubernetes Engine Chat Server Dialogflow SDK / socket.io Kubernetes Engine Back-end CRM Python / Django Kubernetes Engine Container Registry Containers images can be stored in the Container Registry Messaging Publisher Pub/Sub Webhook Router Cloud Function Webhook Container Builder Building Dev Pipelines
  66. Confidential + Proprietary Advanced Chatflow with machine learning and bot

    analytics User types to custom UI or channel Chatbot replies Dialogflow Enterprise Customer Client JS Angular 5 web front-end Kubernetes Engine Chat Server Dialogflow SDK / socket.io Kubernetes Engine Back-end CRM Python / Django Kubernetes Engine Subscription Cloud Function Sensitivity Filter DLP API Sentiment Detector NLP API Data Warehouse BigQuery Messaging Publisher Pub/Sub Webhook Router Cloud Function Webhook
  67. Confidential + Proprietary Confidential + Proprietary Thank you!

  68. Confidential & Proprietary Demo: A TV Guide Do it yourself:

    https://goo.gl/pH86aT Ok Google, let me talk to the TV Guide Welcome, I’m the Dutch TV Guide. You can ask me the following: What’s on a channel? What’s on RTL 4? Sure! Here’s The TV Guide Enter Earcon On RTL 4: At 7am: Nieuws At 7:30am: Nieuws ... Exit Earcon . . .
  69. Confidential + Proprietary ElasticSearch live search results Architecture: Dialogflow with

    live search results Router Cloud Functions Webhook Workflow Management Calls to logics layers Dialogflow Enterprise Elastic Search User types in chat UI Chatbot or User replies On premise banking systems