Tweet
Tweet

Slide 1

Slide 1 text

1 Creating Conversations For Smart Assistants Lee Boonstra Customer Engineer, Google Twitter: @ladysign

Slide 2

Slide 2 text

2 Lee Boonstra @ladysign

Slide 3

Slide 3 text

Use cases in the contact center and beyond ● Customer service ● E-commerce ● Home entertainment ● Auto ● Organizational knowledge ● Onboarding & Helpdesk

Slide 4

Slide 4 text

Consumers have high expectations Based on data collected in Fall 2017 from sample of 1,051 U.S. adults, ages 18-64.

Slide 5

Slide 5 text

5 50% of enterprises spend more time on creating bots than on mobile app development by 2021 —Gartner

Slide 6

Slide 6 text

6 Chatbots are expected to trim business costs by more than $8 billion per year by 2022. - Juniper Research

Slide 7

Slide 7 text

Confidential + Proprietary Confidential + Proprietary Chatbots exists since the 90’s… So why are they popular now?

Slide 8

Slide 8 text

8 1994, the first chatbot was born 2016, Focus on AI First

Slide 9

Slide 9 text

9 Machine Learning How did you learn your first language? or

Slide 10

Slide 10 text

10 It’s easier to make computers learn than to build smarter computers Machine Learning Process of making computers learn Artificial Intelligence Process of building smarter computers

Slide 11

Slide 11 text

Dialogflow users benefit from world-class AI assets & capabilities, including: Google is an AI company Goal: Make AI easy, fast, and useful for enterprises and developers Machine learning leadership Natural language understanding Knowledge graph Search across unstructured data Speech recognition & synthesis TPUs and unrivaled compute infra for ML

Slide 12

Slide 12 text

Cloud AI Less ML expertise More ML expertise Cloud AI solutions Cloud Job Discovery Contact Center Document Understanding Cloud AI building blocks Cloud AI platform Cloud ML Engine Cloud Video Intelligence ML professionals & service partners ASL Professional Services Organization Cloud AutoML Vision Vision Cloud Vision Language Conversation Cloud Natural Language Cloud AutoML NL Dialogflow Enterprise Cloud Translation Cloud Speech-to-Text Cloud Text-to-Speech Cloud AutoML Translation New New New Cloud GPU Cloud TPU Cloud Dataflow Cloud Dataproc Machine & Deep Learning ML accelerators ML libraries Tensorflow Kubeflow Kaggle/datasets Datasets

Slide 13

Slide 13 text

13 Storage Compute

Slide 14

Slide 14 text

14 Bots to improve your customer experience Callbots in callcenters Chatbots in websites, apps or social media platforms. Voice Activated Speakers & smart assistants.

Slide 15

Slide 15 text

15 70% of requests are in natural language Is it gonna rain tomorrow? Tomorrow it will be sunny In Amsterdam!

Slide 16

Slide 16 text

16 A conversation between you and Google that helps you get more done in your world Meet the Google Assistant

Slide 17

Slide 17 text

17 Terminology Google Assistant — The virtual assistant of Google. Out of the box on Android 6+. For iOS available as app. Action — A third party app, running on top of the Google Assistant. Google Home — Voice-activated speaker powered by Voice. Smart Display — Voice-activated speaker with screen powered by Voice and Touch.

Slide 18

Slide 18 text

18 ● Manage tasks Ok Google, what’s my confirmation number for my flight? ● Plan the day Ok Google, how is my commute to the office? ● Get answers: Ok Google, who is the King of the Netherlands? ● Make memories Ok Google, remember I left my keys on the Kitchen table. ● Control your home Ok Google, turn off the bedroom lights. ● Enjoy Entertainment Ok Google, play the Pixies on Spotify. Things you can ask Google...

Slide 19

Slide 19 text

Google | Proprietary & Confidential 19 Idle Listening Recognizing Speaking Thinking

Slide 20

Slide 20 text

Google | Proprietary & Confidential 20 recognitionResult listenEnd recognitionResult voicestart listenStart listenEnd voiceEnd eventName

Slide 21

Slide 21 text

21 3rd party integration Extend the Google Assistant with your own custom actions. Hey Google, let me talk to BookAMeetingRoom Welcome, how can I help you? I want to book a meeting room for 3 persons. Let’s get BookAMeetingRoom Sure, for when? Tomorrow, from 2pm to 3pm.

Slide 22

Slide 22 text

Confidential + Proprietary Confidential + Proprietary How can you build your own action on top of the Google Assistant?

Slide 23

Slide 23 text

23 Sketch (+ Google Assistant UI Toolkit) Prototyping tools Twine

Slide 24

Slide 24 text

24 ● Write the conversation - Dialogflow (Enterprise) ● Deploy on GA+ UX components - Actions on Google Optional: ● Back-end integration - Fulfillment app (dialogflow/aog SDK) ● Communication to back-ends - Your own APIs Tooling

Slide 25

Slide 25 text

25 Testing tools Ngrok secure internet tunnels to localhost ● Actions on Google Simulator ● Device Testing ● Dialogflow Simulator ● Dialogflow Diagnostics ● GCP Stackdriver ● Cloud Functions Emulator ● Ngrok Secure Tunneling ● Dialogflow Training

Slide 26

Slide 26 text

Confidential + Proprietary Dialogflow

Slide 27

Slide 27 text

27 ● Previously known as API.AI ○ (Sept 2016, acquired by Google) ● Powered by Machine Learning: ○ Natural Language Understanding (NLU) ○ Intent Matching ○ Conversation Training ● Cross platform ● Build faster with the Web UI ● Scalable: separate your conversation text from code ● Speech / Voice Integration ● Multi-lingual bot support (20+ languages) ● Part of Google Cloud (60+ cloud services) Development suite for building Conversational UIs.

Slide 28

Slide 28 text

28 Dialogflow has been used by over 600,000+ developers

Slide 29

Slide 29 text

29 Intents

Slide 30

Slide 30 text

30 Intent Responses

Slide 31

Slide 31 text

31 Entities

Slide 32

Slide 32 text

32 ● Google Assistant ● Amazon Alexa ● MS Cortana ● Web ● iOS ● Android Integrations ● Facebook Messenger ● Twitter ● Slack ● Telegram ● Skype ● Cisco Spark ● Dialogflow Phone ● Twilio ● Viber ● Kik ● Line

Slide 33

Slide 33 text

33 Fulfillment

Slide 34

Slide 34 text

34 ● Your application back-end code. ○ Cloud Function / App Engine / GKE ...? ● Router to catch all intents, that have webhooks enabled. ● Application code ● Response messages ● Calls to external / on-premise APIs Fulfillment App

Slide 35

Slide 35 text

Confidential + Proprietary Actions on Google

Slide 36

Slide 36 text

36 ● Bring your agents to smart speakers (Google Home) or phones (Android, iOS app) ● Actions on Google includes: ○ UI toolkit, ○ Audio toolkit ○ Account Linking API ○ SDKs ○ tutorial guides ● UI components such as: ○ Buttons, Images ○ Cards, Carousels, ○ Lists ○ Tables Program for developers of Actions (“apps”) that run via Google Assistant Actions on Google

Slide 37

Slide 37 text

37 SSML Speech Synthesis Markup Language (SSML) is an XML based markup language for defining speech. ● By using SSML, you can make your agent's responses seem more life-like. ● Tags for defining: ○ Breaks ○ Audio snippets ○ Type of text constructs (date, time, alphabet..) ○ Pitch, Speaking rates ○ Emphasis

Slide 38

Slide 38 text

38 ● Formatting Numbers - ($100.000 - > 100000) http://numeraljs.com/ ● Format JS internationalization (dates etc) - https://formatjs.io/ ● Moment JS (dates, times) - https://momentjs.com/ ● Pluralize - https://www.npmjs.com/package/pluralize ● Lightweight Translation Module - (Multi languages in fulfillment apps) - https://github.com/mashpie/i18n-node Handy JS libraries

Slide 39

Slide 39 text

39 Assistance is not just about voice

Slide 40

Slide 40 text

40 Your experience can run across multiple devices.

Slide 41

Slide 41 text

41 Suggestion Chips

Slide 42

Slide 42 text

42 Suggestion Chips

Slide 43

Slide 43 text

43 Basic Cards

Slide 44

Slide 44 text

44 Basic Cards

Slide 45

Slide 45 text

45 Lists

Slide 46

Slide 46 text

46 Lists

Slide 47

Slide 47 text

47 Carousels

Slide 48

Slide 48 text

48 Carousels

Slide 49

Slide 49 text

49 Media Responses

Slide 50

Slide 50 text

50 Media Responses

Slide 51

Slide 51 text

51 Table Responses

Slide 52

Slide 52 text

52 Table Responses

Slide 53

Slide 53 text

53 Action Directory Submit your action to production.

Slide 54

Slide 54 text

Confidential + Proprietary Confidential + Proprietary How to measure your app usage?

Slide 55

Slide 55 text

Confidential + Proprietary Which customers are unhappy and why?

Slide 56

Slide 56 text

Confidential + Proprietary How can I improve the user experience?

Slide 57

Slide 57 text

57 Sure! I’d like to transfer 100 euro to my wife her bank account.” A customer communicates with the Google Assistant

Slide 58

Slide 58 text

58 What happens under the hood...

Slide 59

Slide 59 text

59 Collect real-time chats from Dialogflow SDK

Slide 60

Slide 60 text

60 Mask sensitive Information with DLP API

Slide 61

Slide 61 text

61 Understand the text with NLP API

Slide 62

Slide 62 text

62 Store all data in a data-warehouse

Slide 63

Slide 63 text

63 Optimize your agent

Slide 64

Slide 64 text

Confidential + Proprietary Confidential + Proprietary Demo!

Slide 65

Slide 65 text

Confidential + Proprietary Advanced Chatflow with machine learning bot analytics User types to custom UI or channel Chatbot replies Dialogflow Enterprise Customer Client JS Angular 5 web front-end Kubernetes Engine Chat Server Dialogflow SDK / socket.io Kubernetes Engine Back-end CRM Python / Django Kubernetes Engine Container Registry Containers images can be stored in the Container Registry Messaging Publisher Pub/Sub Webhook Router Cloud Function Webhook Container Builder Building Dev Pipelines

Slide 66

Slide 66 text

Confidential + Proprietary Advanced Chatflow with machine learning and bot analytics User types to custom UI or channel Chatbot replies Dialogflow Enterprise Customer Client JS Angular 5 web front-end Kubernetes Engine Chat Server Dialogflow SDK / socket.io Kubernetes Engine Back-end CRM Python / Django Kubernetes Engine Subscription Cloud Function Sensitivity Filter DLP API Sentiment Detector NLP API Data Warehouse BigQuery Messaging Publisher Pub/Sub Webhook Router Cloud Function Webhook

Slide 67

Slide 67 text

Confidential + Proprietary Confidential + Proprietary Thank you!

Slide 68

Slide 68 text

Confidential & Proprietary Demo: A TV Guide Do it yourself: https://goo.gl/pH86aT Ok Google, let me talk to the TV Guide Welcome, I’m the Dutch TV Guide. You can ask me the following: What’s on a channel? What’s on RTL 4? Sure! Here’s The TV Guide Enter Earcon On RTL 4: At 7am: Nieuws At 7:30am: Nieuws ... Exit Earcon . . .

Slide 69

Slide 69 text

Confidential + Proprietary ElasticSearch live search results Architecture: Dialogflow with live search results Router Cloud Functions Webhook Workflow Management Calls to logics layers Dialogflow Enterprise Elastic Search User types in chat UI Chatbot or User replies On premise banking systems