Slide 1

Slide 1 text

Building Chat Bots for Google Home using API.AI Rebecca Franks Google Developer Expert - Android @riggaroo riggaroo.co.za

Slide 2

Slide 2 text

Who am I?

Slide 3

Slide 3 text

“Ok Google!”

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Direct Actions • Google Assistant handles the entire user interaction • You only handle the fulfillment • Not yet widely available to build “Ok Google, Play Charlie Brown Movie on Chromecast using DStv Now.” Conversation Actions • Two-way dialog with users • Your action manages the conversation • Can be built and deployed today. “Ok Google, Let me talk to Spelling Master” 
 Types of Actions

Slide 6

Slide 6 text

Conversational Actions Source: https://developers.google.com/actions/develop/conversation

Slide 7

Slide 7 text

Two Options • Actions SDK • Manual understanding of requests - raw text parsed to your server. • API.AI SDK • Natural language Processing • Machine Learning AI • Conversation building features

Slide 8

Slide 8 text

API.AI Flow Source: https://developers.google.com/actions/develop/conversation

Slide 9

Slide 9 text

5 Steps to build a Google Action Chat Bot 1. Create API.AI Agent & intents 2. (Optional) Create WebHook fulfillment 3. (Optional) Deploy Webhook 4. Test Action 5. Submit for Approval

Slide 10

Slide 10 text

DStv ChatBot Prototype DStv Hi! I’m Dave from DStv, how can I assist you today? When is the next Liverpool Game on? The next match is on Thursday at 12. Would you like me to set a reminder? Ok Google, let me talk to DStv!

Slide 11

Slide 11 text

API.AI Basic Concepts • Agent - correspond to applications. • Intent - a mapping between what a user says and what action should be taken • Entities - concepts that are often specific to a domain as a way of mapping natural language phrases. • Actions - the steps your application will take when specific intents are triggered by user inputs • Contexts -current context of the user expression. This is useful for differentiating phrases which might be vague and have different meaning depending on what was spoken previously. Source: https://docs.api.ai/docs/key-concepts

Slide 12

Slide 12 text

Create API.AI Agent

Slide 13

Slide 13 text

Create API.AI Intent- Startup Intent

Slide 14

Slide 14 text

Create API.AI Intent- Query content intent

Slide 15

Slide 15 text

API.AI and our Webhook Speech spoken by user POST request to Webhook with JSON API.AI extract { "id": "b4e86431-0336-416f-8d4d-fe1572207e58", "timestamp": "2017-01-07T15:25:29.686Z", "result": { "resolvedQuery": "when is the simpsons showing", "action": "query_content", "parameters": { "program_name": “the simpsons" }, "metadata": { "intentName": "query_content" } } }

Slide 16

Slide 16 text

Create Webhook Fulfillment 1.Download template project from here: https://github.com/ actions-on-google/apiai-webhook-template-nodejs (or create Node JS server project.) 2. Edit app.js file

Slide 17

Slide 17 text

'use strict'; process.env.DEBUG = 'actions-on-google:*'; let Assistant = require('actions-on-google').ApiAiAssistant; let express = require('express'); let bodyParser = require('body-parser'); let app = express(); app.use(bodyParser.json({type: 'application/json'})); Template Code defined in app.js file app.post('/', function (req, res) { const assistant = new Assistant({request: req, response: res}); // Fulfill action business logic … }); if (module === require.main) { // Start the server let server = app.listen(process.env.PORT || 8080, function () { let port = server.address().port; console.log('App listening on port %s', port); }); } module.exports = app;

Slide 18

Slide 18 text

Create WebHook Fulfillment let request = require('request');
 let moment = require(‘moment'); const QUERY_CONTENT_ACTION = 'query_content';
 const ARGUMENT_PROGRAM_NAME = 'program_name';
 
 app.post('/', function (req, res) {
 const assistant = new Assistant({request: req, response: res});
 
 function getNextAiringOfProgram(assistant){
 let showName = assistant.getArgument(ARGUMENT_PROGRAM_NAME);
 request(‘https://gook.com;searchTerm='+ showName, function (error, response, body) {
 }) } 
 if (!error && response.statusCode == 200) {
 var event = JSON.parse(body);
 var dateTimeFormatted = new moment(event.startDateTime);
 assistant.tell(event.title + ' will air on ' + dateTimeFormatted.format('dddd Do [at] hh:mm a'));
 } else {
 assistant.tell('I\'m sorry, there was an error trying to process your request.’);
 let actionMap = new Map();
 actionMap.set(QUERY_CONTENT_ACTION, getNextAiringOfProgram);
 assistant.handleRequest(actionMap);


Slide 19

Slide 19 text

Deploy Webhook 1. Create Google Cloud Project at https://cloud.google.com 2. Run the node server locally from folder 3. Deploy to Google Cloud 4. Set web hook fulfilment URL $ npm install $ npm start $ gcloud init $ gcloud app deploy

Slide 20

Slide 20 text

Preview, Test + Train Action

Slide 21

Slide 21 text

Test on Web Simulator

Slide 22

Slide 22 text

Testing on Google Home

Slide 23

Slide 23 text

Webhook Quirks • Timeout for service response – 5 seconds. • No option to configure this. • Be aware - the timezone you are receiving on the server is not the timezone of the client. • Solution: Request users device location and determine timezone from there

Slide 24

Slide 24 text

Submit for approval 1.Go to Google Cloud Console (https:// console.cloud.google.com), search for “Google Actions API” 2. Enable the API 3. Go to Directory Listing and enter information

Slide 25

Slide 25 text

Submit for approval • Prepare Store Listing items • Choose unique invocation name • Rules are strict, be prepared to be rejected

Slide 26

Slide 26 text

VoiceLabs.co Analytics

Slide 27

Slide 27 text

VoiceLabs.co Analytics

Slide 28

Slide 28 text

Things to consider • Lots of guidelines to read online • Voice UI Guidelines • Authentication - Support for account linking via Google Home app. Using OAuth2. • Error States • VoiceLabs.co Analytics

Slide 29

Slide 29 text

Demo Time

Slide 30

Slide 30 text

Thank you! Rebecca Franks Google Developer Expert - Android @riggaroo / +RebeccaFranksSA riggaroo.co.za