Direct Actions • Google Assistant handles the entire user interaction • You only handle the fulfillment • Not yet widely available to build “Ok Google, Play Charlie Brown Movie on Chromecast using DStv Now.” Conversation Actions • Two-way dialog with users • Your action manages the conversation • Can be built and deployed today. “Ok Google, Let me talk to Spelling Master” Types of Actions
Two Options • Actions SDK • Manual understanding of requests - raw text parsed to your server. • API.AI SDK • Natural language Processing • Machine Learning AI • Conversation building features
DStv ChatBot Prototype DStv Hi! I’m Dave from DStv, how can I assist you today? When is the next Liverpool Game on? The next match is on Thursday at 12. Would you like me to set a reminder? Ok Google, let me talk to DStv!
API.AI Basic Concepts • Agent - correspond to applications. • Intent - a mapping between what a user says and what action should be taken • Entities - concepts that are often specific to a domain as a way of mapping natural language phrases. • Actions - the steps your application will take when specific intents are triggered by user inputs • Contexts -current context of the user expression. This is useful for differentiating phrases which might be vague and have different meaning depending on what was spoken previously. Source: https://docs.api.ai/docs/key-concepts
API.AI and our Webhook Speech spoken by user POST request to Webhook with JSON API.AI extract { "id": "b4e86431-0336-416f-8d4d-fe1572207e58", "timestamp": "2017-01-07T15:25:29.686Z", "result": { "resolvedQuery": "when is the simpsons showing", "action": "query_content", "parameters": { "program_name": “the simpsons" }, "metadata": { "intentName": "query_content" } } }
'use strict'; process.env.DEBUG = 'actions-on-google:*'; let Assistant = require('actions-on-google').ApiAiAssistant; let express = require('express'); let bodyParser = require('body-parser'); let app = express(); app.use(bodyParser.json({type: 'application/json'})); Template Code defined in app.js file app.post('/', function (req, res) { const assistant = new Assistant({request: req, response: res}); // Fulfill action business logic … }); if (module === require.main) { // Start the server let server = app.listen(process.env.PORT || 8080, function () { let port = server.address().port; console.log('App listening on port %s', port); }); } module.exports = app;
Create WebHook Fulfillment let request = require('request'); let moment = require(‘moment'); const QUERY_CONTENT_ACTION = 'query_content'; const ARGUMENT_PROGRAM_NAME = 'program_name';
app.post('/', function (req, res) { const assistant = new Assistant({request: req, response: res});
function getNextAiringOfProgram(assistant){ let showName = assistant.getArgument(ARGUMENT_PROGRAM_NAME); request(‘https://gook.com;searchTerm='+ showName, function (error, response, body) { }) }
if (!error && response.statusCode == 200) { var event = JSON.parse(body); var dateTimeFormatted = new moment(event.startDateTime); assistant.tell(event.title + ' will air on ' + dateTimeFormatted.format('dddd Do [at] hh:mm a')); } else { assistant.tell('I\'m sorry, there was an error trying to process your request.’); let actionMap = new Map(); actionMap.set(QUERY_CONTENT_ACTION, getNextAiringOfProgram); assistant.handleRequest(actionMap);
Deploy Webhook 1. Create Google Cloud Project at https://cloud.google.com 2. Run the node server locally from folder 3. Deploy to Google Cloud 4. Set web hook fulfilment URL $ npm install $ npm start $ gcloud init $ gcloud app deploy
Webhook Quirks • Timeout for service response – 5 seconds. • No option to configure this. • Be aware - the timezone you are receiving on the server is not the timezone of the client. • Solution: Request users device location and determine timezone from there
Submit for approval 1.Go to Google Cloud Console (https:// console.cloud.google.com), search for “Google Actions API” 2. Enable the API 3. Go to Directory Listing and enter information
Things to consider • Lots of guidelines to read online • Voice UI Guidelines • Authentication - Support for account linking via Google Home app. Using OAuth2. • Error States • VoiceLabs.co Analytics
riggaroo
wakamsha
onecareer_tech
tkikuchi1002
coconala_engineer
upura
wakamatsu_takumu
keiko713
knsg16
you
masasuzu
peisuke
hygradme
3n
destraynor
rocio
chriscoyier
eileencodes
sachag
aki_iinuma
mojombo
sstephenson
orderedlist
reverentgeek
mthomps