Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Chat Bots for the Google Assistant / Google Home using API.AI

Building Chat Bots for the Google Assistant / Google Home using API.AI

In this session, Rebecca will explain how to quickly and easily build a custom Google Actions for the Google Home by leveraging API.AI.

Rebecca Franks

January 07, 2017

More Decks by Rebecca Franks

Other Decks in Technology


  1. Building Chat Bots for Google Home using API.AI Rebecca Franks

    Google Developer Expert - Android @riggaroo riggaroo.co.za
  2. Who am I?

  3. “Ok Google!”

  4. None
  5. Direct Actions • Google Assistant handles the entire user interaction

    • You only handle the fulfillment • Not yet widely available to build “Ok Google, Play Charlie Brown Movie on Chromecast using DStv Now.” Conversation Actions • Two-way dialog with users • Your action manages the conversation • Can be built and deployed today. “Ok Google, Let me talk to Spelling Master” 
 Types of Actions
  6. Conversational Actions Source: https://developers.google.com/actions/develop/conversation

  7. Two Options • Actions SDK • Manual understanding of requests

    - raw text parsed to your server. • API.AI SDK • Natural language Processing • Machine Learning AI • Conversation building features
  8. API.AI Flow Source: https://developers.google.com/actions/develop/conversation

  9. 5 Steps to build a Google Action Chat Bot 1.

    Create API.AI Agent & intents 2. (Optional) Create WebHook fulfillment 3. (Optional) Deploy Webhook 4. Test Action 5. Submit for Approval
  10. DStv ChatBot Prototype DStv Hi! I’m Dave from DStv, how

    can I assist you today? When is the next Liverpool Game on? The next match is on Thursday at 12. Would you like me to set a reminder? Ok Google, let me talk to DStv!
  11. API.AI Basic Concepts • Agent - correspond to applications. •

    Intent - a mapping between what a user says and what action should be taken • Entities - concepts that are often specific to a domain as a way of mapping natural language phrases. • Actions - the steps your application will take when specific intents are triggered by user inputs • Contexts -current context of the user expression. This is useful for differentiating phrases which might be vague and have different meaning depending on what was spoken previously. Source: https://docs.api.ai/docs/key-concepts
  12. Create API.AI Agent

  13. Create API.AI Intent- Startup Intent

  14. Create API.AI Intent- Query content intent

  15. API.AI and our Webhook Speech spoken by user POST request

    to Webhook with JSON API.AI extract { "id": "b4e86431-0336-416f-8d4d-fe1572207e58", "timestamp": "2017-01-07T15:25:29.686Z", "result": { "resolvedQuery": "when is the simpsons showing", "action": "query_content", "parameters": { "program_name": “the simpsons" }, "metadata": { "intentName": "query_content" } } }
  16. Create Webhook Fulfillment 1.Download template project from here: https://github.com/ actions-on-google/apiai-webhook-template-nodejs

    (or create Node JS server project.) 2. Edit app.js file
  17. 'use strict'; process.env.DEBUG = 'actions-on-google:*'; let Assistant = require('actions-on-google').ApiAiAssistant; let

    express = require('express'); let bodyParser = require('body-parser'); let app = express(); app.use(bodyParser.json({type: 'application/json'})); Template Code defined in app.js file app.post('/', function (req, res) { const assistant = new Assistant({request: req, response: res}); // Fulfill action business logic … }); if (module === require.main) { // Start the server let server = app.listen(process.env.PORT || 8080, function () { let port = server.address().port; console.log('App listening on port %s', port); }); } module.exports = app;
  18. Create WebHook Fulfillment let request = require('request');
 let moment =

    require(‘moment'); const QUERY_CONTENT_ACTION = 'query_content';
 const ARGUMENT_PROGRAM_NAME = 'program_name';
 app.post('/', function (req, res) {
 const assistant = new Assistant({request: req, response: res});
 function getNextAiringOfProgram(assistant){
 let showName = assistant.getArgument(ARGUMENT_PROGRAM_NAME);
 request(‘https://gook.com;searchTerm='+ showName, function (error, response, body) {
 }) } 
 if (!error && response.statusCode == 200) {
 var event = JSON.parse(body);
 var dateTimeFormatted = new moment(event.startDateTime);
 assistant.tell(event.title + ' will air on ' + dateTimeFormatted.format('dddd Do [at] hh:mm a'));
 } else {
 assistant.tell('I\'m sorry, there was an error trying to process your request.’);
 let actionMap = new Map();
 actionMap.set(QUERY_CONTENT_ACTION, getNextAiringOfProgram);

  19. Deploy Webhook 1. Create Google Cloud Project at https://cloud.google.com 2.

    Run the node server locally from folder 3. Deploy to Google Cloud 4. Set web hook fulfilment URL $ npm install $ npm start $ gcloud init $ gcloud app deploy
  20. Preview, Test + Train Action

  21. Test on Web Simulator

  22. Testing on Google Home

  23. Webhook Quirks • Timeout for service response – 5 seconds.

    • No option to configure this. • Be aware - the timezone you are receiving on the server is not the timezone of the client. • Solution: Request users device location and determine timezone from there
  24. Submit for approval 1.Go to Google Cloud Console (https:// console.cloud.google.com),

    search for “Google Actions API” 2. Enable the API 3. Go to Directory Listing and enter information
  25. Submit for approval • Prepare Store Listing items • Choose

    unique invocation name • Rules are strict, be prepared to be rejected
  26. VoiceLabs.co Analytics

  27. VoiceLabs.co Analytics

  28. Things to consider • Lots of guidelines to read online

    • Voice UI Guidelines • Authentication - Support for account linking via Google Home app. Using OAuth2. • Error States • VoiceLabs.co Analytics
  29. Demo Time

  30. Thank you! Rebecca Franks Google Developer Expert - Android @riggaroo

    / +RebeccaFranksSA riggaroo.co.za