Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Chat Bots for the Google Assistant / Google Home using API.AI

Building Chat Bots for the Google Assistant / Google Home using API.AI

In this session, Rebecca will explain how to quickly and easily build a custom Google Actions for the Google Home by leveraging API.AI.


Rebecca Franks

January 07, 2017


  1. Building Chat Bots for Google Home using API.AI Rebecca Franks

    Google Developer Expert - Android @riggaroo riggaroo.co.za
  2. Who am I?

  3. “Ok Google!”

  4. None
  5. Direct Actions • Google Assistant handles the entire user interaction

    • You only handle the fulfillment • Not yet widely available to build “Ok Google, Play Charlie Brown Movie on Chromecast using DStv Now.” Conversation Actions • Two-way dialog with users • Your action manages the conversation • Can be built and deployed today. “Ok Google, Let me talk to Spelling Master” 
 Types of Actions
  6. Conversational Actions Source: https://developers.google.com/actions/develop/conversation

  7. Two Options • Actions SDK • Manual understanding of requests

    - raw text parsed to your server. • API.AI SDK • Natural language Processing • Machine Learning AI • Conversation building features
  8. API.AI Flow Source: https://developers.google.com/actions/develop/conversation

  9. 5 Steps to build a Google Action Chat Bot 1.

    Create API.AI Agent & intents 2. (Optional) Create WebHook fulfillment 3. (Optional) Deploy Webhook 4. Test Action 5. Submit for Approval
  10. DStv ChatBot Prototype DStv Hi! I’m Dave from DStv, how

    can I assist you today? When is the next Liverpool Game on? The next match is on Thursday at 12. Would you like me to set a reminder? Ok Google, let me talk to DStv!
  11. API.AI Basic Concepts • Agent - correspond to applications. •

    Intent - a mapping between what a user says and what action should be taken • Entities - concepts that are often specific to a domain as a way of mapping natural language phrases. • Actions - the steps your application will take when specific intents are triggered by user inputs • Contexts -current context of the user expression. This is useful for differentiating phrases which might be vague and have different meaning depending on what was spoken previously. Source: https://docs.api.ai/docs/key-concepts
  12. Create API.AI Agent

  13. Create API.AI Intent- Startup Intent

  14. Create API.AI Intent- Query content intent

  15. API.AI and our Webhook Speech spoken by user POST request

    to Webhook with JSON API.AI extract { "id": "b4e86431-0336-416f-8d4d-fe1572207e58", "timestamp": "2017-01-07T15:25:29.686Z", "result": { "resolvedQuery": "when is the simpsons showing", "action": "query_content", "parameters": { "program_name": “the simpsons" }, "metadata": { "intentName": "query_content" } } }
  16. Create Webhook Fulfillment 1.Download template project from here: https://github.com/ actions-on-google/apiai-webhook-template-nodejs

    (or create Node JS server project.) 2. Edit app.js file
  17. 'use strict'; process.env.DEBUG = 'actions-on-google:*'; let Assistant = require('actions-on-google').ApiAiAssistant; let

    express = require('express'); let bodyParser = require('body-parser'); let app = express(); app.use(bodyParser.json({type: 'application/json'})); Template Code defined in app.js file app.post('/', function (req, res) { const assistant = new Assistant({request: req, response: res}); // Fulfill action business logic … }); if (module === require.main) { // Start the server let server = app.listen(process.env.PORT || 8080, function () { let port = server.address().port; console.log('App listening on port %s', port); }); } module.exports = app;
  18. Create WebHook Fulfillment let request = require('request');
 let moment =

    require(‘moment'); const QUERY_CONTENT_ACTION = 'query_content';
 const ARGUMENT_PROGRAM_NAME = 'program_name';
 app.post('/', function (req, res) {
 const assistant = new Assistant({request: req, response: res});
 function getNextAiringOfProgram(assistant){
 let showName = assistant.getArgument(ARGUMENT_PROGRAM_NAME);
 request(‘https://gook.com;searchTerm='+ showName, function (error, response, body) {
 }) } 
 if (!error && response.statusCode == 200) {
 var event = JSON.parse(body);
 var dateTimeFormatted = new moment(event.startDateTime);
 assistant.tell(event.title + ' will air on ' + dateTimeFormatted.format('dddd Do [at] hh:mm a'));
 } else {
 assistant.tell('I\'m sorry, there was an error trying to process your request.’);
 let actionMap = new Map();
 actionMap.set(QUERY_CONTENT_ACTION, getNextAiringOfProgram);

  19. Deploy Webhook 1. Create Google Cloud Project at https://cloud.google.com 2.

    Run the node server locally from folder 3. Deploy to Google Cloud 4. Set web hook fulfilment URL $ npm install $ npm start $ gcloud init $ gcloud app deploy
  20. Preview, Test + Train Action

  21. Test on Web Simulator

  22. Testing on Google Home

  23. Webhook Quirks • Timeout for service response – 5 seconds.

    • No option to configure this. • Be aware - the timezone you are receiving on the server is not the timezone of the client. • Solution: Request users device location and determine timezone from there
  24. Submit for approval 1.Go to Google Cloud Console (https:// console.cloud.google.com),

    search for “Google Actions API” 2. Enable the API 3. Go to Directory Listing and enter information
  25. Submit for approval • Prepare Store Listing items • Choose

    unique invocation name • Rules are strict, be prepared to be rejected
  26. VoiceLabs.co Analytics

  27. VoiceLabs.co Analytics

  28. Things to consider • Lots of guidelines to read online

    • Voice UI Guidelines • Authentication - Support for account linking via Google Home app. Using OAuth2. • Error States • VoiceLabs.co Analytics
  29. Demo Time

  30. Thank you! Rebecca Franks Google Developer Expert - Android @riggaroo

    / +RebeccaFranksSA riggaroo.co.za