$30 off During Our Annual Pro Sale. View Details »

Building Chat Bots for the Google Assistant / Google Home using API.AI

Building Chat Bots for the Google Assistant / Google Home using API.AI

In this session, Rebecca will explain how to quickly and easily build a custom Google Actions for the Google Home by leveraging API.AI.

Rebecca Franks

January 07, 2017
Tweet

More Decks by Rebecca Franks

Other Decks in Technology

Transcript

  1. Building Chat Bots for Google Home
    using API.AI
    Rebecca Franks
    Google Developer Expert - Android
    @riggaroo
    riggaroo.co.za

    View Slide

  2. Who am I?

    View Slide

  3. “Ok Google!”

    View Slide

  4. View Slide

  5. Direct Actions
    • Google Assistant handles the entire
    user interaction
    • You only handle the fulfillment
    • Not yet widely available to build
    “Ok Google, Play Charlie Brown Movie
    on Chromecast using DStv Now.”
    Conversation Actions
    • Two-way dialog with users
    • Your action manages the conversation
    • Can be built and deployed today.
    “Ok Google, Let me talk to Spelling
    Master” 

    Types of Actions

    View Slide

  6. Conversational Actions
    Source: https://developers.google.com/actions/develop/conversation

    View Slide

  7. Two Options
    • Actions SDK
    • Manual understanding of requests - raw text parsed to
    your server.
    • API.AI SDK
    • Natural language Processing
    • Machine Learning AI
    • Conversation building features

    View Slide

  8. API.AI Flow
    Source: https://developers.google.com/actions/develop/conversation

    View Slide

  9. 5 Steps to build a
    Google Action Chat
    Bot
    1. Create API.AI Agent &
    intents
    2. (Optional) Create
    WebHook fulfillment
    3. (Optional) Deploy
    Webhook
    4. Test Action
    5. Submit for Approval

    View Slide

  10. DStv ChatBot Prototype
    DStv
    Hi! I’m Dave from DStv, how
    can I assist you today?
    When is the next Liverpool
    Game on?
    The next match is on Thursday at
    12. Would you like me to set a
    reminder?
    Ok Google, let me talk to DStv!

    View Slide

  11. API.AI Basic Concepts
    • Agent - correspond to applications.
    • Intent - a mapping between what a user says and what action
    should be taken
    • Entities - concepts that are often specific to a domain as a way of
    mapping natural language phrases.
    • Actions - the steps your application will take when specific intents
    are triggered by user inputs
    • Contexts -current context of the user expression. This is useful for
    differentiating phrases which might be vague and have different
    meaning depending on what was spoken previously.
    Source: https://docs.api.ai/docs/key-concepts

    View Slide

  12. Create API.AI Agent

    View Slide

  13. Create API.AI Intent- Startup Intent

    View Slide

  14. Create API.AI Intent- Query content intent

    View Slide

  15. API.AI and our Webhook
    Speech spoken
    by user
    POST request to
    Webhook with JSON
    API.AI extract
    {
    "id": "b4e86431-0336-416f-8d4d-fe1572207e58",
    "timestamp": "2017-01-07T15:25:29.686Z",
    "result": {
    "resolvedQuery": "when is the simpsons showing",
    "action": "query_content",
    "parameters": {
    "program_name": “the simpsons"
    },
    "metadata": {
    "intentName": "query_content"
    }
    }
    }

    View Slide

  16. Create Webhook Fulfillment
    1.Download template project from here: https://github.com/
    actions-on-google/apiai-webhook-template-nodejs (or create
    Node JS server project.)
    2. Edit app.js file

    View Slide

  17. 'use strict';
    process.env.DEBUG = 'actions-on-google:*';
    let Assistant = require('actions-on-google').ApiAiAssistant;
    let express = require('express');
    let bodyParser = require('body-parser');
    let app = express();
    app.use(bodyParser.json({type: 'application/json'}));
    Template Code defined in app.js file
    app.post('/', function (req, res) {
    const assistant = new Assistant({request: req, response: res});
    // Fulfill action business logic

    });
    if (module === require.main) {
    // Start the server
    let server = app.listen(process.env.PORT || 8080, function () {
    let port = server.address().port;
    console.log('App listening on port %s', port);
    });
    }
    module.exports = app;

    View Slide

  18. Create WebHook Fulfillment
    let request = require('request');

    let moment = require(‘moment');
    const QUERY_CONTENT_ACTION = 'query_content';

    const ARGUMENT_PROGRAM_NAME = 'program_name';


    app.post('/', function (req, res) {

    const assistant = new Assistant({request: req, response: res});


    function getNextAiringOfProgram(assistant){

    let showName = assistant.getArgument(ARGUMENT_PROGRAM_NAME);

    request(‘https://gook.com;searchTerm='+ showName, function (error, response,
    body) {

    })
    }

    if (!error && response.statusCode == 200) {

    var event = JSON.parse(body);

    var dateTimeFormatted = new moment(event.startDateTime);

    assistant.tell(event.title + ' will air on '
    + dateTimeFormatted.format('dddd Do [at] hh:mm a'));

    } else {

    assistant.tell('I\'m sorry, there was an error trying to process your
    request.’);

    let actionMap = new Map();

    actionMap.set(QUERY_CONTENT_ACTION, getNextAiringOfProgram);

    assistant.handleRequest(actionMap);


    View Slide

  19. Deploy Webhook
    1. Create Google Cloud Project at https://cloud.google.com
    2. Run the node server locally from folder
    3. Deploy to Google Cloud
    4. Set web hook fulfilment URL
    $ npm install
    $ npm start
    $ gcloud init
    $ gcloud app deploy

    View Slide

  20. Preview, Test + Train Action

    View Slide

  21. Test on Web Simulator

    View Slide

  22. Testing on Google Home

    View Slide

  23. Webhook Quirks
    • Timeout for service response – 5 seconds.
    • No option to configure this.
    • Be aware - the timezone you are receiving on the
    server is not the timezone of the client.
    • Solution: Request users device location and
    determine timezone from there

    View Slide

  24. Submit for approval
    1.Go to Google Cloud Console (https://
    console.cloud.google.com), search for “Google
    Actions API”
    2. Enable the API
    3. Go to Directory Listing and enter information

    View Slide

  25. Submit for approval
    • Prepare Store
    Listing items
    • Choose unique
    invocation name
    • Rules are strict, be
    prepared to be
    rejected

    View Slide

  26. VoiceLabs.co Analytics

    View Slide

  27. VoiceLabs.co Analytics

    View Slide

  28. Things to consider
    • Lots of guidelines to read online
    • Voice UI Guidelines
    • Authentication - Support for account linking via Google
    Home app. Using OAuth2.
    • Error States
    • VoiceLabs.co Analytics

    View Slide

  29. Demo Time

    View Slide

  30. Thank you!
    Rebecca Franks
    Google Developer Expert - Android
    @riggaroo / +RebeccaFranksSA
    riggaroo.co.za

    View Slide