Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Working with Google Assistant

Working with Google Assistant

2307a37297162f815342545a2068b2f1?s=128

Erik Hellman

April 07, 2017
Tweet

More Decks by Erik Hellman

Other Decks in Programming

Transcript

  1. Working with Google Assistant Erik Hellman

  2. Working with Google Assistant Erik Hellman

  3. Working with Actions on Google Erik Hellman

  4. Working with Actions on Google Erik Hellman

  5. Building Chatbots with API.ai Erik Hellman

  6. Building Chatbots with API.ai Erik Hellman

  7. Building Chatbots Erik Hellman

  8. 100% Fragment free session!

  9. My Google Home

  10. Create your own integrations! https://developers.google.com/actions/

  11. Components of Conversation Actions

  12. Direct Actions

  13. Conversation Actions

  14. Chatbot system design

  15. Use Case: The DroidCon Italy 2017 Chatbot!

  16. DroidCon IT Chatbot Diagram

  17. Business Plan 1.Define API.ai intents and entities 2.Create Web Hooks

    using Firebase Functions 3.$$$
  18. API.ai Entities Demo!

  19. Challenges • Refer by (real) value • Variations • Conflicting

    values
  20. Speakers and events // Speakers [ { "bio" : "<p>Wayne

    Piekarski is ...</p>", "company_name" : "Google", "id" : "2808", "name" : "Wayne Piekarski", "personal_url" : "http://it.droidcon.com/2017/speakers/wayne-piekarski/", "photo_url" : "http://it.droidcon.com/2017/wp-content/uploads/2013/12/wayne.jpg" }, ... ] // Events { "69825914" : { "day_id" : "0", "description" : "<p>Google recently announced the Developer Preview for Android Things...</p>", "end_time" : 1491468000000, "experience_level" : "beginner", "id" : "69825914", "name" : "Android Things for IoT", "place_id" : "0", "speaker_ids" : [ "2808" ], "start_time" : 1491465600000, "track_id" : "2", "type" : "talk" }, ... ]
  21. Speaker entities

  22. Speaker entities

  23. Topic entities

  24. Event entities

  25. API.ai Intents

  26. API.ai Intents Demo!

  27. Challenges • Predicting phrases • Mapping entities

  28. Fullfillments Web Hooks using Cloud Functions

  29. Complaining about Gradle build times? $ time firebase deploy ...

    real 0m51.385s user 0m2.944s sys 0m1.433s
  30. Node.JS Fullfilment Web Hook (1) 'use strict'; const ApiAiAssistant =

    require('actions-on-google').ApiAiAssistant; var functions = require('firebase-functions'); const speakers = require('./droidcon-it-2017-chatbot-speakers.json'); const events = require('./droidcon-it-2017-chatbot-events.json')
  31. Node.JS Fullfilment Web Hook (2) exports.droidConItalyFulfillments = functions.https.onRequest(function(req, res) {

    const assistant = new ApiAiAssistant({request: req, response: res}); console.log('Request headers: ' + JSON.stringify(req.headers)); console.log('Request body: ' + JSON.stringify(req.body)); // Lookup event and generate response function lookupSpeakerEvent(assistant) { ... } const actionMap = new Map(); actionMap.set('lookup-speaker-event', lookupSpeakerEvent); assistant.handleRequest(actionMap); });
  32. Node.JS Fullfilment Web Hook (3) function lookupSpeakerEvent(assistant) { const speakerName

    = assistant.getArgument('speaker'); var speaker = findSpeakerByName(speakerName); if (speaker != null) { var event = findEventBySpeaker(speaker); if (event != null) { assistant.tell(speakerName + ' will be talking about ' + event.name + ' at ' + toDayAndTime(event.start_time); } } assistant.tell('No presentation by ' + speakerName + ' could be found...'); }
  33. Find event by topic // Find all events for a

    topic and ask which one the use wants to hear about function findEventByTopic(assistant) { const topic = assistant.getArgument('topic'); var matchingEvents = findEventsByTopic(topic); // Returns array of events var response = 'I found the following events; ' for (var event in matchingEvents) { response += event.name + ' at ' + toDayAndTime(event.start_time) + '. '; } response += 'Which one do you want to hear more about?' // Input prompts when no reply is given const noInputPrompts = ['Which event do you want to hear more about?', 'Say the name of the event.', 'Ok, bye for now!']; assistant.ask(response, noInputPrompts); }
  34. Give event details function giveEventDetails(assistant) { const eventName = assistant.getArgument('event-name');

    var event = findEventByName(eventName); var speakers = findSpeakersForEvent(event); var response = eventName + ' starts at ' + toDayAndTime(event.start_time) + ' and is presented by ' + speakers.join(' and ') + '. ' + 'The description reads; ' + event.description; assistant.tell(response); }
  35. Update action map const actionMap = new Map(); actionMap.set('lookup-speaker-event', lookupSpeakerEvent);

    actionMap.set('find-events-by-topic', findEventByTopic); actionMap.set('give-event-details', giveEventDetails); assistant.handleRequest(actionMap);
  36. ApiAiAssistant API const assistant = new ApiAiAssistant({request: req, response: res});

    // Read a message to the user, ends conversation. assistant.tell(message); // Get an argument from the interpreted voice input. var arg1 = assistant.getArgument("arg1"); // Read a message asking for further input assistant.ask(question, noInputs); // Get the raw input var rawInput = assistant.getRawInput(); // Map function to intent const actionMap = new Map(); actionMap.set('lookup-speaker-event', lookupSpeakerEvent); actionMap.set('find-events-by-topic', lookupEventsByTopic); actionMap.set('give-event-details', giveEventDetails); assistant.handleRequest(actionMap)
  37. DEMO! (everybody hold their breath!)

  38. Designing interactions

  39. Google Voice Design Principles • Keep it short • Give

    them credit • Optimize for relevance • Delight the ear without distracting the mind • Engage beginners and attract experts • Take turns • Don't read minds https://developers.google.com/actions/design/ principles
  40. Erik's Design Tips • Don't annoy the user • Variate

    your output • Avoid long error descriptions
  41. Beyond Google Home

  42. Beyond Google Home https://github.com/api-ai/apiai- android-client

  43. Thank you for listening! @ErikHellman