Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Build Assistant Actions With Dialogflow

Build Assistant Actions With Dialogflow

This talk shows how to utilize Dialogflow for your Google Assistant Actions. It includes a rather larger part of live demo which obviously cannot be shown in the slides.

D0ed2f28dd13ddfb192ebe198623924c?s=128

Wolfram Rittmeyer

September 28, 2019
Tweet

Transcript

  1. None
  2. Dialogflow

  3. None
  4. None
  5. App-Structure

  6. Base concepts • Intents • Entities • Fulfillment • Context

    • Events @RittmeyerW
  7. Permissions • For some things you need the user‘s consent

    – User name – Fine or coarse location – Push actions – Repeated actions @RittmeyerW
  8. Visual Elements • Only on devices with visual interface –

    Cards with images, text and links – Lists – Carousel – Suggestion chips @RittmeyerW
  9. SSML • Makes text to speech more lively • Supports

    – breaks – cardinals / ordinals / date / characters – audio / media – emphasis • Doesn‘t support other languages • https://github.com/mandnyc/ssml-builder @RittmeyerW
  10. Fulfillment • It‘s just JSON going back and forth •

    It has to use TLS • For real projects: Use Cloud Functions / AWS Lambda • For the hackathon: – Use a local server – Expose it via ngrok • Your backend has roughly five seconds time @RittmeyerW
  11. Fulfillment • Google libs for JS: https://github.com/actions-on-google/actions-on-google-nodejs • Google libs

    for Java/Kotlin: https://github.com/actions-on-google/actions-on-google-java • Json Spec https://developers.google.com/actions/build/json/dialogflow-webhook-json @RittmeyerW
  12. None
  13. None
  14. Dialogflow-Demo

  15. Initialisation const { dialogflow, Image, Suggestions, Carousel, //... } =

    require('actions-on-google'); const app = dialogflow(); // or: const app = dialogflow( {debug: true} ); @RittmeyerW
  16. Normal speech app.intent('welcome', conv => { conv.ask('Hi, welcome back to

    Fast Journey?'); conv.ask('You can ask to review booked journeys or start booking a new one.'); conv.ask('Which would you like to do: Revisit an existing booking or booking a new trip?'); }); @RittmeyerW
  17. Parameters app.intent('suggestion', (conv, {number}) => { conv.close(`Here's the number: ${number}`)

    }); @RittmeyerW
  18. Parameters app.intent('suggestion', (conv, params) => { if (params.number !== '')

    { conv.ask(`Your lucky number is: ${params.number}`); } else { conv.ask('I still need your number'); } }); @RittmeyerW
  19. Ending a conversation app.intent('bye', conv => { conv.close('Thanks. It was

    a pleasure talking to you!') }); @RittmeyerW
  20. Rich responses conv.ask(new Carousel({ title: 'Carousel Title', items: { //

    Add the first item to the carousel 'SELECTION_KEY_ONE': { synonyms: [ 'synonym 1', 'synonym 2', ], title: 'Title of First Carousel Item', description: 'This is a description of a carousel item.', image: new Image(/*...*/) , }, // ... } } )); @RittmeyerW
  21. Rich responses @RittmeyerW

  22. Rich responses app.intent('helper.option', (conv, params, option) => { const SELECTED_ITEM_RESPONSES

    = { 'SELECTION_KEY_ONE': 'You selected the first item', 'SELECTION_KEY_GOOGLE_HOME': 'You selected the Google Home!', 'SELECTION_KEY_GOOGLE_PIXEL': 'You selected the Google Pixel!', }; conv.ask(SELECTED_ITEM_RESPONSES[option]); conv.ask('Which response would you like to see next?'); }); @RittmeyerW
  23. Async work app.intent('welcome', conv => { return somePromise() .then(promiseResult =>

    { conv.ask(promiseResult); }); }); app.intent('welcome', async conv => { let result = await somePromise(); conv.ask(result); }); @RittmeyerW
  24. Integration • Probably best for the hackathon is to use

    Express const express = require('express') const bodyParser = require('body-parser') // ... app code here const expressApp = express().use(bodyParser.json()) expressApp.post('/fulfillment', app) expressApp.listen(3000) @RittmeyerW
  25. URLs • Dialogflow Docs https://cloud.google.com/dialogflow/docs/ • Dialogflow console https://dialogflow.cloud.google.com/ @RittmeyerW

  26. Account Linking • If you want to use existing accounts

    (like Hermes does) • Different methods (with or without Oauth) • Not recommended for the hackathon • But definitely for existing services when going live @RittmeyerW
  27. Cross Platform

  28. Libs • AssitantJS https://github.com/webcomputing/AssistantJS • Jovo Framework https://github.com/jovotech/jovo-framework-nodejs @RittmeyerW

  29. Voice UX

  30. Books • Cathy Pearl: Designing Voice User Interfaces • Amir

    Shevat: Designing Bots • Erika Hall: Conversational Design https://abookapart.com/products/conversational-design @RittmeyerW
  31. Online • Recommendations by Google – https://developers.google.com/actions/design/ • Recommendations by

    Amazon – https://developer.amazon.com/ask-resources/guided/conversational- design-workshop#/ @RittmeyerW
  32. None