Build Assistant Actions With Dialogflow

This talk shows how to utilize Dialogflow for your Google Assistant Actions. It includes a rather larger part of live demo which obviously cannot be shown in the slides.

Transcript

2. Dialogflow
5. App-Structure
6. Base concepts • Intents • Entities • Fulfillment • Context • Events @RittmeyerW
7. Permissions • For some things you need the user‘s consent – User name – Fine or coarse location – Push actions – Repeated actions @RittmeyerW
8. Visual Elements • Only on devices with visual interface – Cards with images, text and links – Lists – Carousel – Suggestion chips @RittmeyerW
9. SSML • Makes text to speech more lively • Supports – breaks – cardinals / ordinals / date / characters – audio / media – emphasis • Doesn‘t support other languages • https://github.com/mandnyc/ssml-builder @RittmeyerW
10. Fulfillment • It‘s just JSON going back and forth • It has to use TLS • For real projects: Use Cloud Functions / AWS Lambda • For the hackathon: – Use a local server – Expose it via ngrok • Your backend has roughly five seconds time @RittmeyerW
11. Fulfillment • Google libs for JS: https://github.com/actions-on-google/actions-on-google-nodejs • Google libs for Java/Kotlin: https://github.com/actions-on-google/actions-on-google-java • Json Spec https://developers.google.com/actions/build/json/dialogflow-webhook-json @RittmeyerW
14. Dialogflow-Demo
15. Initialisation const { dialogflow, Image, Suggestions, Carousel, //... } = require('actions-on-google'); const app = dialogflow(); // or: const app = dialogflow( {debug: true} ); @RittmeyerW
16. Normal speech app.intent('welcome', conv => { conv.ask('Hi, welcome back to Fast Journey?'); conv.ask('You can ask to review booked journeys or start booking a new one.'); conv.ask('Which would you like to do: Revisit an existing booking or booking a new trip?'); }); @RittmeyerW
17. Parameters app.intent('suggestion', (conv, {number}) => { conv.close(`Here's the number: ${number}`) }); @RittmeyerW
18. Parameters app.intent('suggestion', (conv, params) => { if (params.number !== '') { conv.ask(`Your lucky number is: ${params.number}`); } else { conv.ask('I still need your number'); } }); @RittmeyerW
19. Ending a conversation app.intent('bye', conv => { conv.close('Thanks. It was a pleasure talking to you!') }); @RittmeyerW
20. Rich responses conv.ask(new Carousel({ title: 'Carousel Title', items: { // Add the first item to the carousel 'SELECTION_KEY_ONE': { synonyms: [ 'synonym 1', 'synonym 2', ], title: 'Title of First Carousel Item', description: 'This is a description of a carousel item.', image: new Image(/*...*/) , }, // ... } } )); @RittmeyerW
21. Rich responses @RittmeyerW
22. Rich responses app.intent('helper.option', (conv, params, option) => { const SELECTED_ITEM_RESPONSES = { 'SELECTION_KEY_ONE': 'You selected the first item', 'SELECTION_KEY_GOOGLE_HOME': 'You selected the Google Home!', 'SELECTION_KEY_GOOGLE_PIXEL': 'You selected the Google Pixel!', }; conv.ask(SELECTED_ITEM_RESPONSES[option]); conv.ask('Which response would you like to see next?'); }); @RittmeyerW
23. Async work app.intent('welcome', conv => { return somePromise() .then(promiseResult => { conv.ask(promiseResult); }); }); app.intent('welcome', async conv => { let result = await somePromise(); conv.ask(result); }); @RittmeyerW
24. Integration • Probably best for the hackathon is to use Express const express = require('express') const bodyParser = require('body-parser') // ... app code here const expressApp = express().use(bodyParser.json()) expressApp.post('/fulfillment', app) expressApp.listen(3000) @RittmeyerW
25. URLs • Dialogflow Docs https://cloud.google.com/dialogflow/docs/ • Dialogflow console https://dialogflow.cloud.google.com/ @RittmeyerW
26. Account Linking • If you want to use existing accounts (like Hermes does) • Different methods (with or without Oauth) • Not recommended for the hackathon • But definitely for existing services when going live @RittmeyerW
27. Cross Platform
28. Libs • AssitantJS https://github.com/webcomputing/AssistantJS • Jovo Framework https://github.com/jovotech/jovo-framework-nodejs @RittmeyerW
29. Voice UX
30. Books • Cathy Pearl: Designing Voice User Interfaces • Amir Shevat: Designing Bots • Erika Hall: Conversational Design https://abookapart.com/products/conversational-design @RittmeyerW
31. Online • Recommendations by Google – https://developers.google.com/actions/design/ • Recommendations by Amazon – https://developer.amazon.com/ask-resources/guided/conversational- design-workshop#/ @RittmeyerW