Building Conversational Experiences

Getting started with Actions on Google and Dialogflow.

Presentation held for GDG Bergen on April 18th, 2018.

Transcript

1. Building Conversational Experiences Hans Petter Eide | Zeros & Ones AS | @hanspeide
2. What is it all about?
3. 400m+ devices supported
4. Japanese German FR French CA French Korean UK English AU English CA English US English Italian Spanish BR Portuguese LATAM Spanish Hindi & IN English Swedish Danish Indonesian Russian Thai Dutch Norwegian 21 languages supported
5. Google Assistant SDK developers.google.com/assistant/sdk
6. Smart Home Device Integration developers.google.com/actions/smarthome/ Dim the lights in the living room a little bit. What lights are on in the living room? Make it warmer. Turn off all the lights. What is the temperature inside? How many lights are on?
7. Hey Google, talk
 to __________. Hey Google, connect me with __________. Hey Google, get __________. We will cover Actions on Google
8. Action { conversation api request } { conversation api response } user input Action response
9. Design
10. Desktop first Mobile first Voice first? ~1960-~2011 ~2011-~2018 ~2018 ?
11. Offer core features rather than feature parity …for now
12. [1] Create a Persona
13. A Persona is conveyed through: • Tone • Word and phrase choices • Functional design • Style • Technique • Voice And it is based on: • Your user population • Their needs • The imagery & qualities associated with your brand
14. [2] Think outside the box
15. code code code code code code code code code code code “dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code
 code code code
 code code code code code code code code code code code code code code code code code CODE STRUCTURE
16. code code code code code code code code code code code “dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code
 code code code
 code code code code code code code code code code code code code code code code code CODE STRUCTURE
17. DIALOG STRUCTURE
18. [3] In a conversation,
 there are no “errors”
19. real life user conditions {
20. When a so-called “error” occurs in a conversation, it should be treated simply as a new turn in the dialog, only with different conditions.
21. g.co/dev/ActionsDesign
22. Let there be code…
23. Node server with AoG SDK { conversation api request } { conversation api response } user input Action response
24. Support speech and display text function simpleResponse (app) {
 app.ask({
 speech: 'Howdy! I can tell you fun facts about ' +
 'almost any number, like 42. What do you have in mind?',
 displayText: 'Howdy! I can tell you fun facts about ' +
 'almost any number. What do you have in mind?'
 });
 }
25. Guide the user function suggestionChips (app) {
 app.ask(app.buildRichResponse()
 .addSimpleResponse({
 speech: 'Howdy! I can tell you fun facts about ' +
 'almost any number like 0, 42, or 100. What number do you have ' +
 'in mind?',
 displayText: 'Howdy! I can tell you fun facts about almost any ' +
 'number. What number do you have in mind?'
 })
 .addSuggestions(['0', '42', '100', 'Never mind'])
 );
 }
26. Display basic cards function basicCard (app) {
 app.ask(app.buildRichResponse()
 .addSimpleResponse('Math and prime numbers it is!')
 .addBasicCard(
 app.buildBasicCard(`42 is an even composite number [..]`)
 .setTitle('Math & prime numbers')
 .addButton('Read more')
 .setImage('https://example.google.com/42.png', ‘Alternate text')
 )
 );
 }
27. Used for
 easy selection <10 items Used for comparison <30 items Carousels and lists for selection
28. Google Home Mobile Device NAME Registered device user’s full name Registered device user’s full name DEVICE_COARSE_LOCATION Zip code and city N/A DEVICE_PRECISE_LOCATION Coordinates and street address Coordinates UPDATE Permission to send updates Permission to send updates Permissions
29. Q: “What’s the weather forecast for tomorrow?” A: “To give you the weather forecast,
 I'll just need to get your location from Google. Is that okay?" let permission = app.SupportedPermissions.DEVICE_COARSE_LOCATION;
 app.askForPermission(‘To give you the weather forecast’, permission); Permissions
30. Meet
31. NLP is hard …very hard
32. “ Intent Matching — Match and categorize sentences spoken by the user to an intent. Entity Extraction — Identify key words and phrases spoken by the user. @
33. Intents
34. Entities User Defined Pre-built
35. Actions and parameters
36. Responses
37. Responses +
38. const app = new DialogflowApp({request: request, response: response}); function getWeather(){ /* ... */ } const actionMap = new Map(); actionMap.set("weather", getWeather); app.handleRequest(actionMap); Mapping intents to code Tying intents and entities to code
39. const app = new DialogflowApp({request: request, response: response}); app.say(`So you're wondering if you need an ${app.getArgument("outfit")} today?`) Extracting entity arguments Tying intents and entities to code
40. Contexts “What’s the weather forecast for tomorrow?” “In Bergen tomorrow you can expect showers, with a low of 8 and a high of 14 degrees.” “What about in Oslo?” “Oslo will get rain in the morning, and be partly cloudy in the afternoon. Temperatures will range from 6 to 11 degrees.”
41. Contexts “What’s the weather forecast for tomorrow?” “What about in Oslo?”
42. const app = new DialogflowApp({request: request, response: response}); let location = app.getContextArgument('weather', ‘date-time’).value; Extracting context arguments
43. Dialogflow Integrations
44. Entities: umbrella | today “Do I need to bring an umbrella today?” User Says Response Webhook “Do I need to bring an umbrella today?” “Rain is expected in the afternoon, so it’s probably not a bad idea to bring an umbrella”