Conversation Actions for Google Assistant

TechTalk given at ChicagoRoboto 2017.

Transcript

1. Conversation Actions for Google Assistant Andrew Kelly @andrew_ke11y Google Developer Expert - Android
2. Actions on Google • Actions on Google let you build for the Google Assistant. • Google Assistant is available today through Google Home, Pixel phones and Allo app - many more in the future. • Direct Actions and Conversation Actions - Direct Actions • Interaction is handled by Google Assistant • Developer handles fulfilment of response • Direct Actions aren't open for developers…yet • “OK Google, Play Rage Against the Machine on Google Play Music”
3. Actions on Google • Direct Actions and Conversation Actions - Conversation Actions • A two-way dialog with users • Developer handles parsing of conversation intent/meaning • Available today for review/deployment • “OK Google, let me talk to Dr Nick”
4. Design Your Persona • Define your use-cases, not everything suits a voice interaction. • Write sample conversation dialogs • Define how your persona will respond - What questions? - What prompts? - Things it won’t say - Whimsical or Serious? • Pick a voice - 2 male, 2 female • Useful when multiple people are working on your persona - consistency
5. Conversation Action
6. Conversation Action • Actions SDK - Handling of raw requests - Developer handles extraction of meaning/intent/entities etc. • API.AI - Conversational User Experience Platform. - Natural Language processing. - AI learning and refinement. - Conversation debugging
7. Conversation Action - API.AI
9. Web hook JSON { "id": "cc5b1b3d-b5b8-416d-819a-684bf2914f7a", "timestamp": "2017-03-21T03:54:01.251Z", "lang": "en", "result": { "source": "agent", "resolvedQuery": "my head is bleeding", "action": "bedside-mannor", "actionIncomplete": false, "parameters": { "body-parts": "head", "symptoms": "bleeding" }, "contexts": [], "score": 1 }, "sessionId": "722df73e-fe82-41a6-b79d-f0061930ea52" }
10. Review 1. Create API.AI agent 2. Setup intents, entities and sample requests 3. Apply AI learnings (optional) 4. Create and deploy web-hook for processing requests • This allows for better responses than the canned examples 5. Test, test, test. 6. Submit for approval.
11. More Info • Actions on Google - https://developers.google.com/actions/ • Cloud Functions - http trigger - https://cloud.google.com/functions/docs/writing/http - https://developers.google.com/actions/reference/ApiAiAssistant - https://developers.google.com/actions/reference/webhook-format • Unofficial Java bindings for Actions - https://github.com/frogermcs/Google-Actions-Java-SDK
12. Google Cloud Next 17 • Extending the Google Assistant with Actions on Google - https://www.youtube.com/watch?v=7e0RGIul8Kk • Your next app might be a bot! Building conversational UX with API.AI - https://www.youtube.com/watch?v=LHX1GqZuaII
13. Thank you! Andrew Kelly @andrew_ke11y Google Developer Expert - Android