Extending the Google Assistant with Actions on Google Guillaume Laforge, Developer Advocate, GCP Brad Abrams, Product Manager, Google Assistant

No content

No content

A conversation between you and Google that helps you get more done in your world

No content

No content

Actions on Google How YOU can extend the Google Assistant Connect with users wherever the Google Assistant is available Help users get things done with your service Innovate with Conversational Interface

Proprietary & Confidential 8

Ok Google, let me talk to Cloud NEXT 2017 Hi, I’ll be your guide to Google Cloud Next, I can help you explore topics or pick a session to attend. What would you like to know? When is the next App Engine talk? Sure! Here’s Cloud NEXT 2017. Enter Earcon The next session about App Engine is “Building Data Driven applications with App Engine” in room 4004 on Tuesday at 4. Is there another topic you’re interested in? Exit Earcon . . .

Agenda Design What to build Develop How to build it Deploy How to deliver it

Design What to build

Why conversation?

Confidential and proprietary How do we deliver on Sci-Fi’s promise?

Confidential and proprietary How do we get there?

Confidential and proprietary A conversation is a contract between two participants with a mutual investment in the outcome

Confidential and proprietary Conversations are hard to codify

No content

Spoken Language 100,000 yrs Writing 5,000 yrs Print 575 yrs Typing 150 yrs TTS 80 yrs Computers 70 yrs Speech recogn. 50 yrs Word processor 40 yrs IVRs, virtual assistants & smartphones 25 yrs Dictation software 20 yrs Chat bots & A.I. assistants <7yrs Internet 30 yrs

1. List out your core brand attributes (what words define the experience you’re shooting for?) 2. Correlate to attributes that will define your functional design principles (how will those manifest in the design?) 3. Define some attributes that you’d want to infuse into the voice, style of writing, and personality of the dialog (what personality traits match your strategy?) 4. Style guide & “bio sketch” (practical application and maintain consistency for longevity of your experience) knowledgeable helpful encouraging data rich recommending proactive geeky eager motivating Create your persona

Example style guide INSTEAD OF... IS MORE LIKELY TO SAY... I found Up for that? Does that sound good? Maybe later While you’re at it... what’s going on I did not receive a response if you feel you have reached this message in error please select from one of the following X options to help us serve you better for questions related to... you have entered that was an invalid… we require that you... please try again for faster answers we’re sorry, we are unable to… I did not understand MIGHT SAY THINGS LIKE... so you can keep up to date on, I’ll look it up right now Sure, that’s coming up Right around the corner from… That session’s full, but… You might like lets need can’t because more about help right now one sec stay allows require unable to due to additional regarding assist currently please hold remain WOULD NEVER SAY...

Write your sample dialogs

Sample dialogs 1. Canonical “Happy path” 2. First time experience 3. Tapered experience (Return user) 4. Repair 5. Personality questions g.co/dev/ActionsChecklist

Develop How to build it

Life of a conversation “Ok Google, talk to Cloud NEXT 2017” Invoke “Cloud NEXT” action “Hi! Welcome to Cloud Next...” Speech to Text “The next Machine Learning Session is…” “I want to hear more about Machine Learning” Text to Speech “Sure, here’s Cloud NEXT 2017” Speech to Text, NLP, Knowledge Graph, ML Ranking, User Profile Text to Speech

Cross-platform tool for building advanced conversational interfaces ListSessions (topics.ComputeEngine); “What GCE breakout sessions are there?”

Demo — API.AI

A “serverless platform for building event-based microservices”. Function-as-a-service approach Great fit for event-oriented architectures, supporting 3 kind of triggers: ● Cloud Storage updates ● Cloud Pub/Sub messages ● Direct HTTP calls Cloud Functions

Cloud Functions Completely serverless & fully managed service ⇒ don’t worry about the ops! Automatic scaling and super-fast ⇒ grows with the success of your project ⇒ cost-effective Open and familiar ⇒ JavaScript / Node.js

Architecture in development

Architecture in development ngrok

Fast feedback loop: Ngrok + Functions emulator Google Cloud Functions emulator Ngrok secure internet tunnels to localhost LIVE RELOADING DEBUG IN CHROME

Demo — Cloud Functions & local development

server.js NPM dependencies ● actions-on-google ● node-fetch Export a function with the ApiAiAssistant handling the requests const Assistant = require('actions-on-google').ApiAiAssistant; const fetch = require('node-fetch'); function listTopicsIntent(assistant) { fetch('https://cloudnext.withgoogle.com/api/v1/categories') .then(response => response.text()) .then(text => { let data = JSON.parse(text.split('\n')[1]); let topics = data.categories .find(cat => cat.name === "Topics") .children.map(topic => topic.name).join(', ')); assistant.ask(`The topics covered are: ${topics}. What do you want to learn?'`); }); } exports.agent = function(request, response) { var assistant = new Assistant({ request, response }); assistant.handleRequest(listTopicsIntent); };

No content

Deploy How to deliver it

Deploying Cloud Functions In production gcloud beta functions deploy agent \ --trigger-http \ --stage-bucket gs://gcp-next-2017-agent/ Locally functions deploy agent --trigger-http

Demo — deploying a Cloud Function

Review & approval Web-based portal ● Triggering Information ● Merchandising and information Approvals ● Automatic and manual policy checks ● Turn around in about 1 week

Discovery Discovery patterns ● Guaranteed invocation ○ “Talk to Cloud Next 2017” ● Discovery Patterns ○ “What’s happening at Next?” Google Home app

Demo — deploying action on Google

No content

Key takeaways 1. You can (and should!) extend the Google Assistant with your custom action 2. Talking with humans is challenging, but API.AI makes it approachable 3. GCP offers a powerful platform for hosting business logic

Other Sessions Covering Conversational Application Development at NEXT BD223: Your next app might be a bot! Building conversational user experiences with API.AI BD225: Extending the Google Assistant with Actions on Google IO225: Best practices for building AI assistant in the enterprise What: How to build a conversational application with API.AI including tips and tricks from other developers Who: ● Dan Aharon, Google ● Cornelius Munz, Bosch ● Petr Netusil, ING Belgium What: How to create an Action that extends Google Assistant and is hosted on GCP Who: ● Brad Abrams, Google ● Guillaume Laforge, Google What: How to use serverless architecture and Cloud Functions to build conversational applications Who: ● Alan Ho, Google ● Amir Shevat, Slack

g.co/dev/ActionsDesign

Questions & answers

Thank you