Jon Markoff - Voice in the enterprise

1. Jon Markoff Developer Advocate Voice in the Enterprise .droidconBerlin
3. What’s possible with voice now? Smart Hotel Rooms (Consumer Focused) • “Have the valet bring my car around please” First Responders (Internal Focused) • “Navigate me to my next emergency”
4. • Real time language translations • Complex dictation • Augmented reality, AI and voice Where is voice going?
5. Voice all the things... Today we will learn 3 things: • Build a voice agent (Voice app or bot) • Enable voice in your Android app • Integrate with the Google Assistant and beyond
6. 1. Let’s build an agent Dialogflow is a great place to start for building rich conversational experiences. • Quickly prototype • Easily test what you’re building • Extend and integrate with many 3rd parties • Agnostic of platform or consumers
7. 1. Build a conference room finder
8. 1.Where is the nearest conference room intent
9. 2. Integrate with a new or existing app Set up the API to v1 in Dialogflow -> Settings -> General
10. 2. App setup - Dialogflow Permissions <uses-permission android:name="android.permission.INTERNET"/> <uses-permission android:name="android.permission.RECORD_AUDIO" /> Gradle additions implementation 'ai.api:sdk:2.0.7@aar' implementation 'ai.api:libai:1.6.12' Record Audio Permission (Automatic EMM) ActivityCompat.requestPermissions(this, permissions, REQUEST_RECORD_AUDIO_PERMISSION);
11. 2. Code Setup Declare member variables private AIConfiguration dfConfig; private AIService aiService; private TextToSpeech textToSpeech; Add to onCreate() / Init dfConfig = new AIConfiguration(“<CLIENT ACCESS TOKEN>”, AIConfiguration.SupportedLanguages.English, AIConfiguration.RecognitionEngine.System); aiService = AIService.getService(getApplicationContext(), dfConfig); aiService.setListener(this); textToSpeech = new TextToSpeech(this, new TextToSpeech.OnInitListener() { … }
12. 2. Listening and Playing back audio Start/Stop listening for audio aiService.startListening(); aiService.stopListening(); Implement AIListener interface @Override public void onResult(ai.api.model.AIResponse result) { String responseText = result.getResult().getFulfillment().getSpeech(); textToSpeech.speak(speech, TextToSpeech.QUEUE_ADD, null, null); String action = result.getResult().getAction(); // The text identifier } @Override public void onError(ai.api.model.AIError error) { Log.e(TAG, "Error, " + error.getMessage()); textToSpeech.speak(“I’m sorry, I didn’t get that”, TextToSpeech.QUEUE_ADD, null, null); }
13. 3.The Google Assistant - Actions on Google The Actions Console provides access to the following: • Manage one or more Actions projects • Manage your actions directory listing • Find health and metrics • Manage deployments to the Assistant
14. 3. Deploy from Dialogflow
15. 3. Test in the Simulator
16. 3.The Google Assistant - Server Side Authentication Link your enterprise or consumer user with the Google Assistant • Configure your server to accept industry standard OAuth 2.0 flows • Enable Account linking which will require scopes and client information • Trigger sign in Verify user has signed in app.intent('ask_for_sign_in', (conv) => { conv.ask(new SignIn()); }); app.intent('ask_for_sign_in_confirmation', (conv, params, signin) => { if (signin.status !== 'OK') { return conv.ask('You need to sign in before using the app.'); } // const access = conv.user.access.token; conv.ask('Great! Thanks for signing in.'); });
17. 3.The Google Assistant - GSuite Configuration How to get started with the Google Assistant - For GSuite Customers: • Enable Web & App Activity under Apps > Additional Google Services • Enable Google Developers Console under Apps > Additional Google Services
18. Circling back to our initial use cases Smart Hotel Rooms - Consumer • Dialogflow Enterprise • Android app integration • Integrated with the Google assistant, leveraging authentication First Responders - Internal • Dialogflow Enterprise • TextToSpeech, Android
19. Thank you! Jon Markoff
20. Q&A