Actions on Google - Workshops - Community Roadshow

Here are some links because video and voice examples are not working :(
You can find codelabs level 1 here bit.ly/aog_lvl1

slide 9 - What is Google Assistant video https://www.youtube.com/watch?v=g410vzUXiyQ

slide 19 - https://g.co/dev/ActionsDesign

slide 27 - SSML 1 - https://docs.google.com/file/d/153vK80HhFYJuT1C6PSu7tjSeL558HsL4/preview

slide 27 - SSML 2 - https://docs.google.com/file/d/1PM2-Y1WIuwy7kSTwhOxh47eZdCghSzrO/preview

slide 28 - advanced SSML https://docs.google.com/file/d/1m0CEOKeeXZ2srYzMLINjJcBTsX8WrXBr/preview

slide 34 -Repository link https://github.com/Deeplocal/mocktailsmixer and video https://docs.google.com/file/d/1wds_ylAR8aRq675Mim3-mYXXYD6JEzFvmw/preview

slide 35 - Project Oasis https://www.youtube.com/watch?v=yyjN_ZwJr8Y

Transcript

1. #CommunityRoadshow Łukasz Byjoś | GDG Rzeszów | @lukaszbyjos
2. #CommunityRoadshow Be excellent to each other. Treat everyone with respect. Participate while acknowledging that everyone deserves to be here — and each of us has the right to enjoy our experience without fear of harassment, discrimination, or condescension, whether blatant or via micro-aggressions. Jokes shouldn’t demean others. Consider what you are saying and how it would feel if it were said to or about you. Practice saying "Yes and" to each other. It’s a theatre improv technique to build on each other’s ideas. We all benefit when we create together. Speak up if you see or hear something. Harassment is not tolerated, and you are empowered to politely engage when you or others are disrespected. The person making you feel uncomfortable may not be aware of what they are doing, and politely bringing their behavior to their attention is encouraged. In case you see or hear something or feel uncomfortable, reach out to any organizer Code of Conduct
3. #CommunityRoadshow 1 - 5 min Welcome 2 - 35 min Building Conversational Experiences 3 - 50 min Codelabs! Build Actions for the Google Assistant (Level 1) Kind of Agenda
4. #CommunityRoadshow Building Conversational Experiences Łukasz Byjoś | GDG Rzeszów | @lukaszbyjos
5. #CommunityRoadshow Ready to help across 500M+ devices
6. #CommunityRoadshow A growing audience: devices
7. #CommunityRoadshow Japanese German FR French CA French Korean UK English AU English CA English US English Italian Spanish BR Portuguese LATAM Spanish Hindi & IN English A growing audience: languages Swedish Danish Norwegian Indonesian Russian Thai Dutch
8. #CommunityRoadshow Hey Google, talk to __________. Hey Google, connect me with __________. Hey Google, get __________.
9. #CommunityRoadshow
10. #CommunityRoadshow Ok but where is the code...
11. #CommunityRoadshow Action { conversation api request } { conversation api response } user input Action response
12. #CommunityRoadshow Ok Google, talk to Personal Chef Sure, here’s Personal Chef What are you in the mood for? What protein would you like to use? Speech to Text NLP Knowledge Graph ML Ranking User Profile Text to Speech Speech to Text Text to Speech ... Invoke Personal Chef action Parse query and generate response Well, it’s kind of cold outside, so I’d like...
13. #CommunityRoadshow Well, it’s kind of cold outside, so I'd like something to warm me up, like a hot soup, and I want it fast.
14. #CommunityRoadshow “ Intent Matching — Match and categorize user utterances to an intent. Entity Extraction — Identify key words and phrases spoken by the user. @
15. #CommunityRoadshow
16. #CommunityRoadshow Ok Google, talk to Personal Chef Sure, here’s Personal Chef What are you in the mood for? What protein would you like to use? Speech to Text NLP Knowledge Graph ML Ranking User Profile Speech to Text Text to Speech ... ... ... Trigger WELCOME event ... NLP: Intent Matching Entity Extraction Well, it’s kind of cold outside, so I’d like...
17. #CommunityRoadshow Entities: homemade cannoli “Find me a recipe for homemade cannoli” User Says Response Webhook “Find me a recipe for homemade cannoli” “Here’s the best one I can find”
18. #CommunityRoadshow Still no code...
19. #CommunityRoadshow g.co/dev/ActionsDesign
20. #CommunityRoadshow AUDIO_OUTPUT SCREEN_OUTPUT Support different surface capabilities developers.google.com/actions/assistant/surface-capabilities
21. #CommunityRoadshow Support speech and display text developers.google.com/actions/assistant/responses conv.ask(new SimpleRespone({ speech: 'Howdy! I can tell you fun facts about ' + 'almost any number, like 42. What do you have in mind?', text: 'Howdy! I can tell you fun facts about ' + 'almost any number. What do you have in mind?' }));
22. #CommunityRoadshow Guide the user (suggestion chips) developers.google.com/actions/assistant/responses conv.ask(new SimpleRespone({ speech: 'Howdy! I can tell you fun facts about ' + 'almost any number, like 42. What do you have in mind?', text: 'Howdy! I can tell you fun facts about ' + 'almost any number. What do you have in mind?' })); conv.ask(new Suggestion(['0', '42', '100', 'Never mind']));
23. #CommunityRoadshow Display basic cards developers.google.com/actions/assistant/responses conv.ask('Math and prime numbers it is!') conv.ask(new BasicCard({ text: '42 is an even composite number. It is composed of three distinct prime ' + 'numbers multiplied together. It has a total of eight divisors. 42 is an ' + 'abundant number, because the sum of its proper divisors 54 is greater ' + 'than itself. To count from 1 to 42 would take you about twenty-one…', title: 'Math & prime numbers', buttons: new Button({ title: 'Read more', url: 'https://example.com', }), image: new Image({ url: 'https://example.google.com/42.png', alt: 'Image alternate text', }), }));
24. #CommunityRoadshow Used for easy selection <10 items Used for comparison <30 items Lists and carousels for selection developers.google.com/actions/assistant/responses
25. #CommunityRoadshow Use helpers developers.google.com/actions/assistant/helpers app.intent('ask_for_datetime_detail', (conv) => { const options = { prompts: { initial: 'When would you like to schedule the appointment?', date: 'What day was that?', time: 'What time?', }, }; conv.ask(new DateTime(options)); });
26. #CommunityRoadshow Google Home Mobile Device NAME Registered device user’s full name Registered device user’s full name DEVICE_COARSE_LOCATION Zip code and city N/A DEVICE_PRECISE_LOCATION Coordinates and street address Coordinates Q: "Recommend me a local bookstore" A: "To find bookstores near you, I'll just need to get your zip code from Google. Is that okay?" Use helpers developers.google.com/actions/assistant/helpers#user_information conv.ask(new Permission({ context: 'To find bookstores near you', permissions: 'DEVICE_COARSE_LOCATION', }));
27. #CommunityRoadshow SSML - Speech Synthesis Markup Language developers.google.com/actions/reference/ssml <speak> Step 1, take a deep breath. <break time="200ms"/> Step 2, exhale. Step 3, take a deep breath again. <break strength="weak"/> Step 4, exhale. </speak> <speak> my SSML content </speak>
28. #CommunityRoadshow SSML - Speech Synthesis Markup Language developers.google.com/actions/reference/ssml <speak> <par> <media xml:id="intro" soundLevel="5dB" fadeOutDur="2.0s"> <audio src="https://upload.wikimedia.org/wikipedia/commons/4/43/Nowhere_Land_%28ISRC_USU AN1600051%29.mp3" clipEnd="5.0s"> <desc>news intro</desc> INTRO </audio> </media> <media xml:id="crowd" soundLevel="5dB" fadeOutDur="1.0s" begin="intro.end-1.0s"> <audio src="https://actions.google.com/sounds/v1/crowds/battle_cry_high_pitch.ogg" clipEnd="3.0s"> <desc>crowd cheering</desc> YEAH! </audio> </media> ... https://medium.com/google-developers/advanced-ssml-for-actions-on-google-5cea45d868c9
29. #CommunityRoadshow Seamless account linking with Google Sign-in Link an account to your OAuth2 server developers.google.com/actions/identity/
30. #CommunityRoadshow • Build orders • Use Google provided payment instrument • Use your payment processor (Stripe, Braintree, Vantiv, more coming) • Update order statu Transact with the user developers.google.com/actions/transactions (check policies and guidelines for availability)
31. #CommunityRoadshow Transact with the user developers.google.com/actions/identity
32. #CommunityRoadshow Smart Home Device Integration developers.google.com/actions/smarthome/ Dim the lights in the living room a little bit. What lights are on in the living room? Make it warmer. Turn off all the lights. What is the temperature inside? How many lights are on?
33. #CommunityRoadshow Google Assistant SDK developers.google.com/assistant/sdk
34. #CommunityRoadshow Google Assistant SDK developers.google.com/assistant/sdk https://github.com/Deeplocal/mocktailsmixer
35. #CommunityRoadshow Google Assistant SDK developers.google.com/assistant/sdk
36. #CommunityRoadshow Templates get you up and running developers.google.com/actions/templates
37. #CommunityRoadshow Reach Users
38. #CommunityRoadshow Ok Google, ask Personal Chef for a hot soup recipe Trigger Phrase Action Name Developer Specified Action Preposition Action Phrase Developer Specified Working Examples: Let’s speak to Domino’s Ask Dr. Doggy if dogs can eat chocolate Ok Google, let’s talk to Personal Chef Trigger Phrase Action Name Developer Specified In Dialogue Discovery: Explicit Triggering
39. #CommunityRoadshow In Dialogue Discovery: Implicit Triggering Hey Google, I want to work out. Hey Google, I want to play a game. Hey Google, what’s the surf report? Hey Google, tell me a joke.
40. #CommunityRoadshow Discovery in the Google Assistant Directory
41. #CommunityRoadshow Link to your Action from anywhere developers.google.com/actions/console/publishing#linking_to_your_actions
42. #CommunityRoadshow Break || codelabs
43. #CommunityRoadshow Actions on Google Community Program developers.google.com/actions/community You are automatically invited to the program with publishing your first Action and will receive a Cool T Shirt + $200 Cloud Credits per month for a year The Assistant team also recognizes milestones: Traction Engagements Globalization Deadline July 31, 2018 at 11:59 PM (PST)
44. #CommunityRoadshow Codelabs codelabs.developers.google.com/io2018 Part 1 codelabs quicklink bit.ly/aog_lvl1
45. #CommunityRoadshow Actions on Google Community Program developers.google.com/actions/community You are automatically invited to the program with publishing your first Action and will receive a Cool T Shirt + $200 Cloud Credits per month for a year The Assistant team also recognizes milestones: Traction Engagements Globalization Deadline July 31, 2018 at 11:59 PM (PST)
46. #CommunityRoadshow Have a great evening! Łukasz Byjoś | GDG Rzeszów | @lukaszbyjos