Introduction to Actions on google

umeda.apk #3

Transcript

1. Introduction to Actions on google 2017/6/16 umeda.apk #3
2. whoami Ryo Yamazaki / Application Engineer twitter: @ymnd github: @ymnder
3. whoami 3
5. Actions on Google Developer Challenge • ԠืకΊ੾Γ: 8/31 • ̍౳৆ʹ͸ I/O 2018ͷνέοτ͕ͭ͘
6. ΍͍͖ͬͯ·͠ΐ͏
7. topics ᶃ What is actions on google ᶄ How to develop app ᶅ Why design is important? ᶆ Appendix
8. topics ᶃ What is actions on google ᶄ How to develop app ᶅ Why design is important? ᶆ Appendix
9. What is actions on google Actions on Google lets you build apps for the Google Assistant. Your apps can help you engage users through Google Home, eligible Android phones, iPhones, and in the future, through every experience where the Google Assistant is available.
10. What is actions on google Actions on Google lets you build apps for the Google Assistant. Your apps can help you engage users through Google Home, eligible Android phones, iPhones, and in the future, through every experience where the Google Assistant is available.
11. Actions on google: console • ΞϓϦͷ؅ཧ • γϛϡϨʔλͰςετʗσόοά
12. DEMO
13. topics ᶃ What is actions on google ᶄ How to develop app ᶅ Why design is important? ᶆ Appendix
14. Actions on googleʹؔΘΔओཁͳΞΫλʔ
15. Actions on googleʹؔΘΔओཁͳΞΫλʔ ͜ͷ෦෼Λ࣮૷͢Δ {
16. Develop Assistant App
17. Develop Assistant App
18. Develop Assistant App
19. Get started
20. Get started • ίʔυΛॻ͔ͳͯ͘΋ಈ͔ͤΔ • API.aiΛϘλϯϙνϙν͢Ε͹࡞ΕΔ • dynamicͳϨεϙϯεΛߦ͏৔߹͸࣮૷͕ඞཁͰ͋Δ • ཪଆͷfulfillment͸node.jsͳͲͰهड़
21. Get started ᶃ actions on googleͷget startedΛ΍Δ • ͱΓ͋͑ͣಈ͔ͤΔʗ֓ཁΛ௫ΊΔ • ։ൃ؀ڥ͕ߏஙͰ͖Δ ᶄ code labsΛ΍Δ • ҰਓͰͰ͖ΔΑ͏ʹ࡞ΒΕ͍ͯΔ ᶅ Google I/OͷSessionΛݟΔ • Building Apps for the Google Assistant
22. Actions on googleʹؔΘΔओཁͳΞΫλʔ
23. Google Assistant • ձ࿩ܕAI/νϟοτϘοτͷػೳΛఏڙ • HomeɾAndroid(6.0+)ɾiOSͳͲͰ͔ͭ͑Δ • ೔ຊޠରԠ͠·ͨ͠!!!! (※ಡΈ্͛͸ࠓޙ)
24. Google Assistant • ݴ༿ͷΏΒ͗ͷ෦෼Λྑ͍ײ͡ʹೝࣝͯ͘͠ΕΔ • Ի੠Λೝࣝͯ͠ඞཁͳ෦෼ΛऔΓग़ͯ͘͠ΕΔ • ࣗ෼ͰԻ੠ೝࣝʹؔ͢Δ࣮૷Λ͠ͳͯ͘ྑ͍ • ͍ΖΜͳͱ͜ΖͰಈ͘
25. Assistant AppͷىಈτϦΨʔ
26. Assistant AppͷىಈτϦΨʔ • Actions on googleͰΞϓϦΛ࡞੒ͨ͠৔߹ • default App name: test app • “ok google, talk to my test app” • App InformationͰApp nameΛࢦఆͰ͖Δ
27. Actions on googleʹؔΘΔओཁͳΞΫλʔ
28. Api.ai • Ի੠Λղੳ͠ɺͦΕʹԠͨ͡ActionΛ࣮ߦ͢ΔAPI • intents • entities • fulfillment
29. Api.ai: Intents
30. Api.ai: Intents • ϢʔβʔͷൃݴΛ࣮ࡍͷActionʹ݁ͼ͚ͭΔ • جຊతͳϨεϙϯε͸ίʔυΛॻ͔ͣʹ࢖͑Δ • ϏδωεϩδοΫ͸fulfillmentͰ࣮૷͢Δ
31. Rich Response • جຊతͳฦ౴ Text Response • Ϧονͳฦ౴ • Card • List • Suggestion Chips • Carousel card • ฦ౴ઌͷ࿈ܞ • Facebook messenger, Slack, Telegram…
32. Rich Response: Card
33. Rich Response: List
34. Rich Response: Suggestion chips
35. Rich Response: Carousel card
36. Api.ai: Entities
37. Api.ai: Entities • நग़͍ͨ͠ޠ۟Λఆٛ͢Δ • Ϣʔβʔͷࣗવݴޠ͔ΒύϥϝʔλΛऔΓग़͢ • ༧૝͞ΕΔೖྗΛ͜͜Ͱఆٛ͢Δ • reference valueʹෳ਺ͷsynonymΛఆٛ͢Δ • ϓϦηοτ΋͍Ζ͍Ζ༻ҙ͞Ε͍ͯΔ • ஍໊ɾ୯Ґɾ࣌ؒɾਓ໊ͳͲ https://docs.api.ai/docs/concept-entities
38. Actions on googleʹؔΘΔओཁͳΞΫλʔ
39. Api.ai: Fulfilment
40. Api.ai: Fulfilment
41. Api.ai: Fulfilment
42. Api.ai: Fulfilment
43. Api.ai: Fulfilment / ಡΈ্͛ • speak • rootʹهड़͢Δ • <speak>Hello World</speak> • break • ϙʔζ͢Δ • <speak>Wait a while <break time="5s" />calm down.</speak>
44. Api.ai: Fulfilment / ಡΈ্͛ • say-as • ޠ۟ͷ৘ใΛهࡌ͢Δ • 1984 • one thousand nine hundred and eighty four • nineteen hundred, eighty four? • can • ͖ΌΜ • ͠ʔ͑ʔ͑͵
45. Api.ai: Fulfilment / ಡΈ্͛ • say-as • ޠ۟ͷ৘ใΛهࡌ͢Δ • 1984 • <say-as interpret-as="cardinal">1984</say-as> • <say-as interpret-as="date" format="yyyy">1984</say-as> • can • <say-as interpret-as="characters">can</say-as>
46. Api.ai: Fulfilment / ಡΈ্͛ • audio • ΦʔσΟΦϑΝΠϧΛ࢖͑Δ • جຊతʹ͸ޮՌԻͳͲΛ໐Β͢ͷʹ࢖͏ʁ • mp3, ogg, wav(deprecated) • ࠷େ̎෼Ҏ಺ • 5MB • httpsͰఏڙ͢Δ͜ͱ
47. Api.ai: Fulfilment / ಡΈ্͛ • sub • ུশͷಡΈํͳͲΛࢦఆ • <sub alias="World Wide Web Consortium">W3C</ sub>
48. Api.ai: Fulfilment / ಡΈ্͛
49. Api.ai: Fulfilment / ಡΈ্͛
50. Api.ai: Fulfilment / ಡΈ্͛
51. Api.ai SDK support language
52. Api.ai: Local࣮ߦ • npm install -g @google-cloud/functions-emulator • npm install • functions start • functions deploy appName --trigger-http • ngrok http 8010
53. Local࣮ߦ
54. Local࣮ߦ
55. Local࣮ߦ -> fulfillment
56. TransactionAPI
57. TransactionAPI
58. topics ᶃ What is actions on google ᶄ How to develop app ᶅ Why design is important? ᶆ Appendix
59. GUI and VUI
60. Design • Voice User Interface (VUI) • ୯७ʹจࣈΛಡΈ্͛Ε͹ྑ͍༁Ͱ͸ͳ͍ • ਓͱձ࿩͢Δͱ͖ͱಉ͡ • ձ࿩͸୺తʹɺ৑௕తͰͳ͘ • creating persona ref: https://developers.google.com/actions/design/
61. Defining Multimodal Interactions: One Size Does Not Fit All
62. Design • Assistant have 3 interaction • audio (for Google Home) • visual (for mobile or TV) • haptic (for mobile) • Input / Output interaction is sometimes not same • Ϣʔβʔͷஔ͔Ε͍ͯΔঢ়گͰมΘΔ • out: eye, ear X in: camera, mic, touch
63. Design (for Home) • Don’t read, listen • ಡΉจষ≠ฉ͘จষ • Avoid information overload • ৘ใΛΘͬͱग़͢ͱϢʔβʔ΋ࠔΔ • ࿩Λ୹͘·ͱΊɺࡉ͔͍ΠϯλϥΫγϣϯΛ • Answer the question • Ͱ͖ͳ͔ͬͨ৔߹Ͱ΋౴͑ํ͕͋Δ
64. Design (for mobile) • if one mode goes away, the other should take over • optimize for the strongest mode, but allow both • Leverage the strengths of each mode & avoid redundancy across modalities
65. topics ᶃ What is actions on google ᶄ How to develop app ᶅ Why design is important? ᶆ Appendix
66. Sessions about Google Assistant • Building Apps for the Google Assistant • Defining Multimodal Interactions: One Size Does Not Fit All • Finding the Right Voice Interactions for your App • In Conversation, There Are No Errors • PullString: Storytelling in the Age of Conversational Interfaces • Applying Built-in Hacks of Conversation to Your Voice UI • Bringing the Google Assistant to Any Device • Transactions with the Google Assistant • Building rich cross-platform conversational UX with API.AI
67. Google Home is Updated
68. Google Home • Google Home is powered by the Google Assistant. • ࠓ೥ͷՆҎ߱ʹ೔ຊ্཮
69. Google Home Updated • Proactive Assistance • Visual Response • hands free calling • Bluetooth Audio Support
70. Google Home Updated: Proactive Assistance • Google Homeʹݺͼ͔͚ͳͯ͘΋৘ใΛڭ͑ͯ͘ΕΔ • ಓ࿏ͷࠞࡶ৘ใΛڭ͑ͯ͘ΕΔσϞ͕͋ͬͨ • χϡʔε΍Ϛʔέοτͷಈ͖Λ఻͑Δ͜ͱ͕Ͱ͖Δʁ • push notificationͷΑ͏ͳ࢓૊Έ͕͋Δ͔ʁ
71. Google Home Updated: Visual Response
72. Google Home Updated: hands free calling • ϋϯζϑϦʔͰͰ͖Δ͜ͱ͕͍Ζ͍Ζ૿͑ͨ • ίϯςΩετ / ൃ࿩ऀͷ൑ผ • Call my mother -> ൃ࿩ऀͷ฼਌Λೝࣝ͢Δ • ΞϝϦΧͱΧφμ͸HomeͰແྉͰి࿩͕͔͚ΒΕΔ
73. Google Home Updated: Bluetooth Audio Support • HomeΛεϐʔΧʔ୅ΘΓʹ࢖͏͜ͱ͕Ͱ͖Δ • ԻָΛฉ͍ͨΓɺcastͨ͠Γ • ΋ͬͱੜ׆ʹ਎ۙͳଘࡏͱͯ͠࢖ΘΕΔΑ͏ʹ
74. ͓ΘΓ
75. References • https://tech.recruit-mp.co.jp/event/post-12166/ • http://tech.mercari.com/entry/2017/05/30/100000 • https://cyberagent.connpass.com/event/55405/ • https://tech.recruit-mp.co.jp/mobile/post-12285/ • https://speakerdeck.com/srym/o-2017-short-report • https://docs.google.com/presentation/d/ 1ea5Touf3c5hgJXmoYL1WmkplF7m6XKYL7YyKPD7OWvY/pub? start=false&loop=false&delayms=60000&slide=id.g22d0f56c70_0_2 7 • https://github.com/vvakame/pubg-assistant/blob/master/ README.md