Introduction to Actions on google

umeda.apk #3

Transcript

1. Introduction to Actions on google
   2017/6/16 umeda.apk #3
   

   View Slide

2. whoami
   Ryo Yamazaki / Application Engineer
   twitter: @ymnd
   github: @ymnder
   

   View Slide

3. whoami
   3
   

   View Slide

4. View Slide

5. Actions on Google Developer Challenge
   • ԠืకΊ੾Γ: 8/31
   • ̍౳৆ʹ͸ I/O 2018ͷνέοτ͕ͭ͘
   

   View Slide

6. ΍͍͖ͬͯ·͠ΐ͏
   

   View Slide

7. topics
   ᶃ What is actions on google
   ᶄ How to develop app
   ᶅ Why design is important?
   ᶆ Appendix
   

   View Slide

8. topics
   ᶃ What is actions on google
   ᶄ How to develop app
   ᶅ Why design is important?
   ᶆ Appendix
   

   View Slide

9. What is actions on google
   Actions on Google lets you build apps for the Google
   Assistant. Your apps can help you engage users through
   Google Home, eligible Android phones, iPhones, and in
   the future, through every experience where the Google
   Assistant is available.
   

   View Slide

10. What is actions on google
    Actions on Google lets you build apps for the Google
    Assistant. Your apps can help you engage users through
    Google Home, eligible Android phones, iPhones, and in
    the future, through every experience where the Google
    Assistant is available.
    

    View Slide

11. Actions on google: console
    • ΞϓϦͷ؅ཧ
    • γϛϡϨʔλͰςετʗσόοά
    

    View Slide

12. DEMO
    

    View Slide

13. topics
    ᶃ What is actions on google
    ᶄ How to develop app
    ᶅ Why design is important?
    ᶆ Appendix
    

    View Slide

14. Actions on googleʹؔΘΔओཁͳΞΫλʔ
    

    View Slide

15. Actions on googleʹؔΘΔओཁͳΞΫλʔ
    ͜ͷ෦෼Λ࣮૷͢Δ
    {
    

    View Slide

16. Develop Assistant App
    

    View Slide

17. Develop Assistant App
    

    View Slide

18. Develop Assistant App
    

    View Slide

19. Get started
    

    View Slide

20. Get started
    • ίʔυΛॻ͔ͳͯ͘΋ಈ͔ͤΔ
    • API.aiΛϘλϯϙνϙν͢Ε͹࡞ΕΔ
    • dynamicͳϨεϙϯεΛߦ͏৔߹͸࣮૷͕ඞཁͰ͋Δ
    • ཪଆͷfulfillment͸node.jsͳͲͰهड़
    

    View Slide

21. Get started
    ᶃ actions on googleͷget startedΛ΍Δ
    • ͱΓ͋͑ͣಈ͔ͤΔʗ֓ཁΛ௫ΊΔ
    • ։ൃ؀ڥ͕ߏஙͰ͖Δ
    ᶄ code labsΛ΍Δ
    • ҰਓͰͰ͖ΔΑ͏ʹ࡞ΒΕ͍ͯΔ
    ᶅ Google I/OͷSessionΛݟΔ
    • Building Apps for the Google Assistant
    

    View Slide

22. Actions on googleʹؔΘΔओཁͳΞΫλʔ
    

    View Slide

23. Google Assistant
    • ձ࿩ܕAI/νϟοτϘοτͷػೳΛఏڙ
    • HomeɾAndroid(6.0+)ɾiOSͳͲͰ͔ͭ͑Δ
    • ೔ຊޠରԠ͠·ͨ͠!!!! (※ಡΈ্͛͸ࠓޙ)
    

    View Slide

24. Google Assistant
    • ݴ༿ͷΏΒ͗ͷ෦෼Λྑ͍ײ͡ʹೝࣝͯ͘͠ΕΔ
    • Ի੠Λೝࣝͯ͠ඞཁͳ෦෼ΛऔΓग़ͯ͘͠ΕΔ
    • ࣗ෼ͰԻ੠ೝࣝʹؔ͢Δ࣮૷Λ͠ͳͯ͘ྑ͍
    • ͍ΖΜͳͱ͜ΖͰಈ͘
    

    View Slide

25. Assistant AppͷىಈτϦΨʔ
    

    View Slide

26. Assistant AppͷىಈτϦΨʔ
    • Actions on googleͰΞϓϦΛ࡞੒ͨ͠৔߹
    • default App name: test app
    • “ok google, talk to my test app”
    • App InformationͰApp nameΛࢦఆͰ͖Δ
    

    View Slide

27. Actions on googleʹؔΘΔओཁͳΞΫλʔ
    

    View Slide

28. Api.ai
    • Ի੠Λղੳ͠ɺͦΕʹԠͨ͡ActionΛ࣮ߦ͢ΔAPI
    • intents
    • entities
    • fulfillment
    

    View Slide

29. Api.ai: Intents
    

    View Slide

30. Api.ai: Intents
    • ϢʔβʔͷൃݴΛ࣮ࡍͷActionʹ݁ͼ͚ͭΔ
    • جຊతͳϨεϙϯε͸ίʔυΛॻ͔ͣʹ࢖͑Δ
    • ϏδωεϩδοΫ͸fulfillmentͰ࣮૷͢Δ
    

    View Slide

31. Rich Response
    • جຊతͳฦ౴ Text Response
    • Ϧονͳฦ౴
    • Card
    • List
    • Suggestion Chips
    • Carousel card
    • ฦ౴ઌͷ࿈ܞ
    • Facebook messenger, Slack, Telegram…
    

    View Slide

32. Rich Response: Card
    

    View Slide

33. Rich Response: List
    

    View Slide

34. Rich Response: Suggestion chips
    

    View Slide

35. Rich Response: Carousel card
    

    View Slide

36. Api.ai: Entities
    

    View Slide

37. Api.ai: Entities
    • நग़͍ͨ͠ޠ۟Λఆٛ͢Δ
    • Ϣʔβʔͷࣗવݴޠ͔ΒύϥϝʔλΛऔΓग़͢
    • ༧૝͞ΕΔೖྗΛ͜͜Ͱఆٛ͢Δ
    • reference valueʹෳ਺ͷsynonymΛఆٛ͢Δ
    • ϓϦηοτ΋͍Ζ͍Ζ༻ҙ͞Ε͍ͯΔ
    • ஍໊ɾ୯Ґɾ࣌ؒɾਓ໊ͳͲ
    https://docs.api.ai/docs/concept-entities
    

    View Slide

38. Actions on googleʹؔΘΔओཁͳΞΫλʔ
    

    View Slide

39. Api.ai: Fulfilment
    

    View Slide

40. Api.ai: Fulfilment
    

    View Slide

41. Api.ai: Fulfilment
    

    View Slide

42. Api.ai: Fulfilment
    

    View Slide

43. Api.ai: Fulfilment / ಡΈ্͛
    • speak
    • rootʹهड़͢Δ
    • Hello World
    • break
    • ϙʔζ͢Δ
    • Wait a while calm
    down.
    

    View Slide

44. Api.ai: Fulfilment / ಡΈ্͛
    • say-as
    • ޠ۟ͷ৘ใΛهࡌ͢Δ
    • 1984
    • one thousand nine hundred and eighty four
    • nineteen hundred, eighty four?
    • can
    • ͖ΌΜ
    • ͠ʔ͑ʔ͑͵
    

    View Slide

45. Api.ai: Fulfilment / ಡΈ্͛
    • say-as
    • ޠ۟ͷ৘ใΛهࡌ͢Δ
    • 1984
    • 1984
    • format="yyyy">1984
    • can
    • can
    

    View Slide

46. Api.ai: Fulfilment / ಡΈ্͛
    • audio
    • ΦʔσΟΦϑΝΠϧΛ࢖͑Δ
    • جຊతʹ͸ޮՌԻͳͲΛ໐Β͢ͷʹ࢖͏ʁ
    • mp3, ogg, wav(deprecated)
    • ࠷େ̎෼Ҏ಺
    • 5MB
    • httpsͰఏڙ͢Δ͜ͱ
    

    View Slide

47. Api.ai: Fulfilment / ಡΈ্͛
    • sub
    • ུশͷಡΈํͳͲΛࢦఆ
    • W3C
    sub>
    

    View Slide

48. Api.ai: Fulfilment / ಡΈ্͛
    

    View Slide

49. Api.ai: Fulfilment / ಡΈ্͛
    

    View Slide

50. Api.ai: Fulfilment / ಡΈ্͛
    

    View Slide

51. Api.ai SDK support language
    

    View Slide

52. Api.ai: Local࣮ߦ
    • npm install -g @google-cloud/functions-emulator
    • npm install
    • functions start
    • functions deploy appName --trigger-http
    • ngrok http 8010
    

    View Slide

53. Local࣮ߦ
    

    View Slide

54. Local࣮ߦ
    

    View Slide

55. Local࣮ߦ -> fulfillment
    

    View Slide

56. TransactionAPI
    

    View Slide

57. TransactionAPI
    

    View Slide

58. topics
    ᶃ What is actions on google
    ᶄ How to develop app
    ᶅ Why design is important?
    ᶆ Appendix
    

    View Slide

59. GUI and VUI
    

    View Slide

60. Design
    • Voice User Interface (VUI)
    • ୯७ʹจࣈΛಡΈ্͛Ε͹ྑ͍༁Ͱ͸ͳ͍
    • ਓͱձ࿩͢Δͱ͖ͱಉ͡
    • ձ࿩͸୺తʹɺ৑௕తͰͳ͘
    • creating persona
    ref: https://developers.google.com/actions/design/
    

    View Slide

61. Defining Multimodal Interactions:
    One Size Does Not Fit All
    

    View Slide

62. Design
    • Assistant have 3 interaction
    • audio (for Google Home)
    • visual (for mobile or TV)
    • haptic (for mobile)
    • Input / Output interaction is sometimes not same
    • Ϣʔβʔͷஔ͔Ε͍ͯΔঢ়گͰมΘΔ
    • out: eye, ear X in: camera, mic, touch
    

    View Slide

63. Design (for Home)
    • Don’t read, listen
    • ಡΉจষ≠ฉ͘จষ
    • Avoid information overload
    • ৘ใΛΘͬͱग़͢ͱϢʔβʔ΋ࠔΔ
    • ࿩Λ୹͘·ͱΊɺࡉ͔͍ΠϯλϥΫγϣϯΛ
    • Answer the question
    • Ͱ͖ͳ͔ͬͨ৔߹Ͱ΋౴͑ํ͕͋Δ
    

    View Slide

64. Design (for mobile)
    • if one mode goes away, the other should take over
    • optimize for the strongest mode, but allow both
    • Leverage the strengths of each mode & avoid
    redundancy across modalities
    

    View Slide

65. topics
    ᶃ What is actions on google
    ᶄ How to develop app
    ᶅ Why design is important?
    ᶆ Appendix
    

    View Slide

66. Sessions about Google Assistant
    • Building Apps for the Google Assistant
    • Defining Multimodal Interactions: One Size Does Not Fit All
    • Finding the Right Voice Interactions for your App
    • In Conversation, There Are No Errors
    • PullString: Storytelling in the Age of Conversational Interfaces
    • Applying Built-in Hacks of Conversation to Your Voice UI
    • Bringing the Google Assistant to Any Device
    • Transactions with the Google Assistant
    • Building rich cross-platform conversational UX with API.AI
    

    View Slide

67. Google Home is Updated
    

    View Slide

68. Google Home
    • Google Home is powered by the Google Assistant.
    • ࠓ೥ͷՆҎ߱ʹ೔ຊ্཮
    

    View Slide

69. Google Home Updated
    • Proactive Assistance
    • Visual Response
    • hands free calling
    • Bluetooth Audio Support
    

    View Slide

70. Google Home Updated: Proactive Assistance
    • Google Homeʹݺͼ͔͚ͳͯ͘΋৘ใΛڭ͑ͯ͘ΕΔ
    • ಓ࿏ͷࠞࡶ৘ใΛڭ͑ͯ͘ΕΔσϞ͕͋ͬͨ
    • χϡʔε΍Ϛʔέοτͷಈ͖Λ఻͑Δ͜ͱ͕Ͱ͖Δʁ
    • push notificationͷΑ͏ͳ࢓૊Έ͕͋Δ͔ʁ
    

    View Slide

71. Google Home Updated: Visual Response
    

    View Slide

72. Google Home Updated: hands free calling
    • ϋϯζϑϦʔͰͰ͖Δ͜ͱ͕͍Ζ͍Ζ૿͑ͨ
    • ίϯςΩετ / ൃ࿩ऀͷ൑ผ
    • Call my mother -> ൃ࿩ऀͷ฼਌Λೝࣝ͢Δ
    • ΞϝϦΧͱΧφμ͸HomeͰແྉͰి࿩͕͔͚ΒΕΔ
    

    View Slide

73. Google Home Updated: Bluetooth Audio Support
    • HomeΛεϐʔΧʔ୅ΘΓʹ࢖͏͜ͱ͕Ͱ͖Δ
    • ԻָΛฉ͍ͨΓɺcastͨ͠Γ
    • ΋ͬͱੜ׆ʹ਎ۙͳଘࡏͱͯ͠࢖ΘΕΔΑ͏ʹ
    

    View Slide

74. ͓ΘΓ
    

    View Slide

75. References
    • https://tech.recruit-mp.co.jp/event/post-12166/
    • http://tech.mercari.com/entry/2017/05/30/100000
    • https://cyberagent.connpass.com/event/55405/
    • https://tech.recruit-mp.co.jp/mobile/post-12285/
    • https://speakerdeck.com/srym/o-2017-short-report
    • https://docs.google.com/presentation/d/
    1ea5Touf3c5hgJXmoYL1WmkplF7m6XKYL7YyKPD7OWvY/pub?
    start=false&loop=false&delayms=60000&slide=id.g22d0f56c70_0_2
    7
    • https://github.com/vvakame/pubg-assistant/blob/master/
    README.md
    

    View Slide