Introduction to Actions on google

934a9e49edc3174d09ab2e09daed5062?s=47 ymnder
June 18, 2017

Introduction to Actions on google

umeda.apk #3

934a9e49edc3174d09ab2e09daed5062?s=128

ymnder

June 18, 2017
Tweet

Transcript

  1. Introduction to Actions on google 2017/6/16 umeda.apk #3

  2. whoami Ryo Yamazaki / Application Engineer twitter: @ymnd github: @ymnder

  3. whoami 3

  4. 4

  5. Actions on Google Developer Challenge • ԠืకΊ੾Γ: 8/31 • ̍౳৆ʹ͸

    I/O 2018ͷνέοτ͕ͭ͘
  6. ΍͍͖ͬͯ·͠ΐ͏

  7. topics ᶃ What is actions on google ᶄ How to

    develop app ᶅ Why design is important? ᶆ Appendix
  8. topics ᶃ What is actions on google ᶄ How to

    develop app ᶅ Why design is important? ᶆ Appendix
  9. What is actions on google Actions on Google lets you

    build apps for the Google Assistant. Your apps can help you engage users through Google Home, eligible Android phones, iPhones, and in the future, through every experience where the Google Assistant is available.
  10. What is actions on google Actions on Google lets you

    build apps for the Google Assistant. Your apps can help you engage users through Google Home, eligible Android phones, iPhones, and in the future, through every experience where the Google Assistant is available.
  11. Actions on google: console • ΞϓϦͷ؅ཧ • γϛϡϨʔλͰςετʗσόοά

  12. DEMO

  13. topics ᶃ What is actions on google ᶄ How to

    develop app ᶅ Why design is important? ᶆ Appendix
  14. Actions on googleʹؔΘΔओཁͳΞΫλʔ

  15. Actions on googleʹؔΘΔओཁͳΞΫλʔ ͜ͷ෦෼Λ࣮૷͢Δ {

  16. Develop Assistant App

  17. Develop Assistant App

  18. Develop Assistant App

  19. Get started

  20. Get started • ίʔυΛॻ͔ͳͯ͘΋ಈ͔ͤΔ • API.aiΛϘλϯϙνϙν͢Ε͹࡞ΕΔ • dynamicͳϨεϙϯεΛߦ͏৔߹͸࣮૷͕ඞཁͰ͋Δ • ཪଆͷfulfillment͸node.jsͳͲͰهड़

  21. Get started ᶃ actions on googleͷget startedΛ΍Δ • ͱΓ͋͑ͣಈ͔ͤΔʗ֓ཁΛ௫ΊΔ •

    ։ൃ؀ڥ͕ߏஙͰ͖Δ ᶄ code labsΛ΍Δ • ҰਓͰͰ͖ΔΑ͏ʹ࡞ΒΕ͍ͯΔ ᶅ Google I/OͷSessionΛݟΔ • Building Apps for the Google Assistant
  22. Actions on googleʹؔΘΔओཁͳΞΫλʔ

  23. Google Assistant • ձ࿩ܕAI/νϟοτϘοτͷػೳΛఏڙ • HomeɾAndroid(6.0+)ɾiOSͳͲͰ͔ͭ͑Δ • ೔ຊޠରԠ͠·ͨ͠!!!! (※ಡΈ্͛͸ࠓޙ)

  24. Google Assistant • ݴ༿ͷΏΒ͗ͷ෦෼Λྑ͍ײ͡ʹೝࣝͯ͘͠ΕΔ • Ի੠Λೝࣝͯ͠ඞཁͳ෦෼ΛऔΓग़ͯ͘͠ΕΔ • ࣗ෼ͰԻ੠ೝࣝʹؔ͢Δ࣮૷Λ͠ͳͯ͘ྑ͍ • ͍ΖΜͳͱ͜ΖͰಈ͘

  25. Assistant AppͷىಈτϦΨʔ

  26. Assistant AppͷىಈτϦΨʔ • Actions on googleͰΞϓϦΛ࡞੒ͨ͠৔߹ • default App name:

    test app • “ok google, talk to my test app” • App InformationͰApp nameΛࢦఆͰ͖Δ
  27. Actions on googleʹؔΘΔओཁͳΞΫλʔ

  28. Api.ai • Ի੠Λղੳ͠ɺͦΕʹԠͨ͡ActionΛ࣮ߦ͢ΔAPI • intents • entities • fulfillment

  29. Api.ai: Intents

  30. Api.ai: Intents • ϢʔβʔͷൃݴΛ࣮ࡍͷActionʹ݁ͼ͚ͭΔ • جຊతͳϨεϙϯε͸ίʔυΛॻ͔ͣʹ࢖͑Δ • ϏδωεϩδοΫ͸fulfillmentͰ࣮૷͢Δ

  31. Rich Response • جຊతͳฦ౴ Text Response • Ϧονͳฦ౴ • Card

    • List • Suggestion Chips • Carousel card • ฦ౴ઌͷ࿈ܞ • Facebook messenger, Slack, Telegram…
  32. Rich Response: Card

  33. Rich Response: List

  34. Rich Response: Suggestion chips

  35. Rich Response: Carousel card

  36. Api.ai: Entities

  37. Api.ai: Entities • நग़͍ͨ͠ޠ۟Λఆٛ͢Δ • Ϣʔβʔͷࣗવݴޠ͔ΒύϥϝʔλΛऔΓग़͢ • ༧૝͞ΕΔೖྗΛ͜͜Ͱఆٛ͢Δ • reference

    valueʹෳ਺ͷsynonymΛఆٛ͢Δ • ϓϦηοτ΋͍Ζ͍Ζ༻ҙ͞Ε͍ͯΔ • ஍໊ɾ୯Ґɾ࣌ؒɾਓ໊ͳͲ https://docs.api.ai/docs/concept-entities
  38. Actions on googleʹؔΘΔओཁͳΞΫλʔ

  39. Api.ai: Fulfilment

  40. Api.ai: Fulfilment

  41. Api.ai: Fulfilment

  42. Api.ai: Fulfilment

  43. Api.ai: Fulfilment / ಡΈ্͛ • speak • rootʹهड़͢Δ • <speak>Hello

    World</speak> • break • ϙʔζ͢Δ • <speak>Wait a while <break time="5s" />calm down.</speak>
  44. Api.ai: Fulfilment / ಡΈ্͛ • say-as • ޠ۟ͷ৘ใΛهࡌ͢Δ • 1984

    • one thousand nine hundred and eighty four • nineteen hundred, eighty four? • can • ͖ΌΜ • ͠ʔ͑ʔ͑͵
  45. Api.ai: Fulfilment / ಡΈ্͛ • say-as • ޠ۟ͷ৘ใΛهࡌ͢Δ • 1984

    • <say-as interpret-as="cardinal">1984</say-as> • <say-as interpret-as="date" format="yyyy">1984</say-as> • can • <say-as interpret-as="characters">can</say-as>
  46. Api.ai: Fulfilment / ಡΈ্͛ • audio • ΦʔσΟΦϑΝΠϧΛ࢖͑Δ • جຊతʹ͸ޮՌԻͳͲΛ໐Β͢ͷʹ࢖͏ʁ

    • mp3, ogg, wav(deprecated) • ࠷େ̎෼Ҏ಺ • 5MB • httpsͰఏڙ͢Δ͜ͱ
  47. Api.ai: Fulfilment / ಡΈ্͛ • sub • ུশͷಡΈํͳͲΛࢦఆ • <sub

    alias="World Wide Web Consortium">W3C</ sub>
  48. Api.ai: Fulfilment / ಡΈ্͛

  49. Api.ai: Fulfilment / ಡΈ্͛

  50. Api.ai: Fulfilment / ಡΈ্͛

  51. Api.ai SDK support language

  52. Api.ai: Local࣮ߦ • npm install -g @google-cloud/functions-emulator • npm install

    • functions start • functions deploy appName --trigger-http • ngrok http 8010
  53. Local࣮ߦ

  54. Local࣮ߦ

  55. Local࣮ߦ -> fulfillment

  56. TransactionAPI

  57. TransactionAPI

  58. topics ᶃ What is actions on google ᶄ How to

    develop app ᶅ Why design is important? ᶆ Appendix
  59. GUI and VUI

  60. Design • Voice User Interface (VUI) • ୯७ʹจࣈΛಡΈ্͛Ε͹ྑ͍༁Ͱ͸ͳ͍ • ਓͱձ࿩͢Δͱ͖ͱಉ͡

    • ձ࿩͸୺తʹɺ৑௕తͰͳ͘ • creating persona ref: https://developers.google.com/actions/design/
  61. Defining Multimodal Interactions: One Size Does Not Fit All

  62. Design • Assistant have 3 interaction • audio (for Google

    Home) • visual (for mobile or TV) • haptic (for mobile) • Input / Output interaction is sometimes not same • Ϣʔβʔͷஔ͔Ε͍ͯΔঢ়گͰมΘΔ • out: eye, ear X in: camera, mic, touch
  63. Design (for Home) • Don’t read, listen • ಡΉจষ≠ฉ͘จষ •

    Avoid information overload • ৘ใΛΘͬͱग़͢ͱϢʔβʔ΋ࠔΔ • ࿩Λ୹͘·ͱΊɺࡉ͔͍ΠϯλϥΫγϣϯΛ • Answer the question • Ͱ͖ͳ͔ͬͨ৔߹Ͱ΋౴͑ํ͕͋Δ
  64. Design (for mobile) • if one mode goes away, the

    other should take over • optimize for the strongest mode, but allow both • Leverage the strengths of each mode & avoid redundancy across modalities
  65. topics ᶃ What is actions on google ᶄ How to

    develop app ᶅ Why design is important? ᶆ Appendix
  66. Sessions about Google Assistant • Building Apps for the Google

    Assistant • Defining Multimodal Interactions: One Size Does Not Fit All • Finding the Right Voice Interactions for your App • In Conversation, There Are No Errors • PullString: Storytelling in the Age of Conversational Interfaces • Applying Built-in Hacks of Conversation to Your Voice UI • Bringing the Google Assistant to Any Device • Transactions with the Google Assistant • Building rich cross-platform conversational UX with API.AI
  67. Google Home is Updated

  68. Google Home • Google Home is powered by the Google

    Assistant. • ࠓ೥ͷՆҎ߱ʹ೔ຊ্཮
  69. Google Home Updated • Proactive Assistance • Visual Response •

    hands free calling • Bluetooth Audio Support
  70. Google Home Updated: Proactive Assistance • Google Homeʹݺͼ͔͚ͳͯ͘΋৘ใΛڭ͑ͯ͘ΕΔ • ಓ࿏ͷࠞࡶ৘ใΛڭ͑ͯ͘ΕΔσϞ͕͋ͬͨ

    • χϡʔε΍Ϛʔέοτͷಈ͖Λ఻͑Δ͜ͱ͕Ͱ͖Δʁ • push notificationͷΑ͏ͳ࢓૊Έ͕͋Δ͔ʁ
  71. Google Home Updated: Visual Response

  72. Google Home Updated: hands free calling • ϋϯζϑϦʔͰͰ͖Δ͜ͱ͕͍Ζ͍Ζ૿͑ͨ • ίϯςΩετ

    / ൃ࿩ऀͷ൑ผ • Call my mother -> ൃ࿩ऀͷ฼਌Λೝࣝ͢Δ • ΞϝϦΧͱΧφμ͸HomeͰແྉͰి࿩͕͔͚ΒΕΔ
  73. Google Home Updated: Bluetooth Audio Support • HomeΛεϐʔΧʔ୅ΘΓʹ࢖͏͜ͱ͕Ͱ͖Δ • ԻָΛฉ͍ͨΓɺcastͨ͠Γ

    • ΋ͬͱੜ׆ʹ਎ۙͳଘࡏͱͯ͠࢖ΘΕΔΑ͏ʹ
  74. ͓ΘΓ

  75. References • https://tech.recruit-mp.co.jp/event/post-12166/ • http://tech.mercari.com/entry/2017/05/30/100000 • https://cyberagent.connpass.com/event/55405/ • https://tech.recruit-mp.co.jp/mobile/post-12285/ •

    https://speakerdeck.com/srym/o-2017-short-report • https://docs.google.com/presentation/d/ 1ea5Touf3c5hgJXmoYL1WmkplF7m6XKYL7YyKPD7OWvY/pub? start=false&loop=false&delayms=60000&slide=id.g22d0f56c70_0_2 7 • https://github.com/vvakame/pubg-assistant/blob/master/ README.md