Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to Actions on google

ymnder
June 18, 2017

Introduction to Actions on google

umeda.apk #3

ymnder

June 18, 2017
Tweet

More Decks by ymnder

Other Decks in Programming

Transcript

  1. Introduction to Actions on google
    2017/6/16 umeda.apk #3

    View Slide

  2. whoami
    Ryo Yamazaki / Application Engineer
    twitter: @ymnd
    github: @ymnder

    View Slide

  3. whoami
    3

    View Slide

  4. 4

    View Slide

  5. Actions on Google Developer Challenge
    • ԠืకΊ੾Γ: 8/31
    • ̍౳৆ʹ͸ I/O 2018ͷνέοτ͕ͭ͘

    View Slide

  6. ΍͍͖ͬͯ·͠ΐ͏

    View Slide

  7. topics
    ᶃ What is actions on google
    ᶄ How to develop app
    ᶅ Why design is important?
    ᶆ Appendix

    View Slide

  8. topics
    ᶃ What is actions on google
    ᶄ How to develop app
    ᶅ Why design is important?
    ᶆ Appendix

    View Slide

  9. What is actions on google
    Actions on Google lets you build apps for the Google
    Assistant. Your apps can help you engage users through
    Google Home, eligible Android phones, iPhones, and in
    the future, through every experience where the Google
    Assistant is available.

    View Slide

  10. What is actions on google
    Actions on Google lets you build apps for the Google
    Assistant. Your apps can help you engage users through
    Google Home, eligible Android phones, iPhones, and in
    the future, through every experience where the Google
    Assistant is available.

    View Slide

  11. Actions on google: console
    • ΞϓϦͷ؅ཧ
    • γϛϡϨʔλͰςετʗσόοά

    View Slide

  12. DEMO

    View Slide

  13. topics
    ᶃ What is actions on google
    ᶄ How to develop app
    ᶅ Why design is important?
    ᶆ Appendix

    View Slide

  14. Actions on googleʹؔΘΔओཁͳΞΫλʔ

    View Slide

  15. Actions on googleʹؔΘΔओཁͳΞΫλʔ
    ͜ͷ෦෼Λ࣮૷͢Δ
    {

    View Slide

  16. Develop Assistant App

    View Slide

  17. Develop Assistant App

    View Slide

  18. Develop Assistant App

    View Slide

  19. Get started

    View Slide

  20. Get started
    • ίʔυΛॻ͔ͳͯ͘΋ಈ͔ͤΔ
    • API.aiΛϘλϯϙνϙν͢Ε͹࡞ΕΔ
    • dynamicͳϨεϙϯεΛߦ͏৔߹͸࣮૷͕ඞཁͰ͋Δ
    • ཪଆͷfulfillment͸node.jsͳͲͰهड़

    View Slide

  21. Get started
    ᶃ actions on googleͷget startedΛ΍Δ
    • ͱΓ͋͑ͣಈ͔ͤΔʗ֓ཁΛ௫ΊΔ
    • ։ൃ؀ڥ͕ߏஙͰ͖Δ
    ᶄ code labsΛ΍Δ
    • ҰਓͰͰ͖ΔΑ͏ʹ࡞ΒΕ͍ͯΔ
    ᶅ Google I/OͷSessionΛݟΔ
    • Building Apps for the Google Assistant

    View Slide

  22. Actions on googleʹؔΘΔओཁͳΞΫλʔ

    View Slide

  23. Google Assistant
    • ձ࿩ܕAI/νϟοτϘοτͷػೳΛఏڙ
    • HomeɾAndroid(6.0+)ɾiOSͳͲͰ͔ͭ͑Δ
    • ೔ຊޠରԠ͠·ͨ͠!!!! (※ಡΈ্͛͸ࠓޙ)

    View Slide

  24. Google Assistant
    • ݴ༿ͷΏΒ͗ͷ෦෼Λྑ͍ײ͡ʹೝࣝͯ͘͠ΕΔ
    • Ի੠Λೝࣝͯ͠ඞཁͳ෦෼ΛऔΓग़ͯ͘͠ΕΔ
    • ࣗ෼ͰԻ੠ೝࣝʹؔ͢Δ࣮૷Λ͠ͳͯ͘ྑ͍
    • ͍ΖΜͳͱ͜ΖͰಈ͘

    View Slide

  25. Assistant AppͷىಈτϦΨʔ

    View Slide

  26. Assistant AppͷىಈτϦΨʔ
    • Actions on googleͰΞϓϦΛ࡞੒ͨ͠৔߹
    • default App name: test app
    • “ok google, talk to my test app”
    • App InformationͰApp nameΛࢦఆͰ͖Δ

    View Slide

  27. Actions on googleʹؔΘΔओཁͳΞΫλʔ

    View Slide

  28. Api.ai
    • Ի੠Λղੳ͠ɺͦΕʹԠͨ͡ActionΛ࣮ߦ͢ΔAPI
    • intents
    • entities
    • fulfillment

    View Slide

  29. Api.ai: Intents

    View Slide

  30. Api.ai: Intents
    • ϢʔβʔͷൃݴΛ࣮ࡍͷActionʹ݁ͼ͚ͭΔ
    • جຊతͳϨεϙϯε͸ίʔυΛॻ͔ͣʹ࢖͑Δ
    • ϏδωεϩδοΫ͸fulfillmentͰ࣮૷͢Δ

    View Slide

  31. Rich Response
    • جຊతͳฦ౴ Text Response
    • Ϧονͳฦ౴
    • Card
    • List
    • Suggestion Chips
    • Carousel card
    • ฦ౴ઌͷ࿈ܞ
    • Facebook messenger, Slack, Telegram…

    View Slide

  32. Rich Response: Card

    View Slide

  33. Rich Response: List

    View Slide

  34. Rich Response: Suggestion chips

    View Slide

  35. Rich Response: Carousel card

    View Slide

  36. Api.ai: Entities

    View Slide

  37. Api.ai: Entities
    • நग़͍ͨ͠ޠ۟Λఆٛ͢Δ
    • Ϣʔβʔͷࣗવݴޠ͔ΒύϥϝʔλΛऔΓग़͢
    • ༧૝͞ΕΔೖྗΛ͜͜Ͱఆٛ͢Δ
    • reference valueʹෳ਺ͷsynonymΛఆٛ͢Δ
    • ϓϦηοτ΋͍Ζ͍Ζ༻ҙ͞Ε͍ͯΔ
    • ஍໊ɾ୯Ґɾ࣌ؒɾਓ໊ͳͲ
    https://docs.api.ai/docs/concept-entities

    View Slide

  38. Actions on googleʹؔΘΔओཁͳΞΫλʔ

    View Slide

  39. Api.ai: Fulfilment

    View Slide

  40. Api.ai: Fulfilment

    View Slide

  41. Api.ai: Fulfilment

    View Slide

  42. Api.ai: Fulfilment

    View Slide

  43. Api.ai: Fulfilment / ಡΈ্͛
    • speak
    • rootʹهड़͢Δ
    • Hello World
    • break
    • ϙʔζ͢Δ
    • Wait a while calm
    down.

    View Slide

  44. Api.ai: Fulfilment / ಡΈ্͛
    • say-as
    • ޠ۟ͷ৘ใΛهࡌ͢Δ
    • 1984
    • one thousand nine hundred and eighty four
    • nineteen hundred, eighty four?
    • can
    • ͖ΌΜ
    • ͠ʔ͑ʔ͑͵

    View Slide

  45. Api.ai: Fulfilment / ಡΈ্͛
    • say-as
    • ޠ۟ͷ৘ใΛهࡌ͢Δ
    • 1984
    • 1984
    • format="yyyy">1984
    • can
    • can

    View Slide

  46. Api.ai: Fulfilment / ಡΈ্͛
    • audio
    • ΦʔσΟΦϑΝΠϧΛ࢖͑Δ
    • جຊతʹ͸ޮՌԻͳͲΛ໐Β͢ͷʹ࢖͏ʁ
    • mp3, ogg, wav(deprecated)
    • ࠷େ̎෼Ҏ಺
    • 5MB
    • httpsͰఏڙ͢Δ͜ͱ

    View Slide

  47. Api.ai: Fulfilment / ಡΈ্͛
    • sub
    • ུশͷಡΈํͳͲΛࢦఆ
    • W3C
    sub>

    View Slide

  48. Api.ai: Fulfilment / ಡΈ্͛

    View Slide

  49. Api.ai: Fulfilment / ಡΈ্͛

    View Slide

  50. Api.ai: Fulfilment / ಡΈ্͛

    View Slide

  51. Api.ai SDK support language

    View Slide

  52. Api.ai: Local࣮ߦ
    • npm install -g @google-cloud/functions-emulator
    • npm install
    • functions start
    • functions deploy appName --trigger-http
    • ngrok http 8010

    View Slide

  53. Local࣮ߦ

    View Slide

  54. Local࣮ߦ

    View Slide

  55. Local࣮ߦ -> fulfillment

    View Slide

  56. TransactionAPI

    View Slide

  57. TransactionAPI

    View Slide

  58. topics
    ᶃ What is actions on google
    ᶄ How to develop app
    ᶅ Why design is important?
    ᶆ Appendix

    View Slide

  59. GUI and VUI

    View Slide

  60. Design
    • Voice User Interface (VUI)
    • ୯७ʹจࣈΛಡΈ্͛Ε͹ྑ͍༁Ͱ͸ͳ͍
    • ਓͱձ࿩͢Δͱ͖ͱಉ͡
    • ձ࿩͸୺తʹɺ৑௕తͰͳ͘
    • creating persona
    ref: https://developers.google.com/actions/design/

    View Slide

  61. Defining Multimodal Interactions:
    One Size Does Not Fit All

    View Slide

  62. Design
    • Assistant have 3 interaction
    • audio (for Google Home)
    • visual (for mobile or TV)
    • haptic (for mobile)
    • Input / Output interaction is sometimes not same
    • Ϣʔβʔͷஔ͔Ε͍ͯΔঢ়گͰมΘΔ
    • out: eye, ear X in: camera, mic, touch

    View Slide

  63. Design (for Home)
    • Don’t read, listen
    • ಡΉจষ≠ฉ͘จষ
    • Avoid information overload
    • ৘ใΛΘͬͱग़͢ͱϢʔβʔ΋ࠔΔ
    • ࿩Λ୹͘·ͱΊɺࡉ͔͍ΠϯλϥΫγϣϯΛ
    • Answer the question
    • Ͱ͖ͳ͔ͬͨ৔߹Ͱ΋౴͑ํ͕͋Δ

    View Slide

  64. Design (for mobile)
    • if one mode goes away, the other should take over
    • optimize for the strongest mode, but allow both
    • Leverage the strengths of each mode & avoid
    redundancy across modalities

    View Slide

  65. topics
    ᶃ What is actions on google
    ᶄ How to develop app
    ᶅ Why design is important?
    ᶆ Appendix

    View Slide

  66. Sessions about Google Assistant
    • Building Apps for the Google Assistant
    • Defining Multimodal Interactions: One Size Does Not Fit All
    • Finding the Right Voice Interactions for your App
    • In Conversation, There Are No Errors
    • PullString: Storytelling in the Age of Conversational Interfaces
    • Applying Built-in Hacks of Conversation to Your Voice UI
    • Bringing the Google Assistant to Any Device
    • Transactions with the Google Assistant
    • Building rich cross-platform conversational UX with API.AI

    View Slide

  67. Google Home is Updated

    View Slide

  68. Google Home
    • Google Home is powered by the Google Assistant.
    • ࠓ೥ͷՆҎ߱ʹ೔ຊ্཮

    View Slide

  69. Google Home Updated
    • Proactive Assistance
    • Visual Response
    • hands free calling
    • Bluetooth Audio Support

    View Slide

  70. Google Home Updated: Proactive Assistance
    • Google Homeʹݺͼ͔͚ͳͯ͘΋৘ใΛڭ͑ͯ͘ΕΔ
    • ಓ࿏ͷࠞࡶ৘ใΛڭ͑ͯ͘ΕΔσϞ͕͋ͬͨ
    • χϡʔε΍Ϛʔέοτͷಈ͖Λ఻͑Δ͜ͱ͕Ͱ͖Δʁ
    • push notificationͷΑ͏ͳ࢓૊Έ͕͋Δ͔ʁ

    View Slide

  71. Google Home Updated: Visual Response

    View Slide

  72. Google Home Updated: hands free calling
    • ϋϯζϑϦʔͰͰ͖Δ͜ͱ͕͍Ζ͍Ζ૿͑ͨ
    • ίϯςΩετ / ൃ࿩ऀͷ൑ผ
    • Call my mother -> ൃ࿩ऀͷ฼਌Λೝࣝ͢Δ
    • ΞϝϦΧͱΧφμ͸HomeͰແྉͰి࿩͕͔͚ΒΕΔ

    View Slide

  73. Google Home Updated: Bluetooth Audio Support
    • HomeΛεϐʔΧʔ୅ΘΓʹ࢖͏͜ͱ͕Ͱ͖Δ
    • ԻָΛฉ͍ͨΓɺcastͨ͠Γ
    • ΋ͬͱੜ׆ʹ਎ۙͳଘࡏͱͯ͠࢖ΘΕΔΑ͏ʹ

    View Slide

  74. ͓ΘΓ

    View Slide

  75. References
    • https://tech.recruit-mp.co.jp/event/post-12166/
    • http://tech.mercari.com/entry/2017/05/30/100000
    • https://cyberagent.connpass.com/event/55405/
    • https://tech.recruit-mp.co.jp/mobile/post-12285/
    • https://speakerdeck.com/srym/o-2017-short-report
    • https://docs.google.com/presentation/d/
    1ea5Touf3c5hgJXmoYL1WmkplF7m6XKYL7YyKPD7OWvY/pub?
    start=false&loop=false&delayms=60000&slide=id.g22d0f56c70_0_2
    7
    • https://github.com/vvakame/pubg-assistant/blob/master/
    README.md

    View Slide