Introduction to Actions on google

Slide 1

Slide 1 text

Introduction to Actions on google 2017/6/16 umeda.apk #3

Slide 2

Slide 2 text

whoami Ryo Yamazaki / Application Engineer twitter: @ymnd github: @ymnder

Slide 3

Slide 3 text

whoami 3

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Actions on Google Developer Challenge • ԠืకΊ੾Γ: 8/31 • ̍౳৆ʹ͸ I/O 2018ͷνέοτ͕ͭ͘

Slide 6

Slide 6 text

΍͍͖ͬͯ·͠ΐ͏

Slide 7

Slide 7 text

topics ᶃ What is actions on google ᶄ How to develop app ᶅ Why design is important? ᶆ Appendix

Slide 8

Slide 8 text

topics ᶃ What is actions on google ᶄ How to develop app ᶅ Why design is important? ᶆ Appendix

Slide 9

Slide 9 text

What is actions on google Actions on Google lets you build apps for the Google Assistant. Your apps can help you engage users through Google Home, eligible Android phones, iPhones, and in the future, through every experience where the Google Assistant is available.

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Actions on google: console • ΞϓϦͷ؅ཧ • γϛϡϨʔλͰςετʗσόοά

Slide 12

Slide 12 text

DEMO

Slide 13

Slide 13 text

topics ᶃ What is actions on google ᶄ How to develop app ᶅ Why design is important? ᶆ Appendix

Slide 14

Slide 14 text

Actions on googleʹؔΘΔओཁͳΞΫλʔ

Slide 15

Slide 15 text

Actions on googleʹؔΘΔओཁͳΞΫλʔ ͜ͷ෦෼Λ࣮૷͢Δ {

Slide 16

Slide 16 text

Develop Assistant App

Slide 17

Slide 17 text

Develop Assistant App

Slide 18

Slide 18 text

Develop Assistant App

Slide 19

Slide 19 text

Get started

Slide 20

Slide 20 text

Get started • ίʔυΛॻ͔ͳͯ͘΋ಈ͔ͤΔ • API.aiΛϘλϯϙνϙν͢Ε͹࡞ΕΔ • dynamicͳϨεϙϯεΛߦ͏৔߹͸࣮૷͕ඞཁͰ͋Δ • ཪଆͷfulﬁllment͸node.jsͳͲͰهड़

Slide 21

Slide 21 text

Get started ᶃ actions on googleͷget startedΛ΍Δ • ͱΓ͋͑ͣಈ͔ͤΔʗ֓ཁΛ௫ΊΔ • ։ൃ؀ڥ͕ߏஙͰ͖Δ ᶄ code labsΛ΍Δ • ҰਓͰͰ͖ΔΑ͏ʹ࡞ΒΕ͍ͯΔ ᶅ Google I/OͷSessionΛݟΔ • Building Apps for the Google Assistant

Slide 22

Slide 22 text

Actions on googleʹؔΘΔओཁͳΞΫλʔ

Slide 23

Slide 23 text

Google Assistant • ձ࿩ܕAI/νϟοτϘοτͷػೳΛఏڙ • HomeɾAndroid(6.0+)ɾiOSͳͲͰ͔ͭ͑Δ • ೔ຊޠରԠ͠·ͨ͠!!!! (※ಡΈ্͛͸ࠓޙ)

Slide 24

Slide 24 text

Google Assistant • ݴ༿ͷΏΒ͗ͷ෦෼Λྑ͍ײ͡ʹೝࣝͯ͘͠ΕΔ • Ի੠Λೝࣝͯ͠ඞཁͳ෦෼ΛऔΓग़ͯ͘͠ΕΔ • ࣗ෼ͰԻ੠ೝࣝʹؔ͢Δ࣮૷Λ͠ͳͯ͘ྑ͍ • ͍ΖΜͳͱ͜ΖͰಈ͘

Slide 25

Slide 25 text

Assistant AppͷىಈτϦΨʔ

Slide 26

Slide 26 text

Assistant AppͷىಈτϦΨʔ • Actions on googleͰΞϓϦΛ࡞੒ͨ͠৔߹ • default App name: test app • “ok google, talk to my test app” • App InformationͰApp nameΛࢦఆͰ͖Δ

Slide 27

Slide 27 text

Actions on googleʹؔΘΔओཁͳΞΫλʔ

Slide 28

Slide 28 text

Api.ai • Ի੠Λղੳ͠ɺͦΕʹԠͨ͡ActionΛ࣮ߦ͢ΔAPI • intents • entities • fulﬁllment

Slide 29

Slide 29 text

Api.ai: Intents

Slide 30

Slide 30 text

Api.ai: Intents • ϢʔβʔͷൃݴΛ࣮ࡍͷActionʹ݁ͼ͚ͭΔ • جຊతͳϨεϙϯε͸ίʔυΛॻ͔ͣʹ࢖͑Δ • ϏδωεϩδοΫ͸fulﬁllmentͰ࣮૷͢Δ

Slide 31

Slide 31 text

Rich Response • جຊతͳฦ౴ Text Response • Ϧονͳฦ౴ • Card • List • Suggestion Chips • Carousel card • ฦ౴ઌͷ࿈ܞ • Facebook messenger, Slack, Telegram…

Slide 32

Slide 32 text

Rich Response: Card

Slide 33

Slide 33 text

Rich Response: List

Slide 34

Slide 34 text

Rich Response: Suggestion chips

Slide 35

Slide 35 text

Rich Response: Carousel card

Slide 36

Slide 36 text

Api.ai: Entities

Slide 37

Slide 37 text

Api.ai: Entities • நग़͍ͨ͠ޠ۟Λఆٛ͢Δ • Ϣʔβʔͷࣗવݴޠ͔ΒύϥϝʔλΛऔΓग़͢ • ༧૝͞ΕΔೖྗΛ͜͜Ͱఆٛ͢Δ • reference valueʹෳ਺ͷsynonymΛఆٛ͢Δ • ϓϦηοτ΋͍Ζ͍Ζ༻ҙ͞Ε͍ͯΔ • ஍໊ɾ୯Ґɾ࣌ؒɾਓ໊ͳͲ https://docs.api.ai/docs/concept-entities

Slide 38

Slide 38 text

Actions on googleʹؔΘΔओཁͳΞΫλʔ

Slide 39

Slide 39 text

Api.ai: Fulﬁlment

Slide 40

Slide 40 text

Api.ai: Fulﬁlment

Slide 41

Slide 41 text

Api.ai: Fulﬁlment

Slide 42

Slide 42 text

Api.ai: Fulﬁlment

Slide 43

Slide 43 text

Api.ai: Fulﬁlment / ಡΈ্͛ • speak • rootʹهड़͢Δ • Hello World • break • ϙʔζ͢Δ • Wait a while calm down.

Slide 44

Slide 44 text

Api.ai: Fulﬁlment / ಡΈ্͛ • say-as • ޠ۟ͷ৘ใΛهࡌ͢Δ • 1984 • one thousand nine hundred and eighty four • nineteen hundred, eighty four? • can • ͖ΌΜ • ͠ʔ͑ʔ͑͵

Slide 45

Slide 45 text

Api.ai: Fulﬁlment / ಡΈ্͛ • say-as • ޠ۟ͷ৘ใΛهࡌ͢Δ • 1984 • 1984 • 1984 • can • can

Slide 46

Slide 46 text

Api.ai: Fulﬁlment / ಡΈ্͛ • audio • ΦʔσΟΦϑΝΠϧΛ࢖͑Δ • جຊతʹ͸ޮՌԻͳͲΛ໐Β͢ͷʹ࢖͏ʁ • mp3, ogg, wav(deprecated) • ࠷େ̎෼Ҏ಺ • 5MB • httpsͰఏڙ͢Δ͜ͱ

Slide 47

Slide 47 text

Api.ai: Fulﬁlment / ಡΈ্͛ • sub • ུশͷಡΈํͳͲΛࢦఆ • _W3C

Slide 48

Slide 48 text

Api.ai: Fulﬁlment / ಡΈ্͛

Slide 49

Slide 49 text

Api.ai: Fulﬁlment / ಡΈ্͛

Slide 50

Slide 50 text

Api.ai: Fulﬁlment / ಡΈ্͛

Slide 51

Slide 51 text

Api.ai SDK support language

Slide 52

Slide 52 text

Api.ai: Local࣮ߦ • npm install -g @google-cloud/functions-emulator • npm install • functions start • functions deploy appName --trigger-http • ngrok http 8010

Slide 53

Slide 53 text

Local࣮ߦ

Slide 54

Slide 54 text

Local࣮ߦ

Slide 55

Slide 55 text

Local࣮ߦ -> fulﬁllment

Slide 56

Slide 56 text

TransactionAPI

Slide 57

Slide 57 text

TransactionAPI

Slide 58

Slide 58 text

topics ᶃ What is actions on google ᶄ How to develop app ᶅ Why design is important? ᶆ Appendix

Slide 59

Slide 59 text

GUI and VUI

Slide 60

Slide 60 text

Design • Voice User Interface (VUI) • ୯७ʹจࣈΛಡΈ্͛Ε͹ྑ͍༁Ͱ͸ͳ͍ • ਓͱձ࿩͢Δͱ͖ͱಉ͡ • ձ࿩͸୺తʹɺ৑௕తͰͳ͘ • creating persona ref: https://developers.google.com/actions/design/

Slide 61

Slide 61 text

Deﬁning Multimodal Interactions: One Size Does Not Fit All

Slide 62

Slide 62 text

Design • Assistant have 3 interaction • audio (for Google Home) • visual (for mobile or TV) • haptic (for mobile) • Input / Output interaction is sometimes not same • Ϣʔβʔͷஔ͔Ε͍ͯΔঢ়گͰมΘΔ • out: eye, ear X in: camera, mic, touch

Slide 63

Slide 63 text

Design (for Home) • Don’t read, listen • ಡΉจষ≠ฉ͘จষ • Avoid information overload • ৘ใΛΘͬͱग़͢ͱϢʔβʔ΋ࠔΔ • ࿩Λ୹͘·ͱΊɺࡉ͔͍ΠϯλϥΫγϣϯΛ • Answer the question • Ͱ͖ͳ͔ͬͨ৔߹Ͱ΋౴͑ํ͕͋Δ

Slide 64

Slide 64 text

Design (for mobile) • if one mode goes away, the other should take over • optimize for the strongest mode, but allow both • Leverage the strengths of each mode & avoid redundancy across modalities

Slide 65

Slide 65 text

topics ᶃ What is actions on google ᶄ How to develop app ᶅ Why design is important? ᶆ Appendix

Slide 66

Slide 66 text

Sessions about Google Assistant • Building Apps for the Google Assistant • Deﬁning Multimodal Interactions: One Size Does Not Fit All • Finding the Right Voice Interactions for your App • In Conversation, There Are No Errors • PullString: Storytelling in the Age of Conversational Interfaces • Applying Built-in Hacks of Conversation to Your Voice UI • Bringing the Google Assistant to Any Device • Transactions with the Google Assistant • Building rich cross-platform conversational UX with API.AI

Slide 67

Slide 67 text

Google Home is Updated

Slide 68

Slide 68 text

Google Home • Google Home is powered by the Google Assistant. • ࠓ೥ͷՆҎ߱ʹ೔ຊ্཮

Slide 69

Slide 69 text

Google Home Updated • Proactive Assistance • Visual Response • hands free calling • Bluetooth Audio Support

Slide 70

Slide 70 text

Google Home Updated: Proactive Assistance • Google Homeʹݺͼ͔͚ͳͯ͘΋৘ใΛڭ͑ͯ͘ΕΔ • ಓ࿏ͷࠞࡶ৘ใΛڭ͑ͯ͘ΕΔσϞ͕͋ͬͨ • χϡʔε΍Ϛʔέοτͷಈ͖Λ఻͑Δ͜ͱ͕Ͱ͖Δʁ • push notiﬁcationͷΑ͏ͳ࢓૊Έ͕͋Δ͔ʁ

Slide 71

Slide 71 text

Google Home Updated: Visual Response

Slide 72

Slide 72 text

Google Home Updated: hands free calling • ϋϯζϑϦʔͰͰ͖Δ͜ͱ͕͍Ζ͍Ζ૿͑ͨ • ίϯςΩετ / ൃ࿩ऀͷ൑ผ • Call my mother -> ൃ࿩ऀͷ฼਌Λೝࣝ͢Δ • ΞϝϦΧͱΧφμ͸HomeͰແྉͰి࿩͕͔͚ΒΕΔ

Slide 73

Slide 73 text

Google Home Updated: Bluetooth Audio Support • HomeΛεϐʔΧʔ୅ΘΓʹ࢖͏͜ͱ͕Ͱ͖Δ • ԻָΛฉ͍ͨΓɺcastͨ͠Γ • ΋ͬͱੜ׆ʹ਎ۙͳଘࡏͱͯ͠࢖ΘΕΔΑ͏ʹ

Slide 74

Slide 74 text

͓ΘΓ

Slide 75

Slide 75 text

References • https://tech.recruit-mp.co.jp/event/post-12166/ • http://tech.mercari.com/entry/2017/05/30/100000 • https://cyberagent.connpass.com/event/55405/ • https://tech.recruit-mp.co.jp/mobile/post-12285/ • https://speakerdeck.com/srym/o-2017-short-report • https://docs.google.com/presentation/d/ 1ea5Touf3c5hgJXmoYL1WmkplF7m6XKYL7YyKPD7OWvY/pub? start=false&loop=false&delayms=60000&slide=id.g22d0f56c70_0_2 7 • https://github.com/vvakame/pubg-assistant/blob/master/ README.md