Slide 1

Slide 1 text

Actions on Google Overview Google Developers ML Summit, Dec 14 2018, Yoichiro Tanaka

Slide 2

Slide 2 text

Yoichiro Tanaka Software Engineer / IT Architect Google Developers Expert (Assistant, Web) twitter.com/yoichiro google.com/+YoichiroTanaka

Slide 3

Slide 3 text

What image do you have against Computers?

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

User Interfaces Character User Interface Graphical User Interface

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

We mainly use eyes and hands.

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

We sometimes cannot use eyes and hands. But, must do something...

Slide 11

Slide 11 text

We almost don't use ears and mouse. If they are available against computers...

Slide 12

Slide 12 text

・・・ ・・・ ? ?

Slide 13

Slide 13 text

・・・ ・・・

Slide 14

Slide 14 text

Ask something by Voice Know something by Voice

Slide 15

Slide 15 text

Voice User Interface

Slide 16

Slide 16 text

I would like to go to the Shibuya Hikarie. Sure. I just set there to the navigation system.

Slide 17

Slide 17 text

Natural Language Processing Morphological analysis Syntax analysis Semantic analysis Context analysis

Slide 18

Slide 18 text

Machine Learning Deep Learning Recently, we can do Natural Language Processing in low cost. Big Data

Slide 19

Slide 19 text

User

Slide 20

Slide 20 text

How far is the moon? The Moon is 384,400 kilometers from earth. User

Slide 21

Slide 21 text

Google Assistant

Slide 22

Slide 22 text

30 months after the Google Assistant launched on Google I/O 2016

Slide 23

Slide 23 text

(Question) (Answer) User

Slide 24

Slide 24 text

Order a hamburger. Is that a small, medium or large? User Your Service Is that a small, medium or large?

Slide 25

Slide 25 text

Actions on Google

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

Conversational Actions Smart Home Actions

Slide 30

Slide 30 text

Devices Actions Webhook Speech Recognition Decision of invoked action Text to Speech

Slide 31

Slide 31 text

NLP Engine You can select your favorite NLP engine. Actions ? Actions ?

Slide 32

Slide 32 text

Dialogflow

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

Devices Dialogflow

Slide 40

Slide 40 text

Devices Actions Dialogflow Webhook

Slide 41

Slide 41 text

Developers can write a fulfillment code with the actions on google nodejs library. 'use strict'; const { dialogflow } = require('actions-on-google'); const functions = require('firebase-functions'); const app = dialogflow({ debug: true }); app.intent('Default Welcome Intent', conv => { conv.close('Hello, world.'); }); export.fulfillment = functions.https.onRequest(app);

Slide 42

Slide 42 text

Order a hamburger. Is that a small, medium or large? User Your Service Is that a small, medium or large?

Slide 43

Slide 43 text

app.intent('Order Hamburger', conv => { conv.ask('Is that a small, medium or large?'); }); You can implement your fulfillment code easily.

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

終わります。 終わりますか? よくわかりません。 期待する回答: 「はい」「いいえ」 期待と違ったので

Slide 46

Slide 46 text

「終わりますか?」 はい。 終わります。 終わるよ。 そうだね。 うん。 だね。 そうします。 終了。 やめる。 やめます。 ですね。 ・・・

Slide 47

Slide 47 text

イチ 予約確認はイチと、 取り消しはニと言って ください。 予約を確認した いです。 予約確認しますか? それとも取り消します か?

Slide 48

Slide 48 text

To implement VUI is easy. To design VUI is very difficult.

Slide 49

Slide 49 text

https://designguidelines.withgoogle.com/conversation

Slide 50

Slide 50 text

What is conversation design? Learn about conversation How do I get started? Is conversation the right fit? Who are your users? Who is your persona? Draft a conversation Design for multiple devices Style guide Conversational components Error handling Visual components https://designguidelines.withgoogle.com/conversation

Slide 51

Slide 51 text

Multimodal

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

Basic card, Suggestion chip, Carousel

Slide 57

Slide 57 text

Media responses, Table, List

Slide 58

Slide 58 text

No content

Slide 59

Slide 59 text

Business (Auth & Purchase)

Slide 60

Slide 60 text

app.intent('Start Signin', conv => { conv.ask(new SignIn('To get your account details')) }) app.intent('Get Signin', (conv, params, signin) => { if (signin.status === 'OK') { const payload = conv.user.profile.payload conv.ask(`I got your account details, ${payload.name}. What do you want to do next?`) } else { conv.ask(`I won't be able to save your data, but what do you want to do next?`) } })

Slide 61

Slide 61 text

You can integrate physical purchase feature in your actions with Google Payment/your own payment.

Slide 62

Slide 62 text

Digital Goods Transactions feature uses In-app products in Google Play.

Slide 63

Slide 63 text

Digital Goods Transactions feature uses In-app products in Google Play.

Slide 64

Slide 64 text

Relationship with Web & Android

Slide 65

Slide 65 text

+

Slide 66

Slide 66 text

No content

Slide 67

Slide 67 text

No content

Slide 68

Slide 68 text

Google Cloud Platform Android, Google Play, Web, etc. Assistant SDK Actions on Google Smart-(phones | watches | speakers | earphones | tvs), PCs, Cars SDKs for Actions, Actions API, Assistant API, Home Graph API Dialogflow

Slide 69

Slide 69 text

Google Assistant is Internet.

Slide 70

Slide 70 text

(Your Brand Name) Search Search Engine Optimization was an important thing.

Slide 71

Slide 71 text

Assistant Optimization will be an important thing than SEO. (Your Brand Name) を知りたい

Slide 72

Slide 72 text

Economic Growth Electricity, Public health, Chemistry, Pharmaceutical industry, Internal-combustion engine and Communication

Slide 73

Slide 73 text

Is AI changing our life?

Slide 74

Slide 74 text

Basic Research/Study Platform/Infrastructure Services/Applications/Use-cases Current Hot Layers Current AI Status Very Important Layers!

Slide 75

Slide 75 text

We just have been starting building an ecosystem of the Google Assistant. We need your cooperations.

Slide 76

Slide 76 text

Google Groups http://bit.ly/assistant-dev-japan Slack http://bit.ly/assistant-dev-slack Assistant Developer Community Japan

Slide 77

Slide 77 text

Thank you. Any questions? Google Developers ML Summit, Dec 14 2018, Yoichiro Tanaka