$30 off During Our Annual Pro Sale. View Details »

VUIが作る未来 〜 NEM9

VUIが作る未来 〜 NEM9

Googleアシスタントを利用可能なデバイスの数は、10億を超えました。皆さんの身近な存在であるGoogleアシスタントは、VUI(Voice User Interface)という新しい体験を人々に提供します。VUIは、コンピュータの使い方を大きく変え、そして人々の生活をより豊かにすることが期待されています。この講演では、VUIやGoogleアシスタントがどのように人々の暮らしを変えようとしているのか、そしてそのエコシステムに参加するにはどうすれば良いのか、といったことをお話いたします。

https://nem.connpass.com/event/122056/

Yoichiro Tanaka

April 20, 2019
Tweet

More Decks by Yoichiro Tanaka

Other Decks in Technology

Transcript

  1. VUIがつくる未来
    Yoichiro Tanaka, NEM #9, April 20 2019

    View Slide

  2. Yoichiro Tanaka
    Software Engineer / IT Architect
    Google Developers Expert (Assistant, Web)
    twitter.com/yoichiro
    google.com/+YoichiroTanaka

    View Slide

  3. Build Actions for Your Community event has been held in the worldwide last year.

    View Slide

  4. Build Actions for your community: I visited at Tokyo, Ishinomaki and Kyoto.

    View Slide

  5. Build Actions for GDG Tokyo

    View Slide

  6. Build Actions for GDG Ishinomaki

    View Slide

  7. Build Actions for GDG Kyoto

    View Slide

  8. Assistant Developer Conference vol.1

    View Slide

  9. I believe that many developers will start building great actions from today in Niigata!
    http://www.city.niigata.lg.jp/shisei/gaiyo/profile/siseigaiyou.images/niigatacitykusatsu.jpg

    View Slide

  10. Today's Agenda

    View Slide

  11. View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. User Interfaces
    Character User Interface Graphical User Interface

    View Slide

  18. View Slide

  19. We mainly use eyes and hands.

    View Slide

  20. View Slide

  21. View Slide

  22. We sometimes cannot use eyes and hands. But, must do something...

    View Slide

  23. We almost don't use ears and mouse. If they are available against computers...

    View Slide

  24. ・・・
    ・・・
    ?
    ?

    View Slide

  25. ・・・
    ・・・

    View Slide

  26. Ask something by Voice
    Know something by Voice

    View Slide

  27. Voice User Interface

    View Slide

  28. View Slide

  29. View Slide

  30. Natural Language Processing
    Morphological analysis
    Syntax analysis
    Semantic analysis
    Context analysis

    View Slide

  31. Machine Learning
    Deep Learning
    Recently, we can do
    Natural Language
    Processing in low cost.
    Big Data

    View Slide

  32. User

    View Slide

  33. How far is the
    moon?
    The Moon is
    384,400 kilometers
    from earth.
    User

    View Slide

  34. Google Assistant

    View Slide

  35. 34 months
    after the Google Assistant launched on Google I/O 2016

    View Slide

  36. 8 → 30 languages
    14 → 80 countries
    The Assistant went global (up from last year).

    View Slide

  37. "active users of the Google Assistant grew four
    times over the past year. By the end of this
    month, we expect the Google Assistant to be
    available on one billion devices, up from 500
    million last May."
    Here’s how the Google Assistant became more helpful in 2018

    View Slide

  38. Google Assistant is in many devices including Android, iOS, PCs, TVs, Cars, Headphones, Watches and Speakers.
    devices where Google Assistant lives.
    1B

    View Slide

  39. View Slide

  40. View Slide

  41. View Slide

  42. View Slide

  43. View Slide

  44. People can order Coffee with AI Assistant provided by Android Auto.

    View Slide

  45. View Slide

  46. Google Home Hub becomes a hub for all IoT devices and an entrance for communication.

    View Slide

  47. View Slide

  48. View Slide

  49. People are currently walking using Smartphone...

    View Slide

  50. People will walk seeing forward, with wearable devices.

    View Slide

  51. 6ooo913

    View Slide

  52. Devices Actions
    Telephone Network Internet

    View Slide

  53. View Slide

  54. How far is the
    moon?
    The Moon is
    384,400
    kilometers from
    earth.
    User

    View Slide

  55. (Question) (Answer)
    User

    View Slide

  56. Order a
    hamburger.
    Is that a small,
    medium or large?
    User Your Service
    Is that a small,
    medium or large?

    View Slide

  57. Actions on Google

    View Slide

  58. View Slide

  59. View Slide

  60. View Slide

  61. Conversational Actions Smart Home Actions

    View Slide

  62. Devices Actions
    Webhook
    Speech Recognition
    Decision of invoked action
    Text to Speech

    View Slide

  63. NLP Engine
    You can select your
    favorite NLP engine.
    Actions
    ?
    Actions
    ?

    View Slide

  64. Dialogflow

    View Slide

  65. View Slide

  66. View Slide

  67. View Slide

  68. View Slide

  69. View Slide

  70. View Slide

  71. Devices Dialogflow

    View Slide

  72. Devices Actions
    Dialogflow
    Webhook

    View Slide

  73. Developers can write a fulfillment code with the actions on google nodejs library.
    'use strict';
    const { dialogflow } = require('actions-on-google');
    const functions = require('firebase-functions');
    const app = dialogflow({ debug: true });
    app.intent('Default Welcome Intent', conv => {
    conv.close('Hello, world.');
    });
    export.fulfillment = functions.https.onRequest(app);

    View Slide

  74. Order a
    hamburger.
    Is that a small,
    medium or large?
    User Your Service
    Is that a small,
    medium or large?

    View Slide

  75. app.intent('Order Hamburger', conv => {
    conv.ask('Is that a small, medium or large?');
    });
    You can implement your fulfillment code easily.

    View Slide

  76. View Slide

  77. 終わります。
    終わりますか?
    よくわかりません。
    期待する回答:
    「はい」「いいえ」
    「はい」「いいえ」では
    なかった!

    View Slide

  78. 「終わりますか?」
    はい。
    終わります。
    終わるよ。
    そうだね。
    うん。
    だね。
    そうします。
    終了。
    やめる。
    やめます。
    ですね。
    ・・・

    View Slide

  79. イチ
    予約確認はイチと、
    取り消しはニと言って
    ください。
    予約を確認した
    いです。
    予約確認しますか?
    それとも取り消します
    か?

    View Slide

  80. To implement VUI is easy.
    To design VUI is very difficult.

    View Slide

  81. https://designguidelines.withgoogle.com/conversation

    View Slide

  82. What is conversation design?
    Learn about conversation
    How do I get started?
    Is conversation the right fit?
    Who are your users?
    Who is your persona?
    Draft a conversation
    Design for multiple devices
    Style guide
    Conversational components
    Error handling
    Visual components
    https://designguidelines.withgoogle.com/conversation

    View Slide

  83. Multimodal

    View Slide

  84. View Slide

  85. View Slide

  86. View Slide

  87. View Slide

  88. Basic card, Suggestion chip, Carousel

    View Slide

  89. Media responses, Table, List

    View Slide

  90. View Slide

  91. Business (Auth & Purchase)

    View Slide

  92. app.intent('Start Signin', conv => {
    conv.ask(new SignIn('To get your account details'))
    })
    app.intent('Get Signin', (conv, params, signin) => {
    if (signin.status === 'OK') {
    const payload = conv.user.profile.payload
    conv.ask(`I got your account details, ${payload.name}. What do you want to do next?`)
    } else {
    conv.ask(`I won't be able to save your data, but what do you want to do next?`)
    }
    })

    View Slide

  93. You can integrate physical purchase feature in your actions with Google Payment/your own payment.

    View Slide

  94. Digital Goods Transactions feature uses In-app products in Google Play.

    View Slide

  95. Digital Goods Transactions feature uses In-app products in Google Play.

    View Slide

  96. Relationship with Web & Android

    View Slide

  97. +

    View Slide

  98. View Slide

  99. View Slide

  100. Google Cloud Platform
    Android, Google Play, Web, etc.
    Assistant SDK
    Actions on Google
    Smart-(phones | watches | speakers | earphones | tvs), PCs, Cars
    SDKs for Actions, Actions API, Assistant API, Home Graph API
    Dialogflow

    View Slide

  101. Google Assistant is Internet.

    View Slide

  102. There are some issues...

    View Slide

  103. Smart Speakers cannot protect privacy information...

    View Slide

  104. AI Assistant currently can't address users...

    View Slide

  105. AI Assistant does not know enough context of the user...

    View Slide

  106. AI Assistant is NOT good match with Advertisement.

    View Slide

  107. Economic Growth
    Electricity, Public health, Chemistry, Pharmaceutical industry,
    Internal-combustion engine and Communication

    View Slide

  108. Is AI changing our life?

    View Slide

  109. Basic Research/Study
    Platform/Infrastructure
    Services/Applications/Use-cases
    Current Hot Layers
    Current AI Status
    Very Important Layers!

    View Slide

  110. We just have been starting building an ecosystem of the Google Assistant. We need your cooperations.

    View Slide

  111. Google Groups
    http://bit.ly/assistant-dev-japan
    Slack
    http://bit.ly/assistant-dev-slack
    Assistant Developer Community Japan

    View Slide

  112. Thank you. Any questions?
    NEM9, Apr 20 2019, Yoichiro Tanaka

    View Slide