VUIが作る未来 〜 NEM9

Googleアシスタントを利用可能なデバイスの数は、10億を超えました。皆さんの身近な存在であるGoogleアシスタントは、VUI（Voice User Interface）という新しい体験を人々に提供します。VUIは、コンピュータの使い方を大きく変え、そして人々の生活をより豊かにすることが期待されています。この講演では、VUIやGoogleアシスタントがどのように人々の暮らしを変えようとしているのか、そしてそのエコシステムに参加するにはどうすれば良いのか、といったことをお話いたします。

https://nem.connpass.com/event/122056/

Transcript

1. VUIがつくる未来
   Yoichiro Tanaka, NEM #9, April 20 2019
   

   View Slide

2. Yoichiro Tanaka
   Software Engineer / IT Architect
   Google Developers Expert (Assistant, Web)
   twitter.com/yoichiro
   google.com/+YoichiroTanaka
   

   View Slide

3. Build Actions for Your Community event has been held in the worldwide last year.
   

   View Slide

4. Build Actions for your community: I visited at Tokyo, Ishinomaki and Kyoto.
   

   View Slide

5. Build Actions for GDG Tokyo
   

   View Slide

6. Build Actions for GDG Ishinomaki
   

   View Slide

7. Build Actions for GDG Kyoto
   

   View Slide

8. Assistant Developer Conference vol.1
   

   View Slide

9. I believe that many developers will start building great actions from today in Niigata!
   http://www.city.niigata.lg.jp/shisei/gaiyo/profile/siseigaiyou.images/niigatacitykusatsu.jpg
   

   View Slide

10. Today's Agenda
    

    View Slide

11. View Slide

12. View Slide

13. View Slide

14. View Slide

15. View Slide

16. View Slide

17. User Interfaces
    Character User Interface Graphical User Interface
    

    View Slide

18. View Slide

19. We mainly use eyes and hands.
    

    View Slide

20. View Slide

21. View Slide

22. We sometimes cannot use eyes and hands. But, must do something...
    

    View Slide

23. We almost don't use ears and mouse. If they are available against computers...
    

    View Slide

24. ･･･
    ･･･
    ?
    ?
    

    View Slide

25. ･･･
    ･･･
    

    View Slide

26. Ask something by Voice
    Know something by Voice
    

    View Slide

27. Voice User Interface
    

    View Slide

28. View Slide

29. View Slide

30. Natural Language Processing
    Morphological analysis
    Syntax analysis
    Semantic analysis
    Context analysis
    

    View Slide

31. Machine Learning
    Deep Learning
    Recently, we can do
    Natural Language
    Processing in low cost.
    Big Data
    

    View Slide

32. User
    

    View Slide

33. How far is the
    moon?
    The Moon is
    384,400 kilometers
    from earth.
    User
    

    View Slide

34. Google Assistant
    

    View Slide

35. 34 months
    after the Google Assistant launched on Google I/O 2016
    

    View Slide

36. 8 → 30 languages
    14 → 80 countries
    The Assistant went global (up from last year).
    

    View Slide

37. "active users of the Google Assistant grew four
    times over the past year. By the end of this
    month, we expect the Google Assistant to be
    available on one billion devices, up from 500
    million last May."
    Here’s how the Google Assistant became more helpful in 2018
    

    View Slide

38. Google Assistant is in many devices including Android, iOS, PCs, TVs, Cars, Headphones, Watches and Speakers.
    devices where Google Assistant lives.
    1B
    

    View Slide

39. View Slide

40. View Slide

41. View Slide

42. View Slide

43. View Slide

44. People can order Coffee with AI Assistant provided by Android Auto.
    

    View Slide

45. View Slide

46. Google Home Hub becomes a hub for all IoT devices and an entrance for communication.
    

    View Slide

47. View Slide

48. View Slide

49. People are currently walking using Smartphone...
    

    View Slide

50. People will walk seeing forward, with wearable devices.
    

    View Slide

51. 6ooo913
    

    View Slide

52. Devices Actions
    Telephone Network Internet
    

    View Slide

53. View Slide

54. How far is the
    moon?
    The Moon is
    384,400
    kilometers from
    earth.
    User
    

    View Slide

55. (Question) (Answer)
    User
    

    View Slide

56. Order a
    hamburger.
    Is that a small,
    medium or large?
    User Your Service
    Is that a small,
    medium or large?
    

    View Slide

57. Actions on Google
    

    View Slide

58. View Slide

59. View Slide

60. View Slide

61. Conversational Actions Smart Home Actions
    

    View Slide

62. Devices Actions
    Webhook
    Speech Recognition
    Decision of invoked action
    Text to Speech
    

    View Slide

63. NLP Engine
    You can select your
    favorite NLP engine.
    Actions
    ?
    Actions
    ?
    

    View Slide

64. Dialogflow
    

    View Slide

65. View Slide

66. View Slide

67. View Slide

68. View Slide

69. View Slide

70. View Slide

71. Devices Dialogflow
    

    View Slide

72. Devices Actions
    Dialogflow
    Webhook
    

    View Slide

73. Developers can write a fulfillment code with the actions on google nodejs library.
    'use strict';
    const { dialogflow } = require('actions-on-google');
    const functions = require('firebase-functions');
    const app = dialogflow({ debug: true });
    app.intent('Default Welcome Intent', conv => {
    conv.close('Hello, world.');
    });
    export.fulfillment = functions.https.onRequest(app);
    

    View Slide

74. Order a
    hamburger.
    Is that a small,
    medium or large?
    User Your Service
    Is that a small,
    medium or large?
    

    View Slide

75. app.intent('Order Hamburger', conv => {
    conv.ask('Is that a small, medium or large?');
    });
    You can implement your fulfillment code easily.
    

    View Slide

76. View Slide

77. 終わります。
    終わりますか？
    よくわかりません。
    期待する回答：
    「はい」「いいえ」
    「はい」「いいえ」では
    なかった！
    

    View Slide

78. 「終わりますか？」
    はい。
    終わります。
    終わるよ。
    そうだね。
    うん。
    だね。
    そうします。
    終了。
    やめる。
    やめます。
    ですね。
    ・・・
    

    View Slide

79. イチ
    予約確認はイチと、
    取り消しはニと言って
    ください。
    予約を確認した
    いです。
    予約確認しますか？
    それとも取り消します
    か？
    

    View Slide

80. To implement VUI is easy.
    To design VUI is very difficult.
    

    View Slide

81. https://designguidelines.withgoogle.com/conversation
    

    View Slide

82. What is conversation design?
    Learn about conversation
    How do I get started?
    Is conversation the right fit?
    Who are your users?
    Who is your persona?
    Draft a conversation
    Design for multiple devices
    Style guide
    Conversational components
    Error handling
    Visual components
    https://designguidelines.withgoogle.com/conversation
    

    View Slide

83. Multimodal
    

    View Slide

84. View Slide

85. View Slide

86. View Slide

87. View Slide

88. Basic card, Suggestion chip, Carousel
    

    View Slide

89. Media responses, Table, List
    

    View Slide

90. View Slide

91. Business (Auth & Purchase)
    

    View Slide

92. app.intent('Start Signin', conv => {
    conv.ask(new SignIn('To get your account details'))
    })
    app.intent('Get Signin', (conv, params, signin) => {
    if (signin.status === 'OK') {
    const payload = conv.user.profile.payload
    conv.ask(`I got your account details, ${payload.name}. What do you want to do next?`)
    } else {
    conv.ask(`I won't be able to save your data, but what do you want to do next?`)
    }
    })
    

    View Slide

93. You can integrate physical purchase feature in your actions with Google Payment/your own payment.
    

    View Slide

94. Digital Goods Transactions feature uses In-app products in Google Play.
    

    View Slide

95. Digital Goods Transactions feature uses In-app products in Google Play.
    

    View Slide

96. Relationship with Web & Android
    

    View Slide

97. +
    

    View Slide

98. View Slide

99. View Slide

100. Google Cloud Platform
     Android, Google Play, Web, etc.
     Assistant SDK
     Actions on Google
     Smart-(phones | watches | speakers | earphones | tvs), PCs, Cars
     SDKs for Actions, Actions API, Assistant API, Home Graph API
     Dialogflow
     

     View Slide

101. Google Assistant is Internet.
     

     View Slide

102. There are some issues...
     

     View Slide

103. Smart Speakers cannot protect privacy information...
     

     View Slide

104. AI Assistant currently can't address users...
     

     View Slide

105. AI Assistant does not know enough context of the user...
     

     View Slide

106. AI Assistant is NOT good match with Advertisement.
     

     View Slide

107. Economic Growth
     Electricity, Public health, Chemistry, Pharmaceutical industry,
     Internal-combustion engine and Communication
     

     View Slide

108. Is AI changing our life?
     

     View Slide

109. Basic Research/Study
     Platform/Infrastructure
     Services/Applications/Use-cases
     Current Hot Layers
     Current AI Status
     Very Important Layers!
     

     View Slide

110. We just have been starting building an ecosystem of the Google Assistant. We need your cooperations.
     

     View Slide

111. Google Groups
     http://bit.ly/assistant-dev-japan
     Slack
     http://bit.ly/assistant-dev-slack
     Assistant Developer Community Japan
     

     View Slide

112. Thank you. Any questions?
     NEM9, Apr 20 2019, Yoichiro Tanaka
     

     View Slide