$30 off During Our Annual Pro Sale. View Details »

I/O Extended 2018 Tokyo@GDG - Assistant

I/O Extended 2018 Tokyo@GDG - Assistant

Session 3 Assistent

Yoichiro Tanaka

May 19, 2018

More Decks by Yoichiro Tanaka

Other Decks in Programming


  1. Assistant Recap (Session 3) Yoichiro Tanaka, I/O Extended 2018 Tokyo@GDG

  2. Yoichiro Tanaka Software Engineer / IT Architect Google Developers Expert

    (Assistant, Web) twitter.com/yoichiro google.com/+YoichiroTanaka
  3. Assistant I/O 2018 I believe that this Google I/O 2018

    may be renamed as "Assistant I/O 2018" !
  4. Google Duplex demo movie.

  5. Continued Conversation allows users not to need to say "Ok

    Google" each time.
  6. Multiple Actions allows users to ask multiple phrases at one

    sentence to the Google Assistant.
  7. By Pretty Please feature, if a child says some phrase

    with "please", Google Assistant replies "Thanks for saying please".
  8. None
  9. Three Smart Display models will be released in July 2018.

  10. The Google Assistant in navigation in Google Maps.

  11. Google Assistant is in many devices, also many devices are

    connected by Google Assistant.
  12. Users use different devices for each situation. Of course, contents,

    services and apps are different for situations.
  13. We must support both voice-activated and visual devices.

  14. Google Assistant is strongly integrated and connected with Android and

  15. App Actions Slices App Links AMP Support Built-in Intents Routine

    Suggestion Table card Theme customization Notification Many features were announced in this I/O to achieve the integration.
  16. App Actions Slices App Links AMP Support Built-in Intents Routine

    Suggestion Table card Theme customization Notification First, talk about each feature to integrate Google Assistant with Android and Web.
  17. Android Apps are suggested on All Apps page based on

    Routines, Context and etc.
  18. When users select some text, related apps are suggested. By

    clicking the suggestion,
  19. The related app is launched and is deep-linked. That is,

    the content about the selected text is shown.
  20. Also, Google Assistant shows related apps for each content.

  21. The AI recognizes the content, then the AI lists up

    actions which the user may do. Finally, related apps are suggested.
  22. Up to now, users select an app to use, then

    some content is consumed in the app. From now on, they will be inverted.
  23. Google prepares many Built-in Intents. Google's AI decides intents from

    a target content.
  24. Each intent can have one or more parameters.

  25. <?xml version="1.0" encoding="utf-8"?> <actions> <action intentName="actions.intent.TAKE_COURSE"> <parameter name="course"> <entity-set-reference entityType="org.schema.type.Course"

    urlFilter="https://www.coursera.org/.*" /> </parameter> <fulfillment urlTemplate="{url}?referrer=actions-on-google-previews" /> </action> </actions> This is an "Actions.xml" file to define actions which the app can treat. Actions on Google recognizes this definitions.
  26. When a user says some phrase like above,

  27. Google Assistant shows the search result and suggests apps which

    can treat the content.
  28. When the user clicks the suggestion, the app is launched

    with the deep-link based on the URL.
  29. Of course, Conversational Actions can handle the Built-in Intents.

  30. In the Dialogflow Console, when adding the "Play game" built-in

    intent to events of some intent,
  31. If the user says "I want to play a game",

    Google Assistant suggests game apps which have the "Play game" intent.
  32. Developers can provide "Sliced" UI for each content on the

    Search Result page and the Google Assistant (later this year).
  33. Users can invoke an action directly by clicking a URL

    provided by Action Links.
  34. Each link for Action Links can be generated on the

    Actions on Google Console.
  35. If you have web contents, you can provide and optimize

    them for Google Assistant without creating Custom Actions.
  36. Content Providers can provide "Content Actions" for Google Assistant.

  37. For Content Actions, content providers need use AMP and Structured

    Data Markup as each content format.
  38. For example, developers can create a content using AMP and

    embed a structured data markup dynamically.
  39. The structured data markup is written by a definition of

    the Schema.org.
  40. Google Assistant renders the content as rich component.

  41. If AMP and Structured Data Markup supported, you will receive

    an email explaining how to claim your directory page.
  42. App Actions Slices App Links AMP Support Built-in Intents Routine

    Suggestion Table card Theme customization Notification The preceding pages were how to integrate Android and Web with Google Assistant.
  43. Related sessions Getting started with App Actions Android Slices: build

    interactive results for Google Search Integrating your Android apps with the Google Assistant Integrating your content with the Google Assistant using AMP and markup An introduction to developing Actions for the Google Assistant How to build a user base for your Actions
  44. App Actions Slices App Links AMP Support Built-in Intents Routine

    Suggestion Table card Theme customization Notification Next, talk about new features of Google Assistant itself.
  45. Users tends to ask Google Assistant multiple tasks at one

    time for each situation.
  46. Default Routines are sets of actions for each event. One

    routine can has one or more actions.
  47. Users cannot only use default routines, but they also create

    a Custom Routine and use it.
  48. conv.ask(new Suggestion("Add to routine")); "Routine Suggestions" is a new feature

    to suggest to add your action to the routine the user has.
  49. For example, users can add the "Order Coffee to Starbucks"

    action to the "Commuting to work" routine.
  50. Visual assistive Another keyword is "Visual assistive".

  51. Visual assistive will become more important concern than ever before.

    For example, Smart Display supporting.
  52. A new rich response named "Table card" to show users

    3x3 information brings more visual and more clear.
  53. conv.ask("Simple response"); conv.ask(new Table({ dividers: true, columns: ["Header1", "Header2", "Header3"],

    rows: [ ["Row1 Item1", "Row1 Item2", …], ["Row2 Item1", "Row2 Item2", …], ["Row3 Item1", "Row3 Item2", …] ] }); Since actions-on-google-nodejs version 2.1.0, the Table card has been supported.
  54. "Theme customization" allows you to customize the design of rich

    responses. When specifying the background image,
  55. The background image is reflected to each rich response like

    a basic card above.
  56. For monetization, Google Sign-in, Transactions API and some features have

    already been available.
  57. Finally, developers can release apps for limited users with Alpha/Beta

    Release feature.
  58. Related sessions Design Actions for the Google Assistant: beyond smart

    speakers, to phones and smart display 10 tips for building better Actions Best practices for testing your Actions Personalize Actions for the Google Assistant Add transactional capabilities to your Actions
  59. Voice User Interface Design & Usecase Building games for the

    Google Assistant on smart displays Creating a persona: what does your product sound like? Build engaging conversations for the Google Assistant using Dialogflow Smart Home, Automotive and others Integrate your smart home device with the Google Assistant What's new in automotive What's new with the Google Assistant SDK for devices
  60. Build Actions for the Google Assistant (Level 1) Creating a

    new project, intent, Using an entity and Handing them by fulfillment code with the inline editor. Build Actions for the Google Assistant (Level 2) Creating own fulfillment code, custom entity, Adding a deep-link, follow-up intent and Using helpers, SSML Smart Home Washer Creating a new project, a washer on local, Adding modes, toggles, request syncs and report states Codelabs
  61. We just have been starting building an ecosystem of the

    Google Assistant. We need your cooperations.
  62. Assistant Developer Community Japan Google Groups http://bit.ly/assistant-dev-japan Slack http://bit.ly/assistant-dev-slack

  63. Any questions?