Points of VUI apps from implementing to releasing after designing it

Transcript

1. Points of VUI apps from implementing to releasing after designing

   it Yoichiro Tanaka, Google Developers Expert

2. Yoichiro Tanaka Software Engineer / IT Architect Google Developers Expert

   (Web) twitter.com/yoichiro google.com/+YoichiroTanaka

3. What does describe in this doc File points while developing

   the "Bulls and Cows" number-guessing app as an app for Google Assistant. • Points while designing • Points while implementing • Points at submitting for a review

4. https://assistant.google.com/services/a/uid/000000b349693dc4?hl=en

5. Happy path of Bulls and Cows

6. None
7. None
8. None
9. None
10. None
11. None
12. None

13. Points during designing

14. Tools for Design Currently, most nothing's tools for VUI designing.

    • Many companies and communities are finding a way. Many tools might be provided from now. • We don't have any tools, but we need some tool to design VUI apps. It is necessary to find some method by own (or refer to recommendations on Actions on Google)

15. My method 1. Write scenarios. 2. Find contexts. 3. Draw

    navigation between contexts. 4. Find intents, and add them to the context navigation. 5. Apply each dialog before and after each intent.

16. 1. Write scenarios

17. Color alternate rows in Google Spreadsheet in advance. Convenience!

18. First, write a Happy Path. Then, write other scenarios. Happy

    Path Scenario 2 Scenario 3

19. Happy Path

20. Play again

21. Help

22. Hint

23. Mistakes three times

24. 2. Find contexts

25. 2. Find contexts Search for a place where the theme

    of the conversation changed. Then extract before and after change as "context". • "Listen to a three-digit number, and answer the judgment result" • "After answering correctly, ask whether play again one more time" • "After three mistakes, ask whether want to hear the hint"

26. Happy Path - The theme changed from "Now, gaming" to

    "Ask whether plays again" -> The context changed.

27. Help - Heard the help in "Gaming", but the theme

    not changed from "Gaming". As the result, the context is still "Gaming".

28. Mistakes 3 times - By mistaken 3 times, the theme

    changed to "Asking to hear a hint". That is, the context changed.

29. 3. Draw navigation between contexts

30. "Now gaming" "Ask whether play again" "Ask whether want to

    hear a hint" Can understand an abstract of the conversation stream by drawing navigation between contexts.

31. 4. Find intents

32. 4. Find intents Find user's "intents (= declaration of intent,

    requirements)". • "Context" is for apps a bit. • "Intent" is for users.

33. Happy Path - The "input.numbers" intent changes a context (from

    "game" to "play_again")

34. Hint - "Hear a hint" = "help_hint" intent

35. Mistakes 3 times - The "input.numbers" intent also changes a

    context to "hear_hint"

36. 4. And add them to the context navigation.

37. Can understand an abstract of a stream of the conversation

    by drawing a chart of changing contexts.
38. None
39. None

40. 5. Apply each dialog before and after each intent.

41. None

42. Ex: Insert intents and apply dialogs between the "game" context

    and the "hear_hint" context.

43. Ex: Insert intents and apply dialogs between the "game" context

    and the "hear_hint" context. 789 1 Bull 0 Cow. Do you need a hint? Yes The sum of three digit numbers is 16.

44. Ex: Insert intents and apply dialogs between the "game" context

    and the "hear_hint" context. 789 Hint 1 Bull 0 Cow. Do you need a hint? Yes The sum of three digit numbers is 16. The sum of three digit numbers is 16. I see. Please say three digit numbers. 1 Bull and 0 Cow I didn't understand well. Please say three digit numbers. No 123 ??? * Rounded corners are Context, Rectangles are Intent.

45. Scenarios • Google SpreadSheet VUI design starts with scenarios VUI

    design • Google SpreadSheet • UML: State-chart diagram • UML: Activity diagram

46. Points during implementing

47. Points during implementing • Mapping VUI design to Dialogflow •

    Find actions from intents, and create a implementation code • Can end a conversation anytime

48. Mapping to Dialogflow

49. 789 1 Bull and 0 Cow. Do you need a

    hint? 1イート0バイトです。 Congratulation! You got three Bulls! Do you want to play again? Training phrases Input context Output context Response Intent

50. Intent Input context Output context Training phrases Response

51. Actions and Fulfillment

52. Find actions What one response is decided against one intent

    is rare. The response will be changed dynamically depending on user's phrases, contexts and app's statuses. → Fulfillment decides them. • Output contexts • Response To specify a process to the fulfillment, define actions.

53. Action name Use Fulfillment

54. Fulfillment The target code which developers write is this fulfillment

    code. • Can use something which can handle HTTP and JSON. • Generally, actions-on-google-nodejs for Node.js is used. ◦ https://github.com/actions-on-google/actions-on-google-nodejs • Fulfillment has an affinity for Cloud Functions. ◦ GCP/Firebase Cloud Functions (Free plan can be used if no external API accesses) ◦ Other alike solutions should also be no problem.

55. Dialogflow Fulfillment • Action name • Arguments (= Parameters) •

    ... HTTPS request

56. const DialogflowApp = require("actions-on-google").DialogflowApp; const ACTION_INPUT_WELCOME = "input.welcome"; exports.fulfillment =

    (req, res) => { const app = new DialogflowApp({request: req, response: res}); const inputWelcome = app => { app.ask("Hello, this is ***. ..."); }; const actionMap = new Map(); actionMap.set(ACTION_INPUT_WELCOME, inputWelcome); app.handleRequest(actionMap); }; This is a template code for Fulfillment.

57. The code which is used to decide the response. //

    Continue the conversation, and say a next phrase from the user. app.ask("Great! You got three bulls. Do you want to play again?"); // Specify re-prompts as 2nd argument for when the user doesn't say anything. app.ask( "Great! You got three bulls. Do you want to play again?", ["Sorry, I can't hear you well. Play again?", ...] ); // End the conversation app.tell("I see. See you again.");

58. The code which is used to decide Output Contexts //

    Specify the output Context. const CONTEXT_PLAY_AGAIN = "play_again"; app.setContext(CONTEXT_PLAY_AGAIN, 1); // The value "1" is for Lifespan // Delete the input Context const CONTEXT_GAME = "game"; app.setContext(GAME, 0); // Deleted by specifying "0"

59. The code which is used to keep information via multiple

    conversations. // Keep information within the conversation session. app.data.answer = "123"; // Keep information for a next conversation after ending the conversation. app.userStorage.answer = "123";

60. const inputNumber = app => { const number = app.getArgument("number");

    const answer = app.data.answer; if (number === answer) { app.setContext("game", 0); app.setContext("play_again", 1); app.ask("Bingo! Do you want to play again?"); } else { app.setContext("game", 1); app.ask("1 bull and 0 cow."); } }; Actual code sample.

61. Must respond within 5 sec. Currently, cannot respond multiple times

    at once. Points of implementing Fulfillment

62. Can end the conversation anytime

63. The previous diagram is actually wrong.

64. The correct diagram is above.

65. Must be able to end the conversation by the user

    in all contexts. → In the review process, the reviewer will confirm whether the conversation can be ended by the phrase "Finish". → Note that the reviewer may reject the application because the reviewer cannot end the conversation with the phrase in the context answering "yes" or "no" with app.askForConfirmation()... Can end the conversation anytime

66. Points at submitting for a review

67. Registration of app information • App name, pronunciation • App

    description • Image, icon • Invocation phrases • Developer information • Privacy policy, Terms of use • Category • Other... Basically, need to register information to satisfied the page of the right image.

68. App name Recommend that should confirm whether your app can

    be invoked with the app name by Google Assistant as your expectation in advance. → Try calling the app name on Google Assistant, and know the text recognized by the assistant. → And, app name is first come, first served.

69. Image, icon The small icon is "rounded corners" icon. →

    Should design the icon which all corners will not be drawn.

70. Invocation phrases The reviewer confirms the app actually can be

    invoked with all registered phrases. → Also, on all devices. If not invoked, the application will be rejected. Especially, it is important to confirm Implicit Invocation on all devices.

71. Privacy policy, Terms of use It seems that the content

    of them is not checked well at the review. → However, it is important to protect me, therefore you must write them with responsibility. The review checks whether "a correct app name" is written in them.

72. Conclusion

73. Conclusion To design, implement and release smoothy your app for

    Google Assistant, you need to know below described in this material in advance: • Points during designing • Points during implementing • Points at submitting for a review

74. Fin.