Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Build accessible conversational Experiences with Actions on Google

Build accessible conversational Experiences with Actions on Google

Build accessible conversational Experiences with Actions on Google
Talk done at GDG Lisbon

James Coggan

May 24, 2018
Tweet

More Decks by James Coggan

Other Decks in Technology

Transcript

  1. Ok Google, ask GDG Lisbon for the event date and

    time Ok Google, let’s talk to GDG Lisbon Trigger Phrase Action Name Developer Specified
  2. Ok Google, ask GDG Lisbon for the event date and

    time Trigger Phrase Ok Google, let’s talk to GDG Lisbon Trigger Phrase Action Name Developer Specified
  3. Ok Google, ask GDG Lisbon for the event date and

    time Trigger Phrase Action Name Developer Specified Ok Google, let’s talk to GDG Lisbon Trigger Phrase Action Name Developer Specified
  4. Ok Google, ask GDG Lisbon for the event date and

    time Trigger Phrase Action Name Developer Specified Action Preposition Ok Google, let’s talk to GDG Lisbon Trigger Phrase Action Name Developer Specified
  5. Ok Google, ask GDG Lisbon for the event date and

    time Trigger Phrase Action Name Developer Specified Action Preposition Action Phrase Developer Specified Ok Google, let’s talk to GDG Lisbon Trigger Phrase Action Name Developer Specified
  6. Ok Google, ask GDG Lisbon for the event date and

    time Trigger Phrase Action Name Developer Specified Action Preposition Action Phrase Developer Specified Ok Google, let’s talk to GDG Lisbon Trigger Phrase Action Name Developer Specified
  7. conv.ask(new SimpleRespone({ speech: 'Howdy! I can tell you fun facts

    about ' + 'almost any number, like 42. What do you have in mind?' }));
  8. conv.ask(new SimpleRespone({ speech: 'Howdy! I can tell you fun facts

    about ' + 'almost any number, like 42. What do you have in mind?', text: 'Howdy! I can tell you fun facts about ' + 'almost any number. What do you have in mind?' }));
  9. conv.ask(new SimpleRespone({ speech: 'Howdy! I can tell you fun facts

    about ' + 'almost any number, like 42. What do you have in mind?', text: 'Howdy! I can tell you fun facts about ' + 'almost any number. What do you have in mind?' }));
  10. conv.ask(new SimpleRespone({ speech: 'Howdy! I can tell you fun facts

    about ' + 'almost any number, like 42. What do you have in mind?', text: 'Howdy! I can tell you fun facts about ' + 'almost any number. What do you have in mind?' })); conv.ask(new Suggestion(['0', '42', '100', 'Never mind']));
  11. conv.ask('Math and prime numbers it is!') conv.ask(new BasicCard({ text: '42

    is an even composite number. It is composed of three distinct prime ' + 'numbers multiplied together. It has a total of eight divisors. 42 is an ' + 'abundant number, because the sum of its proper divisors 54 is greater ' + 'than itself. To count from 1 to 42 would take you about twenty-one…', }));
  12. conv.ask('Math and prime numbers it is!') conv.ask(new BasicCard({ text: '42

    is an even composite number. It is composed of three distinct prime ' + 'numbers multiplied together. It has a total of eight divisors. 42 is an ' + 'abundant number, because the sum of its proper divisors 54 is greater ' + 'than itself. To count from 1 to 42 would take you about twenty-one…', title: 'Math & prime numbers', }));
  13. conv.ask('Math and prime numbers it is!') conv.ask(new BasicCard({ text: '42

    is an even composite number. It is composed of three distinct prime ' + 'numbers multiplied together. It has a total of eight divisors. 42 is an ' + 'abundant number, because the sum of its proper divisors 54 is greater ' + 'than itself. To count from 1 to 42 would take you about twenty-one…', title: 'Math & prime numbers', buttons: new Button(), }));
  14. conv.ask('Math and prime numbers it is!') conv.ask(new BasicCard({ text: '42

    is an even composite number. It is composed of three distinct prime ' + 'numbers multiplied together. It has a total of eight divisors. 42 is an ' + 'abundant number, because the sum of its proper divisors 54 is greater ' + 'than itself. To count from 1 to 42 would take you about twenty-one…', title: 'Math & prime numbers', buttons: new Button({ title: 'Read more', }), }));
  15. conv.ask('Math and prime numbers it is!') conv.ask(new BasicCard({ text: '42

    is an even composite number. It is composed of three distinct prime ' + 'numbers multiplied together. It has a total of eight divisors. 42 is an ' + 'abundant number, because the sum of its proper divisors 54 is greater ' + 'than itself. To count from 1 to 42 would take you about twenty-one…', title: 'Math & prime numbers', buttons: new Button({ title: 'Read more', url: 'https://example.com', }), }));
  16. conv.ask('Math and prime numbers it is!') conv.ask(new BasicCard({ text: '42

    is an even composite number. It is composed of three distinct prime ' + 'numbers multiplied together. It has a total of eight divisors. 42 is an ' + 'abundant number, because the sum of its proper divisors 54 is greater ' + 'than itself. To count from 1 to 42 would take you about twenty-one…', title: 'Math & prime numbers', buttons: new Button({ title: 'Read more', url: 'https://example.com', }), image: new Image({ }), }));
  17. conv.ask('Math and prime numbers it is!') conv.ask(new BasicCard({ text: '42

    is an even composite number. It is composed of three distinct prime ' + 'numbers multiplied together. It has a total of eight divisors. 42 is an ' + 'abundant number, because the sum of its proper divisors 54 is greater ' + 'than itself. To count from 1 to 42 would take you about twenty-one…', title: 'Math & prime numbers', buttons: new Button({ title: 'Read more', url: 'https://example.com', }), image: new Image({ url: 'https://example.google.com/42.png', }), }));
  18. conv.ask('Math and prime numbers it is!') conv.ask(new BasicCard({ text: '42

    is an even composite number. It is composed of three distinct prime ' + 'numbers multiplied together. It has a total of eight divisors. 42 is an ' + 'abundant number, because the sum of its proper divisors 54 is greater ' + 'than itself. To count from 1 to 42 would take you about twenty-one…', title: 'Math & prime numbers', buttons: new Button({ title: 'Read more', url: 'https://example.com', }), image: new Image({ url: 'https://example.google.com/42.png', accessibilityText: 'Image of an abacus, }), }));
  19. app.intent("Intent that needs screen", function (conv) { if (conv.surface.capabilities.has('actions.capability.SCREEN_OUTPUT')) {

    showUiElement(); } else { conv.ask(new NewSurface({ capabilities: ["actions.capability.SCREEN_OUTPUT"], })); } });
  20. app.intent("Intent that needs screen", function (conv) { if (conv.surface.capabilities.has('actions.capability.SCREEN_OUTPUT')) {

    showUiElement(); } else { conv.ask(new NewSurface({ capabilities: ["actions.capability.SCREEN_OUTPUT"], context: "This device doesn't have a screen.", })); } });
  21. app.intent("Intent that needs screen", function (conv) { if (conv.surface.capabilities.has('actions.capability.SCREEN_OUTPUT')) {

    showUiElement(); } else { conv.ask(new NewSurface({ capabilities: ["actions.capability.SCREEN_OUTPUT"], context: "This device doesn't have a screen.", notificationTitle: "MyApp Notification" })); } });
  22. [1]

  23. Cowabunga, dude! What flavors do you crave? The bus is

    approaching your stop now. Your trip will take 45 minutes.
  24. Buy a pizza Getting there soon Purchase a ticket Arriving

    soon Look at your order Review your schedule
  25. [2]

  26. code code code code code code code code code code

    code “ dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code code code
  27. code code code code code code code code code code

    code “ dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code code code
  28. [3]

  29. Avoid force feeding commands. (such as “to reply to a

    message, say ‘reply’” or “to hear that again, say ‘repeat’”.) People hardly ever say nonsense. They just weren’t parsed correctly. There was a misalignment in meaning.
  30. Avoid force feeding commands. (such as “to reply to a

    message, say ‘reply’” or “to hear that again, say ‘repeat’”.) People hardly ever say nonsense. They just weren’t parsed correctly. There was a misalignment in meaning. There’s no such thing as a query with no intent. The user wanted to do something, even if they didn’t say so.
  31. Avoid force feeding commands. (such as “to reply to a

    message, say ‘reply’” or “to hear that again, say ‘repeat’”.) People hardly ever say nonsense. They just weren’t parsed correctly. There was a misalignment in meaning. There’s no such thing as a query with no intent. The user wanted to do something, even if they didn’t say so. Don’t create dead ends when something doesn’t work. If you ask for them to say something again, they will, but rarely exactly the same way. Honor that they’re invested in outcome of a repair attempt.
  32. Where is the user? What are they doing? What type

    of device are they using? What’s their intent? Where is the user’s frame of mind?
  33. Where is the user? What are they doing? What type

    of device are they using? What’s their intent? Where is the user’s frame of mind? What happened right before? What’s happening next? How is the experience influenced over time?
  34. Where is the user? What are they doing? What type

    of device are they using? What’s their intent? Where is the user’s frame of mind? What happened right before? What’s happening next? How is the experience influenced over time? What do users know going in? What data can you use to enrich the experience predictively or to cut out unnecessary steps?
  35. Rapid reprompt Reframe the question → → Ask another way

    Be ready for questions about the question
  36. Rapid reprompt Reframe the question → → Ask another way

    Be ready for questions about the question Be proactive