Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Designing your first voice interface (workshop)

bensauer
June 09, 2017

Designing your first voice interface (workshop)

Learn about the voice design process, get busy crafting dialogue, learn some fundamental principles, design and test and voice interface in a team. You'll see just how different the design methods are, and what the 'Wizard of Oz' testing method is. Come find your inner bot!

bensauer

June 09, 2017
Tweet

More Decks by bensauer

Other Decks in Design

Transcript

  1. Design
    your first
    voice interface
    Product Camp Poland 2017 • Ben Sauer

    View Slide

  2. Organise

    into
    teams of 4
    *each team needs a MacBook
    *your packet has more instructions

    View Slide

  3. Welcome

    View Slide

  4. Today

    View Slide

  5. Warmup

    View Slide

  6. A spectrum 

    of responses

    View Slide

  7. The medium you
    design in is not the
    medium you
    experience

    View Slide

  8. Find out what
    doesn’t work, as
    well as what does

    View Slide

  9. “When is the next train to London Victoria?”
    Too concise: 

    20 minutes.

    View Slide

  10. “When is the next train to London Victoria?”
    Too detailed: 

    The next train from Brighton station to
    London Victoria station leaves in
    twenty minutes, at twenty past ten. It
    has no first class service and divides
    at Haywards Heath. It arrives at
    London Victoria at ten past eleven.

    View Slide

  11. “When is the next train to London Victoria?”
    Just right: 

    The next train from Brighton to
    London Victoria leaves in 20 minutes.

    View Slide

  12. VOICE INTERFACES:
    NOW AND BEYOND

    View Slide

  13. ARISTOTLE
    There is only one condition in which we can
    imagine managers not needing subordinates,
    and masters not needing slaves.
    This condition would be that each instrument
    could do its own work, at the word of
    command or by intelligent anticipation.
    @bensauer

    View Slide

  14. VUI = Voice User Interface
    ASR = Automatic Speech Recognition
    NLU = Natural Language Understanding
    @bensauer
    humans  and  
    machines  talking
    turning  speech  
    into  words
    turning  words  
    into  meaning

    View Slide

  15. View Slide

  16. Curious Rituals
    produced by The Near Future Laboratory

    View Slide

  17. View Slide

  18. Curious Rituals
    produced by The Near Future Laboratory

    View Slide

  19. View Slide

  20. • Limited shared vocabulary
    • Poor skills / co-ordination
    • No capacity to learn
    • Never sure job will get done
    • Turns you into the a$$h0le
    Using VUI is a bit like dealing with Manuel…

    View Slide

  21. https://www.flickr.com/photos/cryptik/23815673664 @bensauer
    ?

    View Slide

  22. Our true calling is to make 

    the system fail gracefully.

    View Slide

  23. REALITIES OF VUI

    View Slide

  24. VUI is deceptively limited.
    @bensauer

    View Slide

  25. @bensauer
    VUI USE MAY BE U-SHAPED
    @bensauer
    AWESOME
    SUCKS
    10 100 ∞
    USE CASES

    View Slide

  26. View Slide

  27. Platforms limit possibilities.
    @bensauer

    View Slide

  28. Transactions over conversations.
    @bensauer

    View Slide

  29. The context of use 

    is everything.

    View Slide

  30. WHERE IS 

    VUI GOING?

    View Slide

  31. The price of a multifunctional touchscreen 

    is the volume of required input.
    VUI is an efficient shortcut.
    @bensauer

    View Slide

  32. The short-term promise of VUI is
    to simplify complex interactions
    into directly expressed goals,
    without the distraction of a screen.
    @bensauer

    View Slide

  33. @bensauer
    BEYOND USE CASES
    “If the doorbell rings while I’m
    in the garden, call me.”
    @bensauer

    View Slide

  34. View Slide

  35. View Slide

  36. View Slide

  37. @bensauer
    @bensauer
    VOICE ASSISTANT CONCEPTS
    SIRI: ’FAMILIAR’ ALEXA: ’THING’
    http://www.dfki.de/LAMDa/2013/accepted/13_ApplinFischer.pdf

    View Slide

  38. @bensauer
    AMAZON’S PLAY FOR UBICOMP
    • TVs (Westinghouse, Element, Seiki)
    • Fridges (LG)
    • Home robots (LG)
    • Wifi speakers (Lenovo)
    • Kid’s speaker (Mattel)
    • Lamps (GE)
    @bensauer
    • Alarm clocks
    • Cars (Ford, VW)
    • Charging docks
    • Watches
    • Cameras
    • Smartphones (Huawei)

    View Slide

  39. @bensauer
    CONVERSATIONAL UI IS PRETTY HYPED
    @bensauer
    WTF!  menus!

    View Slide

  40. AI is getting too much hype.

    View Slide

  41. WHAT IS
    CONVERSATION?

    View Slide

  42. Context
    Language
    Exchange
    Agreement
    Transaction / Action

    View Slide

  43. from Paul Pangaro's ‘Conversation is more than interface’ https://vimeo.com/207866146

    View Slide

  44. WHAT’S DIFFERENT
    ABOUT VUI DESIGN?

    View Slide

  45. Recall over recognition.

    View Slide

  46. The first interaction 

    expresses the goal.

    View Slide

  47. GUI VUI
    THE USER JOURNEY

    View Slide

  48. The medium you design in

    is not the medium you experience.
    @bensauer

    View Slide

  49. Shorter ‘Time to Test’.

    View Slide

  50. A.I. is getting better at
    understanding so that 

    we don’t have to.

    View Slide

  51. The heart of VUI is great writing.
    @bensauer

    View Slide

  52. VUI OPPORTUNITIES

    View Slide

  53. @bensauer
    VUI APPS
    @bensauer

    View Slide

  54. @bensauer
    SEARCH RESULTS (IN APP)
    @bensauer

    View Slide

  55. @bensauer
    SIMPLE GOPHER TASKS
    @bensauer

    View Slide

  56. @bensauer
    PLAYING WITH AMAZON ECHO
    @bensauer
    Raspberry Pi + Image + Lambda + Node
    Sayspring

    View Slide

  57. Our focus
    Design, prototype,
    and test a voice
    interface

    View Slide

  58. Our focus
    Ignore the
    technology!

    View Slide

  59. Our focus
    Use a 

    wake word

    View Slide

  60. Use cases
    s

    View Slide

  61. A great VUI starts
    with a great use case

    View Slide

  62. Speed,
    simplicity,
    convenience

    View Slide

  63. #1
    Answer off the top
    of your head

    View Slide

  64. What time is it?
    What’s the weather?
    Is there a conference
    dinner tonight?

    View Slide

  65. #2
    A way to avoid
    pulling out a phone or
    jotting notes on paper

    View Slide

  66. Set a timer for
    for 20 minutes
    Call Anna

    View Slide

  67. Order a pizza
    from Tramonti’s
    Remind me to make a
    dentist appointment
    tomorrow

    View Slide

  68. #3
    Help for someone
    with their eyes or
    hands occupied

    View Slide

  69. How long will I
    be in traffic?
    How many cups of
    sugar do I add?

    View Slide

  70. Alert people
    to helpful information
    Bonus

    View Slide

  71. Hey Google,
    what’s up?
    There’s a 40 minute
    delay on Highway 17

    View Slide

  72. Activity: 

    find a use case

    View Slide

  73. Step #1

    Pick a decider for today

    View Slide

  74. Step #2

    Write down activities you do
    most weeks
    One activity per post it
    Swimming Cooking Drive  to  work
    Take  kids  to  
    school

    View Slide

  75. Step #3

    Map your week as a group
    Commute Work Lunch Gym
    MONDAY
    Commute W
    TUESDAY

    View Slide

  76. Step 4: Write down questions or tasks
    that a voice interface could help with
    Drive  to  work
    Check  
    traffic
    Shopping
    Find  shelf  
    location
    Getting  
    dressed
    Check  
    weather
    Dinner
    Ordering  
    takeout

    View Slide

  77. Dinner
    Ordering  
    takeout
    Step 5: Vote for your favourites
    Drive  to  work
    Check  
    traffic
    Shopping
    Find  shelf  
    location
    Getting  
    dressed
    Check  
    weather





    View Slide

  78. Step #6
    Choose your idea

    View Slide

  79. Step #7
    Empathise
    Feeling  tired
    Hands  dirty
    Kids  
    screaming
    Make  
    healthy,  easy,  
    tasty  food
    Cooking  
    dinner

    View Slide

  80. Personality

    View Slide

  81. Same voice,
    different roles

    View Slide

  82. Users hear
    personality,
    regardless

    View Slide

  83. Example: Cortana
    Confident, caring, competent,
    loyal
    Helpful, but not bossy
    Functionally and emotionally
    transparent

    View Slide

  84. Define personality

    View Slide

  85. Step #1

    Discuss and capture
    Page 4 of the handout

    View Slide

  86. Breaktime!
    Come back at 11

    View Slide

  87. What do people say first?
    Define personality

    View Slide

  88. What they say,

    and the way they
    say it

    View Slide

  89. View Slide

  90. View Slide

  91. View Slide

  92. View Slide

  93. Activity: 

    capture the first
    question

    View Slide

  94. Step #1

    Open a google doc
    Give it the following section headings:
    •Test Brief
    •Opening questions
    •Sample dialogue (1,2,3,4)

    View Slide

  95. Step #2

    Write a test brief

    View Slide

  96. Step #3

    Write a first sentence
    “Alexa,  find  me  a  steak  
    restaurant  in  Gdansk.”

    View Slide

  97. Step #4

    Choose roles
    •A moderator
    •A participant from another team

    View Slide

  98. Step #5

    Capture the first question

    View Slide

  99. Step #6

    Swap roles and repeat

    View Slide

  100. Step #7

    Swap roles and repeat

    View Slide

  101. Sample dialogue
    Define personality

    View Slide

  102. Dialogue format

    View Slide

  103. Expressing
    personality

    View Slide

  104. Find the best paths

    View Slide

  105. Exercise
    Creating dialogue

    View Slide

  106. Step #1

    Choose roles
    • A user
    • The voice interface
    • One or two transcribers

    View Slide

  107. Step #2

    Improvise!
    No more than six to ten 

    lines of dialogue

    View Slide

  108. Step #3

    Write your own

    View Slide

  109. Step #4

    Share and discuss

    View Slide

  110. WOZ demo time

    View Slide

  111. Lunch!
    See you at 13:30pm

    View Slide

  112. Exercise 6
    Editing dialogue

    View Slide

  113. Principles

    View Slide

  114. Sound natural

    View Slide

  115. Be true
    to your personality

    View Slide

  116. Give people credit

    View Slide

  117. Be cooperative

    View Slide

  118. Keep it short
    What’s the
    temperature?

    View Slide

  119. Keep it short
    How old is
    Prince Harry?

    View Slide

  120. Prince Harry
    is 32 years old
    He’s 32 years old
    32

    View Slide

  121. Take turns

    View Slide

  122. Bonus tips

    View Slide

  123. #1 Use pronouns

    View Slide

  124. #2 Important
    information 

    at the end

    View Slide

  125. #3 Say everything 

    out loud

    View Slide

  126. Exercise: 

    editing dialogue

    View Slide

  127. Goal: 

    make it good, 

    make it testable

    View Slide

  128. Step #1

    Edit in pairs

    View Slide

  129. Step #2

    Identify patterns
    The  user  starts  
    the  recording  
    defining  the  
    duration
    The  user  starts  
    the  recording  
    *without*  defining  a  
    duration
    The  user  stops  the  
    recording
    The  user  plays  
    back  the  meeting
    Example: Recording meetings

    View Slide

  130. Step #3

    Setup ‘Say Wizard’
    see handout page 8
    tip: make text bigger with 

    Shift cmd +

    View Slide

  131. Step #4

    Copy dialogue into script.txt

    View Slide

  132. Step #5

    Remove the user’s lines from
    script.txt

    View Slide

  133. Step #6

    Reorder the dialogue

    View Slide

  134. First answer
    Second
    Third
    Fourth
    Second 

    scenario

    View Slide

  135. Step #6

    Reorder the dialogue
    tip: leave gaps

    View Slide

  136. Wizard of Oz
    Testing

    View Slide

  137. The goal:
    Having all the
    answers

    View Slide

  138. Fail fast,
    Fail cheap

    View Slide

  139. Amazon Echo
    Early design process

    View Slide

  140. The WOZ process

    View Slide

  141. Exercise
    Wizard of Oz testing

    View Slide

  142. Step #1

    Choose roles
    •A wizard
    •A participant from another team
    •Two observers

    View Slide

  143. Step #2

    Run the test

    View Slide

  144. What did you learn?

    View Slide

  145. Step #3

    Reflect & Refine

    View Slide

  146. Step #4

    Swap roles and test

    View Slide

  147. Step #5

    Reflect & Refine

    View Slide

  148. Final test

    Record a video

    View Slide

  149. How was it?

    Record a video

    View Slide

  150. Post-test
    survey
    Annoying Useful Classic Futuristic

    View Slide

  151. Dialogue Flow
    booking 

    a cab

    View Slide

  152. Exercise:
    Map the flow
    tip: don’t re-use the 

    dialogue as titles

    View Slide

  153. Build and test

    View Slide

  154. UTTERANCE
    “Order me some glow-in-the-dark toilet paper.”

    View Slide



  155. “Order me a Nintendo Switch.”
    INTENT
    Buy Product

    View Slide



  156. “Set a timer for 30 seconds.”
    VARIABLE
    Timer Duration

    View Slide

  157. VUI PRINCIPLES

    View Slide

  158. Always be capturing.

    View Slide

  159. Postel’s Law
    Be conservative in what you send,
    be liberal in what you accept.
    @bensauer

    View Slide

  160. I’d really like to visit Llanfairpwllgwyngyll-
    gogerychwyrndrobwllllantysiliogogogoch.
    @bensauer
    USER:
    In North Wales?
    ALEXA:

    View Slide

  161. Design for mundane realities.

    View Slide

  162. Pair Bluetooth!!!!
    @bensauer
    HEAVILY ACCENTED USER FOR THE 3RD TIME:
    I’m really sorry, I still don’t understand.
    Can I suggest using the Alexa app instead?
    ALEXA:

    View Slide

  163. Considerate confirmation.

    View Slide

  164. Did you mean Odeon, Brighton?
    @bensauer
    SYSTEM HAS 50% CONFIDENCE:
    When would you like to go to 

    the Odeon Brighton?
    IMPLICIT:

    NONE:

    View Slide

  165. Human, but not *too* human.

    View Slide

  166. View Slide

  167. RESOURCES
    • Cathy Pearl’s VUI book
    • Wizard of Oz Testing:

    https://chatbotsmagazine.com/stfu-test-your-
    voice-app-idea-in-less-than-an-
    hour-2638cdff2503
    • Say Wizard:

    https://github.com/bensauer/saywizard

    View Slide

  168. MARC WEISER
    “The most profound technologies are
    those that disappear.
    They weave themselves into the
    fabric of everyday life until they are
    indistinguishable from it.”
    @bensauer

    View Slide

  169. Questions?
    Get in touch!
    [email protected]
    @bensauer

    View Slide

  170. Give me feedback
    As a team, write two words on post-
    its that describe your workshop
    experience today and leave them
    behind on your table.

    View Slide