Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Designing your first voice interface (workshop)

bensauer
June 09, 2017

Designing your first voice interface (workshop)

Learn about the voice design process, get busy crafting dialogue, learn some fundamental principles, design and test and voice interface in a team. You'll see just how different the design methods are, and what the 'Wizard of Oz' testing method is. Come find your inner bot!

bensauer

June 09, 2017
Tweet

More Decks by bensauer

Other Decks in Design

Transcript

  1. Design
    your first
    voice interface
    Product Camp Poland 2017 • Ben Sauer

    View full-size slide

  2. Organise

    into
    teams of 4
    *each team needs a MacBook
    *your packet has more instructions

    View full-size slide

  3. A spectrum 

    of responses

    View full-size slide

  4. The medium you
    design in is not the
    medium you
    experience

    View full-size slide

  5. Find out what
    doesn’t work, as
    well as what does

    View full-size slide

  6. “When is the next train to London Victoria?”
    Too concise: 

    20 minutes.

    View full-size slide

  7. “When is the next train to London Victoria?”
    Too detailed: 

    The next train from Brighton station to
    London Victoria station leaves in
    twenty minutes, at twenty past ten. It
    has no first class service and divides
    at Haywards Heath. It arrives at
    London Victoria at ten past eleven.

    View full-size slide

  8. “When is the next train to London Victoria?”
    Just right: 

    The next train from Brighton to
    London Victoria leaves in 20 minutes.

    View full-size slide

  9. VOICE INTERFACES:
    NOW AND BEYOND

    View full-size slide

  10. ARISTOTLE
    There is only one condition in which we can
    imagine managers not needing subordinates,
    and masters not needing slaves.
    This condition would be that each instrument
    could do its own work, at the word of
    command or by intelligent anticipation.
    @bensauer

    View full-size slide

  11. VUI = Voice User Interface
    ASR = Automatic Speech Recognition
    NLU = Natural Language Understanding
    @bensauer
    humans  and  
    machines  talking
    turning  speech  
    into  words
    turning  words  
    into  meaning

    View full-size slide

  12. Curious Rituals
    produced by The Near Future Laboratory

    View full-size slide

  13. Curious Rituals
    produced by The Near Future Laboratory

    View full-size slide

  14. • Limited shared vocabulary
    • Poor skills / co-ordination
    • No capacity to learn
    • Never sure job will get done
    • Turns you into the a$$h0le
    Using VUI is a bit like dealing with Manuel…

    View full-size slide

  15. https://www.flickr.com/photos/cryptik/23815673664 @bensauer
    ?

    View full-size slide

  16. Our true calling is to make 

    the system fail gracefully.

    View full-size slide

  17. REALITIES OF VUI

    View full-size slide

  18. VUI is deceptively limited.
    @bensauer

    View full-size slide

  19. @bensauer
    VUI USE MAY BE U-SHAPED
    @bensauer
    AWESOME
    SUCKS
    10 100 ∞
    USE CASES

    View full-size slide

  20. Platforms limit possibilities.
    @bensauer

    View full-size slide

  21. Transactions over conversations.
    @bensauer

    View full-size slide

  22. The context of use 

    is everything.

    View full-size slide

  23. WHERE IS 

    VUI GOING?

    View full-size slide

  24. The price of a multifunctional touchscreen 

    is the volume of required input.
    VUI is an efficient shortcut.
    @bensauer

    View full-size slide

  25. The short-term promise of VUI is
    to simplify complex interactions
    into directly expressed goals,
    without the distraction of a screen.
    @bensauer

    View full-size slide

  26. @bensauer
    BEYOND USE CASES
    “If the doorbell rings while I’m
    in the garden, call me.”
    @bensauer

    View full-size slide

  27. @bensauer
    @bensauer
    VOICE ASSISTANT CONCEPTS
    SIRI: ’FAMILIAR’ ALEXA: ’THING’
    http://www.dfki.de/LAMDa/2013/accepted/13_ApplinFischer.pdf

    View full-size slide

  28. @bensauer
    AMAZON’S PLAY FOR UBICOMP
    • TVs (Westinghouse, Element, Seiki)
    • Fridges (LG)
    • Home robots (LG)
    • Wifi speakers (Lenovo)
    • Kid’s speaker (Mattel)
    • Lamps (GE)
    @bensauer
    • Alarm clocks
    • Cars (Ford, VW)
    • Charging docks
    • Watches
    • Cameras
    • Smartphones (Huawei)

    View full-size slide

  29. @bensauer
    CONVERSATIONAL UI IS PRETTY HYPED
    @bensauer
    WTF!  menus!

    View full-size slide

  30. AI is getting too much hype.

    View full-size slide

  31. WHAT IS
    CONVERSATION?

    View full-size slide

  32. Context
    Language
    Exchange
    Agreement
    Transaction / Action

    View full-size slide

  33. from Paul Pangaro's ‘Conversation is more than interface’ https://vimeo.com/207866146

    View full-size slide

  34. WHAT’S DIFFERENT
    ABOUT VUI DESIGN?

    View full-size slide

  35. Recall over recognition.

    View full-size slide

  36. The first interaction 

    expresses the goal.

    View full-size slide

  37. GUI VUI
    THE USER JOURNEY

    View full-size slide

  38. The medium you design in

    is not the medium you experience.
    @bensauer

    View full-size slide

  39. Shorter ‘Time to Test’.

    View full-size slide

  40. A.I. is getting better at
    understanding so that 

    we don’t have to.

    View full-size slide

  41. The heart of VUI is great writing.
    @bensauer

    View full-size slide

  42. VUI OPPORTUNITIES

    View full-size slide

  43. @bensauer
    VUI APPS
    @bensauer

    View full-size slide

  44. @bensauer
    SEARCH RESULTS (IN APP)
    @bensauer

    View full-size slide

  45. @bensauer
    SIMPLE GOPHER TASKS
    @bensauer

    View full-size slide

  46. @bensauer
    PLAYING WITH AMAZON ECHO
    @bensauer
    Raspberry Pi + Image + Lambda + Node
    Sayspring

    View full-size slide

  47. Our focus
    Design, prototype,
    and test a voice
    interface

    View full-size slide

  48. Our focus
    Ignore the
    technology!

    View full-size slide

  49. Our focus
    Use a 

    wake word

    View full-size slide

  50. A great VUI starts
    with a great use case

    View full-size slide

  51. Speed,
    simplicity,
    convenience

    View full-size slide

  52. #1
    Answer off the top
    of your head

    View full-size slide

  53. What time is it?
    What’s the weather?
    Is there a conference
    dinner tonight?

    View full-size slide

  54. #2
    A way to avoid
    pulling out a phone or
    jotting notes on paper

    View full-size slide

  55. Set a timer for
    for 20 minutes
    Call Anna

    View full-size slide

  56. Order a pizza
    from Tramonti’s
    Remind me to make a
    dentist appointment
    tomorrow

    View full-size slide

  57. #3
    Help for someone
    with their eyes or
    hands occupied

    View full-size slide

  58. How long will I
    be in traffic?
    How many cups of
    sugar do I add?

    View full-size slide

  59. Alert people
    to helpful information
    Bonus

    View full-size slide

  60. Hey Google,
    what’s up?
    There’s a 40 minute
    delay on Highway 17

    View full-size slide

  61. Activity: 

    find a use case

    View full-size slide

  62. Step #1

    Pick a decider for today

    View full-size slide

  63. Step #2

    Write down activities you do
    most weeks
    One activity per post it
    Swimming Cooking Drive  to  work
    Take  kids  to  
    school

    View full-size slide

  64. Step #3

    Map your week as a group
    Commute Work Lunch Gym
    MONDAY
    Commute W
    TUESDAY

    View full-size slide

  65. Step 4: Write down questions or tasks
    that a voice interface could help with
    Drive  to  work
    Check  
    traffic
    Shopping
    Find  shelf  
    location
    Getting  
    dressed
    Check  
    weather
    Dinner
    Ordering  
    takeout

    View full-size slide

  66. Dinner
    Ordering  
    takeout
    Step 5: Vote for your favourites
    Drive  to  work
    Check  
    traffic
    Shopping
    Find  shelf  
    location
    Getting  
    dressed
    Check  
    weather





    View full-size slide

  67. Step #6
    Choose your idea

    View full-size slide

  68. Step #7
    Empathise
    Feeling  tired
    Hands  dirty
    Kids  
    screaming
    Make  
    healthy,  easy,  
    tasty  food
    Cooking  
    dinner

    View full-size slide

  69. Same voice,
    different roles

    View full-size slide

  70. Users hear
    personality,
    regardless

    View full-size slide

  71. Example: Cortana
    Confident, caring, competent,
    loyal
    Helpful, but not bossy
    Functionally and emotionally
    transparent

    View full-size slide

  72. Define personality

    View full-size slide

  73. Step #1

    Discuss and capture
    Page 4 of the handout

    View full-size slide

  74. Breaktime!
    Come back at 11

    View full-size slide

  75. What do people say first?
    Define personality

    View full-size slide

  76. What they say,

    and the way they
    say it

    View full-size slide

  77. Activity: 

    capture the first
    question

    View full-size slide

  78. Step #1

    Open a google doc
    Give it the following section headings:
    •Test Brief
    •Opening questions
    •Sample dialogue (1,2,3,4)

    View full-size slide

  79. Step #2

    Write a test brief

    View full-size slide

  80. Step #3

    Write a first sentence
    “Alexa,  find  me  a  steak  
    restaurant  in  Gdansk.”

    View full-size slide

  81. Step #4

    Choose roles
    •A moderator
    •A participant from another team

    View full-size slide

  82. Step #5

    Capture the first question

    View full-size slide

  83. Step #6

    Swap roles and repeat

    View full-size slide

  84. Step #7

    Swap roles and repeat

    View full-size slide

  85. Sample dialogue
    Define personality

    View full-size slide

  86. Dialogue format

    View full-size slide

  87. Expressing
    personality

    View full-size slide

  88. Find the best paths

    View full-size slide

  89. Exercise
    Creating dialogue

    View full-size slide

  90. Step #1

    Choose roles
    • A user
    • The voice interface
    • One or two transcribers

    View full-size slide

  91. Step #2

    Improvise!
    No more than six to ten 

    lines of dialogue

    View full-size slide

  92. Step #3

    Write your own

    View full-size slide

  93. Step #4

    Share and discuss

    View full-size slide

  94. WOZ demo time

    View full-size slide

  95. Lunch!
    See you at 13:30pm

    View full-size slide

  96. Exercise 6
    Editing dialogue

    View full-size slide

  97. Sound natural

    View full-size slide

  98. Be true
    to your personality

    View full-size slide

  99. Give people credit

    View full-size slide

  100. Be cooperative

    View full-size slide

  101. Keep it short
    What’s the
    temperature?

    View full-size slide

  102. Keep it short
    How old is
    Prince Harry?

    View full-size slide

  103. Prince Harry
    is 32 years old
    He’s 32 years old
    32

    View full-size slide

  104. #1 Use pronouns

    View full-size slide

  105. #2 Important
    information 

    at the end

    View full-size slide

  106. #3 Say everything 

    out loud

    View full-size slide

  107. Exercise: 

    editing dialogue

    View full-size slide

  108. Goal: 

    make it good, 

    make it testable

    View full-size slide

  109. Step #1

    Edit in pairs

    View full-size slide

  110. Step #2

    Identify patterns
    The  user  starts  
    the  recording  
    defining  the  
    duration
    The  user  starts  
    the  recording  
    *without*  defining  a  
    duration
    The  user  stops  the  
    recording
    The  user  plays  
    back  the  meeting
    Example: Recording meetings

    View full-size slide

  111. Step #3

    Setup ‘Say Wizard’
    see handout page 8
    tip: make text bigger with 

    Shift cmd +

    View full-size slide

  112. Step #4

    Copy dialogue into script.txt

    View full-size slide

  113. Step #5

    Remove the user’s lines from
    script.txt

    View full-size slide

  114. Step #6

    Reorder the dialogue

    View full-size slide

  115. First answer
    Second
    Third
    Fourth
    Second 

    scenario

    View full-size slide

  116. Step #6

    Reorder the dialogue
    tip: leave gaps

    View full-size slide

  117. Wizard of Oz
    Testing

    View full-size slide

  118. The goal:
    Having all the
    answers

    View full-size slide

  119. Fail fast,
    Fail cheap

    View full-size slide

  120. Amazon Echo
    Early design process

    View full-size slide

  121. The WOZ process

    View full-size slide

  122. Exercise
    Wizard of Oz testing

    View full-size slide

  123. Step #1

    Choose roles
    •A wizard
    •A participant from another team
    •Two observers

    View full-size slide

  124. Step #2

    Run the test

    View full-size slide

  125. What did you learn?

    View full-size slide

  126. Step #3

    Reflect & Refine

    View full-size slide

  127. Step #4

    Swap roles and test

    View full-size slide

  128. Step #5

    Reflect & Refine

    View full-size slide

  129. Final test

    Record a video

    View full-size slide

  130. How was it?

    Record a video

    View full-size slide

  131. Post-test
    survey
    Annoying Useful Classic Futuristic

    View full-size slide

  132. Dialogue Flow
    booking 

    a cab

    View full-size slide

  133. Exercise:
    Map the flow
    tip: don’t re-use the 

    dialogue as titles

    View full-size slide

  134. Build and test

    View full-size slide

  135. UTTERANCE
    “Order me some glow-in-the-dark toilet paper.”

    View full-size slide



  136. “Order me a Nintendo Switch.”
    INTENT
    Buy Product

    View full-size slide



  137. “Set a timer for 30 seconds.”
    VARIABLE
    Timer Duration

    View full-size slide

  138. VUI PRINCIPLES

    View full-size slide

  139. Always be capturing.

    View full-size slide

  140. Postel’s Law
    Be conservative in what you send,
    be liberal in what you accept.
    @bensauer

    View full-size slide

  141. I’d really like to visit Llanfairpwllgwyngyll-
    gogerychwyrndrobwllllantysiliogogogoch.
    @bensauer
    USER:
    In North Wales?
    ALEXA:

    View full-size slide

  142. Design for mundane realities.

    View full-size slide

  143. Pair Bluetooth!!!!
    @bensauer
    HEAVILY ACCENTED USER FOR THE 3RD TIME:
    I’m really sorry, I still don’t understand.
    Can I suggest using the Alexa app instead?
    ALEXA:

    View full-size slide

  144. Considerate confirmation.

    View full-size slide

  145. Did you mean Odeon, Brighton?
    @bensauer
    SYSTEM HAS 50% CONFIDENCE:
    When would you like to go to 

    the Odeon Brighton?
    IMPLICIT:

    NONE:

    View full-size slide

  146. Human, but not *too* human.

    View full-size slide

  147. RESOURCES
    • Cathy Pearl’s VUI book
    • Wizard of Oz Testing:

    https://chatbotsmagazine.com/stfu-test-your-
    voice-app-idea-in-less-than-an-
    hour-2638cdff2503
    • Say Wizard:

    https://github.com/bensauer/saywizard

    View full-size slide

  148. MARC WEISER
    “The most profound technologies are
    those that disappear.
    They weave themselves into the
    fabric of everyday life until they are
    indistinguishable from it.”
    @bensauer

    View full-size slide

  149. Questions?
    Get in touch!
    [email protected]
    @bensauer

    View full-size slide

  150. Give me feedback
    As a team, write two words on post-
    its that describe your workshop
    experience today and leave them
    behind on your table.

    View full-size slide