Designing your first voice interface (workshop)

Learn about the voice design process, get busy crafting dialogue, learn some fundamental principles, design and test and voice interface in a team. You'll see just how different the design methods are, and what the 'Wizard of Oz' testing method is. Come find your inner bot!

Transcript

1. Design
   your first
   voice interface
   Product Camp Poland 2017 • Ben Sauer
   

   View Slide

2. Organise

   into
   teams of 4
   *each team needs a MacBook
   *your packet has more instructions
   

   View Slide

3. Welcome
   

   View Slide

4. Today
   

   View Slide

5. Warmup
   

   View Slide

6. A spectrum 

   of responses
   

   View Slide

7. The medium you
   design in is not the
   medium you
   experience
   

   View Slide

8. Find out what
   doesn’t work, as
   well as what does
   

   View Slide

9. “When is the next train to London Victoria?”
   Too concise: 

   20 minutes.
   

   View Slide

10. “When is the next train to London Victoria?”
    Too detailed: 

    The next train from Brighton station to
    London Victoria station leaves in
    twenty minutes, at twenty past ten. It
    has no first class service and divides
    at Haywards Heath. It arrives at
    London Victoria at ten past eleven.
    

    View Slide

11. “When is the next train to London Victoria?”
    Just right: 

    The next train from Brighton to
    London Victoria leaves in 20 minutes.
    

    View Slide

12. VOICE INTERFACES:
    NOW AND BEYOND
    

    View Slide

13. ARISTOTLE
    There is only one condition in which we can
    imagine managers not needing subordinates,
    and masters not needing slaves.
    This condition would be that each instrument
    could do its own work, at the word of
    command or by intelligent anticipation.
    @bensauer
    

    View Slide

14. VUI = Voice User Interface
    ASR = Automatic Speech Recognition
    NLU = Natural Language Understanding
    @bensauer
    humans  and  
    machines  talking
    turning  speech  
    into  words
    turning  words  
    into  meaning
    

    View Slide

15. View Slide

16. Curious Rituals
    produced by The Near Future Laboratory
    

    View Slide

17. View Slide

18. Curious Rituals
    produced by The Near Future Laboratory
    

    View Slide

19. View Slide

20. • Limited shared vocabulary
    • Poor skills / co-ordination
    • No capacity to learn
    • Never sure job will get done
    • Turns you into the a$$h0le
    Using VUI is a bit like dealing with Manuel…
    

    View Slide

21. https://www.flickr.com/photos/cryptik/23815673664 @bensauer
    ?
    

    View Slide

22. Our true calling is to make 

    the system fail gracefully.
    

    View Slide

23. REALITIES OF VUI
    

    View Slide

24. VUI is deceptively limited.
    @bensauer
    

    View Slide

25. @bensauer
    VUI USE MAY BE U-SHAPED
    @bensauer
    AWESOME
    SUCKS
    10 100 ∞
    USE CASES
    

    View Slide

26. View Slide

27. Platforms limit possibilities.
    @bensauer
    

    View Slide

28. Transactions over conversations.
    @bensauer
    

    View Slide

29. The context of use 

    is everything.
    

    View Slide

30. WHERE IS 

    VUI GOING?
    

    View Slide

31. The price of a multifunctional touchscreen 

    is the volume of required input.
    VUI is an efficient shortcut.
    @bensauer
    

    View Slide

32. The short-term promise of VUI is
    to simplify complex interactions
    into directly expressed goals,
    without the distraction of a screen.
    @bensauer
    

    View Slide

33. @bensauer
    BEYOND USE CASES
    “If the doorbell rings while I’m
    in the garden, call me.”
    @bensauer
    

    View Slide

34. View Slide

35. View Slide

36. View Slide

37. @bensauer
    @bensauer
    VOICE ASSISTANT CONCEPTS
    SIRI: ’FAMILIAR’ ALEXA: ’THING’
    http://www.dfki.de/LAMDa/2013/accepted/13_ApplinFischer.pdf
    

    View Slide

38. @bensauer
    AMAZON’S PLAY FOR UBICOMP
    • TVs (Westinghouse, Element, Seiki)
    • Fridges (LG)
    • Home robots (LG)
    • Wifi speakers (Lenovo)
    • Kid’s speaker (Mattel)
    • Lamps (GE)
    @bensauer
    • Alarm clocks
    • Cars (Ford, VW)
    • Charging docks
    • Watches
    • Cameras
    • Smartphones (Huawei)
    

    View Slide

39. @bensauer
    CONVERSATIONAL UI IS PRETTY HYPED
    @bensauer
    WTF!  menus!
    

    View Slide

40. AI is getting too much hype.
    

    View Slide

41. WHAT IS
    CONVERSATION?
    

    View Slide

42. Context
    Language
    Exchange
    Agreement
    Transaction / Action
    

    View Slide

43. from Paul Pangaro's ‘Conversation is more than interface’ https://vimeo.com/207866146
    

    View Slide

44. WHAT’S DIFFERENT
    ABOUT VUI DESIGN?
    

    View Slide

45. Recall over recognition.
    

    View Slide

46. The first interaction 

    expresses the goal.
    

    View Slide

47. GUI VUI
    THE USER JOURNEY
    

    View Slide

48. The medium you design in

    is not the medium you experience.
    @bensauer
    

    View Slide

49. Shorter ‘Time to Test’.
    

    View Slide

50. A.I. is getting better at
    understanding so that 

    we don’t have to.
    

    View Slide

51. The heart of VUI is great writing.
    @bensauer
    

    View Slide

52. VUI OPPORTUNITIES
    

    View Slide

53. @bensauer
    VUI APPS
    @bensauer
    

    View Slide

54. @bensauer
    SEARCH RESULTS (IN APP)
    @bensauer
    

    View Slide

55. @bensauer
    SIMPLE GOPHER TASKS
    @bensauer
    

    View Slide

56. @bensauer
    PLAYING WITH AMAZON ECHO
    @bensauer
    Raspberry Pi + Image + Lambda + Node
    Sayspring
    

    View Slide

57. Our focus
    Design, prototype,
    and test a voice
    interface
    

    View Slide

58. Our focus
    Ignore the
    technology!
    

    View Slide

59. Our focus
    Use a 

    wake word
    

    View Slide

60. Use cases
    s
    

    View Slide

61. A great VUI starts
    with a great use case
    

    View Slide

62. Speed,
    simplicity,
    convenience
    

    View Slide

63. #1
    Answer off the top
    of your head
    

    View Slide

64. What time is it?
    What’s the weather?
    Is there a conference
    dinner tonight?
    

    View Slide

65. #2
    A way to avoid
    pulling out a phone or
    jotting notes on paper
    

    View Slide

66. Set a timer for
    for 20 minutes
    Call Anna
    

    View Slide

67. Order a pizza
    from Tramonti’s
    Remind me to make a
    dentist appointment
    tomorrow
    

    View Slide

68. #3
    Help for someone
    with their eyes or
    hands occupied
    

    View Slide

69. How long will I
    be in traffic?
    How many cups of
    sugar do I add?
    

    View Slide

70. Alert people
    to helpful information
    Bonus
    

    View Slide

71. Hey Google,
    what’s up?
    There’s a 40 minute
    delay on Highway 17
    

    View Slide

72. Activity: 

    find a use case
    

    View Slide

73. Step #1

    Pick a decider for today
    

    View Slide

74. Step #2

    Write down activities you do
    most weeks
    One activity per post it
    Swimming Cooking Drive  to  work
    Take  kids  to  
    school
    

    View Slide

75. Step #3

    Map your week as a group
    Commute Work Lunch Gym
    MONDAY
    Commute W
    TUESDAY
    

    View Slide

76. Step 4: Write down questions or tasks
    that a voice interface could help with
    Drive  to  work
    Check  
    traffic
    Shopping
    Find  shelf  
    location
    Getting  
    dressed
    Check  
    weather
    Dinner
    Ordering  
    takeout
    

    View Slide

77. Dinner
    Ordering  
    takeout
    Step 5: Vote for your favourites
    Drive  to  work
    Check  
    traffic
    Shopping
    Find  shelf  
    location
    Getting  
    dressed
    Check  
    weather
    ✓
    ✓
    ✓
    ✓
    ✓
    

    View Slide

78. Step #6
    Choose your idea
    

    View Slide

79. Step #7
    Empathise
    Feeling  tired
    Hands  dirty
    Kids  
    screaming
    Make  
    healthy,  easy,  
    tasty  food
    Cooking  
    dinner
    

    View Slide

80. Personality
    

    View Slide

81. Same voice,
    different roles
    

    View Slide

82. Users hear
    personality,
    regardless
    

    View Slide

83. Example: Cortana
    Confident, caring, competent,
    loyal
    Helpful, but not bossy
    Functionally and emotionally
    transparent
    

    View Slide

84. Define personality
    

    View Slide

85. Step #1

    Discuss and capture
    Page 4 of the handout
    

    View Slide

86. Breaktime!
    Come back at 11
    

    View Slide

87. What do people say first?
    Define personality
    

    View Slide

88. What they say,

    and the way they
    say it
    

    View Slide

89. View Slide

90. View Slide

91. View Slide

92. View Slide

93. Activity: 

    capture the first
    question
    

    View Slide

94. Step #1

    Open a google doc
    Give it the following section headings:
    •Test Brief
    •Opening questions
    •Sample dialogue (1,2,3,4)
    

    View Slide

95. Step #2

    Write a test brief
    

    View Slide

96. Step #3

    Write a first sentence
    “Alexa,  find  me  a  steak  
    restaurant  in  Gdansk.”
    

    View Slide

97. Step #4

    Choose roles
    •A moderator
    •A participant from another team
    

    View Slide

98. Step #5

    Capture the first question
    

    View Slide

99. Step #6

    Swap roles and repeat
    

    View Slide

100. Step #7

     Swap roles and repeat
     

     View Slide

101. Sample dialogue
     Define personality
     

     View Slide

102. Dialogue format
     

     View Slide

103. Expressing
     personality
     

     View Slide

104. Find the best paths
     

     View Slide

105. Exercise
     Creating dialogue
     

     View Slide

106. Step #1

     Choose roles
     • A user
     • The voice interface
     • One or two transcribers
     

     View Slide

107. Step #2

     Improvise!
     No more than six to ten 

     lines of dialogue
     

     View Slide

108. Step #3

     Write your own
     

     View Slide

109. Step #4

     Share and discuss
     

     View Slide

110. WOZ demo time
     

     View Slide

111. Lunch!
     See you at 13:30pm
     

     View Slide

112. Exercise 6
     Editing dialogue
     

     View Slide

113. Principles
     

     View Slide

114. Sound natural
     

     View Slide

115. Be true
     to your personality
     

     View Slide

116. Give people credit
     

     View Slide

117. Be cooperative
     

     View Slide

118. Keep it short
     What’s the
     temperature?
     

     View Slide

119. Keep it short
     How old is
     Prince Harry?
     

     View Slide

120. Prince Harry
     is 32 years old
     He’s 32 years old
     32
     

     View Slide

121. Take turns
     

     View Slide

122. Bonus tips
     

     View Slide

123. #1 Use pronouns
     

     View Slide

124. #2 Important
     information 

     at the end
     

     View Slide

125. #3 Say everything 

     out loud
     

     View Slide

126. Exercise: 

     editing dialogue
     

     View Slide

127. Goal: 

     make it good, 

     make it testable
     

     View Slide

128. Step #1

     Edit in pairs
     

     View Slide

129. Step #2

     Identify patterns
     The  user  starts  
     the  recording  
     defining  the  
     duration
     The  user  starts  
     the  recording  
     *without*  defining  a  
     duration
     The  user  stops  the  
     recording
     The  user  plays  
     back  the  meeting
     Example: Recording meetings
     

     View Slide

130. Step #3

     Setup ‘Say Wizard’
     see handout page 8
     tip: make text bigger with 

     Shift cmd +
     

     View Slide

131. Step #4

     Copy dialogue into script.txt
     

     View Slide

132. Step #5

     Remove the user’s lines from
     script.txt
     

     View Slide

133. Step #6

     Reorder the dialogue
     

     View Slide

134. First answer
     Second
     Third
     Fourth
     Second 

     scenario
     

     View Slide

135. Step #6

     Reorder the dialogue
     tip: leave gaps
     

     View Slide

136. Wizard of Oz
     Testing
     

     View Slide

137. The goal:
     Having all the
     answers
     

     View Slide

138. Fail fast,
     Fail cheap
     

     View Slide

139. Amazon Echo
     Early design process
     

     View Slide

140. The WOZ process
     

     View Slide

141. Exercise
     Wizard of Oz testing
     

     View Slide

142. Step #1

     Choose roles
     •A wizard
     •A participant from another team
     •Two observers
     

     View Slide

143. Step #2

     Run the test
     

     View Slide

144. What did you learn?
     

     View Slide

145. Step #3

     Reflect & Refine
     

     View Slide

146. Step #4

     Swap roles and test
     

     View Slide

147. Step #5

     Reflect & Refine
     

     View Slide

148. Final test

     Record a video
     

     View Slide

149. How was it?

     Record a video
     

     View Slide

150. Post-test
     survey
     Annoying Useful Classic Futuristic
     

     View Slide

151. Dialogue Flow
     booking 

     a cab
     

     View Slide

152. Exercise:
     Map the flow
     tip: don’t re-use the 

     dialogue as titles
     

     View Slide

153. Build and test
     

     View Slide

154. UTTERANCE
     “Order me some glow-in-the-dark toilet paper.”
     

     View Slide

155. 

     

     “Order me a Nintendo Switch.”
     INTENT
     Buy Product
     

     View Slide

156. 

     

     “Set a timer for 30 seconds.”
     VARIABLE
     Timer Duration
     

     View Slide

157. VUI PRINCIPLES
     

     View Slide

158. Always be capturing.
     

     View Slide

159. Postel’s Law
     Be conservative in what you send,
     be liberal in what you accept.
     @bensauer
     

     View Slide

160. I’d really like to visit Llanfairpwllgwyngyll-
     gogerychwyrndrobwllllantysiliogogogoch.
     @bensauer
     USER:
     In North Wales?
     ALEXA:
     

     View Slide

161. Design for mundane realities.
     

     View Slide

162. Pair Bluetooth!!!!
     @bensauer
     HEAVILY ACCENTED USER FOR THE 3RD TIME:
     I’m really sorry, I still don’t understand.
     Can I suggest using the Alexa app instead?
     ALEXA:
     

     View Slide

163. Considerate confirmation.
     

     View Slide

164. Did you mean Odeon, Brighton?
     @bensauer
     SYSTEM HAS 50% CONFIDENCE:
     When would you like to go to 

     the Odeon Brighton?
     IMPLICIT:
     
     NONE:
     

     View Slide

165. Human, but not *too* human.
     

     View Slide

166. View Slide

167. RESOURCES
     • Cathy Pearl’s VUI book
     • Wizard of Oz Testing:

     https://chatbotsmagazine.com/stfu-test-your-
     voice-app-idea-in-less-than-an-
     hour-2638cdff2503
     • Say Wizard:

     https://github.com/bensauer/saywizard
     

     View Slide

168. MARC WEISER
     “The most profound technologies are
     those that disappear.
     They weave themselves into the
     fabric of everyday life until they are
     indistinguishable from it.”
     @bensauer
     

     View Slide

169. Questions?
     Get in touch!
     ben@clearleft.com
     @bensauer
     

     View Slide

170. Give me feedback
     As a team, write two words on post-
     its that describe your workshop
     experience today and leave them
     behind on your table.
     

     View Slide