Lydia Christine Penkert - Beyond graphical UIs - Researching voice interaction with social robots

Researching voice interaction is challenging, as several factors such as the gender and appearance of a voice impact user behavior, as well as user expectations and current limitations of speech technology. In this talk, you will learn how to approach those challenges in UX Research, along with exemplary insights of my research on social robotics in public spaces, such as the use of gender-neutral voices, investigating the impact of appearances through replication studies or the use of Wizard-of-Oz settings.

Transcript

1. Concept of an Intuitive Human-Robot-Collaboration via Motion Tracking and Augmented Reality
   Dario Luipers and Anja Richert, Cologne Cobots Lab
   TH Köln, Cologne, Germany
   Beyond GUI: researching voice
   interaction with social robots
   Lydia Penkert
   

   View Slide

2. Agenda
   1. Voice Interfaces
   2. Challenges of UXR with Voice
   • Bias in Smart Assistens and Robots Design
   • UXR Methods with Voice
   • Novel and non-users
   3. Q&A
   

   View Slide

3. About me
   Lydia Penkert
   Scientific Researcher & PhD candidate
   @University of Applied Science Cologne
   Freelancer UX Research & Agile Coaching
   Background:
   Cognitive and Media Science
   @University Duisburg- Essen & @University Sunshine Coast
   UX Researcher
   @ kaufland.de (e-Commerce)
   

   View Slide

4. Voice Interaction
   Market Value of US$ 2.9 Billion
   16% growth expected (until 2026)
   Use spread across all age
   groups
   VixenLabs Voice Consumer Index 2021
   

   View Slide

5. Challenges UXR with Voice
   Voice and appearance elicit human characteristics
   1
   2
   3 Users have no prior experience or refuse voice interaction
   Established methods from UXR are not suitable
   

   View Slide

6. Challenges UXR with Voice
   Voice and appearance elicit human characteristics
   1
   2
   3 Users have no prior experience or refuse voice interaction
   Established methods from UXR are not suitable
   

   View Slide

7. Imagine Siri, Cortana or Alexa as a person:
   How do they look?
   

   View Slide

8. Gender bias
   Voice assistants are per
   default female
   West, Kraut & Chew, 2019
   

   View Slide

9. Reason?
   We want technology to help
   us, while remaining in
   control
   Nass & Brave, 2005; West, Kraut & Chew, 2019
   Consequence?
   Gendered design could solidify
   harmful gender stereotypes and
   mirror this behavior in everyday
   conversations
   

   View Slide

10. Gender in Voice
    Can a voice be neutral or
    even genderless?
    Q, 2019
    Gender – ambiguous
    

    View Slide

11. Social robots
    

    View Slide

12. Appearance
    Method
    Interview with pictures
    Quotes
    “What is he doing with his hands?”
    “Looks like Mike from Monster Inc”
    

    View Slide

13. View Slide

14. Racial bias
    Most robots currently being
    sold are either stylized with
    white material or have a
    metallic appearance
    People perceive that robots
    have a race
    Bartneck et al.,2018;
    Addison, Bartneck & Yogeeswaran, 2019
    

    View Slide

15. View Slide

16. Gender bias
    Recent research also shows
    gap in perceived gender
    Perugia et al., 2022
    

    View Slide

17. View Slide

18. View Slide

19. Context
    Do you behave the same in a
    public space than at your
    home?
    Voice assistent usage is 38%
    lower in public spaces
    *VixenLabs Voice Consumer Index 2021
    

    View Slide

20. Context
    Method
    Video online survey with
    different context conditions
    Research Question
    How does crowd density
    impact users perception of
    the robot?
    

    View Slide

21. Interaction happens in a context with a UI which has an
    (perceived) appearence and gender which shape user
    expectations and behavior.
    

    View Slide

22. Where is your product used?
    Is your product female or
    male?
    

    View Slide

23. Challenges UXR with Voice
    Voice and appearance elicit human characteristics
    1
    2
    3 Users have no prior experience or refuse voice interaction
    Established methods from UXR are not suitable
    

    View Slide

24. Usability test
    • Focus on non-verbal
    behaviour
    • Choose wordings carefully
    • Exact sentences are difficult
    to remember
    

    View Slide

25. Field observation
    • Research in incognito modus
    • Observe users (while
    respecting privacy)!
    

    View Slide

26. Field observation
    • Research in incognito modus
    • Observe users (while
    respecting privacy)!
    

    View Slide

27. Voice data
    • Analyse user behaviour
    without Hawthron effect
    • Replicate conversations
    Pokemon go = „Hello Jo“
    

    View Slide

28. • (Video/audio) - surveys with different conditions
    • Focus on non-verbal behavior
    • Field observation in incognito
    • Data analysis
    

    View Slide

29. Challenges UXR with Voice
    Voice and appearance elicit human characteristics
    1
    2
    3 Users have no prior experience or refuse voice interaction
    Established methods from UXR are not suitable
    

    View Slide

30. No experience or rejection
    • Schedule „voice training“
    sessions to avoid bias through
    novelty effect
    • Understand in-depth motivation
    behind rejection
    Great opportunity for foundational
    research and product discovery!
    

    View Slide

31. Augustin et al., 2021
    Non-user personas
    • Traditional persona
    extended with:
    • Product criticism
    • Reasons for non-use
    • Resistance level
    

    View Slide

32. Take home message
    Voice is attributed to human characteristics.
    

    View Slide

33. Take home message
    Voice is attributed to human characteristics.
    Consider gender, appearance and context of your product.
    

    View Slide

34. Take home message
    Voice is attributed to human characteristics.
    Consider gender, appearance and context of your product.
    Combine non-verbal and verbal behavior.
    

    View Slide

35. Take home message
    Voice is attributed to human characteristics.
    Consider gender, appearance and context of your product.
    Combine non-verbal and verbal behavior.
    Include everyone in your research.
    

    View Slide

36. Sources
    Addison, A., Bartneck, C., & Yogeeswaran, K. (2019, January). Robots can be more than black and white: examining racial bias
    towards robots. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (pp. 493-498).
    Augustin, L., Kokoschko, B., Wiesner, M., & Schabacker, M. (2020, May). Toward a comprehensive definition of the non-user.
    In Proceedings of the Design Society: DESIGN Conference (Vol. 1, pp. 1853-1862). Cambridge University Press.
    Augustin, L., Pfrang, S., Wolffram, A., & Beyer, C. (2021). The value of the non-user: developing (non-) user profiles for the
    development of a robot vacuum with the use of the (non-) persona. Proceedings of the Design Society, 1, 3131-3140.
    Bartneck, C., Yogeeswaran, K., Ser, Q. M., Woodward, G., Sparrow, R., Wang, S., & Eyssel, F. (2018, February). Robots and racism.
    In Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction (pp. 196-204).
    Nass, C. I., & Brave, S. (2005). Wired for speech: How voice activates and advances the human-computer relationship (p. 9).
    Cambridge: MIT press.
    Perugia, G., Guidi, S., Bicchi, M., & Parlangeli, O. (2022, March). The Shape of Our Bias: Perceived Age and Gender in the Humanoid
    Robots of the ABOT Database. In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction (pp.
    110-119).
    West, M., Kraut, R., Chew, H.E.: I’d blush if I could: closing gender divides in digital skills through education (2019).
    Q. 2019. The First Genderless Voice. 2019. Meet Q: The First Genderless Voice FULL SPEECH.-
    https://www.youtube.com/watch?v=jasEIteA3Ag
    VixenLabs 2021. https://vixenlabs.co/wp-content/uploads/vci2021email/VixenLabs-VoiceConsumerIndex2021-WhitePaper.pdf
    https://nl.pinterest.com/pin/428334614538814300/
    

    View Slide