Lydia Christine Penkert - Beyond graphical UIs - Researching voice interaction with social robots

Researching voice interaction is challenging, as several factors such as the gender and appearance of a voice impact user behavior, as well as user expectations and current limitations of speech technology. In this talk, you will learn how to approach those challenges in UX Research, along with exemplary insights of my research on social robotics in public spaces, such as the use of gender-neutral voices, investigating the impact of appearances through replication studies or the use of Wizard-of-Oz settings.

Transcript

1. Concept of an Intuitive Human-Robot-Collaboration via Motion Tracking and Augmented Reality Dario Luipers and Anja Richert, Cologne Cobots Lab TH Köln, Cologne, Germany Beyond GUI: researching voice interaction with social robots Lydia Penkert
2. Agenda 1. Voice Interfaces 2. Challenges of UXR with Voice • Bias in Smart Assistens and Robots Design • UXR Methods with Voice • Novel and non-users 3. Q&A
3. About me Lydia Penkert Scientific Researcher & PhD candidate @University of Applied Science Cologne Freelancer UX Research & Agile Coaching Background: Cognitive and Media Science @University Duisburg- Essen & @University Sunshine Coast UX Researcher @ kaufland.de (e-Commerce)
4. Voice Interaction Market Value of US$ 2.9 Billion 16% growth expected (until 2026) Use spread across all age groups VixenLabs Voice Consumer Index 2021
5. Challenges UXR with Voice Voice and appearance elicit human characteristics 1 2 3 Users have no prior experience or refuse voice interaction Established methods from UXR are not suitable
6. Challenges UXR with Voice Voice and appearance elicit human characteristics 1 2 3 Users have no prior experience or refuse voice interaction Established methods from UXR are not suitable
7. Imagine Siri, Cortana or Alexa as a person: How do they look?
8. Gender bias Voice assistants are per default female West, Kraut & Chew, 2019
9. Reason? We want technology to help us, while remaining in control Nass & Brave, 2005; West, Kraut & Chew, 2019 Consequence? Gendered design could solidify harmful gender stereotypes and mirror this behavior in everyday conversations
10. Gender in Voice Can a voice be neutral or even genderless? Q, 2019 Gender – ambiguous
11. Social robots
12. Appearance Method Interview with pictures Quotes “What is he doing with his hands?” “Looks like Mike from Monster Inc”
14. Racial bias Most robots currently being sold are either stylized with white material or have a metallic appearance People perceive that robots have a race Bartneck et al.,2018; Addison, Bartneck & Yogeeswaran, 2019
16. Gender bias Recent research also shows gap in perceived gender Perugia et al., 2022
19. Context Do you behave the same in a public space than at your home? Voice assistent usage is 38% lower in public spaces *VixenLabs Voice Consumer Index 2021
20. Context Method Video online survey with different context conditions Research Question How does crowd density impact users perception of the robot?
21. Interaction happens in a context with a UI which has an (perceived) appearence and gender which shape user expectations and behavior.
22. Where is your product used? Is your product female or male?
23. Challenges UXR with Voice Voice and appearance elicit human characteristics 1 2 3 Users have no prior experience or refuse voice interaction Established methods from UXR are not suitable
24. Usability test • Focus on non-verbal behaviour • Choose wordings carefully • Exact sentences are difficult to remember
25. Field observation • Research in incognito modus • Observe users (while respecting privacy)!
26. Field observation • Research in incognito modus • Observe users (while respecting privacy)!
27. Voice data • Analyse user behaviour without Hawthron effect • Replicate conversations Pokemon go = „Hello Jo“
28. • (Video/audio) - surveys with different conditions • Focus on non-verbal behavior • Field observation in incognito • Data analysis
29. Challenges UXR with Voice Voice and appearance elicit human characteristics 1 2 3 Users have no prior experience or refuse voice interaction Established methods from UXR are not suitable
30. No experience or rejection • Schedule „voice training“ sessions to avoid bias through novelty effect • Understand in-depth motivation behind rejection Great opportunity for foundational research and product discovery!
31. Augustin et al., 2021 Non-user personas • Traditional persona extended with: • Product criticism • Reasons for non-use • Resistance level
32. Take home message Voice is attributed to human characteristics.
33. Take home message Voice is attributed to human characteristics. Consider gender, appearance and context of your product.
34. Take home message Voice is attributed to human characteristics. Consider gender, appearance and context of your product. Combine non-verbal and verbal behavior.
35. Take home message Voice is attributed to human characteristics. Consider gender, appearance and context of your product. Combine non-verbal and verbal behavior. Include everyone in your research.
36. Sources Addison, A., Bartneck, C., & Yogeeswaran, K. (2019, January). Robots can be more than black and white: examining racial bias towards robots. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (pp. 493-498). Augustin, L., Kokoschko, B., Wiesner, M., & Schabacker, M. (2020, May). Toward a comprehensive definition of the non-user. In Proceedings of the Design Society: DESIGN Conference (Vol. 1, pp. 1853-1862). Cambridge University Press. Augustin, L., Pfrang, S., Wolffram, A., & Beyer, C. (2021). The value of the non-user: developing (non-) user profiles for the development of a robot vacuum with the use of the (non-) persona. Proceedings of the Design Society, 1, 3131-3140. Bartneck, C., Yogeeswaran, K., Ser, Q. M., Woodward, G., Sparrow, R., Wang, S., & Eyssel, F. (2018, February). Robots and racism. In Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction (pp. 196-204). Nass, C. I., & Brave, S. (2005). Wired for speech: How voice activates and advances the human-computer relationship (p. 9). Cambridge: MIT press. Perugia, G., Guidi, S., Bicchi, M., & Parlangeli, O. (2022, March). The Shape of Our Bias: Perceived Age and Gender in the Humanoid Robots of the ABOT Database. In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction (pp. 110-119). West, M., Kraut, R., Chew, H.E.: I’d blush if I could: closing gender divides in digital skills through education (2019). Q. 2019. The First Genderless Voice. 2019. Meet Q: The First Genderless Voice FULL SPEECH.- https://www.youtube.com/watch?v=jasEIteA3Ag VixenLabs 2021. https://vixenlabs.co/wp-content/uploads/vci2021email/VixenLabs-VoiceConsumerIndex2021-WhitePaper.pdf https://nl.pinterest.com/pin/428334614538814300/