Designing a Robot-based Customer Feedback Gathering Service for Hospitality Industry ... and what are the HRI community’s recommendations on designing robotic services?

workshop: https://sites.google.com/view/lsi-hri/home/
google slides: https://docs.google.com/presentation/d/1VY1IZ9D-hrjn-D8cvgSOFIlG6jCh_YCMxolmh0IQ_ak/

Transcript

1. Designing a Robot-based Customer Feedback Gathering Service for Hospitality Industry ... and what are the HRI community’s recommendations on designing robotic services? Michael Jae-Yoon Chung, 2022/03/08
2. Mike’s project in grad school Methods from Human-Robot-Interaction Agenda
3. Mike’s project in grad school Methods from Human-Robot-Interaction Agenda
5. Research Questions 1. Can we use robots to gather feedback from guests? 2. How should we design robot behaviors to gather guest feedback effectively? "How was your stay?": Exploring the Use of Robots for Gathering Customer Feedback in the Hospitality Industry (Chung & Cakmak, 2018)
6. Approach 1. Need finding interviews at 5 hotels 2. Online guest scenario survey (n=60) 3. Comparison study of 3 different behaviors 4. Deployment study at 3 hotels (~4 months) A method inspired by sequential exploratory method (creswell, 2003) but wasn’t as (academically) rigorous User Research Evaluation "How was your stay?": Exploring the Use of Robots for Gathering Customer Feedback in the Hospitality Industry (Chung & Cakmak, 2018)
7. 1. Need Finding Interviews What is Need Finding? A product design tool Semi-structured interview & observations at interviewees’ workplace See Pantofaru and Takayama 2011 for more Participants 5 hotel employees Had administrative jobs, used Relay 4 San Francisco Bay area, 1 LA hotels Median # of rooms: 175 [62, 304]
9. Image source: https://www.ihg.com/crowneplaza/hotels/gb/en/milpitas/sfomp/hoteldetail/photos-tours Interview Protocol 1. Introduction 2. Warm-up questions 3. Current Practices for Collecting Guest Feedback 4. Your Experiences with Relay 5. Your Opinions on Collecting Guest Feedback with Relay 6. Tour 7. Wrap-up
10. Need Finding: Analysis & Results Image source: https://www.ihg.com/crowneplaza/hotels/gb/en/milpitas/sfomp/hoteldetail/photos-tours Analysis Method Audio recorded interviews & transcribed them Inductive content analysis to identify themes Discovered Themes 1. Existing Methods 2. Service Recovery Strategies 3. Participants’ Opinions 4. Factors Affecting Robot Usage
11. 2. Online Guest Scenario Survey Goal To understand the effect of a customer feedback collection method on guest’s willingness to respond. Scenario-based Survey Design 60 participants from Amazon MTurk A positive or negative scenario 4 feedback solicitation conditions 1. Email 2. Kiosk 3. Guest Room Delivery Robot 4. Front Desk Robot 5-point Likert scale question Open-ended question
12. 2. Online Guest Scenario Survey: Result
13. 2. Online Guest Scenario Survey: Result “It sounds easy and kind of cool.” “It is too rude to not respond to this cute robot.” “A feedback robot? That's just weird, the world isn't ready for that.” “I wouldn't trust the robot.”
14. 2. Online Guest Scenario Survey: Result Perception of the robot solicitations (In)convenience Privacy issues “Less likely, but I’d feel the staff might be watching over my shoulder.” “Checking out. I just want to leave and get home.”
15. Goal To understand the effect of (1) the environment and (2) the robot behavior on response rate. Conditions 3 times a week for 3 weeks • Monday Breakfast (Mon BF) • Monday Lunch (Mon LU) • Wednesday Breakfast (Wed BF) 3 behaviors • Default • Social • Mobile 3. Robot Behavior Comparison Study
17. Record & annotated videos System logs Customer opinion questionnaire • sent after each session • 4 5-point scale Likert questions • 1 open-ended question 3. Robot Behavior Comparison Study: Data Collection
18. Robot Behavior Week # Mon LU Meal Type WED LU 3. Robot Behavior Comparison Study: Result
19. 4. Deployment Study Goal To understand the value that the robot added, interaction patterns, and real-world challenges in gathering feedback from customers with robots Field Study Design Prototyped with UX designer & hotel manager Passive observations & follow-up interviews during 2 visit days
20. Video
22. 4. Deployment Study: Findings Leverage environmental factors Keep it “simple and fun” UI for customizing behavior by the hotel Follow-up or explain • but respect the guest privacy first Consider the existing workflow & overall changes introduced by a robot
23. Lessons Learned Start from the hardware, if possible, and be honest Leverage user research, product design tools properly • e.g., need to asking right questions to right people Do cheaper, lighter design iterations as much as possible • e.g., storyboarding, WoZ, HRI literature search Do scoping, setting/leveraging the context, defining variables as much as possible • e.g., might be the factor that distinguishes a robot-based service from others
24. Mike’s project in grad school Methods from Human-Robot-Interaction Agenda
25. What is Human-Robot Interaction research about? How is HRI different from human-computer interaction (HCI)? • HCI involving robots? ◦ What is robots? ◦ Agency & intentionality? • What does that imply for studying HRI? ◦ Should robots exploit being perceived as agents? From https://homes.cs.washington.edu/~todorov/courses/cseP590/07_HRI.pdf
26. Methods Discovery • pre-design (formative) Evaluation • during/post-design (summative) From https://homes.cs.washington.edu/~todorov/courses/cseP590/07_HRI.pdf
27. Methods Asking users • Questionnaires, interviews, focus groups, contextual inquiry Observing users • Passive observation, empirical user studies, think-aloud protocol, ethnography, field studies Make users observe themselves • Diaries, experience sampling Ask experts • Heuristic evaluation, cognitive walkthrough From https://homes.cs.washington.edu/~todorov/courses/cseP590/07_HRI.pdf
28. How to use HRI research in industry Apply user research framework • Exploring the role of robots in home organization Take a look at findings • Robots in organizations: the role of workflow, social, and environmental factors in human-robot interaction • "Now, I Have a Body": Uses and Social Norms for Mobile Remote Presence in the Workplace • Most HRI papers focus on specific questions, contexts, … be adaptive!
29. Apply design/evaluation framework • Design Methodology for the UX of HRI: A Field Study of a Commercial Social Robot at an Airport • Enabling building service robots to guide blind people a participatory design approach • Mechanical ottoman: how robotic furniture offers and withdraws support • Which robot behavior can motivate children to tidy up their toys? Design and Evaluation of "Ranger" • … How to use HRI research in industry
30. When checking out HRI research, • see if you can take the framework, methodology • take findings adoptively • mix with industry standard user research, product design, project management, and startup tools & methodologies Now what? • Check out awesome HRI papers! • Tell me about your project! Teach me your/your client’s method! Summary
31. Mike’s project in grad school Methods from Human-Robot-Interaction Agenda
32. Final thoughts What is a robot product/robot-based service? Why is building one challenging? Approaching the program like designing a DSL?