Helping Participants Actually Participate: lessons learned from diary studies

Diary studies enable us to gather unique temporal and longitudinal qualitative data about users and are a valuable tool in the repertoire of a UX Researcher. However, a diary study is also a complex and costly research method. It is longer than other qualitative techniques, takes more time and effort to run, and often requires higher incentives for participants.

The success of a diary study rests on the willingness of participants to stay involved and engaged with tasks and activities. Yet, after taking the time to carefully craft and design our study, we can often overlook the importance of how our participants might really feel about the tasks and activities we have selected.

This presentation covers the strategies and techniques I’ve developed based on my experience with diary studies, including practical tips and tricks you can apply to your next research project.

Transcript

1. Helping participants actually participate! Cara de Ferranti @CdeFerranti Lessons learned from diary studies
2. Cara de Ferranti User Experience Researcher - Google @CdeFerranti
4. 9 days 20 participants 10 mins each day
5. 9 days 20 participants 10 mins each day 4 1-min videos 5 photos 1 survey
6. 9 days 20 participants 10 mins each day 4 1-min videos 5 photos 1 survey
7. Then the emails* started coming in... “I’m not comfortable with the amount of personal information required” “I’ve had some work come up and won’t be able to do all the videos” “I’m going away for the weekend, I did not realise I needed to be in my home to complete this study” *Paraphrased from actual participant responses
9. What I’m going to cover today… / Evaluating & communicating effort / Selecting the right task structure / Thinking outside the participant box / Keeping it human / Giving lots of feedback
10. What I’m going to cover today… / Evaluating & communicating effort / Selecting the right task structure / Thinking outside the participant box / Keeping it human / Giving lots of feedback
11. Evaluating and communicating effort It’s more than just TIME that is important
12. “I work on a tight daily schedule and will not have time to fit this into my day”* *Paraphrased from actual participant responses
13. The effort of switching between tasks vs. Completing one large activity (e.g., long survey) Completing two smaller activities (e.g., short survey + video)
14. The effort of spreading tasks across a day vs. Completing an activity once (one long video in the evening) Completing an activity several times (many short videos across the day)
15. The effort of completing in tasks vs. Reminded to record an entry after using a specific app Remembering to record an entry after using a specific app
16. The effort of recording information during tasks Impersonal methods of reporting (e.g., survey or written responses) Personal methods of reporting (e.g., videos or photographs) vs.
17. What I’m going to cover today… / Evaluating & communicating effort / Selecting the right task structure / Thinking outside the participant box / Keeping it human / Giving lots of feedback
18. Selecting the right task structure Make tasks easy and low effort to complete Structure activities and stagger incentives to maximise response detail and frequency Build variety and interest into the study
19. “The same as yesterday, the day before, and every other time before that...”* *Paraphrased from actual participant responses
20. What I’m going to cover today… / Evaluating & communicating effort / Selecting the right task structure / Thinking outside the participant box / Keeping it human / Giving lots of feedback
21. Thinking outside the participant box Tools available for tasks Physical location of participants Public events during the study
22. “I don’t have pen or paper at home, so will have to do this task when I’m back at work on Monday”* *Paraphrased from actual participant responses
23. What I’m going to cover today… / Evaluating & communicating effort / Selecting the right task structure / Thinking outside the participant box / Keeping it human / Giving lots of feedback
24. Keeping it human Let them know there is a person behind the operation
25. “Hi Susan, thanks for completing your entry today - sounds like you had a pretty tough time getting home!”
26. Let them know there is a person behind the operation Recognise and use preferences for communication Keeping it human
27. “Can you please let me know if this email address is the best method for staying in touch over the next two weeks?”
28. What I’m going to cover today… / Evaluating & communicating effort / Selecting the right task structure / Thinking outside the participant box / Keeping it human / Giving lots of feedback
29. Giving lots of feedback Send many reminders Acknowledge responses Be flexible Provide task guidance Track participation Personalise communications
30. Back to my diary study… what could be improved? Manage task expectations better, overestimate the daily time commitment and describe task structure (not just duration). Confirm where participants were located for the study, not just where they lived. Overemphasise the use of video and photography as the chosen methods of data collection.
31. Cara de Ferranti User Experience Researcher - Google @CdeFerranti Thanks! Helping participants actually participate: lessons learned from diary studies / Evaluating & communicating effort / Selecting the right task structure / Thinking outside the participant box / Keeping it human / Giving lots of feedback