Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Difficult usability testing participants

Difficult usability testing participants

A look at difficult participants through the lease of Mr men and little miss!


Laura Yarrow

March 09, 2019


  1. @laura_yarrow Difficult research participants …and (non-sinister) ways of dealing with

  2. @laura_yarrow Difficult user testing participants 2 ★ Research in labs/in

    office ★ In-house or agency ★ Recruiting participants ★ Organising the day ★ Managing a client and observers This covers:
  3. @laura_yarrow Difficult user testing participants 3 Sometimes, no matter what

    you do, tricky participants just happen.
  4. @laura_yarrow Difficult user testing participants 4 Disclaimer: Most user testing

    participants are reacting to an unnatural situation
  5. @laura_yarrow Difficult user testing participants 5 The many types of

    participant to deal with…
  6. @laura_yarrow Difficult user testing participants 6 The Chatterbox ★ Spend

    all the session off topic ★ Tell you their life story - use the session as a social event ★ Lengthy discussions about the task ★ Don’t take it seriously Tips: ✓ Put a large clock on the desk ✓ Time-boxed script ✓ Don’t be afraid to interrupt ✓ Use closed questions at the end to get to the point MR CHATTERBOX
  7. @laura_yarrow Difficult user testing participants 7 2. The Mute ★

    Nervous, detached, despondent, or naturally quiet ★ Recent trauma or bad situation ★ Won’t talk through what they’re doing, afraid we’re testing them ✓ Be kind ✓ Be ready with open ended questions ✓ Have an introduction at the start that states they’re not being tested MR QUIET Tips:
  8. @laura_yarrow Difficult user testing participants 8 3. The Pleasers ★

    Wants to tell you what you want to hear ★ Won’t criticise anything, even if they struggle ★ If asked to rate anything, will give 10/10 ★ Will rush through everything like a race ✓ Screen for their sentiment towards brands/clients ✓ Emphasise your neutrality ✓ Play devils advocate - say you have to bring back a balanced set of feedback MR PERFECT Tips:
  9. @laura_yarrow Difficult user testing participants 9 4. The Nosey Ones

    ★ Grill you about the client or organisation ★ Find it hard to focus on the tasks and research ★ May start using their phone/your tech to look up details! ✓ Again, reiterate your independence ✓ Let them know you can tell them at the end ✓ Don’t name anyone ✓ Test a range of products/sites/apps to maintain neutrality MR NOSEY Tips:
  10. @laura_yarrow Difficult user testing participants 10 5. The Cheaters ★

    Lied on the screener! ★ Trying to make a profession out of being a research participant ★ Will skew your research findings ★ Say they use the tech or devices required but can’t in the session ✓ Compare previous answers to screeners ✓ Keep a blacklist - take note of names that respond regularly ✓ Always recruit a backup LITTLE MISS NAUGHTY Tips:
  11. @laura_yarrow Difficult user testing participants 11 6. The Undercover Agent

    ★ Refuses to be recorded or observed ★ Highly suspicious, very wary of recording devices ★ May have had a past bad experience ✓ Ask them in the screener if they are happy to be recorded in the session ✓ Give them a copy of the consent form ✓ Let them know exactly what you do with the recordings ✓ In the worst case, fall back to audio recording, or good note-taking! MR WORRY Tips:
  12. @laura_yarrow Difficult user testing participants 12 7. The Free Spirit

    ✓ Time-box your script and tasks ✓ State “I want to spend 5 minutes talking about…” ✓ Keep a clock on the desk and keep referring to it ✓ Send instructions before hand about thinking out loud so they can practice LITTLE MISS SCATTERBRAIN ★ Regularly goes off course, distracted ★ Opening new tabs in browsers, not sticking to tasks ★ Incoherently narrates, can’t think out loud ★ Will show you photos on their phone of their cat/grandkids/plant ★ Possibly eating a MacDonalds lunch throughout Tips:
  13. @laura_yarrow Difficult user testing participants 13 8. The Reluctant Participant

    ★ Treat it like an interview ★ Won’t touch the tech or engage ★ Think it’s a quiz and try and guess the answers to tasks without doing them ✓ Set expectations if you want them to use some devices ✓ “Can you show me how you would do this?” ✓ “Without telling me…” ✓ Gesture at the devices you want them to use… LITTLE MISS STUBBORN Tips:
  14. @laura_yarrow Difficult user testing participants 14 9. The Confused ★

    Not sure why they’re here / never done this before ★ Doesn’t normally use the tech you’re expecting them to use ★ Thinks they can bring their friend/child/dog… MR MUDDLE Tips: ✓ Send a guide before hand ✓ Brief them in at the start (what/why), and have they done this before? ✓ Only one question at a time ✓ Screen for devices they’re familiar with / technical ability
  15. @laura_yarrow Difficult user testing participants 15 10. The Inappropriate Person

    ★ Wants to talk about you more than the tasks ★ Makes inappropriate comments ★ Poor hygiene etc ★ Inadvertently in-appropriate ✓ Don’t go alone, take a colleague who’s always watching ✓ End the test if you feel uncomfortable ✓ You can position them where you want! LITTLE MISS SCARY Tips:
  16. @laura_yarrow Difficult user testing participants 16 11. The No Show

    ★ Doesn’t let you know ★ Doesn’t answer the phone ★ May turn up really late and try and slot themselves in ★ Will suggest a date in 4 weeks when they have a slot free… ✓ Blacklist ✓ Capture their phone number and text the night before ✓ Have a backup participant, or “floater” ✓ Allow breaks between each session where you could squeeze them in MR NOBODY Tips:
  17. @laura_yarrow Difficult user testing participants 17 Summary ✓ Send out

    information before ✓ Have a time-boxed script to follow ✓ Text the night before ✓ Have a backup participant ✓ Screen properly (device, sentiment, job…) ✓ Brief the participant at the start ✓ Hide the tech ✓ Tell them what you do with recordings ✓ Allow for 5 minutes of settling in at the start ✓ Take a colleague MR CLEVER
  18. @laura_yarrow Difficult user testing participants 18 Thanks, any questions? laurayarrow@gmail.com

    Email: Twitter: @laura_yarrow All images courtesy of www.mrmen.com