How to Find and Recruit Amazing Participants

0939baa4deb58f5e1c32560fe3bfc32e?s=47 Nate Bolt
October 11, 2016

How to Find and Recruit Amazing Participants

Talk at Rosenfeld Media's URFE conference on Oct 11th, 2016. http://www.userresearchforeveryone.com/

0939baa4deb58f5e1c32560fe3bfc32e?s=128

Nate Bolt

October 11, 2016
Tweet

Transcript

  1. (for user research)

  2. thx

  3. The Recruiting Spectrum Someone in Your Office Live User Intercept

    Recruiting Agency Realness Friends & Family Facebook Ads / Twitter Literally The First Person You See Panel Craigslist #1 #3 #2 #4 #5 #6 #7 #8
  4. Mainly Because of… Attachment to Your Interface Someone in Your

    Office Live User Intercept Recruiting Agency Friends & Family Literally The First Person You See Panel Craigslist #1 #3 #2 #4 #5 #6 #7 #8 Facebook Ads / Twitter
  5. Mainly Because of… the moment of use Someone in Your

    Office Live User Intercept Recruiting Agency Friends & Family Literally The First Person You See Panel Craigslist #1 #3 #2 #4 #5 #6 #7 #8 Facebook Ads / Twitter
  6. Unless It’s Top Secret Shhhh Someone in Your Office Friends

    & Family #3 #2
  7. Or When It’s Complicated Employees Recruiting External Participants LOL

  8. User Research vs. Design Research vs. UX Research vs. Insights

    Recruiting Participants For…
  9. Watching people use interfaces in order to design better interfaces

  10. 60 YEARS AGO

  11. LET’S RECORD HUMANS USING INTERFACE S 30 YEARS AGO

  12. Vs. 10 YEARS AGO

  13. 1% 
 Humans observing humans 99% 
 Data

  14. None
  15. Analytics Data Science Quantitative Research User Research UX Research Usability

    User Research 1:1 Observation Lean Research
  16. TEXT Recruiting & Team Geography

  17. None
  18. Btw I’m Nate Bolt

  19. None
  20. None
  21. None
  22. PLEASE BUY THIS BOOK

  23. None
  24. ethn.io

  25. Let’s Start With The Recruiting Checklist

  26. What are you testing? Where in the design cycle are

    you? Method(s)? Number of participants? Target audience for the interface? Targeting vs Screening Incentives Scheduling? Stakeholders / Scrutiny? Recruiting Checklist 1. 2. 3. 4. 5. 6. 7. 8. 9.
  27. 1. Literally The First Person You See 2. Someone in

    Your Office 3. Friends & Family 4. Craigslist 5. Recruiting Agency 6. Panel 7. Facebook Ads / Twitter 8. Live User Intercept The Best Methods For Recruiting
  28. 1. Literally The First Person You See

  29. Literally The First Person You See #1 • Little or

    no budget • Time pressure • Broad Audience • No skeptical stakeholders • Direct collaboration with designers/developers/PM When to use
  30. • Broad Audience • Easy • Fast • Free •

    Dynamic Screening • Logistics • Don’t need a screener Literally The First Person You See #1 • Broad Audience • No targeting • Perceived as informal • Geographically limited • No attachment Pros Cons
  31. Targeting vs. Screening

  32. 2. Someone in Your Office 3. Friends & Family
 


    They already love you
  33. • Shhhhh top secret - world can’t see • You

    are the user • Bob the builder culture • Very early in the design cycle • Complete game-changer • No budget or time When to use Someone in Your Office / Friends & Family #2 / #3
  34. • Attachment • Easy to find • Fast • Free

    • It’s their job • Bias • Can be hard to schedule • No incentive • Not seeing external use of your interface Pros Cons Someone in Your Office / Friends & Family #2 / #3
  35. What is a Screener, Exactly?

  36. None
  37. 4. Craigslist 
 Or similar sites/concepts

  38. • Only need a bit of screening - something more

    targeted than methods 1-3 • Just enough budget for incentives • Very little scrutiny on participant sourcing When to use Craigslist #4
  39. • All walks of life • Can be fast •

    Inexpensive incentives • Habit-forming #4 • No attachment • Sometimes slow • Cheaters • Kind of a pain • Embarrassing to admit • Habit-forming • Requires extra vetting Pros Cons Craigslist
  40. 5. Recruiting Agency 
 Money Bags

  41. • $$$$$$$$$$$$$$ ($150/user) • When you have more budget than

    time • The kind of targeting you need is demographics (income, location, age) When to use Recruiting Agency #5
  42. • Hands down the least work • They can write

    the screener • Fairly fast • Someone in charge of no-shows, scheduling, etc. #5 • Expensive • No attachment • Professional tester alert Pros Cons Recruiting Agency
  43. 6. Panel 
 Your own, a tool like usertesting.com, etc

  44. Panel #6 • You’re using an online research tool that

    provides users (dScout, etc) • Repeat testing • Other methods not allowed by your organization When to use
  45. • Vetted • Any technology has been tested • Known

    entity • Demographics Panel #6 • Limited targeting • No attachment • Professional tester alert Pros Cons
  46. 7. Facebook Ads / Twitter 
 Or Social Media, or

    Anything Similar
  47. Facebook Ads / Twitter #6 • You need to reach

    a population you have zero contact with. Triathlete mechanics. • There’s some budget • You have the patience to learn ad targeting When to use
  48. • Can be incredibly targeted • Fast • Geographically diverse

    • Access to populations even agencies might not have access to • Attachment (potentially) • Gets expensive • Professional tester alert • Difficult to learn Pros Cons Facebook Ads / Twitter #6
  49. 8. Live User Intercepts 
 Time-Aware Recruiting, In the moment,

    etc
  50. • Decent traffic web site or app 
 (> 50K

    pageviews/ month) • Moments matter • In-depth screening and targeting • Some budget When to use Live User Intercepts #8
  51. • Attachment • The moment • Fast • Targeting or

    Screening #8 • Scary • Can be Expensive • Requires some engineering resources or high traffic • Totally new users Pros Cons Live User Intercepts
  52. None
  53. Let’s All Try Out This Live User Intercept

  54. ethn.io/77226

  55. None
  56. None
  57. None
  58. None
  59. None
  60. Recruiting Pitfalls And Now Some Common

  61. No Shows

  62. Solution: Schedule Backups

  63. BIAS

  64. Solution: Behavior.

  65. Quiet Users

  66. Garden Variety Demographics

  67. The Quant(s)

  68. • No Shows • Bias • Quiet Users • Garden

    Variety Demographics • The Quant(s) Summary of Recruiting Pitfalls
  69. Scheduling + Recruiting = ?

  70. (for user research) The Amazing Part

  71. • Open-ended questions • Attachment to your interface • Extreme

    scenarios - generative vs formative • Phone calls • Involving stakeholders • Online Consent • Staged Incentives The Amazing Part of Recruiting Participants
  72. None
  73. If NASA can do THIS from 153 million miles away,

    we can meet our users where they are, even if we’re a couple thousand miles away.
  74. Thanks for listening Recruit Like The Wind

  75. fin.

  76. And also the ethn.io code for 20% off is ROSENRAD

    Participant Recruiting Take-Aways remoteresear.ch/urfe
  77. None
  78. nate@ethn.io @ethnio Nate Bolt