Best Practices from 10 Years of UX Research

0939baa4deb58f5e1c32560fe3bfc32e?s=47 Nate Bolt
September 14, 2016

Best Practices from 10 Years of UX Research

For UXPin on Sep 14th, 2016

0939baa4deb58f5e1c32560fe3bfc32e?s=128

Nate Bolt

September 14, 2016
Tweet

Transcript

  1. Best Practices From 10 Years of Remote UX Research

  2. thx

  3. UX Research vs. Remote UX Research vs. User Research

  4. Watching people use interfaces in order to design better interfaces

  5. 60 YEARS AGO

  6. LET’S RECORD HUMANS USING INTERFACES 30 YEARS AGO

  7. Vs. 10 YEARS AGO

  8. 1% 
 Humans observing humans 99% 
 Data

  9. • UX Research (Facebook, Google) • Experience Research (AirBnB) •

    Insights (Netflix) • Design Research (Uber) • User Research (So Many People) INTERCHANGEABLE TERMS
  10. None
  11. Analytics Data Science Quantitative Research User Research UX Research Usability

    User Research 1:1 Observation Lean Research
  12. TEXT Geography of remote UX Research

  13. None
  14. Btw I’m Nate Bolt

  15. None
  16. None
  17. PLEASE BUY THIS BOOK

  18. None
  19. ethn.io

  20. How To Plan, Recruit, and Conduct Remote UX Research

  21. None
  22. 1. Ask the right questions 
 AKA Conversations with Stakeholders

  23. Figuring out the right question(s) is tricky

  24. Quantitative research lets us test our assumptions, and qualitative research

    lets us find out what we don't know. “ ”
  25. What do our customers/ users/humans want? “ ”

  26. • Remoteresear.ch site • Mental Models by Indi Young •

    Just Enough Research by Erika Hall • Cognition in The Wild by Ed Hutchins • Complete Beginner's Guide to Design Research • Observing the User Experience: A Practitioner's Guide to User Research A Deeper Read
  27. 2. Try to Incorporate Data •Surveys •Experiments (A/B, etc) •System

    data •Purchase Behavior
  28. 3. Identify Testing Goals

  29. What do our customers/ users/humans want? “ ”

  30. (and write them down in a G&O and then in

    a Moderator Script)
  31. remoteresear.ch/category/samples

  32. 3. Recruit real humans ethn.io

  33. The Recruiting Spectrum Grab Someone in Your Office Live Customer

    Intercept Recruiting Agency
  34. 4. Combine Tools & Methods

  35. Observe Behavior

  36. 5. Collaborate don’t report

  37. And how does this fit into the design process?

  38. Understand Prototype Build

  39. Understand • review existing research • focus groups • interviews

    • ethnography • surveys • card sorts • data analysis • diary studies
  40. Prototype • participatory design • collaborative design • brainstorming techniques

    • heuristic evaluation • paper prototype testing • static mock testing
  41. Build • usability testing • heuristic evaluation • diary study

    • A/B test • field testing
  42. 1. Identify Testing Goals 2. Incorporate Data 3. Recruit Real

    Humans 4. Combine tools & methods 5. Collaborate Don’t Report How To Plan, Recruit, and Conduct Remote UX Research
  43. Pitfalls And Now Some Common

  44. Decision Theory & Adaptive Systems Group

  45. None
  46. • Hypothetical Questions • A False Premise • Opinions are

    worthless Some Common Pitfalls of Remote UX Research
  47. • Behavior • Context • Behavior Best Practices From 10

    Years of Remote UX Research
  48. • Bandwidth (games) • Security (prototypes) • Translation (time) •

    Rapport (lacking) Even More Pitfalls
  49. How & When to Run Rapid Remote UX Research

  50. None
  51. 46 Million Americans That’s 22 Million Households 15% of the

    country
  52. None
  53. Now what percentage of those 46 million were employed within

    a year of receiving food assistance?
  54. > 80%

  55. So these are working people. People with families. Teachers, etc.

    How do they apply?
  56. 90 Minutes / 53 Days up to

  57. Alan Williams Design Lead, SNAP Caroline Hays Health Focus Area

    Intern Jacob Soloman Policy analyst and project manager
  58. SNAP (Supplemental Nutrition Assistance Program) Let’s Try the Interface for

    a Application
  59. None
  60. None
  61. None
  62. Just Stop

  63. None
  64. None
  65. None
  66. 30 phone interviews 10 minutes each 58 unique experiences in

    California
  67. And 15 Follow-Up Phone Interviews

  68. Quant & Qual

  69. None
  70. None
  71. None
  72. None
  73. 11 Minutes / 53 Days (?)

  74. • 30 Remote Interviews • Real context • 15 Follow-Ups

    • Little to No Report • Input straight to designer & engineer Best Practices From 10 Years of Remote UX Research
  75. • Easy, cheap, & fast phone interviews • Quick “Guerilla”

    Human Observation • No statistical significance • Behavior repeats over small samples • Combine with other inputs Best Practices From 10 Years of Remote UX Research
  76. Top Tools for Remote UX Research

  77. “At least a little behavioral” - Marv Surveys. Web Analytics.

  78. Five Categories •Mobile (spans other categories) •Moderated (screen-sharing + audio)

    •Self-moderated (usertesting.com) •Automated (loop11, usabilla) •Fancy Analytics on Steroids (crazyegg)
  79. remoteresear.ch/tools

  80. Conceptual For A Long Time Self-Moderated Moderated Web Analytics Automated

    Static Automated Live
  81. Tasks On Concepts For A Long Time People Talk To

    A Machine Talk To People Fancy-Ass Analytics Task Bar On Images Task Bar On A Live Site
  82. I.A. & Navigation For Ad Agencies Ammunition Insight Just Better

    Analytics Still More Ammunition More Ammunition
  83. Conceptual For A Long Time Self-Moderated Moderated Web Analytics Automated

    Static Automated Live Deep Behavioral Ux Insight
  84. Opinions Conceptual For A Long Time Self-Moderated Moderated Web Analytics

    Automated Static Automated Live
  85. Ad Agencies Love It Conceptual For A Long Time Self-Moderated

    Moderated Web Analytics Automated Static Automated Live
  86. USABILLA Clicks & Questions Heatmaps You do your own recruiting

    $49 - $199
  87. None
  88. None
  89. None
  90. ethn.io

  91. None
  92. None
  93. None
  94. None
  95. None
  96. If NASA can do THIS from 153 million miles away,

    we can meet our users where they are, even if we’re a couple thousand miles away.
  97. Thanks for attending Go run some awesome remote UX research

  98. fin.

  99. • Remoteresear.ch site • Mental Models by Indi Young •

    Just Enough Research by Erika Hall • Cognition in The Wild by Ed Hutchins • Complete Beginner's Guide to Design Research • Observing the User Experience: A Practitioner's Guide to User Research A Deeper Read
  100. nate@ethn.io @ethnio Nate Bolt