Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PYROS Deep Dive #2 - Usability Testing

Regan Davis
February 17, 2022

PYROS Deep Dive #2 - Usability Testing

Founders need to test their riskiest assumptions before they commit to a costly mistake of building a fully realized product.

In this session, we'll cover:
- what usability and UX is
- common UX mistakes
- how to run a usability test with real users
- how to identify your assumptions to know what to test

Regan Davis

February 17, 2022
Tweet

More Decks by Regan Davis

Other Decks in Design

Transcript

  1. usability testing
    regan davis
    BUILD | deep dive #2
    PYROS

    View Slide

  2. regan davis
    product leader from chicago, il
    BACKGROUND
    EIR @ 1871
    Sr. Director of Product @ SpotHero
    Advisor, Coach
    regandavis.com
    VP of Product @ Jellyvision
    VP of Product @ info.com

    View Slide

  3. case study
    regandavis.com

    View Slide

  4. bill nguyen
    7x founder, Color app
    regandavis.com
    an instant feed of
    photos of everyone
    around you
    Color app (2011):

    View Slide

  5. bill nguyen
    7x founder, Color app
    regandavis.com
    $41 million
    raised:
    all before launching and any users

    View Slide

  6. regandavis.com
    inundated with
    negative reviews

    View Slide

  7. regandavis.com
    “Within 30 minutes I realized,
    ‘Oh my God, it’s broken. Holy
    s——, we totally f——ed up.’ ”
    —Bill Nguyen, Founder of Color

    View Slide

  8. regandavis.com

    View Slide

  9. regandavis.com
    usability
    • Learnability
    • Efficiency
    • Memorability
    • Errors
    • Satisfaction
    let’s talk
    (credit: Nielsen Group)

    View Slide

  10. regandavis.com
    usability
    • Learnability
    • Efficiency
    • Memorability
    • Errors
    • Satisfaction
    let’s talk

    View Slide

  11. regandavis.com
    usability
    • Learnability
    • Efficiency
    • Memorability
    • Errors
    • Satisfaction
    let’s talk

    View Slide

  12. good vs bad ux
    regandavis.com

    View Slide

  13. (credit: Adobe) regandavis.com
    terminology

    View Slide

  14. regandavis.com
    terminology
    • Use clear, understandable language.
    • Match your language to the terms and
    definitions your users are comfortable with.

    View Slide

  15. (credit: Adobe) regandavis.com
    hierarchy

    View Slide

  16. regandavis.com
    hierarchy
    • Structure information in a way that makes it
    clear what’s most important.
    • Make ‘call-to-action’ buttons (CTAs) easy to
    distinguish and press.

    View Slide

  17. (credit: Adobe) regandavis.com
    explain/reduce errors

    View Slide

  18. regandavis.com
    explain/reduce errors
    • Provide clear messaging of an error.
    • Avoid errors by:
    • Hiding irrelevant options that cause errors
    • Providing clear examples of what success looks like
    • Using placeholder text
    • Using consistent patterns, symbols, and placement

    View Slide

  19. regandavis.com
    accessible
    Accessibility: making your product work for all types
    of users.
    15-20% of users have some form of disability.
    usabledesign
    design
    is

    View Slide

  20. regandavis.com
    accessible design
    (credit: Monsido.com)

    View Slide

  21. testing
    regandavis.com

    View Slide

  22. is the most expensive
    way to test your idea.
    regandavis.com
    building production-quality software
    cheap by comparison
    research is

    View Slide

  23. (credit: Teresa Torres, Product Talk) regandavis.com
    ladder of evidence
    Time and Effort
    Value of Learning
    listen
    specific stories of the past
    what they’ve done in the past
    what they would do
    watch
    observe them in a real-life instance
    ask to show you what they did

    View Slide

  24. regandavis.com
    usability testing
    Goal: to find out if your users can effectively
    complete the task that they set out to do.
    NOT: do they ‘like this color’?

    View Slide

  25. regandavis.com
    usability testing
    • A plan.
    • A prototype or sketch.
    • A way of note-taking.
    • A timer or watch.
    • Four to five participants of
    each user type
    • A facilitator.
    • An observer.
    STUFF NEEDED PEOPLE NEEDED
    (credit: Erika Hall, Just Enough Research)

    View Slide

  26. recording
    requires consent
    regandavis.com
    (app: otter.ai)
    “Are you comfortable with me recording
    this conversation so that we can refer to it
    later?
    We won’t plan on sharing any of your
    statements outside of [our company].”

    View Slide

  27. (credit: Erika Hall, Just Enough Research) regandavis.com
    The greatest danger is…
    “…that [founders won’t] sit idly by while
    their creation fails to perform or elicits
    derision from the participant.”

    View Slide

  28. try to avoid
    “If you click that over in the bottom right…”
    “Do you like that you can…”
    regandavis.com
    hints
    leading questions
    filling silence
    user self-blame
    (credit: Erika Hall, Just Enough Research)

    View Slide

  29. regandavis.com
    take note.
    Frustration, non-verbal responses, and quotes
    How long it takes to complete the task.
    If the user failed to complete any step or task.
    Any terminology that presented a stumbling block.
    (credit: Erika Hall, Just Enough Research)

    View Slide

  30. (Source: “How do you know when a design fails a usability test?”, webdesignerdepot.com) regandavis.com

    View Slide

  31. regandavis.com
    how many interviews?
    Only 4-5 interviews per
    round will find ~80% of
    the issues.
    Then, fix the biggest
    issues and test again.
    (Source: Nielsen Norman Group)

    View Slide

  32. regandavis.com
    learn & iterate
    (credit: ”The Founder” movie clip)
    our goal is to

    View Slide

  33. examples
    regandavis.com

    View Slide

  34. regandavis.com
    slide deck

    View Slide

  35. regandavis.com

    View Slide

  36. View Slide

  37. regandavis.com
    mockup + video

    View Slide

  38. regandavis.com
    paper prototype

    View Slide

  39. competitor site

    View Slide

  40. let’s try it.

    View Slide

  41. regandavis.com
    Task #1
    You want to see what delivery options are near you.
    QUESTIONS
    • What do you see on the screen?
    • What do you think this is for?
    Link: lettuce.com

    View Slide

  42. how to know
    what to test.
    regandavis.com

    View Slide

  43. regandavis.com
    user story
    mapping
    • A technique to visually break down user stories into
    prioritized paths for delivering your MVP
    • and… A technique to generate your usability test plan

    View Slide

  44. regandavis.com

    View Slide

  45. regandavis.com
    (Source: Teresa Torres, Continuous Discovery Habits)
    mapping
    assumptions

    View Slide

  46. let’s try it.
    regandavis.com

    View Slide

  47. regandavis.com
    ex: streaming service
    (Source: Teresa Torres, Continuous Discovery Habits)
    Assumptions of:
    • Value
    • Usability
    • Feasibility
    ‘I want to watch live sports’

    View Slide

  48. recap
    regandavis.com

    View Slide

  49. regandavis.com
    “Within 30 minutes I realized,
    ‘Oh my God, it’s broken. Holy
    s——, we totally f——ed up.’ ”
    —Bill Nguyen, Founder of Color

    View Slide

  50. Diagram courtesy of Jussi Pasanen, with acknowledgements to Aarron Walter, Ben Tollady, Ben Rowe, Lexi Thorn and Senthil Kugalur.
    regandavis.com

    View Slide

  51. (credit: Teresa Torres, Product Talk) regandavis.com
    ladder of evidence
    Time and Effort
    Value of Learning
    listen
    specific stories of the past
    what they’ve done in the past
    what they would do
    watch
    observe them in a real-life instance
    ask to show you what they did

    View Slide

  52. regandavis.com
    usability testing
    • A plan.
    • A prototype or sketch.
    • A way of note-taking.
    • A timer or watch.
    • Four to five participants of
    each user type
    • A facilitator.
    • An observer.
    STUFF NEEDED PEOPLE NEEDED
    (credit: Erika Hall, Just Enough Research)

    View Slide

  53. try to avoid
    “If you click that over in the bottom right…”
    “Do you like that you can…”
    regandavis.com
    hints
    leading questions
    filling silence
    user self-blame
    (credit: Erika Hall, Just Enough Research)

    View Slide

  54. regandavis.com
    competitor
    usability options
    mockups
    paper

    View Slide

  55. (Source: “How do you know when a design fails a usability test?”, webdesignerdepot.com) regandavis.com

    View Slide

  56. regandavis.com
    (Source: Teresa Torres, Continuous Discovery Habits)
    mapping
    assumptions

    View Slide

  57. regandavis.com
    assignment:
    1. Take your user story map from Deep Dive #1 and
    underneath each step, add some assumptions.
    2. Write out a usability test with tasks to complete
    • Either use your own prototype, or pick an existing
    product / competitor.
    Next week, you’ll run a user test with each other in breakouts.
    BUILD | deep dive #2

    View Slide

  58. questions?
    NEXT STEPS
    regandavis.com/speaker
    #pyros_build on slack
    @regan on slack
    regandavis.com
    download
    feedback
    contact

    View Slide