PYROS Deep Dive #2 - Usability Testing

Founders need to test their riskiest assumptions before they commit to a costly mistake of building a fully realized product.

In this session, we'll cover:
- what usability and UX is
- common UX mistakes
- how to run a usability test with real users
- how to identify your assumptions to know what to test

Transcript

1. usability testing
   regan davis
   BUILD | deep dive #2
   PYROS
   

   View Slide

2. regan davis
   product leader from chicago, il
   BACKGROUND
   EIR @ 1871
   Sr. Director of Product @ SpotHero
   Advisor, Coach
   regandavis.com
   VP of Product @ Jellyvision
   VP of Product @ info.com
   

   View Slide

3. case study
   regandavis.com
   

   View Slide

4. bill nguyen
   7x founder, Color app
   regandavis.com
   an instant feed of
   photos of everyone
   around you
   Color app (2011):
   

   View Slide

5. bill nguyen
   7x founder, Color app
   regandavis.com
   $41 million
   raised:
   all before launching and any users
   

   View Slide

6. regandavis.com
   inundated with
   negative reviews
   

   View Slide

7. regandavis.com
   “Within 30 minutes I realized,
   ‘Oh my God, it’s broken. Holy
   s——, we totally f——ed up.’ ”
   —Bill Nguyen, Founder of Color
   

   View Slide

8. regandavis.com
   

   View Slide

9. regandavis.com
   usability
   • Learnability
   • Efficiency
   • Memorability
   • Errors
   • Satisfaction
   let’s talk
   (credit: Nielsen Group)
   

   View Slide

10. regandavis.com
    usability
    • Learnability
    • Efficiency
    • Memorability
    • Errors
    • Satisfaction
    let’s talk
    

    View Slide

11. regandavis.com
    usability
    • Learnability
    • Efficiency
    • Memorability
    • Errors
    • Satisfaction
    let’s talk
    

    View Slide

12. good vs bad ux
    regandavis.com
    

    View Slide

13. (credit: Adobe) regandavis.com
    terminology
    

    View Slide

14. regandavis.com
    terminology
    • Use clear, understandable language.
    • Match your language to the terms and
    definitions your users are comfortable with.
    

    View Slide

15. (credit: Adobe) regandavis.com
    hierarchy
    

    View Slide

16. regandavis.com
    hierarchy
    • Structure information in a way that makes it
    clear what’s most important.
    • Make ‘call-to-action’ buttons (CTAs) easy to
    distinguish and press.
    

    View Slide

17. (credit: Adobe) regandavis.com
    explain/reduce errors
    

    View Slide

18. regandavis.com
    explain/reduce errors
    • Provide clear messaging of an error.
    • Avoid errors by:
    • Hiding irrelevant options that cause errors
    • Providing clear examples of what success looks like
    • Using placeholder text
    • Using consistent patterns, symbols, and placement
    

    View Slide

19. regandavis.com
    accessible
    Accessibility: making your product work for all types
    of users.
    15-20% of users have some form of disability.
    usabledesign
    design
    is
    

    View Slide

20. regandavis.com
    accessible design
    (credit: Monsido.com)
    

    View Slide

21. testing
    regandavis.com
    

    View Slide

22. is the most expensive
    way to test your idea.
    regandavis.com
    building production-quality software
    cheap by comparison
    research is
    

    View Slide

23. (credit: Teresa Torres, Product Talk) regandavis.com
    ladder of evidence
    Time and Effort
    Value of Learning
    listen
    specific stories of the past
    what they’ve done in the past
    what they would do
    watch
    observe them in a real-life instance
    ask to show you what they did
    

    View Slide

24. regandavis.com
    usability testing
    Goal: to find out if your users can effectively
    complete the task that they set out to do.
    NOT: do they ‘like this color’?
    

    View Slide

25. regandavis.com
    usability testing
    • A plan.
    • A prototype or sketch.
    • A way of note-taking.
    • A timer or watch.
    • Four to five participants of
    each user type
    • A facilitator.
    • An observer.
    STUFF NEEDED PEOPLE NEEDED
    (credit: Erika Hall, Just Enough Research)
    

    View Slide

26. recording
    requires consent
    regandavis.com
    (app: otter.ai)
    “Are you comfortable with me recording
    this conversation so that we can refer to it
    later?
    We won’t plan on sharing any of your
    statements outside of [our company].”
    

    View Slide

27. (credit: Erika Hall, Just Enough Research) regandavis.com
    The greatest danger is…
    “…that [founders won’t] sit idly by while
    their creation fails to perform or elicits
    derision from the participant.”
    

    View Slide

28. try to avoid
    “If you click that over in the bottom right…”
    “Do you like that you can…”
    regandavis.com
    hints
    leading questions
    filling silence
    user self-blame
    (credit: Erika Hall, Just Enough Research)
    

    View Slide

29. regandavis.com
    take note.
    Frustration, non-verbal responses, and quotes
    How long it takes to complete the task.
    If the user failed to complete any step or task.
    Any terminology that presented a stumbling block.
    (credit: Erika Hall, Just Enough Research)
    

    View Slide

30. (Source: “How do you know when a design fails a usability test?”, webdesignerdepot.com) regandavis.com
    

    View Slide

31. regandavis.com
    how many interviews?
    Only 4-5 interviews per
    round will find ~80% of
    the issues.
    Then, fix the biggest
    issues and test again.
    (Source: Nielsen Norman Group)
    

    View Slide

32. regandavis.com
    learn & iterate
    (credit: ”The Founder” movie clip)
    our goal is to
    

    View Slide

33. examples
    regandavis.com
    

    View Slide

34. regandavis.com
    slide deck
    

    View Slide

35. regandavis.com
    

    View Slide

36. View Slide

37. regandavis.com
    mockup + video
    

    View Slide

38. regandavis.com
    paper prototype
    

    View Slide

39. competitor site
    

    View Slide

40. let’s try it.
    

    View Slide

41. regandavis.com
    Task #1
    You want to see what delivery options are near you.
    QUESTIONS
    • What do you see on the screen?
    • What do you think this is for?
    Link: lettuce.com
    

    View Slide

42. how to know
    what to test.
    regandavis.com
    

    View Slide

43. regandavis.com
    user story
    mapping
    • A technique to visually break down user stories into
    prioritized paths for delivering your MVP
    • and… A technique to generate your usability test plan
    

    View Slide

44. regandavis.com
    

    View Slide

45. regandavis.com
    (Source: Teresa Torres, Continuous Discovery Habits)
    mapping
    assumptions
    

    View Slide

46. let’s try it.
    regandavis.com
    

    View Slide

47. regandavis.com
    ex: streaming service
    (Source: Teresa Torres, Continuous Discovery Habits)
    Assumptions of:
    • Value
    • Usability
    • Feasibility
    ‘I want to watch live sports’
    

    View Slide

48. recap
    regandavis.com
    

    View Slide

49. regandavis.com
    “Within 30 minutes I realized,
    ‘Oh my God, it’s broken. Holy
    s——, we totally f——ed up.’ ”
    —Bill Nguyen, Founder of Color
    

    View Slide

50. Diagram courtesy of Jussi Pasanen, with acknowledgements to Aarron Walter, Ben Tollady, Ben Rowe, Lexi Thorn and Senthil Kugalur.
    regandavis.com
    

    View Slide

51. (credit: Teresa Torres, Product Talk) regandavis.com
    ladder of evidence
    Time and Effort
    Value of Learning
    listen
    specific stories of the past
    what they’ve done in the past
    what they would do
    watch
    observe them in a real-life instance
    ask to show you what they did
    

    View Slide

52. regandavis.com
    usability testing
    • A plan.
    • A prototype or sketch.
    • A way of note-taking.
    • A timer or watch.
    • Four to five participants of
    each user type
    • A facilitator.
    • An observer.
    STUFF NEEDED PEOPLE NEEDED
    (credit: Erika Hall, Just Enough Research)
    

    View Slide

53. try to avoid
    “If you click that over in the bottom right…”
    “Do you like that you can…”
    regandavis.com
    hints
    leading questions
    filling silence
    user self-blame
    (credit: Erika Hall, Just Enough Research)
    

    View Slide

54. regandavis.com
    competitor
    usability options
    mockups
    paper
    

    View Slide

55. (Source: “How do you know when a design fails a usability test?”, webdesignerdepot.com) regandavis.com
    

    View Slide

56. regandavis.com
    (Source: Teresa Torres, Continuous Discovery Habits)
    mapping
    assumptions
    

    View Slide

57. regandavis.com
    assignment:
    1. Take your user story map from Deep Dive #1 and
    underneath each step, add some assumptions.
    2. Write out a usability test with tasks to complete
    • Either use your own prototype, or pick an existing
    product / competitor.
    Next week, you’ll run a user test with each other in breakouts.
    BUILD | deep dive #2
    

    View Slide

58. questions?
    NEXT STEPS
    regandavis.com/speaker
    #pyros_build on slack
    @regan on slack
    regandavis.com
    download
    feedback
    contact
    

    View Slide