PYROS Deep Dive #2 - Usability Testing

Founders need to test their riskiest assumptions before they commit to a costly mistake of building a fully realized product.

In this session, we'll cover:
- what usability and UX is
- common UX mistakes
- how to run a usability test with real users
- how to identify your assumptions to know what to test

1. usability testing regan davis BUILD | deep dive #2 PYROS
2. regan davis product leader from chicago, il BACKGROUND EIR @ 1871 Sr. Director of Product @ SpotHero Advisor, Coach regandavis.com VP of Product @ Jellyvision VP of Product @ info.com
3. case study regandavis.com
4. bill nguyen 7x founder, Color app regandavis.com an instant feed of photos of everyone around you Color app (2011):
5. bill nguyen 7x founder, Color app regandavis.com $41 million raised: all before launching and any users
6. regandavis.com inundated with negative reviews
7. regandavis.com “Within 30 minutes I realized, ‘Oh my God, it’s broken. Holy s——, we totally f——ed up.’ ” —Bill Nguyen, Founder of Color
8. regandavis.com
9. regandavis.com usability • Learnability • Efficiency • Memorability • Errors • Satisfaction let’s talk (credit: Nielsen Group)
10. regandavis.com usability • Learnability • Efficiency • Memorability • Errors • Satisfaction let’s talk
11. regandavis.com usability • Learnability • Efficiency • Memorability • Errors • Satisfaction let’s talk
12. good vs bad ux regandavis.com
13. (credit: Adobe) regandavis.com terminology
14. regandavis.com terminology • Use clear, understandable language. • Match your language to the terms and definitions your users are comfortable with.
15. (credit: Adobe) regandavis.com hierarchy
16. regandavis.com hierarchy • Structure information in a way that makes it clear what’s most important. • Make ‘call-to-action’ buttons (CTAs) easy to distinguish and press.
17. (credit: Adobe) regandavis.com explain/reduce errors
18. regandavis.com explain/reduce errors • Provide clear messaging of an error. • Avoid errors by: • Hiding irrelevant options that cause errors • Providing clear examples of what success looks like • Using placeholder text • Using consistent patterns, symbols, and placement
19. regandavis.com accessible Accessibility: making your product work for all types of users. 15-20% of users have some form of disability. usabledesign design is
20. regandavis.com accessible design (credit: Monsido.com)
21. testing regandavis.com
22. is the most expensive way to test your idea. regandavis.com building production-quality software cheap by comparison research is
23. (credit: Teresa Torres, Product Talk) regandavis.com ladder of evidence Time and Effort Value of Learning listen specific stories of the past what they’ve done in the past what they would do watch observe them in a real-life instance ask to show you what they did
24. regandavis.com usability testing Goal: to find out if your users can effectively complete the task that they set out to do. NOT: do they ‘like this color’?
25. regandavis.com usability testing • A plan. • A prototype or sketch. • A way of note-taking. • A timer or watch. • Four to five participants of each user type • A facilitator. • An observer. STUFF NEEDED PEOPLE NEEDED (credit: Erika Hall, Just Enough Research)
26. recording requires consent regandavis.com (app: otter.ai) “Are you comfortable with me recording this conversation so that we can refer to it later? We won’t plan on sharing any of your statements outside of [our company].”
27. (credit: Erika Hall, Just Enough Research) regandavis.com The greatest danger is… “…that [founders won’t] sit idly by while their creation fails to perform or elicits derision from the participant.”
28. try to avoid “If you click that over in the bottom right…” “Do you like that you can…” regandavis.com hints leading questions filling silence user self-blame (credit: Erika Hall, Just Enough Research)
29. regandavis.com take note. Frustration, non-verbal responses, and quotes How long it takes to complete the task. If the user failed to complete any step or task. Any terminology that presented a stumbling block. (credit: Erika Hall, Just Enough Research)
30. (Source: “How do you know when a design fails a usability test?”, webdesignerdepot.com) regandavis.com
31. regandavis.com how many interviews? Only 4-5 interviews per round will find ~80% of the issues. Then, fix the biggest issues and test again. (Source: Nielsen Norman Group)
32. regandavis.com learn & iterate (credit: ”The Founder” movie clip) our goal is to
33. examples regandavis.com
34. regandavis.com slide deck
35. regandavis.com
37. regandavis.com mockup + video
38. regandavis.com paper prototype
39. competitor site
40. let’s try it.
41. regandavis.com Task #1 You want to see what delivery options are near you. QUESTIONS • What do you see on the screen? • What do you think this is for? Link: lettuce.com
42. how to know what to test. regandavis.com
43. regandavis.com user story mapping • A technique to visually break down user stories into prioritized paths for delivering your MVP • and… A technique to generate your usability test plan
44. regandavis.com
45. regandavis.com (Source: Teresa Torres, Continuous Discovery Habits) mapping assumptions
46. let’s try it. regandavis.com
47. regandavis.com ex: streaming service (Source: Teresa Torres, Continuous Discovery Habits) Assumptions of: • Value • Usability • Feasibility ‘I want to watch live sports’
48. recap regandavis.com
49. regandavis.com “Within 30 minutes I realized, ‘Oh my God, it’s broken. Holy s——, we totally f——ed up.’ ” —Bill Nguyen, Founder of Color
50. Diagram courtesy of Jussi Pasanen, with acknowledgements to Aarron Walter, Ben Tollady, Ben Rowe, Lexi Thorn and Senthil Kugalur. regandavis.com
51. (credit: Teresa Torres, Product Talk) regandavis.com ladder of evidence Time and Effort Value of Learning listen specific stories of the past what they’ve done in the past what they would do watch observe them in a real-life instance ask to show you what they did
52. regandavis.com usability testing • A plan. • A prototype or sketch. • A way of note-taking. • A timer or watch. • Four to five participants of each user type • A facilitator. • An observer. STUFF NEEDED PEOPLE NEEDED (credit: Erika Hall, Just Enough Research)
53. try to avoid “If you click that over in the bottom right…” “Do you like that you can…” regandavis.com hints leading questions filling silence user self-blame (credit: Erika Hall, Just Enough Research)
54. regandavis.com competitor usability options mockups paper
55. (Source: “How do you know when a design fails a usability test?”, webdesignerdepot.com) regandavis.com
56. regandavis.com (Source: Teresa Torres, Continuous Discovery Habits) mapping assumptions
57. regandavis.com assignment: 1. Take your user story map from Deep Dive #1 and underneath each step, add some assumptions. 2. Write out a usability test with tasks to complete • Either use your own prototype, or pick an existing product / competitor. Next week, you’ll run a user test with each other in breakouts. BUILD | deep dive #2
58. questions? NEXT STEPS regandavis.com/speaker #pyros_build on slack @regan on slack regandavis.com download feedback contact