Product Research Rules! MtPCon London 2019

Product leaders are constantly asking if they are building the right features and products. One core reason is that there just isn't enough research conducted toensure that the identified solutions match the problems. This talk covers some rules to help guide you.

Transcript

1. Product
   Research
   C. Todd Lombardo — @iamctodd
   

   View Slide

2. “Are we building the right thing?”
   @iamctodd
   — Martin Eriksson, earlier today
   

   View Slide

3. Meet Bill
   

   View Slide

4. Meet Bill
   Serial entrepreneur
   

   View Slide

5. Meet Bill
   Serial entrepreneur
   Worked for Steve Jobs
   

   View Slide

6. Meet Bill
   Serial entrepreneur
   Worked for Steve Jobs
   Fifteen years in Silicon Valley
   

   View Slide

7. Meet Bill
   Serial entrepreneur
   Worked for Steve Jobs
   Fifteen years in Silicon Valley
   Sold a startup for $850 million… at age 29
   

   View Slide

8. I literally don’t think there’s
   anything to be learned
   from other people’s stuff.”
   — Bill Nguyen
   “
   

   View Slide

9. View Slide

10. “On first blush, there's
    almost zero chance we
    return to this app. Our
    friends aren't in it. It's
    confusing. Sharing photos
    isn't easy or intuitive.
    It's not very fun.
    — Business Insider
    

    View Slide

11. Within 30 minutes I realized.. We totally
    fucked up. I thought we were going to
    build a better Facebook. My reaction was
    like putting your finger into a light socket.
    You know something went very wrong.
    — Bill Nguyen
    “
    

    View Slide

12. Research Sprints 8
    

    View Slide

13. View Slide

14. KILLEDBYGOOGLE.COM
    

    View Slide

15. GCEMETERY.CO
    

    View Slide

16. View Slide

17. We’re Not Good at This…
    STANDISH GROUP
    Always
    7 %
    Often
    13 %
    Sometimes
    16 %
    Rarely
    19 %
    Never
    45 %
    

    View Slide

18. Why Products Fail
    No market need
    Ran out of cash
    Not the right team
    Got outcompeted
    Pricing/cost issue
    Poor product
    Lack a business model
    Poor marketing
    Ignore Customers
    entrepreneur.com & cbinsights.com
    

    View Slide

19. Why Products Fail
    No market need
    Ran out of cash
    Not the right team
    Got outcompeted
    Pricing/cost issue
    Poor product
    Lack a business model
    Poor marketing
    Ignore Customers
    entrepreneur.com & cbinsights.com
    BAD PODUCT RESEARCH
    

    View Slide

20. View Slide

21. Research is expensive!
    

    View Slide

22. Research is expensive!
    We already know.
    

    View Slide

23. Research is expensive!
    We already know.
    It will take too long!
    

    View Slide

24. Do you want waste 45% of your resources??
    No? Then do some f*cking research!!
    @iamctodd
    

    View Slide

25. Product”
    Research?!
    “
    

    View Slide

26. View Slide

27. Studious inquiry or examination; especially
    investigation or experimentation aimed at
    the discovery and interpretation of facts,
    revision of accepted theories or laws in the
    light of new facts, or practical application of
    such new or revised theories or laws.
    — Merriam Webster
    

    View Slide

28. View Slide

29. What people do?
    Qualitative
    Understanding
    User Research
    

    View Slide

30. Market Research
    What markets do?
    Quantitative
    Framing
    

    View Slide

31. Product Analytics
    Usage of a product
    Quantitative
    Trends & Patterns
    

    View Slide

32. User Research
    Market Research
    Product Analytics
    Product
    Research
    

    View Slide

33. User Research Market Research Product Analytics
    Generative Exploratory
    Descriptive Descriptive Descriptive
    Evaluative Causal Diagnostic
    Predictive Predictive
    Prescriptive
    

    View Slide

34. View Slide

35. Start with a beginner’s mind
    @iamctodd
    

    View Slide

36. Meet C. Todd
    

    View Slide

37. Meet C. Todd
    Serial product guy
    

    View Slide

38. Meet C. Todd
    Serial product guy
    Did not work for Steve Jobs
    

    View Slide

39. Meet C. Todd
    Serial product guy
    Did not work for Steve Jobs
    Fifteen years in startups & enterprises
    

    View Slide

40. Meet C. Todd
    Serial product guy
    Did not work for Steve Jobs
    Fifteen years in startups & enterprises
    Grew product from $5M to $36M.. in 30 months
    

    View Slide

41. View Slide

42. Prepare to be wrong
    # 1
    Because you will be.
    Humans prefer to be right than do right.
    Check your ego. Check your assumptions.
    @iamctodd
    

    View Slide

43. I was totally wrong. I was
    just wrong. See, I don’t have
    a problem being wrong.”
    — Bill Nguyen
    “
    

    View Slide

44. In order to be wrong, you need an
    understanding of what you already know.
    @iamctodd
    

    View Slide

45. Meet Catt
    

    View Slide

46. Meet Catt
    Ace designer & developer
    

    View Slide

47. Meet Catt
    Ace designer & developer
    Works for large marketplace platform
    

    View Slide

48. Meet Catt
    Ace designer & developer
    Works for large marketplace platform
    Fourteen years in gaming, design, and tech
    

    View Slide

49. Meet Catt
    Ace designer & developer
    Works for large marketplace platform
    Fourteen years in gaming, design, and tech
    Learned to code when she was 10
    

    View Slide

50. Available Data
    Annual surveys, social
    media, customer forums,
    customer emails, product
    analytics, interviews
    

    View Slide

51. Available Data
    Annual surveys, social
    media, customer forums,
    customer emails, product
    analytics, interviews
    I can't find my listing and search in
    the rankings and results! Did they
    change the search algorithm?
    

    View Slide

52. Identified Problem
    Marketplace sellers didn't
    understand how they were
    being discovered by buyers
    Available Data
    Annual surveys, social
    media, customer forums,
    customer emails, product
    analytics, interviews
    I can't find my listing and search in
    the rankings and results! Did they
    change the search algorithm?
    

    View Slide

53. Identified Problem
    Marketplace sellers didn't
    understand how they were
    being discovered by buyers
    Available Data
    Annual surveys, social
    media, customer forums,
    customer emails, product
    analytics, interviews
    Research Question
    How do sellers learn how to
    market themselves?
    I can't find my listing and search in
    the rankings and results! Did they
    change the search algorithm?
    

    View Slide

54. # 2
    Get Curious with Your Data
    Start with the data you have.
    Generate questions to answer.
    Settle on a research question to answer.
    @iamctodd
    

    View Slide

55. What you ask and how you speak matters.
    @iamctodd
    

    View Slide

56. So, what do you think of this feature?
    Would you use this new feature?
    How would you design this?
    How likely are you to recommend this…
    

    View Slide

57. View Slide

58. CREDIT: MASS CHALLENGE
    

    View Slide

59. I See You Have A
    Coffee Cup. We
    Make Coffee.
    Would You Like To
    Buy A Coffee?
    CREDIT: MASS CHALLENGE
    

    View Slide

60. Transactional
    I See You Have A
    Coffee Cup. We
    Make Coffee.
    Would You Like To
    Buy A Coffee?
    CREDIT: MASS CHALLENGE
    

    View Slide

61. Transactional
    I See You Have A
    Coffee Cup. We
    Make Coffee.
    Would You Like To
    Buy A Coffee?
    Wouldn´t It Be
    Cool If You Had A
    Premium Cold-
    Brew Coffee To
    Fill That Cup?
    CREDIT: MASS CHALLENGE
    

    View Slide

62. Transactional
    I See You Have A
    Coffee Cup. We
    Make Coffee.
    Would You Like To
    Buy A Coffee?
    Confirmational
    Wouldn´t It Be
    Cool If You Had A
    Premium Cold-
    Brew Coffee To
    Fill That Cup?
    CREDIT: MASS CHALLENGE
    

    View Slide

63. Transactional
    I See You Have A
    Coffee Cup. We
    Make Coffee.
    Would You Like To
    Buy A Coffee?
    Confirmational
    Wouldn´t It Be
    Cool If You Had A
    Premium Cold-
    Brew Coffee To
    Fill That Cup?
    Tell Me More
    About That Cup.
    What Do You
    Usually Drink Out
    Of It?
    CREDIT: MASS CHALLENGE
    

    View Slide

64. Transactional
    I See You Have A
    Coffee Cup. We
    Make Coffee.
    Would You Like To
    Buy A Coffee?
    Confirmational
    Wouldn´t It Be
    Cool If You Had A
    Premium Cold-
    Brew Coffee To
    Fill That Cup?
    Diagnostic
    Tell Me More
    About That Cup.
    What Do You
    Usually Drink Out
    Of It?
    CREDIT: MASS CHALLENGE
    

    View Slide

65. CREDIT: MASS CHALLENGE
    

    View Slide

66. Transactional
    Confirmational
    Diagnostic
    CREDIT: MASS CHALLENGE
    

    View Slide

67. Interviews, Not Interrogations
    # 5
    Put your journalist cap on.
    Ask “Tell me about…” questions.
    STFU and listen.
    @iamctodd
    

    View Slide

68. What people say is not as telling as what
    people do. Humans are unreliable.
    @iamctodd
    

    View Slide

69. View Slide

70. Go Beyond Interviews
    # 6
    Interviews only tell so much.
    Walk in their shoes. Literally.
    Learn by participation, your and/or theirs.
    @iamctodd
    

    View Slide

71. Share what you learnt.
    @iamctodd
    

    View Slide

72. Move the search bar from
    bottom left to the top right
    

    View Slide

73. Research Findings
    Hi Perry,
    Please find our research report attached. You can view the
    prototype showing our suggestions at the following link.
    http://prototype.link/dkJd2ao
    Regards,
    C. Todd
    User research report 13-11-2016.docx [Auto-saved]
    

    View Slide

74. View Slide

75. Inspire Actions thru Insights
    # 8
    Don’t just write a report!
    Facts don’t move people. Stories and co-creation do.
    Focus on insights that inspire actions.
    @iamctodd
    

    View Slide

76. You’re only as good as your last performance.
    @iamctodd
    

    View Slide

77. Meet Jeff
    

    View Slide

78. Meet Jeff
    Works at AppCues
    

    View Slide

79. Meet Jeff
    Works at AppCues
    Cut teeth as Wistia's Director of Customer Happiness
    

    View Slide

80. Meet Jeff
    Works at AppCues
    Cut teeth as Wistia's Director of Customer Happiness
    Runs user sessions last Wednesday of every month
    

    View Slide

81. Meet Jeff
    Works at AppCues
    Cut teeth as Wistia's Director of Customer Happiness
    Runs user sessions last Wednesday of every month
    Be like Jeff
    

    View Slide

82. View Slide

83. View Slide

84. “When a user asked, ´What’s this button for?´
    an engineer responded with a perfect non-
    leading follow up: ´I’m not sure. What would
    you think it’s for?´ I nearly fell out of my chair.
    — Tristan Howard, Former Designer at AppCues
    

    View Slide

85. View Slide

86. View Slide

87. Make it a Habit
    # 9
    Continuous delivery discovery!
    Employ a more frequent cadence.
    Each question answered uncovers more questions.
    @iamctodd
    

    View Slide

88. Prepare to
    be Wrong
    Get Curious
    with Data
    Match Method
    to Outcome
    Prepare then
    Improvise
    Interviews, Not
    Interrogations
    Go Beyond
    Interviews
    Analyze
    Together
    Inspire Action
    with Insights
    Make it
    a Habit
    

    View Slide

89. LEARN
    BUILD MEASURE
    PRODUCT
    DATA
    IDEAS
    THE LEAN STARTUP
    What we think happens
    

    View Slide

90. LEARN
    BUILD MEASURE
    PRODUCT
    PRODUCT
    PRODUCT
    CREDIT: WOLF BREUNING
    BUILD
    BUILD
    What actually happens
    

    View Slide

91. View Slide

92. Live ‘em
    Learn ‘em
    Smash ‘em
    

    View Slide

93. ProductResearchRules.com
    @iamctodd
    Q & A @ 2:15pm
    

    View Slide