$30 off During Our Annual Pro Sale. View Details »

Product Research Rules! MtPCon London 2019

Product Research Rules! MtPCon London 2019

Product leaders are constantly asking if they are building the right features and products. One core reason is that there just isn't enough research conducted toensure that the identified solutions match the problems. This talk covers some rules to help guide you.

C. Todd Lombardo

October 18, 2019
Tweet

More Decks by C. Todd Lombardo

Other Decks in Business

Transcript

  1. Product
    Research
    C. Todd Lombardo — @iamctodd

    View Slide

  2. “Are we building the right thing?”
    @iamctodd
    — Martin Eriksson, earlier today

    View Slide

  3. Meet Bill

    View Slide

  4. Meet Bill
    Serial entrepreneur

    View Slide

  5. Meet Bill
    Serial entrepreneur
    Worked for Steve Jobs

    View Slide

  6. Meet Bill
    Serial entrepreneur
    Worked for Steve Jobs
    Fifteen years in Silicon Valley

    View Slide

  7. Meet Bill
    Serial entrepreneur
    Worked for Steve Jobs
    Fifteen years in Silicon Valley
    Sold a startup for $850 million… at age 29

    View Slide

  8. I literally don’t think there’s
    anything to be learned
    from other people’s stuff.”
    — Bill Nguyen

    View Slide

  9. View Slide

  10. “On first blush, there's
    almost zero chance we
    return to this app. Our
    friends aren't in it. It's
    confusing. Sharing photos
    isn't easy or intuitive.
    It's not very fun.
    — Business Insider

    View Slide

  11. Within 30 minutes I realized.. We totally
    fucked up. I thought we were going to
    build a better Facebook. My reaction was
    like putting your finger into a light socket.
    You know something went very wrong.
    — Bill Nguyen

    View Slide

  12. Research Sprints 8

    View Slide

  13. 9

    View Slide

  14. KILLEDBYGOOGLE.COM

    View Slide

  15. GCEMETERY.CO

    View Slide

  16. View Slide

  17. We’re Not Good at This…
    STANDISH GROUP
    Always
    7 %
    Often
    13 %
    Sometimes
    16 %
    Rarely
    19 %
    Never
    45 %

    View Slide

  18. Why Products Fail
    No market need
    Ran out of cash
    Not the right team
    Got outcompeted
    Pricing/cost issue
    Poor product
    Lack a business model
    Poor marketing
    Ignore Customers
    entrepreneur.com & cbinsights.com

    View Slide

  19. Why Products Fail
    No market need
    Ran out of cash
    Not the right team
    Got outcompeted
    Pricing/cost issue
    Poor product
    Lack a business model
    Poor marketing
    Ignore Customers
    entrepreneur.com & cbinsights.com
    BAD PODUCT RESEARCH

    View Slide

  20. View Slide

  21. Research is expensive!

    View Slide

  22. Research is expensive!
    We already know.

    View Slide

  23. Research is expensive!
    We already know.
    It will take too long!

    View Slide

  24. Do you want waste 45% of your resources??
    No? Then do some f*cking research!!
    @iamctodd

    View Slide

  25. Product”
    Research?!

    View Slide

  26. View Slide

  27. Studious inquiry or examination; especially
    investigation or experimentation aimed at
    the discovery and interpretation of facts,
    revision of accepted theories or laws in the
    light of new facts, or practical application of
    such new or revised theories or laws.
    — Merriam Webster

    View Slide

  28. View Slide

  29. What people do?
    Qualitative
    Understanding
    User Research

    View Slide

  30. Market Research
    What markets do?
    Quantitative
    Framing

    View Slide

  31. Product Analytics
    Usage of a product
    Quantitative
    Trends & Patterns

    View Slide

  32. User Research
    Market Research
    Product Analytics
    Product
    Research

    View Slide

  33. User Research Market Research Product Analytics
    Generative Exploratory
    Descriptive Descriptive Descriptive
    Evaluative Causal Diagnostic
    Predictive Predictive
    Prescriptive

    View Slide

  34. View Slide

  35. Start with a beginner’s mind
    @iamctodd

    View Slide

  36. Meet C. Todd

    View Slide

  37. Meet C. Todd
    Serial product guy

    View Slide

  38. Meet C. Todd
    Serial product guy
    Did not work for Steve Jobs

    View Slide

  39. Meet C. Todd
    Serial product guy
    Did not work for Steve Jobs
    Fifteen years in startups & enterprises

    View Slide

  40. Meet C. Todd
    Serial product guy
    Did not work for Steve Jobs
    Fifteen years in startups & enterprises
    Grew product from $5M to $36M.. in 30 months

    View Slide

  41. View Slide

  42. Prepare to be wrong
    # 1
    Because you will be.
    Humans prefer to be right than do right.
    Check your ego. Check your assumptions.
    @iamctodd

    View Slide

  43. I was totally wrong. I was
    just wrong. See, I don’t have
    a problem being wrong.”
    — Bill Nguyen

    View Slide

  44. In order to be wrong, you need an
    understanding of what you already know.
    @iamctodd

    View Slide

  45. Meet Catt

    View Slide

  46. Meet Catt
    Ace designer & developer

    View Slide

  47. Meet Catt
    Ace designer & developer
    Works for large marketplace platform

    View Slide

  48. Meet Catt
    Ace designer & developer
    Works for large marketplace platform
    Fourteen years in gaming, design, and tech

    View Slide

  49. Meet Catt
    Ace designer & developer
    Works for large marketplace platform
    Fourteen years in gaming, design, and tech
    Learned to code when she was 10

    View Slide

  50. Available Data
    Annual surveys, social
    media, customer forums,
    customer emails, product
    analytics, interviews

    View Slide

  51. Available Data
    Annual surveys, social
    media, customer forums,
    customer emails, product
    analytics, interviews
    I can't find my listing and search in
    the rankings and results! Did they
    change the search algorithm?

    View Slide

  52. Identified Problem
    Marketplace sellers didn't
    understand how they were
    being discovered by buyers
    Available Data
    Annual surveys, social
    media, customer forums,
    customer emails, product
    analytics, interviews
    I can't find my listing and search in
    the rankings and results! Did they
    change the search algorithm?

    View Slide

  53. Identified Problem
    Marketplace sellers didn't
    understand how they were
    being discovered by buyers
    Available Data
    Annual surveys, social
    media, customer forums,
    customer emails, product
    analytics, interviews
    Research Question
    How do sellers learn how to
    market themselves?
    I can't find my listing and search in
    the rankings and results! Did they
    change the search algorithm?

    View Slide

  54. # 2
    Get Curious with Your Data
    Start with the data you have.
    Generate questions to answer.
    Settle on a research question to answer.
    @iamctodd

    View Slide

  55. What you ask and how you speak matters.
    @iamctodd

    View Slide

  56. So, what do you think of this feature?
    Would you use this new feature?
    How would you design this?
    How likely are you to recommend this…

    View Slide

  57. View Slide

  58. CREDIT: MASS CHALLENGE

    View Slide

  59. I See You Have A
    Coffee Cup. We
    Make Coffee.
    Would You Like To
    Buy A Coffee?
    CREDIT: MASS CHALLENGE

    View Slide

  60. Transactional
    I See You Have A
    Coffee Cup. We
    Make Coffee.
    Would You Like To
    Buy A Coffee?
    CREDIT: MASS CHALLENGE

    View Slide

  61. Transactional
    I See You Have A
    Coffee Cup. We
    Make Coffee.
    Would You Like To
    Buy A Coffee?
    Wouldn´t It Be
    Cool If You Had A
    Premium Cold-
    Brew Coffee To
    Fill That Cup?
    CREDIT: MASS CHALLENGE

    View Slide

  62. Transactional
    I See You Have A
    Coffee Cup. We
    Make Coffee.
    Would You Like To
    Buy A Coffee?
    Confirmational
    Wouldn´t It Be
    Cool If You Had A
    Premium Cold-
    Brew Coffee To
    Fill That Cup?
    CREDIT: MASS CHALLENGE

    View Slide

  63. Transactional
    I See You Have A
    Coffee Cup. We
    Make Coffee.
    Would You Like To
    Buy A Coffee?
    Confirmational
    Wouldn´t It Be
    Cool If You Had A
    Premium Cold-
    Brew Coffee To
    Fill That Cup?
    Tell Me More
    About That Cup.
    What Do You
    Usually Drink Out
    Of It?
    CREDIT: MASS CHALLENGE

    View Slide

  64. Transactional
    I See You Have A
    Coffee Cup. We
    Make Coffee.
    Would You Like To
    Buy A Coffee?
    Confirmational
    Wouldn´t It Be
    Cool If You Had A
    Premium Cold-
    Brew Coffee To
    Fill That Cup?
    Diagnostic
    Tell Me More
    About That Cup.
    What Do You
    Usually Drink Out
    Of It?
    CREDIT: MASS CHALLENGE

    View Slide

  65. CREDIT: MASS CHALLENGE

    View Slide

  66. Transactional
    Confirmational
    Diagnostic
    CREDIT: MASS CHALLENGE

    View Slide

  67. Interviews, Not Interrogations
    # 5
    Put your journalist cap on.
    Ask “Tell me about…” questions.
    STFU and listen.
    @iamctodd

    View Slide

  68. What people say is not as telling as what
    people do. Humans are unreliable.
    @iamctodd

    View Slide

  69. View Slide

  70. Go Beyond Interviews
    # 6
    Interviews only tell so much.
    Walk in their shoes. Literally.
    Learn by participation, your and/or theirs.
    @iamctodd

    View Slide

  71. Share what you learnt.
    @iamctodd

    View Slide

  72. Move the search bar from
    bottom left to the top right

    View Slide

  73. Research Findings
    Hi Perry,
    Please find our research report attached. You can view the
    prototype showing our suggestions at the following link.
    http://prototype.link/dkJd2ao
    Regards,
    C. Todd
    User research report 13-11-2016.docx [Auto-saved]

    View Slide

  74. View Slide

  75. Inspire Actions thru Insights
    # 8
    Don’t just write a report!
    Facts don’t move people. Stories and co-creation do.
    Focus on insights that inspire actions.
    @iamctodd

    View Slide

  76. You’re only as good as your last performance.
    @iamctodd

    View Slide

  77. Meet Jeff

    View Slide

  78. Meet Jeff
    Works at AppCues

    View Slide

  79. Meet Jeff
    Works at AppCues
    Cut teeth as Wistia's Director of Customer Happiness

    View Slide

  80. Meet Jeff
    Works at AppCues
    Cut teeth as Wistia's Director of Customer Happiness
    Runs user sessions last Wednesday of every month

    View Slide

  81. Meet Jeff
    Works at AppCues
    Cut teeth as Wistia's Director of Customer Happiness
    Runs user sessions last Wednesday of every month
    Be like Jeff

    View Slide

  82. View Slide

  83. View Slide

  84. “When a user asked, ´What’s this button for?´
    an engineer responded with a perfect non-
    leading follow up: ´I’m not sure. What would
    you think it’s for?´ I nearly fell out of my chair.
    — Tristan Howard, Former Designer at AppCues

    View Slide

  85. View Slide

  86. View Slide

  87. Make it a Habit
    # 9
    Continuous delivery discovery!
    Employ a more frequent cadence.
    Each question answered uncovers more questions.
    @iamctodd

    View Slide

  88. Prepare to
    be Wrong
    Get Curious
    with Data
    Match Method
    to Outcome
    Prepare then
    Improvise
    Interviews, Not
    Interrogations
    Go Beyond
    Interviews
    Analyze
    Together
    Inspire Action
    with Insights
    Make it
    a Habit

    View Slide

  89. LEARN
    BUILD MEASURE
    PRODUCT
    DATA
    IDEAS
    THE LEAN STARTUP
    What we think happens

    View Slide

  90. LEARN
    BUILD MEASURE
    PRODUCT
    PRODUCT
    PRODUCT
    CREDIT: WOLF BREUNING
    BUILD
    BUILD
    What actually happens

    View Slide

  91. View Slide

  92. Live ‘em
    Learn ‘em
    Smash ‘em

    View Slide

  93. ProductResearchRules.com
    @iamctodd
    Q & A @ 2:15pm

    View Slide