Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Research Rebooted: Why Market Research is Broken, How Lean Can Help Fix It

The Difference Engine
September 17, 2013
85

Research Rebooted: Why Market Research is Broken, How Lean Can Help Fix It

From Lean Day West, September 17, 2013 in Portland, OR

The Difference Engine

September 17, 2013
Tweet

Transcript

  1. Research Rebooted
    Why Most Market Research Is Broken,
    And How Lean Can Help Fix It

    View full-size slide

  2. what’s wrong with
    research?

    View full-size slide

  3. the value is broken.

    View full-size slide

  4. $30,000,000,000+

    View full-size slide

  5. time.
    it’s in a box.
    and it slips.

    View full-size slide

  6. information request
    research RFP
    study design
    recruitment
    fielding
    Analysis
    Brand/product team
    research & insights
    research company
    recruiter/panel
    moderators
    research company

    View full-size slide

  7. The CEO’s Kids
    “I keep a lot of kids around me and ask them questions and peek around
    the corner when they think I'm not listening. My master plan was I had a
    lot of kids. I have six kids. Three girls that are six, a 15-year-old, 18-year-
    old and 22-year-old. I have the perfect research team in my house."
    http://acupofteawithphd.wordpress.com/2013/06/19/live-from-cannes-cannes-lions-2013-so-farpart-4-its-diddy/

    View full-size slide

  8. the design is broken.

    View full-size slide

  9. The fear of small numbers
    Managers worry about the
    signal-to-noise ratio.
    The smaller the sample, the
    more we worry about noise
    (wrong people, inaccuracies or
    untruths, groupthink, etc.)
    So they compensate with bigger
    sample sizes.
    But more numbers
    make more noise.

    View full-size slide

  10. over targeting leads us astray
    “[M]ost companies with a practiced discipline
    of listening to their best customers and
    identifying new products that promise greater
    profitability and growth are rarely able to
    build a case for investing in disruptive
    technologies until it is too late.”
    - Clay Christensen, The Innovator’s Dilemma

    View full-size slide

  11. research is often designed to confirm biases.
    when it doesn’t, it’s painful.

    View full-size slide

  12. our customer or our design target?

    View full-size slide

  13. research ignores switchers,
    rejectors, fringe & new users
    “Generally, disruptive technologies
    underperform established products in
    mainstream markets. But they have other
    features that a few fringe (and generally new)
    customers value.”
    - Clay Christensen, The Innovator’s Dilemma

    View full-size slide

  14. the process is broken.

    View full-size slide

  15. garbage in, garbage out
    “The research manager’s job is
    to manage the hell out of the research.”
    - Michelle Crawford

    View full-size slide

  16. quant über alles

    View full-size slide

  17. packing 10 lbs into a 5 lb bag
    30 minute surveys
    2 hour focus groups
    10 page discussion guides
    Too many questions, too much data,
    not enough understanding

    View full-size slide

  18. stockpiling data

    View full-size slide

  19. trapping insight in
    powerpoint decks

    View full-size slide

  20. It's really hard to design products
    by focus groups. A lot of times,
    people don't know what they want
    until you show it to them.
    - Steve Jobs

    View full-size slide

  21. you ≠ steve jobs.

    View full-size slide

  22. research ≠ design
    Research is the process by which we
    understand problems.
    Design is the process by which we
    solve problems.

    View full-size slide

  23. how lean can help fix
    research.

    View full-size slide

  24. what is a lean company for?
    1. To specify customer value.
    2. To identify & implement the actions that
    create that value.
    3. To remove anything that doesn’t create value.
    4. To analyze the results (and repeat).

    View full-size slide

  25. why does feedback &
    customer pull matter?
    Feedback and testing (a.k.a., research)
    help teams settle questions and make decisions
    based on evidence not preferences.
    Continuous feedback helps teams see progress.

    View full-size slide

  26. Go and see.
    This is the new intimacy.

    View full-size slide

  27. how do we do lean
    research?

    View full-size slide

  28. the owner has to change

    View full-size slide

  29. empower the
    research manager
    We have to transform the research manager
    from a project manager into a Scrum master.
    Their role is to remove obstacles to progress,
    and to facilitate communication.
    Their responsibility to the team is to facilitate
    hypothesis testing, and to create
    opportunities to challenge assumptions.

    View full-size slide

  30. the tools have to change

    View full-size slide

  31. use right-sized tools

    View full-size slide

  32. Use the tools
    closest to hand

    View full-size slide

  33. Talk to people where
    it’s relevant or convenient
    i.e., NOT HERE

    View full-size slide

  34. talk to them here.

    View full-size slide

  35. the whole team can do this.
    http://giffconstable.com/2010/07/12-tips-for-early-customer-development-interviews/

    View full-size slide

  36. the flow has to change

    View full-size slide

  37. Do less, more often
    Lean research is about continuous
    customer, partner & team feedback.

    View full-size slide

  38. set people up for success

    View full-size slide

  39. the output has to change.

    View full-size slide

  40. data doesn’t “tell” you
    what to do.

    View full-size slide

  41. nothing beats a conversation.
    To make research
    meaningful to the
    whole team, the
    whole team needs:
    Investment
    Access
    Accountability

    View full-size slide

  42. forget the horse’s mouth
    Don’t count on users to make your
    decisions for you - that’s not their job.
    Don’t let the report be the end of your
    learning process - it’s just a way to
    memorialize what you’ve learned and provide
    a reference when you encounter similar
    issues again.

    View full-size slide

  43. information should be
    reusable, not disposable.

    View full-size slide

  44. watch this space.
    and thanks, this was fun.

    View full-size slide