Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Research Rebooted: Why Market Research is Broken, How Lean Can Help Fix It

The Difference Engine
September 17, 2013
79

Research Rebooted: Why Market Research is Broken, How Lean Can Help Fix It

From Lean Day West, September 17, 2013 in Portland, OR

The Difference Engine

September 17, 2013
Tweet

Transcript

  1. Research Rebooted
    Why Most Market Research Is Broken,
    And How Lean Can Help Fix It

    View Slide

  2. what’s wrong with
    research?

    View Slide

  3. the value is broken.

    View Slide

  4. $30,000,000,000+

    View Slide

  5. time.
    it’s in a box.
    and it slips.

    View Slide

  6. information request
    research RFP
    study design
    recruitment
    fielding
    Analysis
    Brand/product team
    research & insights
    research company
    recruiter/panel
    moderators
    research company

    View Slide

  7. The CEO’s Kids
    “I keep a lot of kids around me and ask them questions and peek around
    the corner when they think I'm not listening. My master plan was I had a
    lot of kids. I have six kids. Three girls that are six, a 15-year-old, 18-year-
    old and 22-year-old. I have the perfect research team in my house."
    http://acupofteawithphd.wordpress.com/2013/06/19/live-from-cannes-cannes-lions-2013-so-farpart-4-its-diddy/

    View Slide

  8. the design is broken.

    View Slide

  9. The fear of small numbers
    Managers worry about the
    signal-to-noise ratio.
    The smaller the sample, the
    more we worry about noise
    (wrong people, inaccuracies or
    untruths, groupthink, etc.)
    So they compensate with bigger
    sample sizes.
    But more numbers
    make more noise.

    View Slide

  10. over targeting leads us astray
    “[M]ost companies with a practiced discipline
    of listening to their best customers and
    identifying new products that promise greater
    profitability and growth are rarely able to
    build a case for investing in disruptive
    technologies until it is too late.”
    - Clay Christensen, The Innovator’s Dilemma

    View Slide

  11. research is often designed to confirm biases.
    when it doesn’t, it’s painful.

    View Slide

  12. our customer or our design target?

    View Slide

  13. research ignores switchers,
    rejectors, fringe & new users
    “Generally, disruptive technologies
    underperform established products in
    mainstream markets. But they have other
    features that a few fringe (and generally new)
    customers value.”
    - Clay Christensen, The Innovator’s Dilemma

    View Slide

  14. the process is broken.

    View Slide

  15. garbage in, garbage out
    “The research manager’s job is
    to manage the hell out of the research.”
    - Michelle Crawford

    View Slide

  16. quant über alles

    View Slide

  17. packing 10 lbs into a 5 lb bag
    30 minute surveys
    2 hour focus groups
    10 page discussion guides
    Too many questions, too much data,
    not enough understanding

    View Slide

  18. stockpiling data

    View Slide

  19. trapping insight in
    powerpoint decks

    View Slide

  20. It's really hard to design products
    by focus groups. A lot of times,
    people don't know what they want
    until you show it to them.
    - Steve Jobs

    View Slide

  21. View Slide

  22. you ≠ steve jobs.

    View Slide

  23. research ≠ design
    Research is the process by which we
    understand problems.
    Design is the process by which we
    solve problems.

    View Slide

  24. how lean can help fix
    research.

    View Slide

  25. what is a lean company for?
    1. To specify customer value.
    2. To identify & implement the actions that
    create that value.
    3. To remove anything that doesn’t create value.
    4. To analyze the results (and repeat).

    View Slide

  26. why does feedback &
    customer pull matter?
    Feedback and testing (a.k.a., research)
    help teams settle questions and make decisions
    based on evidence not preferences.
    Continuous feedback helps teams see progress.

    View Slide

  27. Go and see.
    This is the new intimacy.

    View Slide

  28. how do we do lean
    research?

    View Slide

  29. the owner has to change

    View Slide

  30. empower the
    research manager
    We have to transform the research manager
    from a project manager into a Scrum master.
    Their role is to remove obstacles to progress,
    and to facilitate communication.
    Their responsibility to the team is to facilitate
    hypothesis testing, and to create
    opportunities to challenge assumptions.

    View Slide

  31. validation

    View Slide

  32. the tools have to change

    View Slide

  33. use right-sized tools

    View Slide

  34. Use the tools
    closest to hand

    View Slide

  35. View Slide

  36. Talk to people where
    it’s relevant or convenient
    i.e., NOT HERE

    View Slide

  37. talk to them here.

    View Slide

  38. the whole team can do this.
    http://giffconstable.com/2010/07/12-tips-for-early-customer-development-interviews/

    View Slide

  39. the flow has to change

    View Slide

  40. Do less, more often
    Lean research is about continuous
    customer, partner & team feedback.

    View Slide

  41. set people up for success

    View Slide

  42. the output has to change.

    View Slide

  43. data doesn’t “tell” you
    what to do.

    View Slide

  44. View Slide

  45. nothing beats a conversation.
    To make research
    meaningful to the
    whole team, the
    whole team needs:
    Investment
    Access
    Accountability

    View Slide

  46. forget the horse’s mouth
    Don’t count on users to make your
    decisions for you - that’s not their job.
    Don’t let the report be the end of your
    learning process - it’s just a way to
    memorialize what you’ve learned and provide
    a reference when you encounter similar
    issues again.

    View Slide

  47. information should be
    reusable, not disposable.

    View Slide

  48. watch this space.
    and thanks, this was fun.

    View Slide