$30 off During Our Annual Pro Sale. View Details »

Lean Research Will Set You Free

Lean Research Will Set You Free

Farrah's Lean Day London talk about how Lean approaches qualitative research to make is more useful and scalable.

The Difference Engine

March 25, 2014
Tweet

Transcript

  1. lean research 

    will set you free*
    #leandaylondon
    @farrahbostic
    * but first it will piss you off

    View Slide

  2. happy birthday

    View Slide

  3. Hi, I’m
    @farrahbostic
    !
    !
    I run...

    View Slide

  4. i come from advertising.

    View Slide

  5. we bet only on holes-in-one

    View Slide

  6. this guy.

    View Slide

  7. but…

    View Slide

  8. $30,000,000,000+

    View Slide

  9. information request
    research RFP
    study design
    recruitment
    fielding
    Analysis
    Brand/product team
    research & insights
    research company
    recruiter/panel
    moderators
    research company

    View Slide

  10. why?

    View Slide

  11. because marketers want roi,
    but avoid defining what that is.
    You’d think it might be cost per acquisition, 

    or customer lifetime value, or revenue.
    But often, KPIs are less about business
    results, 

    and more about media performance.

    View Slide

  12. exhibit A: 

    “passion” is a kpi.
    http://www.cmo.com/content/cmo-com/home/articles/2013/9/26/KPIs_CWTK.html

    View Slide

  13. we’re not designing for data.

    View Slide

  14. “those numbers won’t
    count themselves.”

    View Slide

  15. what are we using the data for?

    View Slide

  16. how (a lot of) 

    marketing people deal with data

    View Slide

  17. data v.
    prestige
    Who cares about affluent baby boomers & 

    the advertisers who love them? 

    I want the 18-24 demo!!

    View Slide

  18. there’s a fear of small numbers
    Marketers worry about the signal-to-noise ratio.
    The smaller the sample, the more they worry
    about noise (wrong people, inaccuracies or
    untruths, groupthink, etc.)
    So they compensate with bigger sample sizes.
    But more numbers often make more noise.

    View Slide

  19. 20,000 people surveyed in three countries.
    100 shop-alongs and at-home interviews.
    Post-hoc analytics.
    More?
    leading to… analysis paralysis

    View Slide

  20. how marketers 

    ‘get out of the building’

    View Slide

  21. “why do we even do focus
    groups anymore? it’s so
    weird.”

    View Slide

  22. because, tradition.

    View Slide

  23. because, tradition.
    but also, because, bias.

    View Slide

  24. “statistically significant”
    “rigorous sample method”
    “information with authority”

    View Slide

  25. “The data will tell us what to do.”

    View Slide

  26. market research isn’t as
    sciencey as it pretends to be.

    View Slide

  27. research ≠ design
    Research is the process by which we understand problems.
    Design is the process by which we solve problems.

    View Slide

  28. The CEO’s Kids
    “I keep a lot of kids around me and ask them questions and peek around the corner
    when they think I'm not listening. My master plan was I had a lot of kids. I have six kids. 

    Three girls that are six, a 15-year-old, 18-year-old and 22-year-old. 

    I have the perfect research team in my house."
    http://acupofteawithphd.wordpress.com/2013/06/19/live-from-cannes-cannes-lions-2013-so-farpart-4-its-diddy/

    View Slide

  29. It's really hard to design products
    by focus groups. A lot of times,
    people don't know what they want
    until you show it to them.
    - Steve Jobs

    View Slide

  30. View Slide

  31. you ≠ steve
    jobs.

    View Slide

  32. over targeting leads us astray
    “[M]ost companies with a practiced discipline of
    listening to their best customers and identifying new
    products that promise greater profitability and growth
    are rarely able to build a case for investing in
    disruptive technologies until it is too late.”
    - Clay Christensen, The Innovator’s Dilemma

    View Slide

  33. research ignores switchers,
    rejectors, fringe & new users
    “Generally, disruptive technologies underperform
    established products in mainstream markets.
    But they have other features that a few fringe
    (and generally new) customers value.”
    - Clay Christensen, The Innovator’s Dilemma

    View Slide

  34. it’s complicated.

    View Slide

  35. “people lie.”

    View Slide

  36. it’s not these people’s fault
    focus groups are artificial.

    View Slide

  37. people are great.

    View Slide

  38. 1. we ask them to.

    View Slide

  39. 2. but often, they show us
    how we’re lying to ourselves.

    View Slide

  40. we’ve sacrificed the
    truth for ‘science’.

    View Slide

  41. how lean can help fix
    research.

    View Slide

  42. what is a lean company for?
    1. To specify customer value.
    2. To identify & implement the actions that create value.
    3. To remove anything that doesn’t create value.
    4. To analyze the results (and repeat).

    View Slide

  43. why does feedback & customer pull matter?
    Feedback and testing (a.k.a., research) 

    help teams settle questions and make decisions
    based on evidence not preferences.
    Continuous feedback helps teams see progress.

    View Slide

  44. people are data, too.
    (and vice versa)

    View Slide

  45. cool story, bro.

    View Slide

  46. we need to understand people and
    behavior so we can make better
    predictions and measure outcomes
    more effectively.

    View Slide

  47. theory v. information
    “Who needs theory when you have so
    much information? But this is categorically
    the wrong attitude to take toward
    forecasting, especially… where the data
    is so noisy. Statistical inferences are
    much stronger when backed up by
    theory or at least some deeper
    thinking about their root causes.”

    View Slide

  48. we should always be checking our bias

    View Slide

  49. and we should use data to trigger behavior

    View Slide

  50. validation

    View Slide

  51. scripts

    View Slide

  52. statistical 

    significance

    View Slide

  53. stop stockpiling data

    View Slide

  54. stop trapping insight
    in powerpoint decks

    View Slide

  55. empower the
    research manager
    We have to transform the research manager from a
    project manager into a Scrum master.
    Their role is to remove obstacles to progress, and to
    facilitate communication.
    Their responsibility to the team is to facilitate hypothesis
    testing, and to create opportunities to challenge
    assumptions.

    View Slide

  56. Do less, more often
    Lean research is about continuous
    customer, partner & team feedback.

    View Slide

  57. Use right-sized tools
    that are closest to hand

    View Slide

  58. Go and see.
    This is the new intimacy.

    View Slide

  59. the whole team can do this.
    http://giffconstable.com/2010/07/12-tips-for-early-customer-development-interviews/

    View Slide

  60. forget the horse’s mouth
    Don’t count on users to make
    your decisions for you - 

    that’s not their job.

    Don’t let the report be the end
    of your learning process - it’s just
    a way to memorialize what you’ve
    learned and provide a reference
    when you encounter similar
    issues again.

    View Slide

  61. To make research meaningful to the whole team, 

    the whole team needs:
    Investment
    Access
    Accountability
    nothing beats a conversation.

    View Slide

  62. View Slide

  63. make information
    reusable, not disposable.

    View Slide

  64. data should make us more empathetic
    If we design for data that helps us make decisions, 

    that helps us create value for our customers, 

    and that helps us develop empathy for people so
    deep that we can anticipate solutions to problems
    they can’t yet express…
    That will enable the kind of analysis sufficiently
    advanced to be indistinguishable from magic.

    View Slide

  65. be nice, and listen.

    View Slide

  66. what’s next?
    #leandaylondon
    @farrahbostic

    View Slide