new product that has more qualitative data than quantitative data to track, how do you go about selecting your KPIs? Submitted by: Someone from Motivate Design Perspectives from Naira Musallam THE PRODUCT COMMUNITY Steven Cohn Tristan Kromer Mark Hurst Sam Hysell Josh Wexler
on early stage software innovation. As a Solutions Director at Originate, he develops new business and helps potential partners shape their ideas into projects. He has advised global consulting firms, financial services companies, and early-stage entrepreneurs throughout every stage of the software development process, ranging from ideation and validation to fundraising rounds and commercial launches. JOSH WEXLER Josh’s Perspective The way I think about this is that quantitative data is so dangerous because it’s got this built-in feature that it looks so definite, and it can provide cover for whatever story you want to tell. Most data is all correlation data anyway - you’re not actually proving anything. So I always think you need qualitative data to do it right, and you have it work together with the quantitative data. But I actually think you need qualitative data to figure out what the KPIs should be once you have quantitative data. That’s how I think about them together, once you have a product. But KPIs for purely qualitative information is really difficult, because you’re trying to apply a rigid process to something that will likely produce a lot of unstructured information. You could make hosting the interviews and collecting information the KPI, so that it ensures you’re constantly talking to users. But it’s difficult to measure the output of each interview itself.
be a good father, husband and entrepreneur . He sold his first two companies to LivingSocial and TripAdvisor. Validately is Steven's third company. Validately's mission is to help teams build better products, by making it easy and affordable to test prototypes and live sites on customers or potential customers. Try it for free at Validately.com. Message Steven on LinkedIn with further questions STEVEN COHN Steven’s Perspective First, I would say, don't just pick a KPI for the sake of it. If it doesn't actually link to product growth/usage than it is not worth tracking. Second, if the metric truly can't be quantified, then look for metrics that demonstrate success with that metric. For example, let's say you want to learn track the qualitative metric of "happiness with the product". One way is to ask in a survey. But people don't always match survey responses with real actions. A better approach is to give people an easy way to Tweet about your product or invite a friend. Then track the number of customers who socially share your product. This is a good metric to indicate "happiness" as people generally do not share a product that they don't like (unless they bash it).
at New York University, Naira teaches Applied Statistics, Analytical Skills, and National Security and Middle East Affairs at the Center for Global Affairs. She also serves as an Adjunct Professor at Columbia University and in its collaborated program at the United States Military Academy at West Point where she teaches graduate level research courses. Message Naira on LinkedIn about opportunities NAIRA MUSALLAM Naira’s Perspective Qualitative Data is rich in nature, and allows product managers to bring their success stories to life. One of the advantages of qualitative data is that you are able to code it into both qualitative and quantitative performance indicators. Qualitative indicators require thematic coding that basically categorizes the qualitative data into categories that are meaningful, while quantitative indicators, following the thematic analysis, transform categories into numeric codes that are relevant. This enables clients to then conduct various useful analysis ranging from descriptive statistics (such as rating of a product, or frequency of a specific product theme) , correlations, and even predictive models about customer's preferences, factors effecting them and so forth. The extent to which one is able to conduct a certain analysis is largely dependent on the nature and complexity of the qualitative data you have. Below is a link that provides an example of how to quantify/code qualitative data that comes mainly from interviews: http://www.uniteforsight.org/global-health- university/quantify-research
startup coach, he works with innovation teams to run at least one experiment/research per week to improve their product and business model. TRISTAN KROMER Tristan’s Perspective Stop thinking about KPIs. The "I" stands for Indicator. e.g. Our 60 day retention rate might be a strong indicator of customer satisfaction. We quantitatively measure the 60 day retention rate, but the goal is to measure customer satisfaction. The retention rate itself is not the goal. The goal is what we want to learn. It's a question. In this case, the goal is, "Are customers satisfied with out product?” If we can measure customer satisfaction by getting on the phone and calling 5 customers for qualitative feedback, then let's do that. If we can double check that qualitative data with a small sample size quantitative experiment, even better. Running multiple qual & small sample size quant experiments towards the same learning goal can allow you to triangulate the true User Experience. Read more best practices on Usecase.org
improve the customer experience. Since then the Creative Good team has brought our strategy consulting to premier clients in media, finance, retail, consumer products, travel & hospitality, health care, automotive, information services, and other fields. MARK HURST Mark’s Perspective At Creative Good we always recommend that KPIs be based in the full business context... that is to say, before evaluating the data (whether qual or quant), look at the entire situation: what are the short- and medium-term goals for the product? What's the competitive landscape? How do you define success? etc. ... once this is fully established (along with a consensus among the decision makers) it is generally much easier to select or define the KPIs that should be tracked. Read more best practices on Usecase.org
and leverage lean startup principles. He is currently helping the Moves the Needle community succeed by helping large organizations operationalize innovation efforts. Message Sam on LinkedIn with further questions SAM HYSELL Sam’s Perspective First step is to define what you're seeking to learn. We all hear what gets measured gets managed and the main objective here is to learn and generate insights that will help grow our business and make our customers more successful. The most common learning objectives I see stem from the following three questions: Who is my customer? What do they need? Why do they need it? When it comes to determining your KPIs simply track as to how much information are you getting that helps you answer those questions better, and if you're able to take action and implement those insights. Sometimes that means calling and speaking to your customers. Other times, realize that whatever the qualitative data is, qualitative date quickly becomes quantitative when 5 out of 10 people mention the same thing. As you generate more qualitative, see what trends emerge. Those trends are often where the greatest insights lie. In closing, before measuring anything, determine what you want to learn? Then measure whatever helps you better understand that and find trends. Lastly, seek to take action. The value of learning and data lies in it's application. All the insights in the world don't mean anything until action is taken, but the right insights and right action can mean everything. Read more best practices on Usecase.org
alphaux
addyosmani
sachag
tanoku
samanthasiow
notwaldorf
chrislema
aki_iinuma
speakerdeck
sugarenia
ammeep
chrisshort
