efficient hard-working team • But we're not focused on what problems we're solving and why; instead we get caught in the how (implementation details) • The Lean Startup will help us gain clarity on what we should and shouldn't build, and help us fail faster • We will be able to track our progress empirically • Which will help us prioritize valueable features; AND gain back confidence from the rest of the company and executive team • And will ultimately make us more confident in the product and in our longterm goals
rigorously and whole-heartedly Lean as we have with Agile • Implement a process within our weekly iteration to incorporate Lean standards and recognize that this will be a team effort •Help me develop tangible next steps so we can start being Lean TODAY
how to build a sustainable business. learning is validated scientifically by running frequent experiments that allow you to test elements of your vision. Startups require a new kind of management. geared towards working and learning in extreme uncertainty. Build, measure, learn. turn an idea into a product, measure how customers respond, learn whether to pivot or persevere. Innovation accounting (the boring stuff). a new type of accounting to hold innovators accountable.
seems counterintuitive to "manage" a startup, but this this is not true. focus and methodology = success. startups are fun, we don't like the boring stuff! many people think of process & management as boring where startups are supposed to dynamic & exciting... you know what's really exciting? to see startups succeed and change the world. the passion, energy and vision that people bring to these new ventures are resources too precious to waste.
stuff overload of features, detail, complexity make it so even tiny errors in assumptions = catastrophic outcomes tuning over time vs. precision make constant adjustments over time instead of having a complex plan to execute don't "achieve failure" it is possible to successfully, faithfully, rigorously execute a plan that was utterly flawed
good story to hide failure what it is the process of demonstrating empirically that a team has discovered valuable truths about a startup's present and future business prospects
to... see waste and then systematically eliminate it waste anything that does not contribute to a form of learning value what provides benefit to the customer everything else is waste examples of waste > debate and prioritization of effort that goes into features that customers don't like or don't use > incorrect strategic assumptions
customers a chance to try something, then measure their behavior learning is the essential unit of progress any effort not absolutely necessary for learning what customers want can be eliminated don't kid yourself you don't know what customers want until you prove it back up your validated learning collect empirical data from real customers
vision and what customers will accept; not to capitulate to what customers thought they wanted, or to tell customers what they ought to want understand our customers better this is the ONLY way to improve our product everything is an experiment every product, feature, marketing campaign (everything we do) is understood to be an experiment designed to achieve validated learning
have the problem you're trying to solve? 2. If there was a solution, would they buy it? 3. Would they buy it from us? 4. Can we build a solution to that problem? the common tendency is to skip straight to #4 and build a solution based on our assumptions that customers have the problem and we already know how to solve it kodak "Success is not delivering a feature; success is learning how to solve the customer's problem"
that emphasizes one element only our team's goal focus our energies as a team on minimizing the TOTAL time through this feedback loop the only way is through experimenting all the time we need to use experiments to help determine what techniques will work to solve our customer's problem strategy is not experimentation the only role of strategy is to help us figure out the right questions to ask
to be true our goal is to test them as QUICKLY as possible build an organization... that tests assumptions systematically, quickly, all the time perform rigorous testing without losing sight of our overall vision leaps of faith are what lead ideas; if they end up being true - awesome! but if not, we risk total failure what is needed is experimentation and testing to make sure we're on the right track
be sure you really understand any part of the business problem unless you go and see for yourself firsthand; it is unacceptable to take anything for granted or rely on the reports of others" story about the toyota sierra isn't it obvious who the target audience is? real key was the "kid appeal"
purpose is to start the process of learning... not to end it! early adopters use their imagination they don't want additional features or polish this is a waste of time and resources (and also hard to accept!) we can't be tied to the perfect vision in our heads an early, buggy, incomplete product seems like an unacceptable compromise; think about it in terms of learning and an iterative process
doubt, simplify! you WILL dramatically overestimate the amount of features needed in an MVP don't build a product to see how customers react this is backwards! and leads to a lot of waste the quality principle if we don't know who the customer is, we do not know what "quality" is; leverage "low-quality" design to find out what customers really care about have the courage! to put your assumptions to the test!
often provide a much-needed dose of reality don't blame a false negative visionaries often blame customer rejection on a flawed MVP because it's too small, too limited what is the solution? continue being dedicated to validated learning & ITERATION!
something (world-changing product), traditionally management says: there are only 2 explanations, a failure of execution or a failure to plan appropriately how do we report progress? and show learning? don't define progress by number of features, show progress by delivering value towards solving your user's problems, in a testable way with the goal of improving your core metrics the answer INNOVATION ACCOUNTING
our product is right now, confronting the hard truths that assessment reveals and then 2. devise experiments to learn how to move the real numbers closer to the ideal reflected in the business plan
enables you to prove objectively that you are learning how to grow a sustainable business and holds the leaders accountable the model supports accountability even when the model changes we have to turn our leap-of-faith assumptions into a quantitative business model
to establish real data on where the company is right now without a clear-eyed picture of your current status - no matter how far from the goal you may be - you cannot begin to track your progress 2. startups must tune the engine from the baseline toward the ideal this may take many attempts; once you exhaust micro changes and optimizations, you reach a decision point 3. pivot or persevere
learning milestone, the MVP the metrics you get from the MVP allow you to fill in baseline data this is valuable as the foundation for learning about customers and their reactions to a product even if it begins with extremely bad news test the riskiest assumptions first
targeted at improving one of the drivers of our growth model ex. improving design to make it easier for new customers to use BUT this presupposes that activation rate of new customers is the driver of growth; & that the baseline is lower than the company would like > to demonstrate validated learning, the design changes must improve the activation rate > if it doesn't, the new design is deemed a failure good design changes customer behavior for the better. period.
clear baseline metric, hypothesis on what they want to improve, and a set of experiments to test the hypothesis startup #2: sits around debating what would improve the product, implements several changes/features at once, and celebrates if there is any positive increase in any of the numbers which startup is more likely to be doing effective work and
we declare failure this creates this motivation, context, and space for more qualitative research these investigations produce new ideas which call for new hypotheses to be tested... leading to a possible pivot repeat this simple rhythm: establish the baseline, tune the engine, and make a decision to pivot or persevere
optimizing the product or its marketing will not yield significant results it's a high bar we're aiming for we're trying to elicit evidence that a sustainable business can be built around our product this standard can only be assessed once we've made clear, tangible predictions ahead of time and without these predictions... product and strategy decisions are far more difficult and time-consuming
the business is not seeing results" logical conclusion (old school) = team is not working effectively or efficiently this creates a downward spiral = more planning, specs more detailed, batch size increases, feedback is delayed, team morale decreases, etc. etc. learning milestones prevent this spiral! by emphasizing the more likely possibility: the team is executing - with discipline! - a plan that does not make sense
ad buys • whiz-bang demos • vanity metrics (metrics that give a rosy picture) • focusing on topline numbers anything you spend time on that makes people think you are being successful rather than spending that energy towards serving customers innovation accounting requires us to avoid the temptation to use these tricks you may be adding new customers, but you're not improving the yield on each group
• in the process of validating the problem • backlog • actively being built • done-done (tech) • in the process of being verified ◦ knowing whether a story was a good idea to have been done in the first place user stories are not considered complete until they lead to validated learning
OR customer interviews Kanban limits the number of stories in this bucket; only when a story has been validated can it be removed and "accepted" if the validation fails and it turns out the story is a bad idea, the relevant feature is removed from the product the key is to include the validation exercise at story writing so the whole team can be more productive test ALL changes, even "minor" ones team begins to measure productivity by validated learning, not production of new features
cause and effect ◦don't use vanity metrics 2. Accessible: understood by the employees and managers who are supposed to use them to guide their decision making ◦make reports as simple as possible (1 page, in English) ◦tangible, human, concrete units (website hit vs. person visiting the website) ◦cohorts: among the people who used our product in this period, here's how many of them exhibited each of the behaviors we care about ◦IMVU: 4 metrics - downloading product, logging in from computer, engaging in chat, upgrading to the paid version
to get what you want rather than as genuine feedback to guide future actions ◦data is credible to employees ◦"metrics are people, too" (need way to test data by hand by talking to real customers) ◦bonus: able to gain insights into WHY customers are behaving the way the data indicates
of entrepreneurship is the fun stuff and the glory story: the big idea, the business model, the whiteboard strategizing, and splitting up of the spoils 95% is the gritty work that is measured in innovation accounting: product prioritization decisions, deciding which customers to target or listen to, and having the courage to subject a grand vision to constant testing and feedback
progress to believe original strategic hypothesis is correct, or do we need to make a change? • misconception that LS offers formula for making this decision • no way to remove the human element - vision, intuition, judgement - from entrepreneurship, nor is that desirable • scientific approach channels human creativity into most productive form • no bigger destroyer of creative potential than misguided decision to persevere
or features it is about aligning our efforts with a business and product that are working to create value and drive growth successful pivots put us on a path towards growing a sustainable business
big leaps of faith: 1. Customers would be interested enough in the social network to sign up (Acquisition) 2. Votizen would be able to verify them as registered voters (Activation) 3. Customers who were verified would engaging in activism tools over time (Retention) 4. Engaged customers would tell their friends and recruit them to civic causes (Referral)
of feedback: •"I always wanted to get more involved; this makes it so much easier" •"The fact that you prove I'm a voter matters" •"There's no one here. What's the point of coming back?" zoom-in pivot - refocusing the product on what had previously been considered just one feature of a larger whole pivoted to @2gov, a social lobbying platform numbers went up
amount of time remaining in which a startup must achieve lift-off or fail •amount of remaining cash in the bank divided by monthly burn rate •true measure of runway is the number of opportunities you have left to make a fundamental change to your business strategy
you they wish they had made the decision sooner, for 3 reasons: 1. vanity metrics can skew conclusions and alter reality; robs the team of the belief that it's necessary to change 2. you have an unclear hypothesis which makes it impossible to experience complete failure, which gives you no impetus to make a radical change; "build it and see what happens" approach 3. you are afraid; biggest fear is having vision be wrong; even worse is if it's deemed wrong without having been given a real chance to prove itself
objective mindset tell-tale signs of the need to pivot: 1. the decreasing effectiveness of product experiments 2. general feeling that product development should be more productive whenever you see those symptoms, consider a pivot similar to agile product demo - ask your stakeholders, are you happy with our progress and may we continue?
of structured change designed to test a new fundamental hypothesis about the product, business model, and/or engine for growth zoom-in pivot: one feature becomes the product zoom-out pivot: product becomes feature of larger product customer segment pivot: solving right problem but for different customer customer need pivot: discover other problems that are more important to solve for the customer
you deliver the solution business architecture pivot: high margin low volume to low margin high volume engine growth pivot: viral, sticky or paid growth channel pivot: deliver solution through different channel with greater effectiveness technology pivot: more common in traditional businesses
features one at a time (agile) TDD/continuous deployment = andon cord pull, don't push respond to pull requests in the form of experiments that need to be run 1. formulate the hypothesis 2. run the experiment as quickly as possible Build, Measure, Learn planning works in REVERSE it's our hypothesis about the customer that is the pull request
simple rule: New customers come from the actions of past customers. 1. WOM 2. As a side effect of product usage (fashion, Venmo) 3. funded advertising (cost of acquiring that customer must be lower than revenue from that customer) 4. through repeat purchase or use
user starts using the product, they will continue to do so •track attrition or churn rate very carefully •if the rate of new customer acquisition exceeds the churn rate, the product will grow •the focus should be on improving customer retention above all else
share of the marketing (Hotmail) •powered by a feedback loop that can be quantified (viral loop) - its speed is determined by the viral coefficient •how many friends will come with a new user? •0.1 = one in every ten customers will recruit one friend •focus on increasing viral coefficient more than anything else •monetary exchange does not drive new growth; it is useful only as an indicator that customers value the product
cost to sign up a new customer? •to increase rate of growth, increase the revenue from each customer or drive down cost of acquiring a new customer •lifetime value of a customer is the feedback loop •you can't lose money on each customer and still grow •use advertising or salesforce to get new customers •tune over time so you can continue to monetize customers (e.g. if CPA goes up)
growth engine, but the most successful startups usually focus on just 1 •eric ries strongly recommends focusing on just 1 •entrepreneurs should have a strong hypothesis about which engine is most likely to work; if they do not, time spent out of the building with customers will quickly suggest one that seems profitable •pursue 1 and then pivot if necessary
you're not there yet the concept of the engine of growth can put the idea of product/market fit on more rigorous footing e.g. a startup with a viral coefficient of .9 or more is on the verge of success what matters isn't the raw numbers... what matters is the direction and degree of progress with Innovation Accounting
and performance for the current conditions •use the 5 whys to make incremental investments and evolve a startup's processes gradually •tie investments to the prevention of the most problematic symptoms •it's difficult to do although it sounds easy - MAKE IT A HABIT •can reveal very unexpected root causes (not technology but human; not human but process; etc)
issue, start taking incremental steps proportional to the impact of the problem •incremental improvements = incremental benefits •over time, the improvements compound, freeing up time and energy that was previously lost to firefighting and crisis management
integrate Lean into our daily lives, like we have with Agile? 2. What are our current assumptions about GEL? What is our engine of growth? 3. What should our first step be towards validated learning? 4. How can we leverage innovation accounting to report our progress to the rest of the company? 5. How can we manage this process as a team and support each other?
integrate Lean into our daily lives, like we have with Agile? - Learning demos (present hypothesis, experiments, results) as part of iteration - 4 states of stories: • in the process of validating the problem • backlog • actively being built • done-done (tech) • in the process of being verified ◦ knowing whether a story was a good idea to have been done in the first place user stories are not considered complete until they lead to validated learning
assumptions about GEL? What is our engine of growth? - Assumptions audit - Tomorrow morning - List of assumptions tonight (anything we haven't proven yet based on what we've developed) - What our users want, who our users are - What are our riskiest assumptions, so we can focus on them first Engine of growth - determine revenue model that we're aiming for (VPP) - what engine of growth fits with this
step be towards validated learning? 1. Do consumers recognize they have the problem you're trying to solve? 2. If there was a solution, would they buy it? 3. Would they buy it from us? 4. Can we build a solution to that problem? - Hypothesis on problem we're trying to solve - Success criteria for each hypothesis
innovation accounting to report our progress to the rest of the company? - Include learning demos are part of iteration demo for whole company (be transparent about hypothesis, experiments and results) - Make metrics available to everyone - Determine the correct metrics to track
this process as a team and support each other? - Lean Startup book club - Running Lean book club - Entrepreneur's Guide to CustDev (tbd) - Learning demos to keep us accountable - Take Innovation Accounting really seriously - Everyone holds each other accountable (you're not being lean!) - Be humble, check egos at the door, be open to criticism - Prepare sources to validate arguments
kgardnr
ysknsid25
kazuhitotakahashi
kitsuya0828
oracle4engineer
4su_para
techiro
takumiogawa
iotengineer22
mshindo
narirou
sugamasao
lynnandtonic
jponch
csswizardry
eileencodes
colly
morganepeng
iamctodd
davidbonilla
lara
aarron
palkan
notwaldorf
