Using Lean Startup &
Innovation Accounting
LUKAS FITTL
@lfittl
spark59.com
ENABLING
INNOVATION
Lean Startup is trademarked by Eric Ries and used with permission.
Business Model Canvas is created by Alex Osterwalder and licensed under CC-BY-SA.
Slide 2
Slide 2 text
About Myself & Spark59
Founded 2 tech startups,
worked with many others
Slide 3
Slide 3 text
Innovation in an
Agile Environment
Slide 4
Slide 4 text
Waterfall
Release!
Risk
Time
BUILD
BUILD
BUILD
Slide 5
Slide 5 text
Agile / XP / SCRUM
Risk
Time
Release!
BUILD
Release!
BUILD
Release!
BUILD
Release!
BUILD
Slide 6
Slide 6 text
Are we making our
product
better or worse?
Slide 7
Slide 7 text
Are we efficient?
vs
Are we effective?
Slide 8
Slide 8 text
The Definition of Done
should include testing
customer behaviour
Slide 9
Slide 9 text
Validated Learning
Slide 10
Slide 10 text
IDEAS
PRODUCT
DATA
MEASURE
BUILD
LEARN
Experiment
Slide 11
Slide 11 text
“State, before you do the work,
this is what I believe will happen,
and when you do the the work
see whether it happens or not.”
- Jabe Bloom
Slide 12
Slide 12 text
Hypotheses
instead of
Requirements
Slide 13
Slide 13 text
Requirement:
Users can create 3D avatars.
Slide 14
Slide 14 text
We believe that users will create a 3D
avatar and spend more time on other
users profiles.
Hypothesis:
Slide 15
Slide 15 text
50% of newly signed up users will create a 3D avatar,
Avg time spent on user profiles will go from 30s to 1min+
Falsifiable Hypothesis:
Slide 16
Slide 16 text
Be closer to reality,
and avoid
bias & comfort zones
Slide 17
Slide 17 text
User Stories != Experiments
Missing the “Measure” Stage
No Way to Validate the Learning
Slide 18
Slide 18 text
Deploy != Launch
if $rollout.active?(:chat, current_user)
...
else
...
end
Slide 19
Slide 19 text
Deploy Every Commit
Release To A Small Set of Customers
Launch The New Feature When Ready
Slide 20
Slide 20 text
Experiments need to be
shown to actual customers
Not just to your team!
Slide 21
Slide 21 text
Lean Stack:
Experiment Reports
Tools & Tactics
Slide 22
Slide 22 text
Background
EXPERIMENT REPORT Author:
Title: Created:
Experiment Scope
Minimum Build Plan
Lean Stack by Spark59.com
Falsifiable Hypothesis
Validated Learning
Results
Next Action
What are you trying to learn or achieve?
List the build plan step by step to get your measurement and experiment.
How many and/or how long will the experiment last?
[DATE]
[NAME]
[TITLE]
Your list of statements on the expected outcome of the experiment.
Use this format:
[Specific Repeatable Action] will [Expected Measurable Outcome]
Enter the data.
Summarize your learning from the experiment.
VALIDATED or INVALIDATED
Whatʼs the next experiment?
Slide 23
Slide 23 text
Background
EXPERIMENT REPORT Author:
Title: Created:
Experiment Scope
Minimum Build Plan
Lean Stack by Spark59.com
Falsifiable Hypothesis
Validated Learning
Results
Next Action
What are you trying to learn or achieve?
List the build plan step by step to get your measurement and experiment.
How many and/or how long will the experiment last?
[DATE]
[NAME]
[TITLE]
Your list of statements on the expected outcome of the experiment.
Use this format:
[Specific Repeatable Action] will [Expected Measurable Outcome]
Enter the data.
Summarize your learning from the experiment.
VALIDATED or INVALIDATED
Whatʼs the next experiment?
Slide 24
Slide 24 text
Give Context.
Why is this relevant?
Background
What are you trying to learn or achieve?
Slide 25
Slide 25 text
Quantify Your Goals
Falsifiable Hypothesis
Your list of statements on the expected outcome of
the experiment.
Use this format:
[Specific Repeatable Action] will
[Expected Measurable Outcome]
Slide 26
Slide 26 text
Set Scope & Timebox
Experiment Scope
How long will the experiment last?
Minimum Build Plan
List the build plan step by step to get your
measurement and experiment.
Slide 27
Slide 27 text
1. Context - Why is this relevant?
2. Quantify Your Goals
3. Set Scope & Timebox
Planning an Experiment
Slide 28
Slide 28 text
Test your
Value and your Growth
Hypothesis first
Slide 29
Slide 29 text
Validated Learning:
Define Hypothesis,
Build & Launch Experiment,
Learn from the Results
Slide 30
Slide 30 text
Concepts for making this work:
1. Product Ownership
2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing
“Shared understanding is about
getting everyone to walk in the
other person’s shoes a little.”
Bill Scott - UX at Netflix, PayPal
This includes the business owner’s shoes!
Slide 43
Slide 43 text
Are we making progress
towards our business goal?
Slide 44
Slide 44 text
Company-Wide
Metrics Dashboard
Tools & Tactics
Slide 45
Slide 45 text
No content
Slide 46
Slide 46 text
Actionable
Accessible
Auditable
Slide 47
Slide 47 text
Actionable
Tie specific repeatable actions to
observed results.
Slide 48
Slide 48 text
Accessible
Let everyone in the company
see and understand whats going on.
Slide 49
Slide 49 text
Auditable
Tie the numbers back to actual
customers, so you can verify the metric
and believe in it.
Slide 50
Slide 50 text
You need all three
AUDITABLE ACCESSIBLE
ACTIONABLE
Slide 51
Slide 51 text
Unit Economics
instead of
Gross Revenue
Slide 52
Slide 52 text
ACQUISITION
ACTIVATION
RETENTION
REVENUE
REFERRAL
How do users find you?
Do users have a reat first experience?
Do users come back?
How do you make money?
Do users tell others?
Dave McClure’s AARRR Metrics
Slide 53
Slide 53 text
ACQUISITION
ACTIVATION
RETENTION
REVENUE
REFERRAL
Value Metrics
How do users find you?
Do users have a great first experience?
Do users come back?
How do you make money?
Do users tell others?
Slide 54
Slide 54 text
ACQUISITION
ACTIVATION
RETENTION
REVENUE
REFERRAL
How do users find you?
Do users have a great first experience?
Do users come back?
How do you make money?
Do users tell others?
Growth Metrics
Slide 55
Slide 55 text
Cross-Functional
Pairing
Tools & Tactics
Slide 56
Slide 56 text
Weekly Strategy Meeting
Part 1: Ideation
Tools & Tactics
Slide 57
Slide 57 text
Avoid Brainstorming.
It causes
Premature Convergence
Around One Idea.
Slide 58
Slide 58 text
http://www.lostgarden.com/2010/08/visualizing-creative-process.html
By Daniel Cook
Slide 59
Slide 59 text
Design Studio
by Will Evans
@semanticwill
Slide 60
Slide 60 text
Observe & Explain
To The Group
Ideate
Alone
Discuss
In the Group
Sketch &
Write down!
Slide 61
Slide 61 text
Review Experiments
Only In Weekly Meeting
Focus On Efficiency
In Daily Stand-ups
Background
EXPERIMENT REPORT Author:
Title: Created:
Experiment Scope
Minimum Build Plan
Lean Stack by Spark59.com
Falsifiable Hypothesis
Validated Learning
Results
Next Action
What are you trying to learn or achieve?
List the build plan step by step to get your measurement and experiment.
How many and/or how long will the experiment last?
[DATE]
[NAME]
[TITLE]
Your list of statements on the expected outcome of the experiment.
Use this format:
[Specific Repeatable Action] will [Expected Measurable Outcome]
Enter the data.
Summarize your learning from the experiment.
VALIDATED or INVALIDATED
Whatʼs the next experiment?
Slide 71
Slide 71 text
Actual data you Measured,
based on your Hypothesis.
Results
Enter the data.
Slide 72
Slide 72 text
Validated
Invalidated
Inconclusive
Validated Learning
Summarize your learning from the experiment.
+ Why?
Slide 73
Slide 73 text
What will you do next
based on your learning?
Next Action
Whatʼs the next experiment?
Slide 74
Slide 74 text
✘
Failure is good.
Validated Learning is our
Measurement of Progress.
Inside-the-Building Comfort Zone
Customer Interview
(Paper) Prototype
MVP
“The Perfect Product”
Slide 82
Slide 82 text
Customer Interview
(Paper) Prototype
MVP
“The Perfect Product”
Get-out-of-the-Building Reality
Slide 83
Slide 83 text
Justify the Cost of MVPs
with Proof
Slide 84
Slide 84 text
Understand Problem:
Explorative Customer Interviews
De ne Solution:
User Testing With Prototypes
Validate Qualitatively:
Release MVP and hand-collect feedback
10s to 100s of customers
Verify Quantitatively:
Partial Rollout or Split Test and measure improvement
100s to 10,000s of customers