Actionable Metrics

Talk at eClub in Prague, 28th March 2013

1. Are you making your product better or worse? LUKAS FITTL @lfittl spark59.com ACTIONABLE METRICS Lean Startup is trademarked by Eric Ries and used with permission. Business Model Canvas is created by Alex Osterwalder and licensed under CC-BY-SA. Friday, 29 March 2013
2. About Myself & Spark59 Founded 2 tech startups, worked with many others Friday, 29 March 2013
3. Friday, 29 March 2013
4. “Am I making progress into the right direction?” Friday, 29 March 2013
5. $$$/users/etc Time Friday, 29 March 2013
6. Product/ Market Fit 1. Deliver Value 2. Accelerate Growth Friday, 29 March 2013
7. Product/ Market Fit 1. Value 2. Growth Friday, 29 March 2013
8. Product/Market Fit: The moment when you can consistently deliver value to the right people. Friday, 29 March 2013
9. Focus on Outcome, not Tasks. Friday, 29 March 2013
10. $10000 / month $2000 / month Now June 2013 ? Friday, 29 March 2013
11. Break Revenue Targets, down into Unit Economics. Friday, 29 March 2013
12. Dave McClure’s AARRR ACQUISITION ACTIVATION RETENTION REVENUE REFERRAL How do users find you? Do users have a reat first experience? Do users come back? How do you make money? Do users tell others? Friday, 29 March 2013
13. $2000 / month ACQ ACT 20% to paid at $35 RET REV 20% stay for 1 week 8% activate 5% sign up 2 paid $70 / day 10 retained 50 activated 625 signed up 12500 unique visitors Each Day: Friday, 29 March 2013
14. Cohort Analysis Friday, 29 March 2013
15. Cohort: Group of people that share a common characteristic over a period of time. Friday, 29 March 2013
16. Friday, 29 March 2013
17. Track people, not events. Friday, 29 March 2013
18. Group users by sign-up date and track their lifecycle. Friday, 29 March 2013
19. Friday, 29 March 2013
20. Friday, 29 March 2013
21. Cohort Analysis for Retention Friday, 29 March 2013
22. Friday, 29 March 2013
23. Cohort Analysis w Google Analytics http://danhilltech.tumblr.com/post/12509218078/startups-hacking-a-cohort-analysis-with-google Friday, 29 March 2013
24. $10000 / month $2000 / month Now June 2013 ? Friday, 29 March 2013
25. $2000 / month ACQ ACT 20% to paid at $35 RET REV 20% stay for 1 week 8% activate 5% sign up 2 paid $70 / day 10 retained 50 activated 625 signed up 12500 Unique Visitors / day Cohort of February 12th, 2013 Friday, 29 March 2013
26. $2000 / month Now ACQ ACT 20% to paid at $35 RET REV 20% stay for 1 week 8% activate 5% sign up $70 / day 12500 Uniques $10000 / month June 2013 20% to paid at $35 20% stay for 1 week 20% activate 10% sign up $350 / day 12500 Uniques 2 10 50 625 10 50 250 1250 Friday, 29 March 2013
27. $2000 / month Now ACQ ACT 20% to paid at $35 RET REV 20% stay for 1 week 8% activate 5% sign up $70 / day 12500 Uniques $10000 / month June 2013 20% to paid at $35 20% stay for 1 week 20% activate 10% sign up $350 / day 12500 Uniques 2 10 50 625 10 50 250 1250 Friday, 29 March 2013
28. Delivering Value Friday, 29 March 2013
29. Customers need to experience the value, otherwise they will leave. Friday, 29 March 2013
30. ACQUISITION ACTIVATION RETENTION REVENUE REFERRAL Value Metrics How do users find you? Do users have a great first experience? Do users come back? How do you make money? Do users tell others? Friday, 29 March 2013
31. Network effect vs Marketplace vs SaaS Who delivers the actual value? Critical Mass of People The Right People Your Product Friday, 29 March 2013
32. ACQUISITION ACTIVATION RETENTION REVENUE REFERRAL Value Metrics How do users find you? Do users have a great first experience? Do users come back? How do you make money? Do users tell others? Friday, 29 March 2013
33. Feature Usage of successful & unsuccessful customers Friday, 29 March 2013
34. Time to Complete Useful for Email Campaigns! Friday, 29 March 2013
35. Request Demo Commit to become paying customer Visit landing page Create Account Provide billing info for 14-day trial Get welcome email with link to docs Schedule weekly meetings Review docs Establish success metrics Map user actions Send first production event ACTIVATION SIGNUP ACQUISITION Baseline AARRR metrics Identify key metric to improve Define experiments First meeting UPGRADE RETENTION Send events to staging Day 1 Day 2 Day 3 Week 2 Week 3 Week 4 Customer Journey Maps Friday, 29 March 2013
36. Usability Tests http://www.slideshare.net/JohnnyForeigner/active-testing-sketches-leancamp-bcn Friday, 29 March 2013
37. ACQUISITION ACTIVATION RETENTION REVENUE REFERRAL Value Metrics How do users find you? Do users have a great first experience? Do users come back? How do you make money? Do users tell others? Friday, 29 March 2013
38. Did Key Activity More than Once “Created at least 3 cards on the Kan-ban board” “Wrote at least 2 blog posts” “Wrote at least 5 restaurant reviews” Friday, 29 March 2013
39. Customer Happiness Index (CHI) Created Blog Post = 5 Points Shared Post on Twitter = 10 Points Visited Site = 0.1 Points Sum together per User in Ranked List & Reset Each Week Visualises your Power Users Friday, 29 March 2013
40. 1-day 52% 7-day 31% 14-day 22% 30-day 14% Friday, 29 March 2013
41. Friday, 29 March 2013
42. Accelerating Growth Friday, 29 March 2013
43. ACQUISITION ACTIVATION RETENTION REVENUE REFERRAL Growth Metrics How do users find you? Do users have a great first experience? Do users come back? How do you make money? Do users tell others? Friday, 29 March 2013
44. The engine of growth is the mechanism that startups use to achieve sustainable growth. -Eric Ries, The Lean Startup Friday, 29 March 2013
45. Sticky Organic Growth > Churn Rate Friday, 29 March 2013
46. Paid 3x Acquisition Cost < Lifetime Value Friday, 29 March 2013
47. Viral Viral Coefficient > 1.0 Friday, 29 March 2013
48. 3x Acquisition Cost < Lifetime Value Content Marketing vs Direct Sales vs Advertising Avg Length of Sales Cycle Monthly Churn Customer Happiness Index 7/14/30 day retention Avg Revenue / User Typical SaaS Company Friday, 29 March 2013
49. Taking Action Friday, 29 March 2013
50. Metrics are not a “Task”. ☑ Built a Metrics Dashboard ☐ Profit! ☐ ? Friday, 29 March 2013
51. IDEAS PRODUCT DATA MEASURE BUILD LEARN Experiment Friday, 29 March 2013
52. Hypotheses instead of Requirements Friday, 29 March 2013
53. Requirement: Users can add a Cover Photo to their Profile. Friday, 29 March 2013
54. We believe that users will add a cover photo and spend more time on other profiles. Hypothesis: Friday, 29 March 2013
55. 50% of newly signed up users will upload a cover photo, Avg time spent on user profiles will go from 30s to 1min+ Falsifiable Hypothesis: Friday, 29 March 2013
56. Experiment Reports Friday, 29 March 2013
57. 1. Context - Why is this relevant? 2. Quantify Your Goals 3. Set Scope & Timebox Planning an Experiment Friday, 29 March 2013
58. Learning Goal EXPERIMENT REPORT Author: Title: Created: Experiment Scope Minimum Build Plan Lean Stack by Spark59.com Falsifiable Hypothesis Validated Learning Results Next Action What are you trying to learn or achieve? List the build plan step by step to get your measurement and experiment. How many and/or how long will the experiment last? [DATE] [NAME] [TITLE] Your list of statements on the expected outcome of the experiment. Use this format: [Specific Repeatable Action] will [Expected Measurable Outcome] Enter the data. Summarize your learning from the experiment. VALIDATED or INVALIDATED Whatʼs the next experiment? Friday, 29 March 2013
59. Learning Goal EXPERIMENT REPORT Author: Title: Created: Experiment Scope Minimum Build Plan Lean Stack by Spark59.com Falsifiable Hypothesis Validated Learning Results Next Action What are you trying to learn or achieve? List the build plan step by step to get your measurement and experiment. How many and/or how long will the experiment last? [DATE] [NAME] [TITLE] Your list of statements on the expected outcome of the experiment. Use this format: [Specific Repeatable Action] will [Expected Measurable Outcome] Enter the data. Summarize your learning from the experiment. VALIDATED or INVALIDATED Whatʼs the next experiment? Left Side Defines The Experiment Friday, 29 March 2013
60. Learning Goal EXPERIMENT REPORT Author: Title: Created: Experiment Scope Minimum Build Plan Lean Stack by Spark59.com Falsifiable Hypothesis Validated Learning Results Next Action What are you trying to learn or achieve? List the build plan step by step to get your measurement and experiment. How many and/or how long will the experiment last? [DATE] [NAME] [TITLE] Your list of statements on the expected outcome of the experiment. Use this format: [Specific Repeatable Action] will [Expected Measurable Outcome] Enter the data. Summarize your learning from the experiment. VALIDATED or INVALIDATED Whatʼs the next experiment? Right Side Analyses The Results Friday, 29 March 2013
61. Give Context. Why is this relevant? Background What are you trying to learn or achieve? Friday, 29 March 2013
62. Quantify Your Goals Falsifiable Hypothesis Your list of statements on the expected outcome of the experiment. Use this format: [Specific Repeatable Action] will [Expected Measurable Outcome] Friday, 29 March 2013
63. Set Scope & Timebox Experiment Scope How long will the experiment last? Minimum Build Plan List the build plan step by step to get your measurement and experiment. Friday, 29 March 2013
64. 1. Context - Why is this relevant? 2. Quantify Your Goals 3. Set Scope & Timebox Planning an Experiment Friday, 29 March 2013
65. Learning Goal EXPERIMENT REPORT Author: Title: Created: Experiment Scope Minimum Build Plan Lean Stack by Spark59.com Falsifiable Hypothesis Validated Learning Results Next Action What are you trying to learn or achieve? List the build plan step by step to get your measurement and experiment. How many and/or how long will the experiment last? [DATE] [NAME] [TITLE] Your list of statements on the expected outcome of the experiment. Use this format: [Specific Repeatable Action] will [Expected Measurable Outcome] Enter the data. Summarize your learning from the experiment. VALIDATED or INVALIDATED Whatʼs the next experiment? Right Side Analyses The Results Friday, 29 March 2013
66. Actual data you Measured, based on your Hypothesis. Results Enter the data. Friday, 29 March 2013
67. Validated Invalidated Inconclusive Validated Learning Summarize your learning from the experiment. + Why? Friday, 29 March 2013
68. What will you do next based on your learning? Next Action Whatʼs the next experiment? Friday, 29 March 2013
69. ✘ Failure is good. Validated Learning is our Measurement of Progress. Friday, 29 March 2013
70. IDEAS PRODUCT DATA MEASURE BUILD LEARN Build Measure Learn ✓ ✘ Kan-ban Board Friday, 29 March 2013
71. Build Measure Learn ✓ ✘ Friday, 29 March 2013
72. 1 experiment per team member Friday, 29 March 2013
73. Separate Task Board for tracking the details Swimlane for each Experiment Friday, 29 March 2013
74. Timeboxing Friday, 29 March 2013
75. Build Measure Learn ✓ ✘ Friday, 29 March 2013
76. Build Measure Learn ✓ ✘ Experiment Scope Experiment Scope How long will the experiment last? Friday, 29 March 2013
77. Interviews/Prototypes: 1-2 Weeks Testing your MVP: 1 Month A/B Tests: 2-3 Months Recommended Timeboxes: Friday, 29 March 2013
78. Weekly Strategy Meeting Friday, 29 March 2013
79. Observe & Explain To The Group Ideate Alone Discuss In the Group Everyone writes down their idea of an Experiment! Friday, 29 March 2013
80. Review Experiments Only In Weekly Meeting Focus On Efficiency In Daily Stand-ups Friday, 29 March 2013
81. Thanks! Learn more: http://j.mp/s59-metrics Friday, 29 March 2013