metrics
regan davis
BUILD | deep dive #3
PYROS
creating products that are
made to measure
Slide 2
Slide 2 text
regan davis
product leader from chicago, il
BACKGROUND
EIR @ 1871
Sr. Director of Product @ SpotHero
Advisor, Coach
regandavis.com
VP of Product @ Jellyvision
VP of Product @ info.com
Slide 3
Slide 3 text
regandavis.com
CORE METRICS
TRACKING PLAN
OPTIMIZATION
“DATA DRIVEN”
agenda
Slide 4
Slide 4 text
regandavis.com
what,
not why
analytics tell you
Slide 5
Slide 5 text
regandavis.com
compare &
contrast
data’s purpose:
Slide 6
Slide 6 text
regandavis.com
pirate metrics
(Credit: Dave McClure, Melanie Blake via Medium)
Slide 7
Slide 7 text
regandavis.com
ACTIVATION
ACQUISITION
RETENTION
REFERRAL
REVENUE
(Credit: Dave McClure, Melanie Blake via Medium)
Slide 8
Slide 8 text
regandavis.com
ACQUISITION how do people
fi
nd you and
(eventually) turn into customers?
how good is their
fi
rst customer
experience?
ACTIVATION
(Credit:Melanie Blake via Medium)
regandavis.com
Let’s try it.
Customer journey sheet
Slide 11
Slide 11 text
regandavis.com
User/Logo retention: % of new users
who are still active 3-6 mo. later
L7/L30 retention: Number of days
that users are active per week/month
RETENTION
REFERRAL Invite rate: % of new users who sent at
least one invite in the
fi
rst X days
Invite conversion: % of received
invites that sign-up in the next X days
Virality factor: % of new users who
have come from an invite
(Credit:Melanie Blake via Medium)
Slide 12
Slide 12 text
regandavis.com
funnels
ACQUISITION
TRACKING PLAN
Slide 13
Slide 13 text
regandavis.com
conversion
Slide 14
Slide 14 text
regandavis.com
start with
What’s the main goal
of your user?
Look
Book
Park
compare parking options
reserve and pre-pay for parking
show up at garage to guaranteed spot
Slide 15
Slide 15 text
regandavis.com
next
What are the paths they take
to get there?
• capture an event for each step in the path
• create a funnel from the steps that are common
Slide 16
Slide 16 text
regandavis.com
track
Events and Properties
• Screen
• Trigger
• Action (Event)
• Extra info (Properties)
• Description
regandavis.com
hypothesis driven
development
“DATA DRIVEN”
Slide 24
Slide 24 text
measure your bets.
regandavis.com
Slide 25
Slide 25 text
regandavis.com
“The only difference between
screwing around and science is
writing down your results.“
— Adam Savage, Mythbusters
Slide 26
Slide 26 text
meet Shannon.
regandavis.com
Shannon Rashedi
Sr. Product Manager, Mobile
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
account holders
1. higher conversion rates
2. more frequent customers
3. better customer service
Slide 29
Slide 29 text
regandavis.com
what,
not why
analytics tell you
Slide 30
Slide 30 text
talk to people
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
regandavis.com
What do we think will happen?
OPPORTUNITY
METRIC
hypothesis
PROBLEM
HYPOTHESIS
Slide 33
Slide 33 text
regandavis.com
ten
ideas.
2 experiments.
Slide 34
Slide 34 text
regandavis.com
quick
rebook
Slide 35
Slide 35 text
regandavis.com
We hypothesize that showing Quick
Rebook to guest users will help
customers [see the value of an
account] and increase [account signup
X%*]
• Quick Rebook Click-through Rate
(from home screen)
• Account sign-up rate
METRIC
BET (HYPOTHESIS)
(*Note: numbers are obfuscated.)
Quick Rebook
Slide 36
Slide 36 text
regandavis.com
cancel
button
Slide 37
Slide 37 text
regandavis.com
We hypothesize that asking for an
account when users try to cancel will
… increase [account signup Y%*]
• Account sign-up rate
• Customer service calls
• Cancellation rate
METRIC
BET (HYPOTHESIS)
(*Note: numbers are obfuscated.)
Quick Rebook
Slide 38
Slide 38 text
so did it work?
Slide 39
Slide 39 text
regandavis.com
quick rebook
• major boost in signup
• more use of the app overall
✅
cancel button
• it did increase signups…
• but likely caused more issues
😱
Slide 40
Slide 40 text
“Experiment with multiple
solutions. When one of those
experiments is a failure, you're
not starting from scratch again.”
—Shannon Rashedi, Sr. PM, Mobile
regandavis.com
Slide 41
Slide 41 text
put the user at the heart of your ideas
use analytics to measure if your guess was right
keep learning — talk to your users often
focus on who
measure what
ask why
regandavis.com
Slide 42
Slide 42 text
FAILURE SUCCESS
regandavis.com
Slide 43
Slide 43 text
regandavis.com
“genius” actual genius
Slide 44
Slide 44 text
regandavis.com
Slide 45
Slide 45 text
regandavis.com
Slide 46
Slide 46 text
questions?
NEXT STEPS
regandavis.com/feedback
[email protected]
regandavis.com
feedback
contact