they sign in once and never return. 02 We get traffic to our landing page but people aren’t signing up for the product. 01 People use our free product, but they don’t upgrade to a paid product. 03
— trying random things from loud blogs without learning about our users. Tal Raviv | Patreon, Product Manager PatreonHQ.com | “That’s Not a Hypothesis”
Shown to 25% Variant 3: Shown to 25% We will run the experiment for the CRM sign up flow for 14 days and need at least 20,000 exposures to reach statistical significance.
of the signup flow will increase by 5 percentage points, resulting in 300 more signups per week. At an upgrade rate of 10% and an average sales price of $20, that will result in an additional $600 in monthly recurring revenue.* If our hypothesis is true… *Fake numbers.
wrong? Why did the experiment turned out the way it did? (For both successes and failures.) What are follow up experiments to continue testing this hypothesis (if it’s still worth testing)?
questions in our sign up flow. If we display social proof in the signup flow, then people will feel reassured and finish signing up. Increase the number of people signing up for HubSpot’s free products. If our hypothesis is true, we expect to see a 3 percentage point improvement in signup completion, translating to 1,000 additional signups per week and $10,000 in additional MRR.* Hypothesis Objective Expected Outcome Prediction
negatively impact the signup completion rate. We should think about how different types of social proof may perform in different regions. We should think about other, more impactful ways, to improve the signup completion rate.
seconds to load. The signup app backend is split into two parts and sometimes the second part doesn’t load. The design of the app is inconsistent with the website style guide and may confuse the user. What’s preventing the signup completion rate from being better?
HubSpot CRM Free account are getting fatigued and in some cases confused when signing up for an account. If we improve the signup load time with an improve design, the signup completion rate will improve. The main objective of this experiment is to understand whether improving the signup load time with a visually consistent design will impact ability to sign up for HubSpot. If our hypothesis is true then we predict that we will see at least a 5% lift in signup completion rate, resulting in in 10,000 more signups a month and $200,000 additional revenue. Hypothesis Objective Expected Outcome Prediction
productledalliance1
msymt
matleenalaakso
lwug
furuhashilab
signer
pimenko
hadley
lsuzuki
ichimichi
seantibor
samuraicurry
malarkey
lauravandoore
cassininazir
dotmariusz
dougneiner
skipperchong
productmarketing
mongodb
kneath
notwaldorf
swwweet
bkeepers
