Clearhead / Insound.com - A Case Study in A/B Testing

Clearhead / Insound.com - A Case Study in A/B Testing

Transcript

1. bInsound, we have a winnerB Insound INS-002 November 1, 2012
2. 2 Insound INS-002 November 1, 2012 Insound: Shipping/Billing Button Text Hypothesis I believe cart abandonment would decrease if the large CONTINUE button said something more enticing, like "FINALIZE ORDER." Test type A/B/n Primary KPI ‣ Revenue Optimization Metric ‣ Total clicks on “Continue” (variation) button Timeframe ‣ 10/19/12 to 10/30/12 Total Visitors Exposed ‣ 3,605 Medium impact Low effort http://bit.ly/QhBsNP
3. Summary review. 3 Quick results. The “Review Order” button significantly outperformed all other variations, including the original. We found the data to be statistically significant and pushed the winning variation to 100%. Recommendation. We recommend coding the “Review Order” button in production immediately. Additionally, we suggest monitoring conversion rate on the next (final) page of checkout in order to confirm that conversion rate improvement confirms the initial results. 30.5% 39.4% 29.7% 27.5% 0% 10% 20% 30% 40% 50% Original Page Review Order Almost Done! Submit % of Visits that Clicked Button Text The error bars represent the range in which there is 95% confidence that the true results fall Insound INS-002 November 1, 2012
4. Variation details. 4 Review Order 39.4% Click Rate Continue (Original) 30.5% Click Rate Insert Mini Submit 29.7% Click Rate Almost Done 27.5% Click Rate Visitors Tested 1,165 Rationale Describes what to expect on next page but reassures that order is not being placed. Visitors Tested 817 Rationale “Logical” description of what the button would do – continue to the next step Visitors Tested 817 Rationale Based on one-step checkout, this confirms that information is being committed. Visitors Tested 779 Rationale Assures that the process is almost complete. Insound INS-002 November 1, 2012
5. Insight. ‣ Insound visitors appear more likely to continue through checkout when the call to action is most clear & descriptive of the next step. - The “Review Order” button describes what will occur on the next page and reassures the customer that the order is not yet being placed.
6. Next steps. ‣ Produce “Review Order” button in production. ‣ Monitor and compare conversion rate through order for new button to historical baseline. ‣ The Shipping/Billing page represents the largest opportunity for optimization in the checkout flow. We recommend additional tests such as: - Test “Next” buttons - Develop hypotheses around additional form interactions and calls to action