A/B Tests with React

Split testing allows us to compare variants – typically an original and a proposed change – via a controlled experiment, in order to see how changes affect user behavior with regard to an outcome of interest.

In this talk, we will describe how we perform A/B-testing with an isomorphic React application, and how we can learn from it.

=====================================================
Follow us and join our Slack community:

- ReactJS Norway's site: http://bit.ly/2ka4hC8
- Facebook: http://bit.ly/2kxmHxW
- Instagram: http://bit.ly/2kpysnJ
- Twitter: http://bit.ly/2kaehvl
- Youtube: http://bit.ly/2kANuJl
- Slack Community: https://goo.gl/YwUUKJ

Sponsored by Stooic Labs (http://bit.ly/2oTE4dF)
Organizer: Domenico Solazzo (http://bit.ly/2jRxj8j)

Transcript

1. A/B Testing with React Eivind Arvesen and Lars Dølvik
2. o Introduction o MatSans • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing? • react-ab-test (library) ▪ Isomorphism o Evaluation • Measurements (GA: scrolling and events…) • Interpreting the numbers o Practical demonstration Outline
3. o MatSans is a recipe focused application o Runs on Google Cloud Platform o Isomorphic o Built with React and Relay in front o GraphQL as data query language o Uses ElasticSearch to hold our content o Enonic CMS to publish content o Hypothesis- and goal-driven development/Workflow MatSans
4. o Tactical goal o Action What? o Hypothesis Why? o Measurement How? o Estimated effect MatSans
5. Story-time
6. Story-time
7. Story-time The original Variation 1 Variation 2
8. A/B-TESTING!
9. A form of statistical hypothesis testing. What is A/B testing? Illustration by Maxime Lorant
10. Usual steps: • Have an idea or supposition about our solution • Verify or falsify the hypothesis via experiment(s) ◦ Collect various measurements What is A/B testing?
11. Why? It’s normal to be wrong. Measuring how our development impacts our users makes us able to make decisions based on facts! What is A/B-testing?
12. React-AB-Test (on npm) https://github.com/pushtell/react-ab-test What is A/B-testing?
13. ” The only true wisdom is in knowing you know nothing. Socrates - allegedly... • Be skeptical • Don’t measure too much • Have a good basis for comparison
14. o Google Analytics API o Scroll and time tracking o Event tracking o Tracking the correct event o Follow a standard o Write documentation o Less is more(ish) o Evaluating the numbers Evaluation
15. o A good grounds for comparison o Avoid noise in your data o Supplement qualitatively... "Lies, damned lies, and statistics" Evaluation
16. Evaluation What is the user’s real intent?
17. Evaluation The original Variation 1 Variation 2
18. Evaluation
19. Evaluation
20. Let’s see how we can perform A/B-testing in practice: look at code and run it in the browser. https://github.com/eivind88/ab-testing-react-meetup/tree/mast er Demo
21. o It is not as complicated as it might seem • The thought process is more the half the job o Keep it simple and structured • Follow a standard and write documentation o Keep your experiments limited o You don’t know it if you haven’t proven it eivind.arvesen@bouvet.no lars.dolvik@bouvet.no In summary