Introduction to MVT

Presented on 5th July 2016 at Tech talks #3

Transcript

1. MULTIVARIATE TESTING INTRODUCTION TO Stanislav Cmakal 4th July 2016
3. UK number 1 online fashion store for twenty- somethings 150 million of visitors each month 30 million page views a day WHAT IS ASOS
4. WHAT IS MVT?
5. MVT offers way of evaluating business and design decisions
6. MVT allows to run two or more variants to statistically determine the best performing test
7. Not all tests can be measured in monetary value but they can still be measured
8. WHERE DO WE START?
9. Only a small portion of total traffic will be used for test (for example ASOS would use 10% for one test)
10. Test confidence in results will improve with the duration of the test and number of visitors
11. Control test and each variation must get equal traffic to get meaningful data
12. THE GOOD PARTS
13. The code can be very specific var dropdown = $('#ctl00_ContentMainPage_ctlSeparateProduct_drpdwnSize'); dropdown.on('change', function(){ var selected = $("option:selected", this).text(); $(window).trigger("analytics.productPageAction", ["select size|" + selected]); });
14. We can target specific audience, browsers, platforms…
15. The results can have great value to the business (the results alone often outweigh the cost of running the MVT programme)
16. THE BAD PARTS
17. The tools for developers are not very developer friendly
18. It’s easy to abuse the tools with hacky code
19. To get meaningful data in meaningful timeframe most tests will require large traffic
20. Results are never black and white
21. THE REALITY
22. Think before you code. Think about your workflow. Don’t over-engineer.
23. LOCAL ENVIRONMENT BUILD AND TEST ON
24. USE GRUNT, MODULARISE AUTOMATE PRODUCTION
25. UNIT TEST NO EXCUSES
26. WRITE GOOD CODE MIND THE QUALITY
27. YOU CAN MAKE BIG DIFFERENCE ENJOY THE POWER OF KNOWLEDGE
28. STANISLAV CMAKAL INTERFACE DEVELOPER AND WEB CONSULTANT STANISLAV.CMAKAL@GMAIL.COM INTERFACE-DEVELOPER.COM