The benefits and drawbacks of online unmoderated user testing

Evidence-based product design and emerging technology is changing the way we measure user behaviour. Learn about our experiences designing exemplar government digital services using:
- unmoderated remote usability testing
- rapid prototyping
- lean UX and hypothesis driven development

Presented by Lana Giorgi at UX Australia 2016

Transcript

1. Anyone who can successfully navigate deserves to have their hecs debt wiped @annoyedmemberofthecommunity / April 7 2016
2. How data driven design is improving government online services Australian Taxation Office Lana Giorgi
3. UX fail = UX success = Embarrassment Unwelcome scrutiny Undermined senior executives Hurt legislation Compliance Growth Advocacy
4. Empathy Define Ideate Prototype Test
6. Where we started ATO usability testing team UX Blog CMS testing platform UI designers & developers UX researchers Usability testers Data analysts Content strategists 12 months
8. Almost 1,000 volunteer usability testers sign-up in 12 months
11. Our methodology Test Discover Define MVP Validate Iterate Agile framework
13. Large data sets Faster turnaround Less chance of bias testing a user in their natural environment Available 24/7 and free from scheduling concerns Easier recruitment No overheads 24/7 The benefits of unmoderated usability testing
14. Marrying quantitative & qualitative data Using both Quantitative and Qualitative evaluation ensures that the limitations of one type of data is balanced by the strengths of another Quantitative Hypothesis driven Data can be measured Based on numbers Larger samples Usually online Qualitative Empathy driven Sense and observe results Based on opinions and experiences Smaller samples Usually face-to-face
15. Landing page design death by 1,000 A/B tests Test 1: Add social proof Test 2: Change button colour Test 3: Add security icons Test 4: Add more social proof Test 5: Add money-back guarantee
16. beta.ato.gov.au
17. Data HELPS you tell stories. People tell stories.