ME'13 Presentation Slides

Transcript

1. Technische Universität München Towards Automated Detection of Mobile Usability Issues Dennis Pagano Daniel Bader
2. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Modern usability evaluation – What should it be like? 2
3. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 1| Strong software support → Automation allows you to involve (more) real users 3
4. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 2| Continue after deployment →Context of use after delivery? 4
5. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Our goal Automated detection of low discoverability usability issues at application runtime (and after deployment) 5
6. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 6 User can’t find certain UI controls → may not reach some of the app’s functionality One of the main UI challenges for touch-based mobile devices [Jakob Nielsen] Low Discoverability
7. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Low discoverability example 7 Image source: http://android.cyrilmottier.com/?p=860
8. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Our contribution ▪ A simple heuristic that detects low discoverability issues in mobile applications ▪ A sample implementation for iOS ▪ An evaluation of the heuristic’s performance 8
9. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 How the heuristic works ▪ Input: a sequence of view change events 9 View Change Event View Identifier Timestamp
10. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 How the heuristic works ▪ Input: a sequence of view change events 10 View Change Event View Change Event View Change Event View Change Event View Change Event View Change Event Time
11. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 How the heuristic works ▪ Look for problematic navigational patterns that indicate low discoverability issues 11 View Change Event View Change Event View Change Event View Change Event View Change Event View Change Event Time
12. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Problematic navigational patterns 12 Navigational Loops The user visited at least one view several times in the last N view changes Low Retention Times The user visited each of the last N views only briefly Low Discoverability ∧
13. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 The framework ▪ A proof of concept implementation of the heuristic ▪ Easy to integrate into existing iOS applications – [[UsabilityTracker sharedTracker] enterView:@”Example”] ▪ Source code and demo project available 13
14. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 The framework 14
15. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Evaluation results ▪ Is using such a simple heuristic worthwhile? ▪ How does it perform with a real application? ▪ How will it scale with respect to application complexity and number of users? 15
16. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Evaluation results ▪ Evaluation setup: – 9 test users – a real world iPhone application of medium complexity →Promising results with respect to the simplicity of the approach • Precision = 0.61, Recall = 0.73 • MCC and Kappa Scores of ~ 0.66 16
17. Technische Universität München 26.02.2013 ME’13 – Daniel Bader and Dennis Pagano Outlook Scale is a double-edged sword → App complexity: ? → Many users: ✓ 17
18. Technische Universität München 26.02.2013 ME’13 – Daniel Bader and Dennis Pagano Outlook Scale is a double-edged sword → App complexity: ? → Many users: ✓ … may be used to complement existing approaches in the real world 18
19. Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Summary ▪ A simple way to detect usability issues at application runtime (even after deployment) ▪ Slides and sample implementation available at http://dbader.org/me13-paper ▪ Keep in touch with us on Twitter: – Daniel Bader: @dbader_org – Dennis Pagano: @dennispagano 19