ME'13 Presentation Slides

Slide 1

Slide 1 text

Technische Universität München Towards Automated Detection of Mobile Usability Issues Dennis Pagano Daniel Bader

Slide 2

Slide 2 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Modern usability evaluation – What should it be like? 2

Slide 3

Slide 3 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 1| Strong software support → Automation allows you to involve (more) real users 3

Slide 4

Slide 4 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 2| Continue after deployment →Context of use after delivery? 4

Slide 5

Slide 5 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Our goal Automated detection of low discoverability usability issues at application runtime (and after deployment) 5

Slide 6

Slide 6 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 6 User can’t ﬁnd certain UI controls → may not reach some of the app’s functionality One of the main UI challenges for touch-based mobile devices [Jakob Nielsen] Low Discoverability

Slide 7

Slide 7 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Low discoverability example 7 Image source: http://android.cyrilmottier.com/?p=860

Slide 8

Slide 8 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Our contribution ▪ A simple heuristic that detects low discoverability issues in mobile applications ▪ A sample implementation for iOS ▪ An evaluation of the heuristic’s performance 8

Slide 9

Slide 9 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 How the heuristic works ▪ Input: a sequence of view change events 9 View Change Event View Identifier Timestamp

Slide 10

Slide 10 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 How the heuristic works ▪ Input: a sequence of view change events 10 View Change Event View Change Event View Change Event View Change Event View Change Event View Change Event Time

Slide 11

Slide 11 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 How the heuristic works ▪ Look for problematic navigational patterns that indicate low discoverability issues 11 View Change Event View Change Event View Change Event View Change Event View Change Event View Change Event Time

Slide 12

Slide 12 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Problematic navigational patterns 12 Navigational Loops The user visited at least one view several times in the last N view changes Low Retention Times The user visited each of the last N views only brieﬂy Low Discoverability ∧

Slide 13

Slide 13 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 The framework ▪ A proof of concept implementation of the heuristic ▪ Easy to integrate into existing iOS applications – [[UsabilityTracker sharedTracker] enterView:@”Example”] ▪ Source code and demo project available 13

Slide 14

Slide 14 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 The framework 14

Slide 15

Slide 15 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Evaluation results ▪ Is using such a simple heuristic worthwhile? ▪ How does it perform with a real application? ▪ How will it scale with respect to application complexity and number of users? 15

Slide 16

Slide 16 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Evaluation results ▪ Evaluation setup: – 9 test users – a real world iPhone application of medium complexity →Promising results with respect to the simplicity of the approach • Precision = 0.61, Recall = 0.73 • MCC and Kappa Scores of ~ 0.66 16

Slide 17

Slide 17 text

Technische Universität München 26.02.2013 ME’13 – Daniel Bader and Dennis Pagano Outlook Scale is a double-edged sword → App complexity: ? → Many users: ✓ 17

Slide 18

Slide 18 text

Technische Universität München 26.02.2013 ME’13 – Daniel Bader and Dennis Pagano Outlook Scale is a double-edged sword → App complexity: ? → Many users: ✓ … may be used to complement existing approaches in the real world 18

Slide 19

Slide 19 text

Technische Universität München ME’13 – Daniel Bader and Dennis Pagano 26.02.2013 Summary ▪ A simple way to detect usability issues at application runtime (even after deployment) ▪ Slides and sample implementation available at http://dbader.org/me13-paper ▪ Keep in touch with us on Twitter: – Daniel Bader: @dbader_org – Dennis Pagano: @dennispagano 19