Testing for Accessibility, SwiftConf 2022

An accessible mobile app is an app that everyone can use and understand, no matter his or her abilities or disabilities. We can all agree, that's a great thing to do, but realistically who has the time to spend before every release manually testing the entire app with VoiceOver?

While automated testing cannot guarantee your app is 100% accessible, why not automate as much as we can?

In this talk we'll take a critical look at what we can and can't test in terms of accessibility. We'll discuss what facets are best tested with UI tests, unit tests, or even SwiftLint. We'll explore some existing tools, from Apple and from third parties, highlighting their abilities and their limitations. Finally, we'll explore how testing for accessibility can be incorporated into your existing testing pyramid.

Presented at SwiftConf 2022

Transcript

1. Testing for Accessibility
2. I’m Robin Kanatzar. You can find me on Twitter @RobinKanatzar Testing for Accessibility 2
3. what we can and can’t test tools process & testing pyramid 3 The plan
4. 4 Accessibility (A11y) in 30 seconds Assistive Technology - VoiceOver - Voice Control - Switch Control - …
5. What we can and can’t test with a machine 1. 5
6. References ▪ Web Content Accessibility Guidelines (WCAG) ▪ Apple’s Human Interface Guidelines (HIG)
7. Color Contrast 4.5 : 1 Dark mode & light mode
8. Touch Target 44 x 44 pt XCTAssert(element.frame.size.height >= 44) XCTAssert(element.frame.size.width >= 44)
9. Overlapping Elements Overlapping Controls element1.frame .intersects(element2.frame)
10. Conflicting Traits E.g. button & link if element.traits.contains(.button) && element.traits.contains(.link) { // traits are conflicting }
11. Dynamic Type “without loss of content or functionality” - WCAG 1.4.4 app.launchArguments += [ "-UIPreferredContentSizeCategoryName", "UICTContentSizeCategoryAccessibilityXXXL" ] app.launch() scrollView .descendants( matching: .staticText ) (Source: A11yUITests)
12. Orientation “without loss of content or functionality” XCUIDevice .shared .orientation = .landscapeLeft
13. Localization accessibilityLabel, not accessibilityIdentifier app.launchArguments += [“-AppleLanguages”, “(fr)”] app.launchArguments += [“-AppleLocale”, “fr_FR”] app.buttons["Start"] app.buttons["Démarrer"]
14. Element Labels Exists No duplicates on page Doesn’t end in “.” Begins with upper case < 20 Characters No special characters No “button”/“image” Does this label make sense? Is it redundant? myButton.accessibilityLabel = “Play” Button() .accessibility(label: “Play”)
15. Element Ignoring Decorative elements “waste time for users and add to their cognitive load without providing any benefit” - Apple myImage.isAccessibilityElement = false Image() .accessibility(hidden: true) Image(decorative: "SomeImage")
16. Element Grouping Semantic Groups
17. Element Ordering Top left => bottom right
18. 18 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Orientation Localization Element labels Element ignoring Element grouping Element ordering … and more What we can and can’t test
19. Tools What they’re good at, and not so good at 2. 19
20. Tools ▪ Automated Testing Tools ▪ Auditing & Inspection Tools ▪ Linting Tools 20
21. Automated Testing Tools ▪ Evinced SDK ($) ▪ Deque SDK ($) ▪ A11yUITests (free) ▪ XCUITest (free) 21
22. Evinced XCUI-SDK ▪ Easy setup (Cocoapods, SPM) ▪ SwiftUI & UIKit ▪ Doesn’t fail or slow down your UI test ▪ Reports in test results, organize by severity 22 ▪ Paid (14 day free trial)
24. 24 24 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Localization - Orientation - Element labels (existence) Element labels (formatting) Element ignoring
25. axeDevToolsXCUI (Deque) ▪ Easy setup (SPM) ▪ UIKit & SwiftUI ▪ Doesn’t fail or slow down your UI test ▪ Reports directly on web dashboard ▪ Experts, x-platform at Deque 25 ▪ Paid ▪ New to Deque
27. 27 27
28. 28 28 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Localization - Orientation - Element labels (existence) Element labels (formatting) Element ignoring
29. A11yUITests ▪ Free ▪ Open source ▪ UIKit & SwiftUI ▪ Easy integration ▪ Choose testing suite ▪ Choose views to test 29 ▪ Slow if you have a lot of elements Rob Whitaker - mobilea11y.com - @RobRWAPP 29 29
30. 30 30 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Localization - Orientation - Element labels (existence) Element labels (formatting) Element ignoring
31. XCUITest (DIY) ▪ Free ▪ Built into Xcode ▪ Only check what you want to check ▪ Learn by doing 31 ▪ Time to create/maintain
32. ▪ Accessibility Inspector (free) ▪ Accessibility Menu (free) ▪ Evinced Flow Analyzer (free) ▪ Reveal ($) 32 Auditing & Inspection Tools
33. Accessibility Inspector ▪ Free ▪ Built into Xcode ▪ UIKit and SwiftUI 33 ▪ Buggy Xcode > Open Developer Tools > Accessibility Inspector
34. 34 34
35. 35 35 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring
36. Accessibility Menu ▪ Free ▪ Built into Xcode ▪ Shows order of elements ▪ Shows label, value, identifier, traits 36 ▪ SwiftUI only ▪ Buggy
37. 37 37
38. 38 38 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Localization - Orientation - Element labels (existence) Element labels (formatting) Element ignoring
39. Evinced Flow Analyzer ▪ Free ▪ SwiftUI & UIKit ▪ Report ▪ iOS and Android ▪ Can check any app ▪ Can check flow instead of individual screen 39 ▪
40. 40 40
41. 41 41 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Localization - Orientation - Element labels (existence) Element labels (formatting) Element ignoring
42. Reveal ▪ SwiftUI & UIKit ▪ VoiceOver & Voice Control modes ▪ Color modes ▪ WCAG AA, AAA color contrast checks 42 ▪ Paid ▪ Beta
43. 43 43
44. 44 44 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring Assistive Technology ▪ VoiceOver ▪ Voice Control
45. Linting Tools ▪ XibLint (free) ▪ SwiftLint (free) 45
46. XibLint ▪ Free ▪ Open source ▪ Can make your own rules (in python) 46 ▪ Setup hard ▪ UIKit only ▪ Must use .xib/.storyboard ▪ Only three rules 46 46
47. SwiftLint ▪ Free ▪ Probably already in your project 47 ▪ Easy setup ▪ Only one accessibility rule ▪ Only SwiftUI accessibility_label_for_image
48. 48 48 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring
49. Tools Automated ▪ Evinced XCUI-SDK ▪ axeDevToolsX CUI SDK ▪ A11yUITests ▪ XCUITest Audit & Inspect ▪ Accessibility Inspector ▪ Accessibility Menu ▪ Evinced Flow Analyzer ▪ Reveal Lint ▪ XibLint ▪ SwiftLint
50. Development Process And testing pyramid 3. 50
51. Small Team, No Resources Startup, indie dev No time, no budget, no manpower
52. Unit Unit & Lint Manual Prod UI Design Develop QA Publish Develop ▪ Accessibility Inspector ▪ Accessibility Menu QA ▪ Evinced Mobile Flow Analyzer Develop Review QA 52 52
53. 53 53 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring ▪ Feedback from users in prod UIKit Assistive Technology ▪ VoiceOver
54. Small Team, Few Resources Time for UI tests $ for paid accessibility tester
55. Unit & Lint Manual Prod Design Develop QA Publish Accessibility UI Tests ▪ Dynamic type ▪ Orientation ▪ Localization ▪ A11yUITests Accessibility Manual Tests Accessibility QA ▪ Paid tester who uses assistive technology Review UI Accessibility UI Accessibility Manual A11y QA 55 55
56. 56 56 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring ▪ One paid tester (real user of assistive technology) Assistive Technology ▪ VoiceOver
57. Medium Team, Some Resources Multiple developers QA team CI/CD Dev Tools
58. Unit & Lint Manual Prod Design Develop QA Publish UI Tests (CI/CD) ▪ More Develop ▪ Reveal Review UI Accessibility UI Accessibility Manual A11y QA Develop Manual Testing with Assistive Technology ▪ Train QAs to test with one type of assistive tech
59. 59 59 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring ▪ One paid tester (real user of assistive technology) ▪ Trained QAs Assistive Technology ▪ VoiceOver ▪ Voice Control
60. Large Team, Unlimited Resources Dream big.
61. Manual Prod Design Develop QA Publish Accessibility UI Tests ▪ Deque SDK (unit + ui) ▪ Evinced SDK Accessibility QA ▪ QAs who use assistive technology ▪ Test multiple ATs Review UI Accessibility UI Accessibility Manual A11y QA Unit & Lint
62. 62 62 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring ▪ Many paid testers (real user of assistive technology) ▪ Trained QAs Assistive Technology ▪ VoiceOver ▪ Voice Control ▪ … and more!
63. 63 Conclusion what we can and can’t test tools process & testing pyramid
64. Thanks! Any questions? You can find me at ▪ @RobinKanatzar on Twitter ▪ hello@robinkanatzar.com 64 Feedback