Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing for Accessibility, SwiftConf 2022

Testing for Accessibility, SwiftConf 2022

An accessible mobile app is an app that everyone can use and understand, no matter his or her abilities or disabilities. We can all agree, that's a great thing to do, but realistically who has the time to spend before every release manually testing the entire app with VoiceOver?

While automated testing cannot guarantee your app is 100% accessible, why not automate as much as we can?

In this talk we'll take a critical look at what we can and can't test in terms of accessibility. We'll discuss what facets are best tested with UI tests, unit tests, or even SwiftLint. We'll explore some existing tools, from Apple and from third parties, highlighting their abilities and their limitations. Finally, we'll explore how testing for accessibility can be incorporated into your existing testing pyramid.

Presented at SwiftConf 2022

Robin Kanatzar

August 19, 2022
Tweet

More Decks by Robin Kanatzar

Other Decks in Technology

Transcript

  1. Dynamic Type “without loss of content or functionality” - WCAG

    1.4.4 app.launchArguments += [ "-UIPreferredContentSizeCategoryName", "UICTContentSizeCategoryAccessibilityXXXL" ] app.launch() scrollView .descendants( matching: .staticText ) (Source: A11yUITests)
  2. Element Labels Exists No duplicates on page Doesn’t end in

    “.” Begins with upper case < 20 Characters No special characters No “button”/“image” Does this label make sense? Is it redundant? myButton.accessibilityLabel = “Play” Button() .accessibility(label: “Play”)
  3. Element Ignoring Decorative elements “waste time for users and add

    to their cognitive load without providing any benefit” - Apple myImage.isAccessibilityElement = false Image() .accessibility(hidden: true) Image(decorative: "SomeImage")
  4. 18 Color contrast Touch targets Overlapping elements Conflicting traits Dynamic

    type Orientation Localization Element labels Element ignoring Element grouping Element ordering … and more What we can and can’t test
  5. Automated Testing Tools ▪ Evinced SDK ($) ▪ Deque SDK

    ($) ▪ A11yUITests (free) ▪ XCUITest (free) 21
  6. Evinced XCUI-SDK ▪ Easy setup (Cocoapods, SPM) ▪ SwiftUI &

    UIKit ▪ Doesn’t fail or slow down your UI test ▪ Reports in test results, organize by severity 22 ▪ Paid (14 day free trial)
  7. 23

  8. 24 24 Color contrast Touch targets Overlapping elements Conflicting traits

    Dynamic type Localization - Orientation - Element labels (existence) Element labels (formatting) Element ignoring
  9. axeDevToolsXCUI (Deque) ▪ Easy setup (SPM) ▪ UIKit & SwiftUI

    ▪ Doesn’t fail or slow down your UI test ▪ Reports directly on web dashboard ▪ Experts, x-platform at Deque 25 ▪ Paid ▪ New to Deque
  10. 28 28 Color contrast Touch targets Overlapping elements Conflicting traits

    Dynamic type Localization - Orientation - Element labels (existence) Element labels (formatting) Element ignoring
  11. A11yUITests ▪ Free ▪ Open source ▪ UIKit & SwiftUI

    ▪ Easy integration ▪ Choose testing suite ▪ Choose views to test 29 ▪ Slow if you have a lot of elements Rob Whitaker - mobilea11y.com - @RobRWAPP 29 29
  12. 30 30 Color contrast Touch targets Overlapping elements Conflicting traits

    Dynamic type Localization - Orientation - Element labels (existence) Element labels (formatting) Element ignoring
  13. XCUITest (DIY) ▪ Free ▪ Built into Xcode ▪ Only

    check what you want to check ▪ Learn by doing 31 ▪ Time to create/maintain
  14. ▪ Accessibility Inspector (free) ▪ Accessibility Menu (free) ▪ Evinced

    Flow Analyzer (free) ▪ Reveal ($) 32 Auditing & Inspection Tools
  15. Accessibility Inspector ▪ Free ▪ Built into Xcode ▪ UIKit

    and SwiftUI 33 ▪ Buggy Xcode > Open Developer Tools > Accessibility Inspector
  16. 35 35 Color contrast Touch targets Overlapping elements Conflicting traits

    Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring
  17. Accessibility Menu ▪ Free ▪ Built into Xcode ▪ Shows

    order of elements ▪ Shows label, value, identifier, traits 36 ▪ SwiftUI only ▪ Buggy
  18. 38 38 Color contrast Touch targets Overlapping elements Conflicting traits

    Dynamic type Localization - Orientation - Element labels (existence) Element labels (formatting) Element ignoring
  19. Evinced Flow Analyzer ▪ Free ▪ SwiftUI & UIKit ▪

    Report ▪ iOS and Android ▪ Can check any app ▪ Can check flow instead of individual screen 39 ▪
  20. 41 41 Color contrast Touch targets Overlapping elements Conflicting traits

    Dynamic type Localization - Orientation - Element labels (existence) Element labels (formatting) Element ignoring
  21. Reveal ▪ SwiftUI & UIKit ▪ VoiceOver & Voice Control

    modes ▪ Color modes ▪ WCAG AA, AAA color contrast checks 42 ▪ Paid ▪ Beta
  22. 44 44 Color contrast Touch targets Overlapping elements Conflicting traits

    Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring Assistive Technology ▪ VoiceOver ▪ Voice Control
  23. XibLint ▪ Free ▪ Open source ▪ Can make your

    own rules (in python) 46 ▪ Setup hard ▪ UIKit only ▪ Must use .xib/.storyboard ▪ Only three rules 46 46
  24. SwiftLint ▪ Free ▪ Probably already in your project 47

    ▪ Easy setup ▪ Only one accessibility rule ▪ Only SwiftUI accessibility_label_for_image
  25. 48 48 Color contrast Touch targets Overlapping elements Conflicting traits

    Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring
  26. Tools Automated ▪ Evinced XCUI-SDK ▪ axeDevToolsX CUI SDK ▪

    A11yUITests ▪ XCUITest Audit & Inspect ▪ Accessibility Inspector ▪ Accessibility Menu ▪ Evinced Flow Analyzer ▪ Reveal Lint ▪ XibLint ▪ SwiftLint
  27. Unit Unit & Lint Manual Prod UI Design Develop QA

    Publish Develop ▪ Accessibility Inspector ▪ Accessibility Menu QA ▪ Evinced Mobile Flow Analyzer Develop Review QA 52 52
  28. 53 53 Color contrast Touch targets Overlapping elements Conflicting traits

    Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring ▪ Feedback from users in prod UIKit Assistive Technology ▪ VoiceOver
  29. Unit & Lint Manual Prod Design Develop QA Publish Accessibility

    UI Tests ▪ Dynamic type ▪ Orientation ▪ Localization ▪ A11yUITests Accessibility Manual Tests Accessibility QA ▪ Paid tester who uses assistive technology Review UI Accessibility UI Accessibility Manual A11y QA 55 55
  30. 56 56 Color contrast Touch targets Overlapping elements Conflicting traits

    Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring ▪ One paid tester (real user of assistive technology) Assistive Technology ▪ VoiceOver
  31. Unit & Lint Manual Prod Design Develop QA Publish UI

    Tests (CI/CD) ▪ More Develop ▪ Reveal Review UI Accessibility UI Accessibility Manual A11y QA Develop Manual Testing with Assistive Technology ▪ Train QAs to test with one type of assistive tech
  32. 59 59 Color contrast Touch targets Overlapping elements Conflicting traits

    Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring ▪ One paid tester (real user of assistive technology) ▪ Trained QAs Assistive Technology ▪ VoiceOver ▪ Voice Control
  33. Manual Prod Design Develop QA Publish Accessibility UI Tests ▪

    Deque SDK (unit + ui) ▪ Evinced SDK Accessibility QA ▪ QAs who use assistive technology ▪ Test multiple ATs Review UI Accessibility UI Accessibility Manual A11y QA Unit & Lint
  34. 62 62 Color contrast Touch targets Overlapping elements Conflicting traits

    Dynamic type Localization Orientation Element labels (existence) Element labels (formatting) Element ignoring ▪ Many paid testers (real user of assistive technology) ▪ Trained QAs Assistive Technology ▪ VoiceOver ▪ Voice Control ▪ … and more!