Only 30% of accessibility issues on a website can be caught by automated testing, according to the GDS accessibility team. But those 30% are a great baseline to operate from, and we should strive to make it as easy as possible for everyone to get there.
While every situation is different and every project faces unique challenges, the basics are often the same: we start small and try to scale our testing approach. We need to integrate new accessibility testing tools into our existing processes and pipelines. And we need the test results to be actionable and easy to understand.
In this talk, I tell the story of how we’re trying to get to the first 30% at ResearchGate: how we approached those issues, the obstacles we’re facing, and what new exciting possibilities await us all in the future.
No more excuses
Getting the first 30% of accessibility issues
Hello good* people
I am Tim von Oldenburg
UX Engineer at ResearchGate, the professional
network for scientists.
Email me at [email protected]
Get in touch: https://twitter.com/sweeneytimm
* if you’re interested in making products accessible,
you’re probably not a bad person
I work in the team that creates Nova, our internal design system
In 2017, the GDS ran 13 accessibility testing tools on the world’s least accessible website
Only 30% of accessibility issues are found by automated tools.
Does this mean automated testing is
Of course not! Automated testing is still very valuable.
These 30% may not sound like much, but they are a
solid baseline that you can operate from.
It’s also a baseline that everyone can achieve, no
matter how big and diverse their organization is, or if
they have dedicated resources for user research.
The value of
“The web can only become an accessible, inclusive
space if developers are empowered to take
responsibility for accessibility testing and accessible
Automated accessibility testing is a huge timesaver, it
doesn't require special expertise, and it allows teams
to focus expert resources on the accessibility issues
that really need them.”
Everyone can fix at least 30% of
problems. Let’s get started!
your team, and
How to educate
Local community: Attend A11y Meetup Berlin :-)
Online resources: Video courses, blogs, books
Existing solutions: Copy other people’s work
Ask: Get help from Stanley*
* anybody who knows more than you do
How to spread
knowledge to your
Give a code review and suggest accessibility
improvements while changes are being made anyway.
This feels less like additional work than a separate
Give a workshop to raise the general accessibility
knowledge in the team, or to work on a specific
Are other teams or people in your organisation
struggling with accessibility?
Offer to help them.
Do you have an internal company blog? Do you have a
Write about what you’re doing to make your product
more accessible. Share relevant articles and news.
How to create
awareness in your
Use tools you already
Which tools are you
Use Selenium for end-to-end tests? Use it to test
Collect Lighthouse performance metrics? There is an
accessibility audit built into Lighthouse as well.
Design using Sketch? There is a plugin to check for
Use a test runner? Don’t add another one to your
Testing setup at ResearchGate
Test in as real
circumstances as possible.
From fake DOM to
Selenium gives you a real browser. However, it’s slow,
required additional infrastructure if you use a CI, and
might run slightly unstable.
Instead, you can run your tests in a headless browser,
such as puppeteer, directly from your command line or
Continuous Integration server.
(Firefox also has a headless mode)
Testing a button component...
...and axe gives us a list of problems
Make the results
axe gives us a list of problems
How to make the connection from test results to the
We insert a highlight around the problematic element.
The terminal output tells us what the problem is.
The highlight in the screenshot tells us where the
Take it one issue at a time
Don’t be discouraged by too many problems.
Either approach it section by section (inputs,
navigation, …) or problem by problem (color contrast,
Take it one issue at
Go beyond the 30%
Go beyond the 30% Automated testing is really useful. Automate what you
Just don’t use “we are using automatic testing” as an
Aim to design and test with real people. There is no
replacement for the human factor when creating a
Addressing the first 30% of accessibility issues means
“getting the obvious issues out of the way”. Now you
can focus on the real ones.
Recap - Educate yourself, your team, and your org
- Use tools you already know well
- Test in as real circumstances as possible
- Make the results actionable
- Take it one issue at a time
- Go beyond the 30%
No more excuses
References GDS Accessibility Testing tools study:
Web Content Accessibility Guidelines (WCAG):
References (2) Inclusive Design Principles:
WAI-ARIA Authoring Practices: