How is accessibility testing perceived in the testing process?

Question

I am studying ways to add accessibility testing in the automated testing process of the company that I am currently working.

I've read some articles, watched some talks about accessibility in digital products (mostly done by ux designers and managers) and I am curious to know how testers/QA analysts deal with this subject on a daily basis.

I am curious to know if you guys a) have ever structured the testing strategy considering accessibility testing; b) use automated tools to check accessibility regularly, locally or as part of the testing script; c) have ever postponed a release because the product did not have the quality needed to pass the accessibility criteria.

Answer 1

Accessibility testing (a11y) is an important part of testing any application. In my experience, it's not done often enough, although more and more companies are getting involved here given legal issues that have cropped up in the last couple of years.

Like anything else that is designed by UX or developed by Developers, a11y needs to be tested.

Structure:

If you are going from zero a11y to some a11y, then lots of testing needs to occur and lots of bugs can be found. The types of testing that's typical here is functional testing, regression testing. Because code is modified to assist in a11y, you still need to ensure the application is functional and doesn't cause any regressions. a11y is really about adding more testing to your process. This can be a large effort in getting started.

Accessibility is a standard protocol by the W3C listed under WCAG guidelines and a lot of people focus on AA level (there are 3 levels: A, AA, AAA).

There are lots of things to consider when a11y testing, including (non-exhaustive list):

• Visual acuity (blind, colorblind, restrictive vision)
• Physical dexterity (ex: keyboard-only navigation, not everyone can use a mouse)
• Hearing (ex: do videos/audio contain captions, subtitles)
• Screen sizes (is your app mobile responsive?)
• Language (localization, internationalization can be considered an accessibility concern)

How do you test a11y?

You can manually test a lot of this using tools:

• Use screen readers. There is paid software for this at the OS level or ChromeVox as a Chrome extension. Phones also have built-in a11y controls.
• Change color contrast on your device. This tests for different colorblindness.
• Zoom in/out on your app
• Google Lighthouse in Chrome DevTools can test for some accessibility, but not all.
• Chrome DevTools Elements tab also shows some ally interpretation when you use the select tool to select any element on the page.
• Keyboard only navigation. Turn your mouse off. The biggest bugs here are "keyboard traps".
• Use aXe. This is considered a de facto, best of breed tool for a11y automation. It also can be used with Selenium WebDriver.

I caution you to not let yourself get caught up by "automate everything" approach. Not everything can or should be automated and there are lots in a11y that can't be. Use automation and other tools where feasible.

What do you test for a11y?

• All software, including desktop apps, web apps, mobile apps.

Who performs a11y testing?

• Testers can perform a11y testing
• Can outsource to 3rd party companies that specialize in a11y testing

Do you postpone testing if there are a11y bugs?

• This depends. I'd lean towards no, but depends on the bug. If there are bugs that need to be addressed, you can fix them in your SDLC/Agile process for the next release (fast follow up).

Answer 2

Partial answer-

1. Use Chrome's lighthouse, it's part of the dev tools and does some accessibility tests

2. Some features can be easily tested by automation like tabbing order or usage of correct structure of menus

Answer 3

Accessibility testing is a type of usability testing and it's main motive to make software's and applications compatible with various user requirements and needs. With the assistance of accessibility testing testers ensure that the applications are suitable for all users and fulfil their requirements. Moreover, it allows them to develop a software in accordance with the Web Content Accessibility Guidelines (WCAG), which make sure that a particular software and application can be accessed effortlessly by a person with disabilities.

Testing can be either conducted manually or via automated tools.

Manual Testing: Manual testing involves team of testers who validate the software based on their skills and knowledge. This is an effective form of testing, as it is capable of detecting issues and defects that cannot be tracked programmatically.

Automated Testing: Automated testing is most efficient way of testing with extremely high importance in accessibility testing.

Accessibility testing is performed by two groups i.e. experts and users. The experts test the basic code and technical aspects of the software, while the users help validate the quality and effectiveness of the accessibility testing.

1. Expert Testing: This part of accessibility testing targets the below five basic components: Structural Inspection Screening & Assistive Perception Operability Understandability

2. User Testing: This type of testing is done with end users perspective, helping the team to uncover and resolve undetected issues. User testing is mainly performed with the help of assistive technologies, focusing on the software or the web page that offering accessibility to people.