What are the options for automated testing of screen reader in Web apps?

Question

Our team's resources are limited, still we need to make sure our site is accessible (government site, EU). Manual testing includes a lot of fiddling around with JAWS, sometimes other screen readers, Lunar Plus for visually constrained users etc. We would like to automate as many of those tasks as possible.

While automating semantic checks (like for attribute on labels or correct heading order) is pretty trivial with Selenium or PhantomJS, getting correct output from JAWS is a completely different matter. Is there any way to actually write a test script and verify the output of a screen reader?

Answer 1

Actually using a screen reader is a difficult skill to master. It is unlikely that you would be able to get a QA resource who is expert enough in using a screen reader to know whether a problem using the tool is actually a bug or not. In my experience, you would be much better off focusing on ensuring you are following best practices than by trying to simulate the use of accessibility tools.

IE's developer tools include a "Validate | Accessibility" menu option.

There are tons of other free tools that will validate HTML for you with a focus on accessibility and most do a pretty good job. Many are listed here: http://www.w3.org/WAI/ER/tools/complete.html

Edit: After reading a few of the other answers, I wanted to expand on my answer. Let's suppose you do use Jaws to test the accessibility of your web site. Let's say that Jaws has a bug, a flaw in the way it uses the accessibility data on your site. You see that there is a problem, assume it is a bug with your web site's accessibility and "fix it" for Jaws, thereby breaking it for all other screen readers. Jaws has flaws, just like any other software, especially with the HTML 5 changes being rolled out and implemented differently in different browsers. Accessibility tools are currently undergoing a lot of churn and change and will definitely have bugs that need to be addressed.

So, I will again stick with my assertion that ensuring you are following accessibility best practices is more cost efficient, more reliable and simpler.

Answer 2

A website become modern face of business for many companies but not many of them know on how they have been read by users with disabilities.

Banks and Governments spend billions on Digital Transformation but they don't give a damn about adhering to accessibility guidelines of WCAG at all.

It is hurting and painful when a customer struggling to book medical appointments in hospital website using screen reader.

Best practices:

Attended a wonderful program that teach screen reader automation testing best practices at: https://engineers-hub.teachable.com/p/istqb-usability-testing-certification-training

Some points I noted recently from the classes:

-Use JAWS or NVDA for every journey being built on web

-Test color contrast

-Test Keyboard Shortcuts compulsory

-Test A,AA level WCAG guidelines bare minimum!

Honestly in my opinion, following points are missing in many famous websites: enter image description here

key notes: You can not automate screen readers. But you can automate objects that read by screen readers. Reference: https://books.google.co.uk/books/about/Advanced_Selenium_Web_Accessibility_Test.html?id=pTCPDwAAQBAJ&printsec=frontcover&source=kp_read_button&redir_esc=y

Answer 3

If you need to support a screen reader for accessibility issues, make sure that the web site support is baked in early on and not bolted on later. As far as I know there are no available off the shelf tools for testing the output (computer generated words). Sometimes you really need to hear what sounds are being generated by JAWS because at times the text used can come out pretty garbled or may even sound like a different word all together.

I tested a JAWS (Job Access With Speech) implementation for a major web site frequently for well over a year. Once you get past the first few hours of listening to the computer generated voice and learn the navigation keys, JAWS testing can be quite easy and quick.

Good luck with your implementation, someone out there will appreciate it!

Answer 4

Screen Reader Types:

If we need to automate the screen reader types, it has the complexity of automating the UI of those readers which is not required if we need to focus on testing the test object which is Website (rather then screen reader)

So we need to focus on the below:

• Elements such as title,alt-text,aria-label which are read by screen readers
• Links, Texts,Navigations which are followed by screen readers

How to Automate?

Use Selenium to automate them. Please see the example taken from a text book:

driver.find_element_by_xpath(""" //div*[@aria-label='Text Box of User Name'] """).click()

Reference: Toolsets such as ZoomText,Selenium,WAT, Pa11y,Axe,JAWS,NVDA used for Test Automation

If you want to try xpath which uses aria-label:

driver.find_element_by_xpath("//div[@aria-label='Main Link for the button ABC']/div[@class='mn-hd-txt' and text()='Any time']");

If you want to write similar scripts in cypressIO,use the below:

cy.get('div[aria-label="Main Link for the button ABC"]').first().click()