How to analyse 'Element not Found' exceptions when working with Selenium

Question

When you are writing Selenium/WebDriver automation scripts you will probably run into the exception NoSuchElementException. You think your selector is working, but somehow the element cannot be found.

What strategy should I take to analyse different reasons that this might be the case?

What should I research and how? Good answers would contain example situations and how to resolve them. (e.g. verify selectors, frames, waiting, etc)

Answer 1

1. Trust your code and doubt SUT (Software under test):

If everything was working fine and the test starts to fail suddenly. Instead of debugging your code for issues, start with checking the actual product. Do some visual inspection and see whether the development team has modified the element or the element is no longer being displayed.

2. Trust your code and doubt the environment:

If everything was working fine locally and failed as soon as you integrated to CI/CD. Then investigate the product behaviour in the test server. Mostly due to OS and configuration difference product won't work as it would in local ( Raise a Bug)

3. Now doubt your scripts (Using absolute XPATH):

You might be using absolute XPATH, this causes flaky tests when DOM structure changes. Use relative XPATH (CSS would be more recommended). Never use xpath/ if you have a unique ID/name to identify an element.

4. Now doubt your scripts (Not using explicit wait):

Sometimes scripts lack explicit wait and try to interact with dynamic elements, this causes the test to fail because it tries to interact with the element before it is even available in DOM.

5. Now doubt your scripts (Handling spinners):

Sometimes spinner takes time to get displayed. SO, if you are just checking for the invisibility condition then it will return true and try to interact with next element before the actual spinner event completes

So, first, check the visibility of spinner and then check for invisibility before interacting with other dynamic elements.

6. Now doubt your scripts (Not handling iFrames):

Sometimes element will be inside iframes and scripts won't switch between frames before interacting with these elements.

Check whether, any parent element contains the tag frame or iframe to determine whether the element is inside an iframe

*********************Add On******************************************

7. Now doubt your scripts (Not disabling wait for angular):

Sometimes products use spinners to wait for asynchronous operations to complete behind the scene. For example, you click login and spinner comes up and will not disappear till the background tasks are not completed.

In this case, make sure you are not waiting for the asynchronous operations to complete (eg waitforangular flag set to true in protractor) before interacting with the elements in the temporary overlay.

This is because, if you set waitforangular to true, then scripts wait till all the tasks are completed and by then the temporary overlay (say spinner) will be removed from DOM

Answer 2

PDHide's answer is good in general, but since you specifically ask about verifying selectors, I would like to add that you can test them in a browser console. For CSS selectors, you can test with

document.querySelector(selector);

For XPath, you can test with

document.evaluate(selector, document).iterateNext();

Each of these will return the first matching element if there is one, or null if there is none. They will both also helpfully throw a SyntaxError if your selector is not legal CSS/XPath.

Answer 3

First of all, we need to understand the root cause of the exception.

As written in the Selenium docs, it's thrown by FindElement/FindElements methods.

That might be caused by:

1. Invalid locator. In this case, you have to manually verify its validity on the page.
2. Absence of element within the DOM at the moment of the FindElement call. In other words, your code attempts to find an element while it's not loaded yet.
3. Absence of element within the DOM on the page. You want to find an element on the page where it's actually not present. Update your code according to business logic/app changes.

The second point can be more tricky, and in order to prove that it's the actual root cause, try to set a break-point to occur just before before you want to access (find) the element. In that way, you'll be able to see how the page is loaded in the browser and add specific conditional waits that can help you.

Answer 4

In addition to the other answers, for me the most efficient way to take a screenshot when a test fails and attach to the report. (When using cucumber its easily done with the After hook)

This way I can go to the CI, view the report and quickly understand what happened.