Selenium / Java: how to handle test case execution on a flaky network connection?

Question

What is the best strategy to handle test case execution when there is a possibility of losing network connectivity during execution? In my case, I have to run 300 test cases which usually take around one hour. It is a waste of time if I find out after an hour that some of my test cases failed due to an intermittent internet issue. I have to check the failed test cases manually, or I have to run the automated tests again, which is inefficient.

In Selenium, how would I detect when a test failed due to a network resource being inaccessible and retry rather than failing? What are good practices for writing durable tests that use network resources?

Answer 1

A strategy I use is restarting the test if it fails. Currently we restart tests three times before we deem it as a failed test.

There are a list of reasons why a test can possibly fail (Page loaded too slow, too fast, network dropped, server was bumped, ajax not responding as quickly while developing the test, etc..)

A hook could be created to check every page to see if its a 404, but or connection reset, then restart - but I would advise against this.

I'd rather restart the test on any failure, log the exception, and take a screenshot. That way if the test passes the second time I know it is something with my test/network issue and the CI suite still passes.

There are plenty of times where my tests restarted and logged enough information for me to go back and add additional waits to wait appropriately in failing situations.

Answer 2

The first solution to this problem is implementing the re-execution logic. Many software testing companies are using the same to avoid the flaky connection failures.

Further there are two ways of implementing re-execution logic.

1. Re-execute the script as soon as it fails.
2. Re-execute the failed scripts after the complete suite execution.

There is one more way to avoid the fake failures due to network connection. In this way if an element is not found once, then we can again check for the visibility of the same element by keeping the element in loop and validating it for 3 to 4 times.