Random Selenium WebDriver Tests inconsistently Fail and Pass on IE browser

Question

Does anyone have any suggestions on how I can improve this situation? I have one suite of tests that I use for all three Firefox, Chrome, and IE browsers. The results I get for the browsers differs - suite passes for Firefox and Chrome, but fails for IE.

I noticed that despite running my suite of tests many times, IE 11 randomly fails tests, but if I rerun them it passes sometimes. Somehow if I run it in a batch in testNG suite it fails more likely, but if I run it as individual test it passes. Not the same tests fail on each run. On first run, test A would fail, but on second run, test B would fail, then on third run, test A and test C would fail. I even set all IE security zones to protected and I also use IEDriverServer 32 bit, which improved things by allowing my test suite to complete instead of hanging/crashing the browser and not logging my test results. However there are still issues with my tests running in IE.

I also noticed a specific test when I run it on Firefox it fails the first time, but passes the 2nd time. This is same behavior with IE. But a few days earlier my entire suite passed for Firefox and Chrome. I'm just trying to get it working for IE now. I don't know if my tests are getting this intermittent fail for all browsers with the exception of IE seeing it more frequently than other browsers. Or if it is just an IE issue. Or if I need to add more timing. But it already takes 1.5hr to 2hr to run 34 test scripts in my suite.

Thanks for your help. Any suggestions or tips would be good? Thanks.

Answer 1

Do you use explicit or implicit wait? Converting my tests to 100% explicit waits eliminated flakiness like you describe.

See also https://stackoverflow.com/questions/15164742/combining-implicit-wait-and-explicit-wait-together-results-in-unexpected-wait-ti by Jim Evans, member of the Selenium team

Answer 2

As Peter mentioned your problem could be that you are using implicit waits instead of explicit waits. Here is another resource on the difference between the two.

The crazy thing about Chrome, Firefox and IE is that all three run at different speeds. I've found Chrome to be the fastest, Firefox to be the middle ground and IE to be the slowest. That means I typically have to build in enough explicit waits so they pass for Chrome's speed (otherwise I get elements aren't clickable) but make sure my timeouts are high enough so that my IE test runs don't timeout.