Is refreshing multiple times a test case for web applications?

Question

I have been faced with a QA test case where refreshing a page multiple times almost instantly (6x, 7x, 8x) creates a bug. This is totally random and every time the number of refreshes require to reproduce changes.

So my question is that should this be even a test case?

Edited It is caused due to uninterrupted page load. Further it is suspected due to DOM elements not rendered fully

Answer 1

Yes, while rather crude and possibly not the most graceful of test cases, such 'abusive' test cases are an important tool in stepping off 'happy path' thinking and testing.

• Effectively a test like this is a crude load/response test.

While in theory it shouldn't happen in the real world environment, the fact that a bug was discovered with such abusive input should be raising red flags for investigation even if ultimately no resources ever get devoted to fixing the issue. ["Known issue, won't fix" is totally a legit 'solution' to some problems. I guess...]

• Why does this non-standard input/interaction cause an unexpected state/bug to be entered?
• Does the above answer expose potential security risks [Such as the system grabbing incorrect, possibly sensitive, data for the wrong user?] or performance spirals? [Such as momentary excessive loads corrupting sessions for some users, who then all end up running multiple hard-refreshes to clear the issue, which in turn creates more server load and causes more user sessions to corrupt?*]

Tests like this go right along with things like navigating back and forth between two screens/states over and over again. The software isn't supposed to be able to fail on things like that, and the low level unit tests should catch it, but at the end of the day what something should do isn't nearly as important to QA as what it actually is doing.

"Something goes wrong when..." are important points to find in the software, and while we may prefer to find more graceful ways to discover such issues the truth is that at times the easiest way is to simply hit things 'at random' with a metaphorical hammer to see if anything breaks.

As a project matures and testing methodologies improve, then such a case may become defunct in favour of more targeted and graceful solutions.

---

It is also important to remember the dangers of thinking "the user will never do something like that in the real world..." because users are mythical creatures and will seemingly do everything in their power to do things no one on a dev team could ever imagine them doing...

• Are they 'aggressively impatient', and hammering on the refresh button out of anger if something goes wrong?
• Do they have a mobility issue or faulty hardware that results in a surge of unexpected or unintended clicks?
• Does the 'rapid refresh' bug kick in if the server is under load, and the user is actually refreshing fairly casually once a minute or so trying to get your site to respond?

---

At the same time realities of resource management need to be considered. Test cases like this may not be well suited to being constant 'front and centre' operations that all get run day to day.

• Maybe group them as a pool, then periodically run a subset of them against a given build rather than the entire pool.
• Possibly useful as part of morale events: Pull the team together with their food and drinks of choice, and make a team game of 'who can break things in the weirdest way'.

---

* I have seen one client reduce their server requirements to nearly an 8th of what they had been using after resolving the software issue behind a problem like this, rather than continue to throw hardware money at it.

Answer 2

Probably not. I like to challenge defects like this with a simple question: Would an actual user execute this behavior?

In most situation I would not expect users to refresh a regular page ten times in a row. Some pages might only update the page content on a refresh, forcing the users to refresh often to get the latest data. I think that is more a UX-issues instead.

So talk to the users. Do they refresh often?

• Yes. It is testcase you should run every time.
• No. Ignore the issue. Why swamp developers with defects that users never run into?

I do like to add a test like this to a set of heuristics. A list you sometimes mix during an explorary testing session.

update: In this case I do wonder if the bug really only happens on quick refreshes, or might it also happen the first time, but only 1 out of 10 times. Is it the load on the server? Is it database locking? What happens with 10 users at the same time.. Ohnoo my brain is on overdrive. Maybe you are on to a real issue. Do research it.

Answer 3

Probably, refresh is not a simple operation as it seems so that's a good system test.

Having said that I would give it low priority and execute it only once in a while since it would significantly slow down the rest of the tests

Answer 4

Every software testing company follow the process of test case creation while testing any feature.

Test cases should be created for the functionalities that actual user will be using on live environment.

This situation only occurs if page is not auto refreshing after applying changes in the application and user has to refresh the application 2-3 times to load all the changes done in the application. For this case, test case creation is mandatory to cover the testing of application changes and it should be the part of regression.