I asked a similar question on Katalon Studio forum, but it didn't get much traction: https://forum.katalon.com/discussion/comment/15556#Comment_15556
Anyway, in order to deal with test flakiness this is what I usually do:
I have a Test Suite consisting of cca 15 Test Cases (obviously, there is more than one TS, but I'm making an example). What usually happens when I run the TS is that two or three TCs fail because of network load issues or varying timeouts but when I run them for the second time they pass.
Now, Katalon Studio console mode has the option of repeating only the failed tests.
So, my idea would be something like this:
- run the TS, let's say 3/15 TCs fail
- re-run the three failed TCs, say 2/3 fail
- re-run the 2 remaining cases, those that fail are really failed
Does this seem like a sound re-test strategy? Are there any best practices?