QF-Test - Conditionally rerun scheduled test suite based on result

Question

We have about 10 test suites (QF-Test GUI Testing tool) for a java application. At the moment the testsuits are run every night with a batch-script triggered by a scheduled job. The tests are not very stable - so every day we have some failures. QF-Tests gives back the test-result in the exit code (0=OK, 1=Warning, 2=Error, etc.). I would like to rerun the tests if they fail.

Is there a tool to run the test suite and run other commands depending on the result?

Example:

• If Exitcode=1: Run Report-Command
• If Exitcode=2: Delete the results and rerun until 5 retries reached.
• If Exitcode=3: ...

I know I could do this programmatically, but I wonder if there is an easier solution for this.

Thanks for any advice.

Answer 1

I don't know any tool parsing exit codes.

---

Anyway, I suppose "Programmatically" (writing own script) is the best solution.

At least you know, what's happening and can fix it yourself quick.

---

Here's batch script as an example:

qftest.exe ...
if %ERRORLEVEL% EQU 1 ( command1; command2; etc )
if %ERRORLEVEL% EQU 2 ( command1; command2; etc )

Answer 2

Updated based on comments. I have not used any tool but a little custom code for below approach helped me achieve rerunning failed cases.

Through Custom Code

• Easier option is to track the flags after test execution in a simple DB table (TestResults - TestcaseId, TestrunId, Result) - Update it after each run, or a custom xml to save the results
• After Test run parse through flagged (failed) entries in tables / failed cases and re-run them selectively
• Since you execute from command line, it should be easy to group / execute by flagging failed cases and retry it later
• Also, You can run it for configured number of retries

By using TestNG, Junit you can leverage in-built features for re-running failed cases.

From TestNG blog - http://testng.org/doc/documentation-main.html#rerunning. Rerun with failed XML testng-failed.xml

Using TestNG - Another approach listed in post - http://seleniumworks.blogspot.in/2013/12/re-run-failed-tests-automatically.html