How to run TestNG failed test cases in Jenkins for Maven project?

Question

I am new to Jenkins and I have configured it on my Windows PC. I have to run failed test cases twice before considering that they are failing. I am using TestNG and Selenium Webdriver for writing test cases.

I have created a job in Jenkins which maps to my workspace and runs the testng.xml. I have specified the path of testng.xml in pom.xml. I have created a maven project and I run the pom.xml for running my test cases.

Now after a successful run of all test cases, I get surefire reports of the tests. I want to run the failed test cases i.e testng-failed.xml twice. I have tried running it using the following command in the post build section:

cd path to workspace
mvn test -DtestNG.file=/target/surefire-reports/testng-failed.xml

But when I run the above command it runs testng.xml instead of running testng-failed.xml. I am not sure if I am on the correct path for running failed test cases.

I need help guys. I am stuck at this point.

Answer 1

You have to do something like this

<plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.17</version><!--$NO-MVN-MAN-VER$ -->
            <configuration>
                <!-- Sets the VM argument line used for unit tests are run. -->
                <suiteXmlFiles>
                    <suiteXmlFile>src/test/resources/testng/testng.xml</suiteXmlFile>
                    <suiteXmlFile>target/surefire-reports/testngfailed.xml</suiteXmlFile>
                </suiteXmlFiles>
            </configuration>
 </plugin>

Answer 2

Form above explanation it seems that failed-testng.xml gets deleted when you re-run your tests in Post task under Jenkins, because when you run "mvn clean" it will delete the target folder where failed-testng.xml was present. To verify this you need to access the Jenkins workspace, but your software testing company might not have the access to it.

Further follow these steps they will solve the issue:

First your pom should be configurable to run the tests from command line:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.19.1</version>
    <configuration>
      <forkCount>0</forkCount>
      <suiteXmlFiles>
        <suiteXmlFile>src/test/resources/${suiteXmlFile}</suiteXmlFile>
      </suiteXmlFiles>
    </configuration>
  </plugin>

Then configure jenkins for the Post build(which is you already doing) to run the failed-testng.xml.

But first you have to copy/move the failed-testng.xml to from target folder to any other folder within framework(except target and test-output folder, because these folders gets deleted when you run the command "mvn clean").

Below is the Maven plugin to copy/move file from one location to another location:

<plugin>
    <artifactId>maven-resources-plugin</artifactId>
    <version>3.0.1</version>
    <executions>
      <execution>
        <id>copy-resources</id>
        <!-- here the phase you need -->
        <phase>validate</phase>
        <goals>
          <goal>copy-resources</goal>
        </goals>
        <configuration>
          <outputDirectory>${basedir}/target/failed-testng.xml</outputDirectory>
          <resources>          
            <resource>
              <directory>src/</directory>
              <filtering>true</filtering>
            </resource>
          </resources>              
        </configuration>            
      </execution>
    </executions>
  </plugin>

Note: This plugin must be executed after the tests execution.

Answer 3

Is switching to jUnit an option?

With jUnit and the surefire plugin for maven, you can can start your tests with a parameter that will restart your testcase in case it failed.

mvn -Dsurefire.rerunFailingTestsCount=5 test where 5 is the maximum number of restart on fail accepted until it will generate an error.

more info can be found here: https://maven.apache.org/components/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html

Answer 4

See there is another way to check your failed testcase is really failed due to your mistake or any other things(e.g. if element is not getting due to synchronization ot stale element is seen then those issues are in your script) Any way you have to use Analyzer and transformer interfaces of testng to run failed testcase more than one.