How to disable specific Maven and TestNG test listeners when running tests from groups?

Question

I am using Maven and TestNG to support my Selenium testing framework for a project.

I am using a custom test listener to report results into TestRail (our TCM tool). I am also using another custom test listener to handle cleaning up HAR files leftover by only a few tests.

Most of my test classes have only the TCM test listener specified like this:

@Test(groups = {"a", "b", "c"})
@Listeners({TestNGCaseResultsListener.class})
public class MyTestClass extends SeleniumTestRailTest { ... }

My Maven Failsafe POM includes section is:

<includes>
   <include>**/folderOne/*.java</include>
   <include>**/folderTwo/*.java</include>
   <include>**/folderThree/**/**.java</include>
   <include>**/folderFour/tests/*.java</include>
</includes>

I am not specifying test listeners in the POM, via the command line, nor am I using a TestNG suite XML file. The only times the listeners are specified is as annotations for the class as appropriate (see above).

If I issue this command to run one of the TCM-listener-only tests:

mvn -Dgroups=a verify

My HARfile cleanup listener (which is not attached to the test class that is running) will get fired off anyway when a test passes or fails.

Note that there is a test class that matches my Failsafe includes, which is annotated with the 'unwanted' listener. That class is in folderFour/tests/HARfiletest.java (following my includes described above.)

Is that listener being loaded up even though that class isn't being used? Is there a way to avoid this from happening? I only want test listeners annotated on classes that actually have tests run, not from any other classes.

Answer 1

As noted in TestNG documentation, this behavior is indeed as expected:

Note that the @Listeners annotation will apply to your entire suite file, just as if you had specified it in a testng.xml file. If you want to restrict its scope (for example, only running on the current class), the code in your listener could first check the test method that's about to run and decide what to do then.

It then goes on to describe an example (see the above link).

As an additional example of this approach, the listener could implement IInvokedMethodListener and add some attribute of the test result to indicate that it applies to the specific test method.

In the following example, I add a Boolean attribute to the test result, which can be used in other methods that have access to the result object to check whether any work should be done by the listener:

public class TestListener2 implements IInvokedMethodListener, ITestListener {

    @Override
    public void beforeInvocation(IInvokedMethod iInvokedMethod, ITestResult iTestResult) {
        ConstructorOrMethod consOrMethod = iInvokedMethod.getTestMethod().getConstructorOrMethod();
        Listeners listeners = consOrMethod.getMethod().getDeclaringClass().getAnnotation(Listeners.class);
        List<Class<? extends org.testng.ITestNGListener>> listenerList = (listeners != null && listeners.value() != null) ? Arrays.asList(listeners.value()) : null;
        Boolean valid = false;
        if (listenerList != null) {
            for (Class<?> listenerClass : listenerList) {
                if (listenerClass.getName().equals(this.getClass().getName())) {
                    valid = true;
                }
            }
        }
        iTestResult.setAttribute(this.getClass().getName(), valid);
    }

    @Override
    public void onTestSuccess(ITestResult result) {
        if ((Boolean)result.getAttribute(this.getClass().getName())) {
            // Do the work
        }
    }
}