TestNG not executing all tests in suite

Question

TestNG fails to execute all of my tests. XML file looks like this:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">

<suite name="Guru Test Suite">
    <parameter name="category" value="land_category"/>
    <parameter name="testwebsite" value="staging"/>
    <parameter name="locale" value="en"/>

    <listeners>
        <listener class-name="helpers.RetryListenerClass"/>
        <listener class-name="helpers.TestStatusListenerClass"/>
        <listener class-name="helpers.OnFailureScreenShotListener"/>
    </listeners>

    <test name="GuruTests" preserve-order="true">

        <groups>
            <define name="allLoginChecks">
                <include name="allLoginChecks"/>
            </define>
            <define name="allListingsChecks">
                <include name="allListingsChecks"/>
            </define>
            <define name="allFeaturedChecks">
                <include name="allFeaturedChecks"/>
            </define>
            <define name="allChecks">
                <include name="allChecks"/>
            </define>
            <define name="allProfileChecks">
                <include name="allProfileChecks"/>
            </define>

            <define name="allTests">
                <include name="allChecks"/>
                <include name="allLoginChecks"/>
                <include name="allListingsChecks"/>
                <include name="allProfileChecks"/>
                <include name="allFeaturedChecks"/>
            </define>

            <run>
                <include name="allTests"/>
                <!--<include name="allLoginChecks"/>-->
                <!--<include name="allChecks"/>-->
                <!--<include name="allListingsChecks"/>-->
                <!--<include name="allProfileChecks"/>-->
                <!--<include name="allFeaturedChecks"/>-->
            </run>
        </groups>
        <classes>
            <class name="guru_tests.DriverSingleton"/>
            <class name="guru_tests.DataProviders"/>
            <class name="guru_tests.AddListingTests"/>
            <class name="guru_tests.LoginLogoutTests"/>
            <class name="guru_tests.ListingsPortfolioTests"/>
            <class name="guru_tests.AgentProfileTests"/>
            <class name="guru_tests.FeaturedListingsTests"/>
            <class name="helpers.GuruHelper"/>
            <class name="helpers.AddListings"/>
        </classes>
    </test>
</suite>

I try to execute it either as it is now or by using the commented lines in "run" section. In both cases, only some of my tests are executed (45/140).

All of the classes and methods are public. There is no "enabled = false" on them. When executing only one out of defined groups everything runs smoothly and all of its methods are executed. I even tried to add a new group and assign it to all methods, but still, the result remains the same.

Not all methods are executed from each group (some groups don't get executed at all) and there are cases where a group doesn't run the @AfterClass when finishing up.

Since each individual group is assigned to all methods of a single class, I assume I could try executing the classes instead, but my @BeforeSuite lies in a completely different class and I kinda like the grouping options of testNG, because I want to able to execute different sets of combinations of these methods in future.

This happens when executing directly from intellij Community Edition or from the terminal. As you can see there in no option for parallel execution set and the preserve-order="true" is not respected as well.

I, also, tried updating testNG and Selenium to latest versions.

Answer 1

It may not helpful, but I googled about testNG grouping.

To run a test case, specify a group name as a parameter at Test annotation.

@Test(groups = {'groupname1', 'groupname2'})
public void test_01();

Do your Test cases have proper annotation parameters?