Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I have a class with dependent methods using the TestNG annotation

dependsOnMethods

The test runs fine 100% of the time if I simply run it as a TestNG Test from the package.

When I include the test in a TestNG Suite, the methods run out of order. Yes, I am using:

<test name="Test" preserve-order="true">

in my .xml file.

Every other test in the suite respects the method ordering and run without issue. Is there any known information on why this might be occurring?

Test code:

@Test(groups={ "Administration"})
public class RoleCrudTest extends AbstractIntegrationTest
{
protected static SeleniumActionHelper action;

    @Test
    public void inactiveRole() throws Exception
    {
        SeleniumHelper helper = new SeleniumHelper();
        action = new SeleniumActionHelper(driver);

        helper.login();

        String roleUrl = navigateToUrl("role/roles.xhtml");
        driver.get(roleUrl);

        assertEquals("Role:", findElementBySelector("span.portletButtonHeader").getText());

        WebElement roleName = findElementById("roleName");
        assertFalse(roleName.isEnabled());

        WebElement deptId = findElementById("deptid");
        assertFalse(deptId.isEnabled());
    }

    @Test(dependsOnMethods = "inactiveRole")
    public void createRole() throws Exception
    {
        WebElement addButton = findElementById("add");
        addButton.click();

        waitUntilAjaxRequestCompletes();

        WebElement roleName = findElementById("roleName");
        roleName.click();
        roleName.sendKeys("AAAAAAAA");

        WebElement deptId = findElementByXpath("(//button[@type='button'])[3]");
        deptId.click();

        WebElement dept = findElementByXpath("//div[@id='department_panel']/ul/li[2]");
        dept.click();

        WebElement checkbox = findElementByXpath("//li[@id='privileges:1']/div/span/div/div");
        checkbox.click();

        Thread.sleep(1000);

        WebElement save = findElementById("save");
        save.click();

        assertEquals("Role saved successfully", findElementBySelector("div.ui-growl-message > p").getText());
    }

    @Test(dependsOnMethods = "createRole")
    public void editUndo() throws Exception
    {
        Thread.sleep(1000);

        WebElement tableSort = findElementByXpath("//th[@id='tableSection:rolesListWrapped:j_idt85']/div/span[2]");
        tableSort.click();

        Thread.sleep(1000);

        WebElement createdRole = findElementByXpath("//tbody[@id='tableSection:rolesListWrapped_data']/tr[1]/td/div");
        createdRole.click();

        Thread.sleep(1000);

        WebElement roleName = findElementById("roleName");
        roleName.click();
        roleName.clear();
        roleName.sendKeys("edited");

        WebElement deptId = findElementByXpath("(//button[@type='button'])[3]");
        deptId.click();

        WebElement dept = findElementByXpath("//div[@id='department_panel']/ul/li[3]");
        dept.click();

        WebElement checkbox = findElementByXpath("//li[@id='privileges:1']/div/span/div/div/span");
        checkbox.click();

        WebElement checkbox2 = findElementByXpath("//li[@id='privileges:0']/div/span/div/div");
        checkbox2.click();

        Thread.sleep(1000);

        WebElement undo = findElementById("cancel");
        undo.click();

        String text = findElementById("roleName").getAttribute("value");
        String oldtext = "AAAAAAAA";

        assertTrue(text.equals(oldtext));
    }

    @Test(dependsOnMethods = "editUndo")
    public void editRole() throws Exception
    {
        Thread.sleep(1000);

        WebElement createdRole = findElementByXpath("//tbody[@id='tableSection:rolesListWrapped_data']/tr[1]/td/div");
        createdRole.click();

        Thread.sleep(1000);

        WebElement roleName = findElementById("roleName");
        roleName.click();
        roleName.clear();
        roleName.sendKeys("AAAAAAAAedited");

        WebElement deptId = findElementByXpath("(//button[@type='button'])[3]");
        deptId.click();

        WebElement dept = findElementByXpath("//div[@id='department_panel']/ul/li[3]");
        dept.click();

        WebElement checkbox = findElementByXpath("//li[@id='privileges:1']/div/span/div/div/span");
        checkbox.click();

        WebElement checkbox2 = findElementByXpath("//li[@id='privileges:0']/div/span/div/div");
        checkbox2.click();

        Thread.sleep(1000);

        WebElement save = findElementById("save");
        save.click();

        assertEquals("Role saved successfully", findElementBySelector("div.ui-growl-message > p").getText());
    }

    @Test(dependsOnMethods = "editRole")
    public void deleteRole() throws Exception
    {
        Thread.sleep(1000);

        WebElement deleteButton = findElementById("tableSection:delete");
        deleteButton.click();

        WebElement deleteConfirm = findElementById("confirmDelete:yes");
        deleteConfirm.click();

        Thread.sleep(500);

        assertEquals("Role deleted successfully", findElementBySelector("div.ui-growl-message > p").getText());

        waitUntilAjaxRequestCompletes();
    }
}

And here is the entire TestNG XML file (please note that this is a simple one with only 3 classes, the behavior is the same if I run an XML with 20 classes). No matter the amount of classes or ordering, the roleCrud test always runs the methods out of order, every other class runs fine:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="none">
  <test name="Test" preserve-order="true">
    <classes>
      <class name="com.x.c4i.view.security.RoleCreateErrorCheckTest"/>
      <class name="com.x.c4i.view.security.GroupCrudTest"/>
      <class name="com.x.c4i.view.security.RoleCrudTest"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->
share|improve this question
    
test code snippet, more of your testng.xml may help others in helping you –  Tarun Jul 20 '12 at 7:06
    
test code attached –  squeemish Jul 20 '12 at 13:16
    
Don't see any thing obviously wrong, can you post testng.xml file and order in which you see method execution in this class. Is the order always random for this class? –  Tarun Jul 23 '12 at 18:38
    
Updated with the entire TestNG.xml file. The method execution I see is inactiveRole->editUndo->editRole->createRole. However, sometimes editUndo and editRole get switched in position –  squeemish Jul 23 '12 at 20:06
    
Yet I don't see any thing wrong. I suggest you to post it here - groups.google.com/forum/#!forum/testng-users with a reproducible test case which demonstrate this issues. –  Tarun Jul 24 '12 at 8:32

1 Answer 1

I have reported this on GitHub https://github.com/cbeust/testng/issues/261. This issue https://github.com/cbeust/testng/issues/97 might be related too, if not duplicate.

In my opinion it is a bug waiting to be fixed.

share|improve this answer
    
Mods, could you convert this to a comment? It's not really an answer. –  Kate Paulk 4 hours ago
    
when you've got more reputation, you'll be able to add comments to others posts. –  Kate Paulk 4 hours ago
    
+1, I would comment if I could –  yurloc 3 hours ago

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.