Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I'm hoping someone can advise me on some best practices to make my tests more robust.

The problem I'm having is that every so often my tests will fail at the point of WebDriver waiting for an expected condition, e.g. new WebDriverWait(chrome, 10).until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("ul.componentList.j-componentList"))); and I know the element is in the DOM as it will pass more often that not.

So if I was to run this overnight, it could fail when it would pass 9 times out of 10. But the point Im making is that my code seems a bit weak in places and not as robust as it could be. Where am I going wrong?

I've pasted a small snippet of my code to ask where I can make it better so that WebDriver waits and finds the elements before timing out.

You might notice I have my tests in alphabetical order prefixed with a,b,c, etc. Im using the new annotation type FixMethodOrder, just in case anyone asks why I have them alphabetical.

Any constructive advice is most welcome.

@BeforeClass
public static void launchBrowser(){
    String currentDir = System.getProperty("user.dir");
    String chromeDriverLocation = currentDir + "/../../tools/chromedriver/chromedriver.exe";
    System.setProperty("webdriver.chrome.driver", chromeDriverLocation);

    chrome = new ChromeDriver();
}

@Test
public void aLogin(){
    chrome.manage().window().maximize();
    chrome.navigate().to("http://privateweb.com/thisisfordemopurposes");

    new WebDriverWait(chrome, 10).until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("input[type='text'][id='usernameInput']")));

    WebElement usernameField = chrome.findElement(By.cssSelector("input[type='text'][id='usernameInput']"));
    usernameField.sendKeys("fake");

    WebElement passwordField = chrome.findElement(By.cssSelector("input[id='passwordInput']"));
    passwordField.sendKeys("fake");

    chrome.findElement(By.cssSelector("input#login")).click();
}

@Test
public void bSelectBlankProject(){
    new WebDriverWait(chrome, 10).until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//*[@id=\"templateGrid\"]/li[2]/img[1]")));

    WebElement item1 = chrome.findElement(By.xpath("//*[@id=\"templateGrid\"]/li[2]/img[1]"));
    WebElement item2 = chrome.findElement(By.xpath("//*[@id=\"templateGrid\"]/li[2]/img[2]"));
    WebElement item3 = chrome.findElement(By.xpath("//*[@id=\"templateGrid\"]/li[2]/header/span"));
    WebElement item4 = chrome.findElement(By.xpath("//*[@id=\"templateGrid\"]/li[2]"));

    Actions click = new Actions(chrome);
    click.moveToElement(item1).moveToElement(item2).moveToElement(item3).moveToElement(item4).click().build().perform();

    System.out.println("Blank Project has been selected");
}

@Test
public void cDragCloseButtonOnToTheCanvas(){

    new WebDriverWait(chrome, 10).until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("ul.componentList.j-componentList")));

    WebElement listContainerClose = chrome.findElement(By.xpath("//*[@id=\"componentsDiv\"]/div/div/div"));
    WebElement componentListClose = chrome.findElement(By.xpath("//*[@id=\"componentsDiv\"]/div/div/div/ul"));
    WebElement closeButton = chrome.findElement(By.xpath("//*[@id=\"componentsDiv\"]/div/div/div/ul/li[2]"));
    WebElement componentThumb = chrome.findElement(By.xpath("//*[@id=\"componentsDiv\"]/div/div/div/ul/li[2]/div[1]"));
    WebElement imageHolderCloseButton = chrome.findElement(By.xpath("//*[@id=\"componentsDiv\"]/div/div/div/ul/li[2]/div[1]/div"));
    WebElement componentNameCloseButton = chrome.findElement(By.xpath("//*[@id=\"componentsDiv\"]/div/div/div/ul/li[2]/div[2]"));

    WebElement canvas = chrome.findElement(By.cssSelector("div#page-c3"));

    Actions dragAndDrop = new Actions(chrome);

    dragAndDrop.clickAndHold(closeButton)
            .moveToElement(listContainerClose)
            .moveToElement(componentListClose)
            .moveToElement(componentThumb)
            .moveToElement(imageHolderCloseButton)
            .moveToElement(componentNameCloseButton)
            .moveToElement(canvas)
            .release().perform();

    WebElement draggableCloseButton = chrome.findElement(By.cssSelector("div.t-componentImg-A.component.closeButtonComponent.draggableComponent.ui-draggable"));

    Assert.assertEquals("closeButton", draggableCloseButton.getAttribute("data-type"));
}
 @Test
public void dRenamingCloseButtonComponent(){
    WebElement name = chrome.findElement(By.cssSelector("input.t-textInput-A[name=\"name\"]"));

    new WebDriverWait(chrome, 5).until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("input.t-textInput-A[name=\"name\"]")));

    String selectAll = Keys.chord(Keys.CONTROL, "a");

    name.sendKeys(selectAll);
    name.clear();
    name.sendKeys("Close me");
    Assert.assertEquals("Close me", name.getAttribute("value"));
    System.out.println("Renamed the close button to something else");
} 
@Test
public void eMoveComponent(){
    String selectAll = Keys.chord(Keys.CONTROL, "a");

    new WebDriverWait(chrome, 5).until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("input.t-textInput-A.number[name=\"x\"]")));

    WebElement xAxis = chrome.findElement(By.cssSelector("input.t-textInput-A.number[name=\"x\"]"));
    xAxis.sendKeys(selectAll);
    xAxis.clear();
    xAxis.sendKeys("240");
    Assert.assertEquals("240",xAxis.getAttribute("value") );

    new WebDriverWait(chrome, 5).until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("input.t-textInput-A.number[name=\"y\"]")));

    WebElement yAxis = chrome.findElement(By.cssSelector("input.t-textInput-A.number[name=\"y\"]"));
    yAxis.sendKeys(selectAll);
    yAxis.clear();
    yAxis.sendKeys("10");
    Assert.assertEquals("10", yAxis.getAttribute("value"));
}

EDIT 1: Comments in relation to @ErkiM answer.

I have just run one of my classes and received a failed test. `@Test public void jGoToPage(){

    new WebDriverWait(chrome, 5).until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("select.t-select-A[name=\"page\"]")));
    List<WebElement> option = chrome.findElements(By.cssSelector("select.t-select-A[name=\"page\"]"));

    option.get(0).click();
    option.get(1).click();

}`

The stacktrace I receive is as follows:

org.openqa.selenium.TimeoutException: Timed out after 5 seconds waiting for visibility of element located by By.selector: select.t-select-A[name="page"] Build info: version: '2.34.0', revision: '11cd0ef93615408e0b6b3bfa28defe125906461a', time: '2013-08-06 11:43:14' System info: os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.7.0_45' Driver info: driver.version: unknown at org.openqa.selenium.support.ui.FluentWait.timeoutException(FluentWait.java:259) at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:228) at AdBuilderComponents.CloseButtonPropertiesPanel.jGoToPage(CloseButtonPropertiesPanel.java:308) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.junit.runner.JUnitCore.run(JUnitCore.java:160) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120) Caused by: org.openqa.selenium.NoSuchElementException: no such element (Session info: chrome=31.0.1650.63) (Driver info: chromedriver=2.6.232923,platform=Windows NT 6.1 SP1 x86_64) (WARNING: The server did not provide any stacktrace information) Command duration or timeout: 2.07 seconds For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html Build info: version: '2.34.0', revision: '11cd0ef93615408e0b6b3bfa28defe125906461a', time: '2013-08-06 11:43:14' System info: os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.7.0_45' Session ID: c211c5ddfe42b768b3bf697b80ce1508 Driver info: org.openqa.selenium.chrome.ChromeDriver

share|improve this question
    
The exception is self-explanatory: Timed out after 5 seconds waiting for visibility of element located by By.selector: select.t-select-A[name="page"]. There can be several reasons, for example your locator is wrong or it takes more than 5 seconds to load that component. First of all, i usually use longer timeout as i am testing for functionality, not performance. Secondly, always test your locators, for firefox there are several plugins available. –  Erki M. Jan 6 at 8:37
    
Anyway, split your statement and see if By.cssSelector("select.t-select-A[name=\"page\"]") returns anything you need. –  Erki M. Jan 6 at 8:38

3 Answers 3

Hm. Did I understand correctly that you are indeed receiving a timeoutexception? Could you provide any stacktrace?

"I know the element is in the DOM as it will pass more often that not.

So if I was to run this overnight, it could fail when it would pass 9 times out of 10."

This statement is not good enough. The best you can say is that you can be sure, that the element was there for at least 9 times out of ten.

So whats happening in visibilityOfElementLocated:

  /**
   * An expectation for checking that an element is present on the DOM of a page
   * and visible. Visibility means that the element is not only displayed but
   * also has a height and width that is greater than 0.
   *
   * @param locator used to find the element
   * @return the WebElement once it is located and visible
   */
  public static ExpectedCondition<WebElement> visibilityOfElementLocated(
      final By locator) {
    return new ExpectedCondition<WebElement>() {
      @Override
      public WebElement apply(WebDriver driver) {
        try {
          return elementIfVisible(findElement(locator, driver));
        } catch (StaleElementReferenceException e) {
          return null;
        }
      }
     // skipped some sht
  }

  /**
   * @return the given element if it is visible and has non-zero size, otherwise
   *         null.
   */
  private static WebElement elementIfVisible(WebElement element) {
    return element.isDisplayed() ? element : null;
  }

So to sum it up, this code checks if an element is present in DOM and its height and width are > 0, if so, it will return the WebElement, otherwise null. It is straightforward, if you are getting a timeout, you can be pretty sure the element is not in the DOM.

Anyway, what is happening before this:

ExpectedConditions.visibilityOfElementLocated(By.cssSelector("ul.componentList.j-componentList")));

some databinding? other stuff that could perhaps cause the staleness of the webelement? Try this piece of code:

new FluentWait<WebDriver>(driver)
        .withTimeout(60, TimeUnit.SECONDS)
        .pollingEvery(1, TimeUnit.SECONDS)
        .ignoring(NoSuchElementException.class)
        .ignoring(StaleElementReferenceException.class)
        .until(new Function<WebDriver, Boolean>() {
            @NotNull
            @Override
            public Boolean apply(WebDriver webDriver) {
                WebElement element = driver.findElement(yourElement);
                return element != null && element.isDisplayed();
            }
        });

This is definetly overly verbose, but i wrote it all out to make it more understandable. If it doesn't help you, i suggest adding more logging to better understand what is going on there. Perhaps 10 sec is not enough some times? Add longer timeout? Perhaps get a screenshot if the test is failing?

As you are using JUnit to execute your test and you were also talking about general robustness, perhaps implement a TestWatcher? Something like:

public class TestRules extends TestWatcher {

    @Override
    protected void failed(Throwable e, Description description) {
        // This will be called whenever a test fails.
        // Take screenshot, log error, go hot wild.
    }

And in your test class simply call it:

public class testClass{

@Rule
public TestRules testRules = new TestRules();

@Test
public void doTestSomething() throws Exception{
    // If the test fails for any reason, it will be caught by testrules.
}

EDIT: Also, as robustness was mentioned - when you run into maintenance nightmare, look into this article: http://code.google.com/p/selenium/wiki/PageObjects

EDIT2: I looked thru the sample code again and i have been there. this spaghetti code will be a maintenance nightmare in no time. Seriously consider any test architecture.

share|improve this answer
    
I have added comments to the end of the question. –  Django_Tester Dec 31 '13 at 11:23
  • You can move to testng and use dependsOnMethods. You will have to change few things in your code to make it compatible.

It makes sure your tests in the same class are executed in the good order. In the following example "secondTest" will start only when "firstTest" is done.

@Test
public void firstTest() {
    //code
}
@Test (dependsOnMethods = { "firstTest" })
public void secondTest() {
    //code
}
share|improve this answer
    
Thanks for your response @David although I'm not looking for a different method sorter to execute my methods. I'm looking for some advice around making my code loop until it try's and catches the element in question. As in my question, I mentioned that occasionally WebDriverWait will timeout after the allocated time, so Im asking what is the best way to loop this without timing out. –  Django_Tester Dec 17 '13 at 16:16
    
At least now you know a better way to sort your test than a,b,c :D. I am not sure if it is the cleaner way but if your are blocked add a timer after your actions and before your checked as: Thread.sleep(10000); –  David Dec 17 '13 at 16:25
    
I was using 'try { Thread.sleep(5000); } catch (InterruptedException e) { e.printStackTrace(); }' but this is not great practice, so Im led to believe. Same goes for listing tests in alphabetical ordering as they should be independant tests. Perhaps using annotations BeforeClass, Before, After, AfterClass is the best way. But again, I'm looking for some advice as I dont have a mentor to learn from. –  Django_Tester Dec 17 '13 at 16:31
    
@David "add a timer after your actions and before your checked as: Thread.sleep(10000);" <- this is seriously bad advice. What happens, if "the thing" takes 10001 ms? Your code will fail. If it takes 9999ms, then you are wasting time. –  Erki M. Dec 17 '13 at 21:46

I advice you to keep your tests simple and they will be more robust.
You should specialize your test:
- UI components (Verify the display of your page)
- UI actions (Verify button are working as expected for example)
- Workflows
More you are verifying things in your test and more the portability it fails will increased.
Don't hesitate to use @BeforeClass and @BeforeTest to keep your code more clear. After your test don't forget to use @AfterClass or @AfterTest if you need to close a database connection or driver.
You can use Selenium IDE to generate your JUnit code if you want to see others ways to make your test.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.