1

I am running Data-driven tests driven from a spreadsheet which are input to TestNG test Framework. In a nutshell, the framework traverses through outlook mailbox and looks for mails based on the subject. Each row in the spreadsheet is the mail subject which is an input to the test.

String comparisonResult = GetMail.compareMails(scenarioName);
    Assert.assertNotNull(comparisonResult,"Mail does not exist");

Under the hood, comparemails method compares the subject of the mail passed as a parameter to the method and looks for that mail in the mailbox.

I am asserting for null value when the mail does not exist and reporting with an assertion message - "Mail does not exist in the mailbox." However the assertion message is not displayed after each test failure, instead is reported a the end of all test runs i.e., after all the rows of the spreadsheet are processed.

When the assertion fails, I would expect it to print the assertion message on the console immediately and process the next test from the spreadsheet.

However, it prints the message at the end of the test all at once like below, assuming there were 5 failures:

[name=null](com.dms.testsuite.Test)  Time elapsed: 66.905 sec  <<< FAILURE!
java.lang.AssertionError: Mail does not exist expected object to not be null
at org.testng.Assert.fail(Assert.java:94)
at org.testng.Assert.assertNotNull(Assert.java:404)
at com.dms.testsuite.Test.submitJob(Test.java:110)

[name=null](com.dms.testsuite.Test)  Time elapsed: 83.24 sec  <<< FAILURE!
java.lang.AssertionError: Mail does not exist expected object to not be null
at org.testng.Assert.fail(Assert.java:94)
at org.testng.Assert.assertNotNull(Assert.java:404)
at com.dms.testsuite.Test.submitJob(Test.java:110)

[name=null](com.dms.testsuite.Test)  Time elapsed: 51.064 sec  <<< FAILURE!
java.lang.AssertionError: Mail does not exist expected object to not be null
at org.testng.Assert.fail(Assert.java:94)
at org.testng.Assert.assertNotNull(Assert.java:404)
at com.dms.testsuite.Test.submitJob(Test.java:110)

[name=null](com.dms.testsuite.Test)  Time elapsed: 51.735 sec  <<< FAILURE!
java.lang.AssertionError: Mail does not exist expected object to not be null
at org.testng.Assert.fail(Assert.java:94)
at org.testng.Assert.assertNotNull(Assert.java:404)
at com.dms.testsuite.Test.submitJob(Test.java:110)

[name=null](com.dms.testsuite.Test)  Time elapsed: 67.177 sec  <<< FAILURE!
java.lang.AssertionError: Mail does not exist expected object to not be null
at org.testng.Assert.fail(Assert.java:94)
at org.testng.Assert.assertNotNull(Assert.java:404)
at com.dms.testsuite.Test.submitJob(Test.java:110)

and the results are reported as follows:

Failed tests: 
com.dms.testsuite.Test.[name=null](com.dms.testsuite.Test)
  Run 1: Test.submitJob:110 Mail does not exist expected object to not be null
  Run 2: PASS
  Run 3: Test.submitJob:110 Mail does not exist expected object to not be null
  Run 4: PASS
  Run 5: Test.submitJob:110 Mail does not exist expected object to not be null
  Run 6: PASS
  Run 7: Test.submitJob:110 Mail does not exist expected object to not be null
  Run 8: Test.submitJob:110 Mail does not exist expected object to not be null
1

Because testng doesnt really expect you to be looking at the results while its running. If you want to say debug some test just temporarily, you can print to System.out

or you can write a test listener to get access to testng metadata of events

import org.testng.IExecutionListener;

public class ExecutionListener1 implements IExecutionListener {
    private long startTime;



@Override
    public void onExecutionStart() {
        startTime = System.currentTimeMillis();
        System.out.println("TestNG is going to start");     
    }

    @Override
    public void onExecutionFinish() {
        System.out.println("TestNG has finished, took around " + (System.currentTimeMillis() - startTime) + "ms");
    }
}

then specifying the listener in your test class

@Listeners({ com.example.ExecutionListener1.class})
public class MyTest {
...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.