How to generate test cases for JUnit?

Question

Explanation

We are currently using a self-made testing framework/tool, to test services of an client-server application. The framework currently works with own implemented JUnit like assert methods and reports.

The frameworks works like this:

• Test cases are defined as CSV files
• Each line of a CSV file contains a test step
• Each test step has a unique name and parameters

• Simplified Example:

  Create Product;name;price;count
  ;PlayStation 3;300;10
  Validate Product;name;price;count
  ;PlayStation 3;300;10

• Each test step is defined as a Java class

The framework parse the CSV file, find the test steps, create a test case to run it and create a report.

What I want

I want to execute the test cases and report the results by native JUnit.

What is the question?

How can I generate JUnit compatible test cases by CSV files and test steps?

Useful information

Because of reasons, we are not able to switch to a completely new framework like Cucumber or anything similar. The goal is to have the JUnit assert methods, because they are more reliable than self-made and to have the JUnit reports, that tools like Bamboo, Jenkins and other are able to get accurate reports in the compatible JUnit format.

I am currently try different things to implement JUnit in the framework, but haven't found a solution yet. I hope I am able to solve this issue with your support.

If you know any other framework or library to solve this issue, please also let me know.

If you have any questions to understand the question or context better, let me know.

Thanks for your support.

---

Update

I tried to create a custom JUnit runner. The issue is that a JUnit Runner expects a test class and in this case a test class doesn't exists. I looked into projects like Cucumber to see how they implemented the JUnit Runner but wasn't able to figure out how it's working yet.

Answer 1

In your position I would look at creating a single JUnit test that loops through the csv rows and behaves accordingly based on the parameters given in each row.

I assume you haven't done it like this for reportability reasons? You want to be able to get feedback on each row or something like that?

It is possible to hack at the TestRunner to do something like that for you. But it would be a lot less painful for you to run it in a single test, then analyse the results in some lovely format at the end (you could look into TAP results for example)

This is what a hacky implementation would look like

@Test
public void shouldCsvItUp() throws Exception
{
    YourCsvTestRunner csvTestRunner = new YourCsvTestRunner("my-csv-file.csv");

    MyTest test;
    // create empty array of test results
    List<MyTestResult> errorsResults = new ArrayList<>();


    // have your csv test runner parse the csv and parse it into a java object that contains a runnable
    while((test = csvTestRunner.getNextTest()) != null)
    {
        try
        {
            // run the test runnable
            test.run();
        }
        catch(Throwable e)
        {
            // if the test failed for whatever reason (including an assertion error) add that to the errors array to
            errorsResults.add(MyTestResult.errorResult(test, e));
        }
    }

    // assert no errors, if there are, then display all the errors in the csv test run
    assertTrue(Arrays.toString(errorsResults.toArray()), errorsResults.isEmpty());

}

If you insist on hacking the test runner I would look into extending BlockJUnit4ClassRunner, and in your childrenInvoker it has an argument called 'Run notifier'. The methods in that object are what JUnit listens to as results.

e.g. fireTestStarted, fireTestFinished, fireTestIgnored

Answer 2

You can use the RunWith and Parameterized annotations to run a test suite that is backed by a spreadsheet, database result set or delimited file.

import java.io.IOException;

import javax.xml.parsers.ParserConfigurationException;

import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;
import org.w3c.dom.Document;

@RunWith(Parameterized.class)
public class TableTest
{
    @Parameters
    public static Iterable<String[]> parameters() throws IOException
    {
        // use some library to parse the CSV into a table or list
        return new TableParser().parse(TableTest.class.getResourceAsStream("sheet3.csv"));
    }

    private String testCaseId;
    private Document document;

    public InferenceTestProcess(String[] fields)
    {
        super();
        this.testCaseId = record.getField(0);
        this.document = XmlAssertionGenerator.toXml(fields); // I convert to an XML document, but you could use some custom class or just preserve the original array of strings
        return;
    }

    @Test
    public void test() throws Exception
    {
        // You could use an external class or keep the test logic here.
        String result = runTestCase(this.testCaseId, this.xml);
        Assert.assertEquals("SUCCESS", result);
    }

    private static String runTestCase(String testCaseId, Document document)
    {
        return "SUCCESS"; // test goes here
    }
}