JUnit ParallelComputer Runs All Test Classes As One

Question

I'm building automation testing for web application by using JUnit 4.9 and Selenium Webdriver. It's time consuming. So I'm trying to experiment with ParallelComputer.

I'm wondering how can I get result of each test. In my current code below, it will run as one test and will pass even though there's an assertion error in one of the test classes. I'm new to the framework and parallel. Thanks!

public class ParallelComputerTest 
{

    @Test
    public void test() 
    {
        Class[] cls={19 test classes in here};

        //Parallel among classes
        JUnitCore.runClasses(ParallelComputer.classes(), cls);
    }

}

Answer 1

Inspired by JUnit's experimental ParallelComputer runner I've built my own ParallelSuite and ParallelParameterized runners. Using these runners one can easily parallelize test suites and parameterized tests.

ParallelSuite.java

public class ParallelSuite extends Suite {
    public ParallelSuite(Class<?> klass, RunnerBuilder builder) throws InitializationError {

        super(klass, builder);

        setScheduler(new RunnerScheduler() {

            private final ExecutorService service = Executors.newFixedThreadPool(4);

            public void schedule(Runnable childStatement) {
                service.submit(childStatement);
            }

            public void finished() {
                try {
                    service.shutdown();
                    service.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
                } catch (InterruptedException e) {
                    e.printStackTrace(System.err);
                }
            }
        });
    }
}

ParallelParameterized.java

public class ParallelParameterized extends Parameterized {

    public ParallelParameterized(Class<?> arg0) throws Throwable {

        super(arg0);

        setScheduler(new RunnerScheduler() {

            private final ExecutorService service = Executors.newFixedThreadPool(8);

            public void schedule(Runnable childStatement) {
                service.submit(childStatement);
            }

            public void finished() {
                try {
                    service.shutdown();
                    service.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
                } catch (InterruptedException e) {
                    e.printStackTrace(System.err);
                }
            }
        });
    }
}

Usage is simple. Just change @RunWith annotations value to one of these Parallel* classes.

@RunWith(ParallelSuite.class)
@SuiteClasses({ATest.class, BTest.class, CTest.class})
public class ABCSuite {}

Answer 2

The test is marked as passed because you're are invoking JUnitCore from within a test class. As soon as JUnitCore.runClasses has completed execution, the current test, ie 'test' in 'ParallelComputerTest', is finished succesfully since no AssertionError were thrown during that test, and that was a single test as far as your IDE's test runner is concerned.

Now, you can achieve your requirement like so:

public class TestRunner{

  public static void main(String[] args){
    Class[] cls = { TestCase1.class, TestCase2.class, ..[as many test classes as you want] };
    Result result = JUnitCore.runClasses(ParallelComputer.classes(), cls);
    List<Failure> failures = result.getFailures();     
  }   
}

and then invoke the test like so :

TestRunner.main(args);

The list failures will have a bunch of Failure objects which you can parse and log. The Result object will have the overall test details, viz. execution time, tests started, ignore, finished and failed.

Please note that ParallelComputer also has other methods apart from classes() which will allow you to have a somewhat higher degree of control over what is executed in parallel.