Running tests in parallel using Jenkins Grid plugin

Question

I have Jenkins 1.499 running Watir-Webdriver tests on a Windows 2008 (64bit) server alongside the Grid 2 standalone server (and Jenkins Selenium Grid plugin).

I can run my tests through the standalone server (or plugin) via Jenkins, but they don't run in parallel. What do I need to do to get them running in parallel?

I've tried out the parallel_tests gem, but fork() doesn't seem to be implemented very well for a Windows environment. I think I might need to use TestNG to manage the parallelism, but I'm not really sure and could use some advice if anyone out there knows?

Answer 1

If you are using Ruby, I believe you should be able to use green threads to avoid fork()

http://www.ruby-doc.org/docs/ProgrammingRuby/html/tut_threads.html

While technically not true parallelism, it should work fine if you are just firing off Watir-Webdriver tests. The idea is to start each thread up separately and have each thread run a test script.

Answer 2

In TestNG.xml file there is an attribute "parallel" that could be used to configure running several test methods/classes at the same time in parallel

<suite name="mySuite" parallel="methods" thread-count="5">
<suite name="mySuite" parallel="tests" thread-count="5">
<suite name="mysuite" parallel="classes" thread-count="5">
<suite name="mySuite" parallel="instances" thread-count="5">

The article http://technologyandleadership.com/testng-running-tests-in-parallel/ gives an indepth description of all these values.

Answer 3

You can fork test tasks from your build tool, whatever it is. With Maven, its called "surefire plugin", with Gradle, you just set "maxParallelTasks" in your test task, with TestNG, you use the thread count like @Aruna suggested. It all depends on what you are using to execute your tests. When selenium tests are forked, then multiple sessions are opened on the grid hub and it will open multiple browser sessions on the Grid node that it is mapped to.

Answer 4

Just for anyone who stumbles across this question.

The solution I have integrated into my environment is the following, and its quite easy.

I have used the Build Flow plugin to create a Build Flow job.

I have created 3 Maven style jobs, each job pointing to a different XML file which contain different test classes.

In the Build Flow job I was able to add in the following code to make the 3 Maven jobs run parallel.

parallel (
// job 1, 2 and 3 will be scheduled in parallel.
{ build("job1") },
{ build("job2") },
{ build("job3") }
)

What this basically does is when you run the Build Flow job it will then initiate the job1, job2 and job3 to run and run parallel.

One problem I encountered with this was it would only run 2 jobs parallel and for the 3rd to run it would have to wait for one of the others to finish.

I found out that my Jenkins had only 2 executors specified ( I dunno if this is the started specification in Jenkins).

This basically meant that I could only run 2 jobs at a time as there was only 2 job executors.

To solve this go to Manage Jenkins > Configure System > # of executors

Here you can specify the number of executors. I changed mine to 5 so if I wanted I could have up to 5 jobs running parallel.

That is it, its quite easy to implement. What I found useful also about this method is that it gives you more control over everything that is going on since its a hierarchical structure. It enables you to have control over each stage of the test process. And also enables you to isolate problems for trouble shooting.