How can I avoid conflicts running Selenium tests in parallel, when they must exercise an underlying REST API?

Question

I have a web application which needs to be tested in multiple browsers in multiple environments (i.e. Chrome, Firefox, and Internet Explorer in both Windows and Linux* (*with the obvious exception of Internet Explorer)).

Tests have been written in Java using JBehave, Selenium, and SerenityBDD (Thucydides)). These tests exercise an underlying REST API, testing if objects may be successfully created and deleted using the UI.

I am using Selenium Grid, and would like to run the tests on parallel nodes; however, the concern is that as the tests exercise an underlying REST API, there could be conflicts.

Is it possible to pass in parameters to the tests as a parameter within the Jenkins job configuration which runs the tests, so that there is a slight difference in the tests dependent on the node in which they are executing? (e.g. An object named 'MYOBJECT-CHROME' is created on Chrome, versus an object named 'MYOBJECT-FIREFOX' on Firefox, meaning any REST API conflicts can be avoided?)

(This seems similar to this question, however it addresses running different tests in parallel so they may be executed faster; rather than running the same tests in parallel on different environements; due to these differences the answers do not seem to apply.)

Answer 1

An alternative is to use a universally unique identifier (UUID) that enables unique identification of information in distributed systems without significant central coordination (so no need to pass parameters to a Jenkins job). In this case each test would take care to create IDs itself:

"MYOBJECT" + UUID.randomUUID()

This could solve your problem, unless you want identifiers of your objects to contain explicit information about a node where the test creating an object has run, I would add hostname:

"MYOBJECT" + hostname + UUID.randomUUID()

hostname can be read as described here.

Answer 2

The question seems to imply that each browser/environment combination is a separate Jenkins job.

Jenkins jobs can have parameters. They are manifested as environment variables. How you propagate those environment variables into your test depends on how you run your test; you will need to read your Jenkins. See the Jenkins documentation for details.

An alternative might be to code the test to generate unique object identifiers. You could write a function that takes the browser type and environment as input, and uses that information plus the current timestamp to generate a unique ID. How you implement that function depends on the syntax of your object IDs.

Answer 3

Given the main title one option is to not use Jenkins and use CircleCI, another CI server.
I used to Jenkins myself and switched recently to CircleCI (via a job move). In Jenkins we had several issues (can't remember details) that prevented us from running in Parallel but in CircleCI it is easier and is working well. We use 4 slots so our 24 minute suite runs in about 6 or 7 minutes which is very nice.

Answer 4

I think the true, but difficult, answer here is that the test system needs a different approach.

This issue can be addressed by making sure that all unit tests mock and stub the database where the objects are stored. This means all data, even reference data is created for each single test.

This is a radically different approach to previous practices but I have seen it implemented as a fairly standard approach with my last 3 employers.

The other big effect from this is speed. Instead of tests taking many seconds, many tests can run per second.

Answer 5

If you are passing browser as a Jenkins/maven parameter, you may use currentTimeStamp and browser name in your java code and append it to the object name which you wish to create or you may use selenium session ids as your unique constraint.

Answer 6

Jenkins parameters can be set as environment variables, you can write a wrapper which based on the environment variables sets up the testing environment. Refer to How to set environment variables in Jenkins? for setting environment variables in Jenkins.

In the Java source code you can read these environment variables something like below

import java.util.Map;

public class EnvMap {
    public static void main (String[] args) {
        Map<String, String> env = System.getenv();
        for (String envName : env.keySet()) {
            System.out.format("%s=%s%n",
                              envName,
                              env.get(envName));
        }
    }
}

Based on the environment variable setup the selenium desirable capabilities like below:

desiredCapabilities.setCapability("browser", browserName);
        desiredCapabilities.setCapability("browser_version", browserVersion);
        desiredCapabilities.setCapability("os", operatingSystem);
        desiredCapabilities.setCapability("os_version", OSVersion); 
        desiredCapabilities.setCapability("resolution", screenResolution);