1

I have a serial E2E test suite that's growing and getting slower.

To solve this issue, I have to run the tests in parallel. But creating a user and all the required configurations is very complicated in this project.

Having one user per test is also not ideal, as some pieces of the configuration cost money.

So I was thinking of using something similar to Jenkins Lockable Resources or, where each user is a resource that can be "locked" per test and "unlocked" at the end.

Is there a simple API for achieving this? (Or another solution)

1 Answer 1

0

The answer from @IAmMilinPatel is a great start, but it needs some improvements to be really fully distributed, be able to lock multiple resources and be flexible enough.

Multiple resources refers to tests requiring more than one resources where resource can be a physical thing or a feature of the system under test, some examples could be Android AND Android Version > 12 AND Pixel 7 phone, Chrome Browser AND Linux Red Hat or Database Was Recently Updated AND Software Version N Installed AND User of of type A.

An extension can be having a lockable resource of a certain type. To fulfil "I want a user that can add files to the system" you can lock one existing user, but also assign one from a list or create one on the fly and then lock it.

Why is that important? Efficiency. You don't want test to wait long for a resource to be randomly released, or even worst test can be deadlocked waiting for multiple resources- two tests waiting for resources A & B, test 1 holds A and test 2 holds B- they will never run.

The idea, shown here from King's tech blog and implemented in somewhat similar way in my team using The Actor Model, is:

  • A job queue manager holds and dispatches tests according to available resource and an algorithm about allocating them. The queue manager should have an API to add tests/jobs and report the results synchronously/asynchronously.

  • A resource manager communicate with the resources' device manager and can mark them as available, in use or "broken".

  • A device manager. This can be a computer connected via USB to phones (the phone is the SUT), a computer running VMs (the VM is the SUT) or the device itself running a small manager software (the typical use case are Apple PCs that don't have good VM solution). The device manager stores and follows the device's state and configurations and communicates them to the resource manager upon request, this is needed so the resource manager don't have to know the device's internals.

  • Some resource might have a special kind of manager, a good example are test users. The use manager can create them on the fly and remove them when needed, it can also clean up or load data into the user's data.

  • A logical-test runner. This is another special resource to be allocated, in some cases the logic of the test needs to run on an agent other than the SUT and trigger action on the device. For example when you test a mobile device or test using multiple devices, think of an end to end video call. Again, the runners can be of different types and have various features.

  • Finally you need to set up a communication methods and protocols between the parts of the system

To answer the original question, I am not aware of any such system ready "out of the box" and it is certainly not easy to develop and maintain.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.