Do I have to run all tests for each user group?

Question

Let's say I want to test a "Change Password" page, and I have 3 user roles: admin, superadmin, and player. I use selenium + pytest, each test can be parameterized to run as many times as there are roles.

Do I have to run every test for each role? I just don't want to wait several hours for the pipelines to pass.

I want to test the change password permissions for each role, and then run negative tests, for example, for the new password length, only for one role. Is it the right approach?

Answer 1

Do I have to run every test for each role?

I think a lot of people will say, "it depends." This isn't wrong per se. But, I'll jump in and say, no, you don't have to test a password change for every type of role.

Many years ago, I worked on a web app that had 15 different roles and even more different permissions that could be attached to 1 role, usually more than 1 role. I forget the actual numbers of roles and permissions...let's just say, it was a lot!

I want to test the change password permissions for each role, and then run negative tests, for example, for the new password length, only for one role. Is it the right approach?

Anyway, regardless of using test automation or not, you don't have to test every role here. It comes back to context.

• Why do you want to test every role? What's different in regards to password changes for each role?
• Do they have different password criteria?
• Do they use a different login page?
• Is the underlying method the same? Can you review the code and see that every role uses the same changePassword method?
• Is the "change password" feature different for each role? Then you test every role since there is a 1-for-1 matchup.
• Is the "change password" feature the same for each role? Then you can test one role. You can even setup your test automation to randomize which role is used to test, that way, you have higher confidence over time that different roles are being utilized.

What you're testing here is functionality, not roles.

The concepts, techniques, and principles that matter in this situation are: Boundary Value Analysis (BVA) and Equivalence Classes. These are classic and timeless concepts and so much as been written about them. If you're not familiar, I'll briefly describe them and encourage you to seek out more info.

Boundary Value Analysis: a test design technique where you create tests based on boundary values. While normally used for input fields, it can be utilized for any boundary. Your boundary here is: using a feature/method for password changes. Like the questions I listed above, is the method used separately for each role type, or is the same method used for each role type?

Equivalence Classes: a subset of values within a "class" that are expected to be treated the same way. For password criteria, all numbers are the same, all letters are the same, all casing is the same, etc. You don't have to test all numbers, all letters, that's too exhaustive. In this case, you have many role types, they create a class of similarities that can be treated the same way.

By using these techniques, you prevent yourself from doing exhaustive testing, which isn't needed most of the time.

Answer 2

You need to decide whether a functionality might be affected with a different user role. If yes, repeating it for each role will make sense.

Typically the password length check implementation is independent of user role. It makes sense to run it for one user role only (suggested to use lowest privileged user).

You will want to repeat the core functionality tests like password change for each role. As you would want to check whether each type of user can change his/her password successfully.

There will be tests that will apply only to the higher privileged user.

Hope this helps.

Answer 3

It depends on the requirements of your project and the specific goals of your test cases. If the goal of the test case is simply to verify that the "Change Password" button can be clicked and that the page loads correctly, then a single test case that covers both user groups may be sufficient.

On the other hand, if the goal of the test case is to specifically test the behavior of the "Change Password" button when accessed from the dashboard or the other pages, then it may be necessary to write two separate test cases to cover both scenarios.

Ultimately, the decision of whether to write one or two test cases will depend on the specific requirements and goals of your project, and the trade-offs between the time and effort required to create and maintain multiple test cases versus the increased coverage and thoroughness that multiple test cases can provide.