Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It only takes a minute to sign up.
Sign up to join this community
We have an application (Product) that, when it's deployed in UAT for any User/Client, a completely new server (Infrastructure) / Domain is set up along with the database, but the code base is the same as for the earlier User/Client.
If we run a regression suite for User/Client A, then do we also need to run the regression suite again when User/Client B is onboarded?
You should perform a risk assessment to decide if it's necessary. If the environment - the hardware, the versions of all dependencies, the configuration of all tools - is identical, then you may find that the risk is low and regression testing is not necessary. However, if you determine that the hardware is different (perhaps due to obsolescence) or a dependency has changed (perhaps due to security patching or new versions being released and old versions being end-of-life), you may want to assess what has changed and think more carefully about if and what to test.
I would suggest at least a smoke test to verify that the system, all its dependencies, and all configurations are correct. A smoke test would be a small subset of the whole set of regression tests selected based on their ability to confirm the installation and configuration of key components of the system before handing it over for end users to perform UAT activities.
It depends on how different and separate the setups are.
You may want to regress areas that are dependent on client configurations. After all, you don't want to Acme Amalgamated see branding for Great Group, or vice versa.
If they share common components, like a single database, you may want to regress more deeply to make sure that user permissions don't leak over, or other security issues.
If there's absolutely nothing in common, you may be able to get away with just checking that all the appropriate pieces of each system are talking to their own system and not something else.
Generally you don't need a full regression if you aren't changing code. You definitely want to make sure your configurations are correct and that the systems are truly separate.
It is generally a good practice to run regression tests whenever there are any changes made to the code base or the infrastructure, even if the changes are small. This is because even seemingly small changes can have unintended consequences, and running regression tests can help to ensure that the application is still functioning correctly after the changes are made.
In your case, since the code base is the same for both User/Client A and User/Client B, you may not need to run the entire regression suite again for User/Client B. However, it is still a good idea to at least run a subset of the regression tests to ensure that the application is functioning correctly on the new infrastructure and with the new database.
It is also a good idea to have a process in place for determining which tests should be run in each situation so that you can ensure that you are testing the most important and relevant parts of the application without spending too much time on unnecessary tests.
Regression testing is performed to verify that changes made to the code base or to the system do not negatively impact existing functionality. In the scenario you described, the infrastructure of the application changes with each new client, but the code base remains the same.
Whether or not regression testing needs to be performed in this case depends on the extent of the infrastructure changes. If the changes are minimal and do not affect the functionality of the code, then regression testing may not be necessary. However, if the changes could potentially affect the code's behavior, it would be advisable to run regression tests to ensure that everything is functioning as expected.
It's important to consider the risks and benefits of regression testing in each scenario. On one hand, running regression tests for every client could be time-consuming and add unnecessary overhead. On the other hand, if the tests are not run and a critical issue arises, it could result in significant downtime and loss of credibility.
Ultimately, the decision to run regression tests in this scenario should be based on a risk assessment, taking into account the scope of the infrastructure changes, the criticality of the application, and the impact of any potential failures.
