0

In our organization, there are 5 environments

  1. Dev
  2. QA -- For QA
  3. UAT -- Demo and client verification
  4. Stage -- Contains all approved task by the client that needs to be deployed on production 5.production

The issue is that every time we are asked to run regression on QA, UAT, AND STAGE this is causing an overhead. Can someone guide us on whether we need to completely retest or if we can run smoke or sanity tests?

2
  • Welcome to the community. Why do you have the need for stage and pre-production? Oftentimes, these are the same environment. How are they different for your org?
    – Lee Jensen
    Dec 5, 2022 at 22:34
  • edited , its production Dec 5, 2022 at 22:41

5 Answers 5

1

Having multiple code environments is a common practice.

Other aspects that we aren't aware of:

  • How do the environments differ by branching strategy? What is expected to be deployed on each environment? Do you have a roll-back/roll-forward policy?
  • How often do you deploy to production? Are you doing CI/CD? Daily deploys? Weekly/Bi-weekly deploys?
  • Do you have any test automation setup? If so, what type of tests: unit, integration, end-to-end tests?

As you can see with these additional questions, the answer really depends on policy and specifics of infrastructure setup.

Generally in my experience, no, you don't need to retest everything in each environment. You'll test most, if not everything, in the QA environment; this is an environment that contains all the new changes and is always in flux; it's where your day-to-day work occurs as a tester.

In stage/pre-production, here is where you can do testing just for deployment or integration purposes. If you have multiple, independent branches where code/features were tested in isolation, in stage, you integrate them into a deployment package. Now you need to ensure everything is integrated and works together. Some regression testing, some smoke testing, really any testing to ensure you're ready for production deployment, but it shouldn't be everything.

If you have test automation, you'll want to ensure you have that running in your pipeline and ensure you setup quality gates. If you have confidence in your test automation, it could be possible that running only the test automation will be needed on stage.

1
  • 1
    To add a bit to you good answer, a possible setup could be- on DEV test basic functionality, on QA test it again because it is stable now and verify integration of all system parts, on UAT get the business people involved to test and verify that they actually got what they have asked for. Stage usually requires a sanity test to make sure nothing is broken on the way.
    – Rsf
    Dec 6, 2022 at 14:53
0

If you have sufficient control over the software and infrastructure and the infrastructure is comparable in each environment, I don't see a reason you would need to fully retest after the QA environment. In my experience in regulated industries, it's common to perform something like an installation qualification, verification of the integration processes, or a smoke test, but these are a very narrow subset of tests explicitly designed to confirm that all components are installed, have the correct version installed, and are configured properly.

4
  • A good reason to re-test on "higher" environments of big and complex products is that they are more integrated and closer to production, on "lower" environments you can get away with mocking of dependencies, internal and external, while as you go "up" you will try to be as close as possible to production.
    – Rsf
    Dec 6, 2022 at 14:50
  • @Rsf You shouldn't be mocking your dependencies if you're doing formal verification activities. It may be fine for development environments to balance costs or environmental complexity, but if you're in an environment where you need to pass through a gated system test process, that system test process better be done without mocking dependencies. Otherwise, it's not a valid test of the system. Dec 6, 2022 at 16:13
  • you are totally correct but reality doesn't always agree with you, in my big banking system we are dependant on external systems which not always have test environments or have all kinds of limitations.
    – Rsf
    Dec 8, 2022 at 13:59
  • 1
    @Rsf I hope your risk management process addresses those risks and how they are mitigated or if they are accepted. But I have also worked with third-party vendors who don't offer a development or test integration and the only solution is to buy a second license (or set of licenses) for lower environments and sometimes the answer is to document and accept those risks. Dec 8, 2022 at 14:03
0

As a software testing company, the process is to perform Regression Testing only on Staging environment once before code is deployed to Production. As there are multiple environments in your scenario, than we recommend your team the below:

  1. Sanity Testing on Dev, QA and UAT environments.
  2. Regression Testing on Stage environment
  3. Smoke/Sanity Testing on Production environment.

Moreover, in case you have automation in your project than that can also be leveraged for executing Smoke/Sanity test executions.

0

It seems you are looking for help on how to approach testing in your organization's multiple environments. In general, it is best practice to run regression tests in all environments where changes have been made, including QA, UAT, and Stage. This is because each environment may have different configurations, data, and dependencies that can impact the behavior of the system.

That being said, it may be possible to minimize the overhead of testing by implementing a risk-based testing approach. This involves identifying and prioritizing the most critical functionality and testing that thoroughly, while possibly reducing the scope of testing for less critical functionality.

One way to do this is to implement smoke or sanity testing in these environments. Smoke testing is a type of testing that verifies that the most important functionality of a system is working correctly. Sanity testing is a type of testing that is used to verify that a small, specific part of a system is working as expected. Both of these approaches can help to reduce the time and effort required for testing, while still providing some level of assurance that the system is functioning correctly.

It's also worth considering whether it is necessary to test everything in every environment. For example, if certain functionality has already been thoroughly tested in QA and UAT and has not changed in Stage, it may be appropriate to skip re-testing it in Stage and focus on testing only the new or modified functionality.

Ultimately, the approach to testing in multiple environments will depend on the specific requirements and constraints of your organization and the risk tolerance for changes. It may be helpful to work with your team to identify the most effective and efficient approach to testing in your multiple environments.

0

The decision on whether to perform complete regression testing or just smoke or sanity testing depends on several factors, including:

  1. The magnitude of changes: If the changes are major, then it might be necessary to perform full regression testing, while if the changes are minor, then smoke or sanity testing might suffice.
  2. The risk involved: Higher risk situations require more comprehensive testing, while lower risk situations can be tested more briefly.
  3. The impact of a potential failure: If the failure of a feature could have a significant impact on the end-users or the business, then full regression testing should be performed.
  4. Time constraints: Full regression testing is a time-consuming process, so if there are time constraints, then smoke or sanity testing may be a better option.
  5. The cost of testing: The cost of testing is a significant consideration in deciding which type of testing to perform. Full regression testing is more expensive than smoke or sanity testing.

In summary, the decision on which type of testing to perform depends on a thorough risk assessment, cost-benefit analysis, and the urgency of the testing. The objective should be to balance the need for thorough testing with the time and resource constraints of the project.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.