2

We do end-end Regression testing of the web portal before any Production release cycle but focus of our QA team is mainly on high-level functionality verification of major modules/area of the product/website.

Due to this sometimes QA miss bugs which then land in Production. How much detailed testing should be done in Regression testing before any major Release of a product/website etc.

Is High-level verification approach good enough or QA should always check each and every little details like every field/element on the web form, say for example and test every possible scenario?

Then wouldn't that be functional testing?

1
  • This is a good question, but I'm not sure if it's answerable - like, how long is a piece of string? It really depends on your product, the release cycle time, e2e automation, unit test coverage, team size, capabilities, etc... it depends.
    – dvniel
    Jan 31 at 11:25

3 Answers 3

1

To answer this, we need to normalize on a few terms and concepts:

  • Verification is confirming that the requirements, as they are understood by the development team, have been satisfied. Verification includes all types of testing - regression, functional, performance, security, and more.
  • Regression testing is testing performed on unmodified parts of the system to determine if changes could lead to failures in these unmodified parts of the system.
  • Functional testing is testing that focuses exclusively on inputs, outputs, and conditions. That is, it asserts that in a given set of conditions and a specified input, the correct outputs are produced by a system.
  • Other types of testing may focus on specific quality attributes of the system, such as security, performance, accessibility, recoverability, etc.

Regression testing is not something that you do prior to verification. Regression testing is a part of verification. It refers specifically to test cases that are not directly associated with a change but are still executed to reduce the risks associated with the release. I don't believe that it makes sense to separate "regression testing" from "verification".

The question about how much regression testing to do depends on effort and risk. In a perfect world, you will have fast, automated test cases for regression, and the vast majority of your test cases will be automated. However, this perfect world scenario doesn't always exist. There are plenty of times where test cases aren't automated, for one reason or another. In these cases, the amount of human effort dedicated to regression testing can be risk-based.

Unfortunately, I don't think anyone can assess the risks. There are technical aspects. For example, if you made a change to Component Y and you know that Component Y is used in Module A and Module B, but not in Module C, the risk of an impact to Module A or Module B is much higher than the risk of an impact to Module C, so you may want to prioritize more regression testing in Module A and B over C. However, this may not get your test execution time down to an acceptable level, either. Module B may be more important - perhaps it's associated with a critical functionality or used by more customers or users - which would indicate that regression testing of Module B would take priority over Module A.

At the end of the day, you need to decide where to invest your time. My recommendation would be to invest the time in automated testing. Having fast feedback is key. This varies by context, but having unit, integration, and system tests that can be run frequently and quickly can find and stop issues before a release much earlier in the development cycle than a late testing phase.

0

System size, complexity and time allocated to complete a regression cycle manually are always the only ingredients for a successful test run.

I would never expect a manual regression run to ever pick up every issue. Manual tests tend to stay static in my opinion, especially with a mature system. People tend to leave the task of updating as a "ill do it soon" type of action, because new tasks take their attention. As such, it creates gaps. And testers tend to test what they have always tested in manual runs.

I have found the best approach is to automate your functional tests and the core tests that fall under your idea of "regression tests". This way, the deviations and granularity are tested always, as well as the main areas having coverage. Then your manual run can consist of more focused testing in the areas relating to that release, allowing less scope for test-creep. Plus, going automation first allows you to run your tests nightly if possible in a CI, getting bugs identified each day, narrowing down the cause to be code added between day x and y

0

The extent of functionality verified during regression testing before a release cycle varies depending on factors such as the size and complexity of the application, the changes introduced since the last release, and the criticality of the features involved. However, here are some general principles that QA teams often follow:

Core Functionality: The most critical and frequently used features of the application should always be included in regression testing. These are the functionalities that users rely on the most, and any regression in these areas could have a significant impact on the user experience.

Recently Modified Features: Regression testing should prioritize features that have been recently modified or enhanced. Changes in these areas are more likely to introduce regressions, so they require thorough testing to ensure that existing functionality has not been affected.

Integration Points: If the application interacts with external systems or services, regression testing should include verification of these integration points. Changes in external dependencies can impact the application's behavior, so it's essential to validate these interactions.

Critical Paths: Identify critical user workflows or paths through the application and ensure that these are thoroughly tested during regression testing. These paths often touch multiple features and are vital for the application's usability.

Commonly Reported Bugs: If there are known issues or recurring bugs in the application, ensure that regression testing covers these areas to prevent the reintroduction of previously fixed issues.

Boundary Cases and Error Handling: Regression testing should include tests for boundary cases, error handling, and edge conditions to ensure that the application behaves correctly under various scenarios.

Compatibility and Cross-Browser Testing: If the application supports multiple platforms or browsers, regression testing should include verification of compatibility across these environments.

Performance and Scalability: Depending on the application's requirements, regression testing may also include performance and scalability testing to ensure that the application can handle expected loads without degradation.

QA teams need to prioritize their regression testing efforts based on risk, impact, and available resources. Test automation can also help streamline regression testing by allowing teams to efficiently execute a comprehensive set of tests across multiple releases. Ultimately, the goal of regression testing is to provide confidence that the application's existing functionality has not been compromised by recent changes or updates.

New contributor
Abhishek Kalotra is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.