Sign up ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I am in a small Agile development team with no automated testing. We use Scrumwise to keep up with development. The above answers address the question with automated tests and hardening testing at the end of the project. I got dinged in regression testing after each sprint.

Should I do regression testing at the end of each sprint?

share|improve this question
Can you describe what you mean by regression testing? Also, what do you mean by, "I got dinged in regression"? – user246 Sep 3 '12 at 1:15

7 Answers 7

What is your confidence level that regression issues won't be introduced ? What is the risk of releasing with possible regression issues ? Why is there no automated testing (checking) ? How do the devs know their code works, what testing are they doing ? Why do you think you are Agile if your code is not backed up with automation ? If you can't move forward with confidence it doesn't sound very agile to me...

share|improve this answer
The Agile Manifesto says, "Individuals and interactions over processes and tools." That tells me it is possible to be agile without using automation. – user246 Sep 4 '12 at 13:41
But what is your definition of 'agile' ? Can you without automation quickly make changes and react with confidence ? The original poster is claiming he is working in an Agile environment, I'm trying to establish what makes them 'agile' – Phil Kirkham Sep 4 '12 at 13:56
I like the Agile Manifesto's definition of Agile. Every organization needs to interpret those principles in light of their own circumstances. I agree that it would help for the poster to provide more details. – user246 Sep 4 '12 at 14:18
In my opinion, this answer should be a comment under the original question as you are requesting additional details to the question and not explicitly answering the question. – David Kaczynski Sep 6 '12 at 12:33
answering these questions will allow the OP to answer his own question – Phil Kirkham Sep 6 '12 at 14:43

Regression testing should ideally happen on every single code commit (and if you've got a good build pipeline, this means doing testing on every single build). This ensures that if a bug has been introduced in the latest commit, that it found as quickly as possible. If you only have to go back one commit to fix a problem, that's super easy to fix and troubleshoot.

share|improve this answer

I assume that you have a suite of unit tests the developers are running which is essentially your first line regression suite.

The purpose of a functional regression test suite is to help ensure that code churn didn't destabilize existing functionality, especially across dependencies. In general, the functional regression test suite should be ran every build (e.g., daily if you are getting daily builds). Having an automated regression suite helps achieve that goal.

In your current situation where you are running manual tests I would recommend that you still run them after each build. But, you may have to limit the size of your regression suite to critical tests, then execute lower priority/critical tests as time permits. You can also limit your suite by running tests in functional areas that had code churn and any dependent areas as well. This is not ideal, but it may expose you to less risk then running your regression tests once per sprint.

share|improve this answer
I'm missing one thing in you otherwise exceleny answer- when will she run a full set of regression tests ? – Rsf Oct 11 '12 at 4:49
Well, we really don't have enough context here. We don't know if they are getting daily builds. One could argue that with daily builds the entire suite could be exercised over cumulative builds, but that could potentially overlook something. We also don't know the cost (time to run) of the regression suite, or if the tests in the regression suite are the appropriate tests or just a bunch of things accumulated into a set over some period of time. Also, I will say "agile with no automated tests" is a big red flag to me that there are probably bigger problems. – Bj Rollison Oct 11 '12 at 16:18

It seems you already know the answer. We don't have automation and I can't manually test every build but when I test at the end I find problems too late (you should test them all the time).

You need to develop automation!

Notice I didn't say specifically test automation. Automation is really the key to delivering quickly. Adding pieces to your automation pipeline will increase speed and reduce the feedback loop. Start with small stories and build your way up. Break it up into small chucks, don't try to solve everything at once. Automate a build, add unit testing to the build process, automate deployment, create automated smoke tests, integrate them to deployments. Automate regression tests and run them as often as you can, expand, expand, expand.

share|improve this answer

My recommendation is a core set of automated regression testing should be part of the build process, if you don't have a CI pipeline. Or at least it is executed on regular basis based on the build cycle and the cost of execution and result investigation. This core set of regression test should be executed and maintained through the whole development cycles.

Noticed that this is a old question. It would be helpful if the owner could share the approaches taken and how it went.

share|improve this answer

Automation would definitely help. It will help in reducing the time for regression testing. But if you have Sprints running in tight schedules, you won't have much time for automating the test scripts. In such a case:

a) Define your testing acceptance criteria at the start of the Sprint.

b) Run a round of system testing on the application / new feature when released for QA.

c) Focus on testing the main functionality and detect critical bugs as early as possible.

d) Your best bet to start regression would be at the start of the second round of testing, i.e. when the defect fixes start rolling in. This will help to ensure that existing functionality is not impacted by the scope change / new feature.

e) If your regression test suite is large, you might need to prioritize the tests to be run. You definitely don't need to run the full gamut of tests. Test only those areas which are directly impacted by the scope change.

In the Long run, Automation is definitely the best bet. If not Supported by the client, take some time off and try automating the app. Even if it is not supported by the client (budget issues etc.), explain to your PM about the value add for automation.

share|improve this answer

When working with Agile, you should be continually testing throughout the sprint. If you have been involved during the planning stages (which you should be) you will have a good idea of what is coming up, how long each task will last and when you are likely to be able to start testing.

After each story is completed, you should have the developer provide a quick handover and allow you to start testing straight away. This will not only spread your workload out, but more importantly allow you to identify any problems early on in the sprint rather than right at the end.

It is still a good idea to perform a manual regression test at the end of the sprint (regardless of how many automated tests you have), but as by then you should have hopefully already covered all the new functionality, this can just be a quick sweep of any critical areas of the software / website that is being tested.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.