I am in a small Agile development team with no automated testing. We use Scrumwise to keep up with development. The above answers address the question with automated tests and hardening testing at the end of the project. I got dinged in regression testing after each sprint.
Should I do regression testing at the end of each sprint?
I assume that you have a suite of unit tests the developers are running which is essentially your first line regression suite.
The purpose of a functional regression test suite is to help ensure that code churn didn't destabilize existing functionality, especially across dependencies. In general, the functional regression test suite should be ran every build (e.g., daily if you are getting daily builds). Having an automated regression suite helps achieve that goal.
In your current situation where you are running manual tests I would recommend that you still run them after each build. But, you may have to limit the size of your regression suite to critical tests, then execute lower priority/critical tests as time permits. You can also limit your suite by running tests in functional areas that had code churn and any dependent areas as well. This is not ideal, but it may expose you to less risk then running your regression tests once per sprint.
Regression testing should ideally happen on every single code commit (and if you've got a good build pipeline, this means doing testing on every single build). This ensures that if a bug has been introduced in the latest commit, that it found as quickly as possible. If you only have to go back one commit to fix a problem, that's super easy to fix and troubleshoot.
It seems you already know the answer. We don't have automation and I can't manually test every build but when I test at the end I find problems too late (you should test them all the time).
You need to develop automation!
Notice I didn't say specifically test automation. Automation is really the key to delivering quickly. Adding pieces to your automation pipeline will increase speed and reduce the feedback loop. Start with small stories and build your way up. Break it up into small chucks, don't try to solve everything at once. Automate a build, add unit testing to the build process, automate deployment, create automated smoke tests, integrate them to deployments. Automate regression tests and run them as often as you can, expand, expand, expand.
What is your confidence level that regression issues won't be introduced ? What is the risk of releasing with possible regression issues ? Why is there no automated testing (checking) ? How do the devs know their code works, what testing are they doing ? Why do you think you are Agile if your code is not backed up with automation ? If you can't move forward with confidence it doesn't sound very agile to me...
My recommendation is a core set of automated regression testing should be part of the build process, if you don't have a CI pipeline. Or at least it is executed on regular basis based on the build cycle and the cost of execution and result investigation. This core set of regression test should be executed and maintained through the whole development cycles.
Noticed that this is a old question. It would be helpful if the owner could share the approaches taken and how it went.
Automation would definitely help. It will help in reducing the time for regression testing. But if you have Sprints running in tight schedules, you won't have much time for automating the test scripts. In such a case:
a) Define your testing acceptance criteria at the start of the Sprint.
b) Run a round of system testing on the application / new feature when released for QA.
c) Focus on testing the main functionality and detect critical bugs as early as possible.
d) Your best bet to start regression would be at the start of the second round of testing, i.e. when the defect fixes start rolling in. This will help to ensure that existing functionality is not impacted by the scope change / new feature.
e) If your regression test suite is large, you might need to prioritize the tests to be run. You definitely don't need to run the full gamut of tests. Test only those areas which are directly impacted by the scope change.
In the Long run, Automation is definitely the best bet. If not Supported by the client, take some time off and try automating the app. Even if it is not supported by the client (budget issues etc.), explain to your PM about the value add for automation.
When working with Agile, you should be continually testing throughout the sprint. If you have been involved during the planning stages (which you should be) you will have a good idea of what is coming up, how long each task will last and when you are likely to be able to start testing.
After each story is completed, you should have the developer provide a quick handover and allow you to start testing straight away. This will not only spread your workload out, but more importantly allow you to identify any problems early on in the sprint rather than right at the end.
It is still a good idea to perform a manual regression test at the end of the sprint (regardless of how many automated tests you have), but as by then you should have hopefully already covered all the new functionality, this can just be a quick sweep of any critical areas of the software / website that is being tested.
