Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It only takes a minute to sign up.
Sign up to join this community
One of the most frequently asked questions of QA by team leads or other business representatives is:
We're getting close to the planned release date. What do you think - is the software ready? Can we move forward with the release?
As far as we know, no software can be made 100% bug free. What should be the answer given by QA in this case?
The question of "Is this application/feature/fix ready for release?" is one that can't be answered accurately, because - as the OP said - there's no way to be sure that there are no bugs in the software.
More than that, depending on the situation, the business could have reasons for releasing despite the concerns of QA or testers that the software is not ready for production (I've been there).
Our role in this situation is simply this: Give the business the information they need to make an informed decision whether or not to release. That can be in the form "Yes, I think it's good to go - there are no critical bugs and I've tested everything I can think of", it can be a matrix of test case results, or it can be something else entirely.
Ultimately, we give the business a risk assessment from our perspective, ideally looking at these areas:
The format you use to communicate the information isn't relevant: what matters is that it's something everyone involved understands.
The answer on if a product / website / software is ready to ship should come from the entire team and not just the software tester or QA team.
A developer will most likely have unit tests and these will give a good indication on if the software is stable. (The test coverage will of course determine how much weight these have in any final decision)
A developer will also have a gut feeling as to how good / stable the code is. They are the ones who have seen the inner workings of the product and know if there are some possible areas that could cause a problem.
Any client or product owner should also be involved. Have their requirements been met, have they done any uat testing.
What external factors (e.g. an ad campaign) are happening which could make you want to wait until its over before releasing any new update and reduce risk during high traffic periods.
The designer will also have an opinion. They will always be able to spot small but important things which are not right which more than often can be put right very quickly.
Of course a software tester will have a very good feeling as to how stable the software is and confidence will grow with the more time they can spend testing, but their feedback shouldn't be combined with the entire teams thoughts.
That question is the entire reason that the QA team exists. As QA engineers it is our job to test the application and get an idea of its quality and whether or not it is fit for purpose. As such, there are three answers possible:
You should always make it clear to your client and/or team lead what exactly you will be testing, which areas of the application you will be focussing on more heavily, etc. so that they can know what to expect.
Having said that, inevitably there will be issues with the application that you did not catch. When this happens, it is important to analyse the situation: Is this an issue that you should have caught? If yes, is it something that should normally have come out in the tests you did or was there a gap in the testing?
It's in the name: QA is about Quality Assurance. This means that not only should you continuously evaluate the quality of the application, you should also keep evaluating the quality (and quantity) of your own testing to ensure you are still covering what needs to be covered and that the depth of your testing corresponds with what's required in each area.
When the software meets the definition of done it is ready.
So the answer to your question is that all the folks involved need to agree on the definition of done. Formally called 'requirements gathering' before Agile took over the world.
Get together and, for for better or worse agree on how the system should perform in terms of features, usability, performance and bugs.
One example:
In an Agile world these will be quite minimal as the idea is that you will discover what to work on each week with a meeting of minds of developers, qa and product ("3 amigos").
I have been asked this question a countless times during countless releases by nervous and excited project managers. Theoretically, this question does not make sense. If the PM well and truly diligently reads the daily testing status report sent by the QA team (which PM's rarely do), he should be well aware of the 'state' the application is in. But being the project manager(especially in a service based company where he/she is handling multiple projects), he/she is always surrounded by countless issues and rarely gets the time to read reports in full detail. In that case, I kind of forgive the PM and tell my interpretation.
My criteria is based on how close the application meets the test closure criteria at the end stages:
If all the above hold true, then I would say that the app looks good.
