- I wonder how can we apply the defect arrival rate to discover the progress in our testing projects from developers and testers side ?
- Is there a risk by applying such approach ?
Add a comment
|
1 Answer
There aren't any black and white rules in testing, everything is project and product dependent.
https://www.istqb.org/images/Articles/kale_The%20key%20principle%20of%20testing.pdf
"Testing is context dependent"
The decision to use metrics in a project should consider many factors, a few would be:
- How critical is the product
- Do the organization and product follow a zero bug policy
- Can we release the product with known bugs
- Team maturity
- Release pace (How fast a product needs to be released)
Defect arrival rate:
Defect arrival date shows how many new defects were identified during the specified unit of time. Example, the number of new bugs per sprint, or per month etc.
In most companies more than defect arrival rate, number of open bugs is considered for making release decision. Meaning how many bugs under different severity and priority are there still there to be fixed.
If company has a zero high bugs and ignore low bug policy, then the product will be released only after all high bugs are fixed.
Advantages of Defect arrival curve:
- It helps in deciding whether development sprint is shorter than required, meaning do we need to have a longer sprint for ensuring quality development.
- Do we bring more tasks or user stories to the current sprint than we could deliver, this might be affecting the quality of sprint.
- Do we need split user stories again to smaller user stories, and need to have a more efficient effort estimation process.
- Do we need to restructure our test pyramid and ensure testing is pushed to the left of the pipeline. Meaning more unit, integration and API testing.
- Having identified more defects (even minor) in the last sprint before release could be an indicator that there may be defect masking in the system. Meaning minor bugs may be masking more critical ones, do we need to spend more time testing it rigorously.
- Is there more defect spillage from unit test > integration > backend > Front end. This indicates we need to improve our testing approaches in the left of the pipeline.
Tips:
- All KPIs should be used for process change and release decision making, it should not be taken as a way to validate the efficiency of developer or tester.
- Open bugs are sometimes more efficient than defect arrival curve in agile teams but can use it to improve the agile process
This post in Linkedin by @Avinash Magdum looks interesting and could give more metrics relating to defect:
https://www.linkedin.com/pulse/software-quality-through-defect-metrics-avinash-magdum/
-
2
-
@PDhide this is really interesting ! Do you belive it's a good metric to use ? I find that although all those advantages it has lots of negative effects I see that may be testers will try to have big number of bugs at the start of the sprint so that the risk of trivial and non coherent bugs will increase and also developers will be stressed if important defects are found at the end. What do you think about possible risks with this metric ? Commented May 30, 2020 at 9:32
-
1@EmnaAyadi this approach makes sense as the project gains maturity , for a project that is being build from scratch and just been few Sprints would should high bugs initially . But during mid project cycle this can be highly effective to know whether we on the right track.– PDHideCommented May 30, 2020 at 11:27
-
1And develoment team feeling stressed , is when we work as different entities . When the management sees everyone has a team , Importance is on fixing the bug than blaming the dev team . This I already mentioned in the comment , like more Sprint length , more unit tests , it's all chance to improve the process .– PDHideCommented May 30, 2020 at 11:29
-