Imagine you are responsible for testing a web application. Your team is preparing a release of a new feature. Just before the release you spot a bug.
What next steps would you take next to resolve the situation ?
Please take into consideration different scenarios and circumstances.
Be open.
Assess the situation as best you can but be sure to pull in the rest of the team and get their input, advice and opinions. Err on the side of over-communicating and most of all don't let fear stop you from being honest. Using a good communication form - slack, teams, etc. may be better than long emails chains
Determine that, going forward, you wish to change things and not be in this situation again, as far as is possible:
Learn from the incident.
Suggest a post mortem and develop any procedures or checklists that you believe are needed.
Mistakes are common.
The error comes when you don't communicate about them or help assess their severity.
Determine severity.
Use knowledge of the product, domain, users and business to help determine potential severity so that you can... yup, commuicate that to others.
Talk about effect in business terms.
Not so much 'the slider doesn't work' but more 'all IE users will be unable to place orders (averaging $12,000 per hour...)
Use or establish procedures for release bugs.
Use an agreed list of people to notify, steps to take, severity levels and what to do for each of them, etc.
Practice
Practice dealing with release bugs ahead of time with play acting so you know the procedures.
Release more often
Release early and often and you will be forced to constantly improve to quickly fix problems.
Measure user interactions.
Measure engagement, web site visits, customer/order conversions etc. so that even if you miss or incorrectly categorize a bug you will still know about the effects from the data about your users.
It is clearly a hypothetical question and answer differ based on scenarios here:
Scenario1:
Defect Spotted: Severity 2 (vulnerability defect in a password field by performing SQL injection)
Days before release: found 3 Days before release in 50 days cycle
Protocol: I will reach to application owners, BA,Product Owners to be alerted about delays caused in fixing this defect and retesting it or postpone the release.
In short, I would tell them to take it serious and take decision accordingly (since it affects your customers):
Scenario 2:
Defect Spotted: Severity 5 (spelling mistake as an example)
Days before release: found 1 Day before release in 50 days cycle
Protocol: I will reach to application owners, BA,Product Owners to approve to go ahead with release if this defect is of less severity.
Steps:
Post Scenario: We have to fix one important issue here. It is the delay caused in life cycle approach while deploying,developing and testing.
If it is an accessibility bug (as an example), the current state can be,
After improving life cycle,it has to be:
Benefits: Lot of days cut down and cost reduction while fixing such last minute defects.
Source:
Link where the images copied for this illustration: https://books.google.co.uk/books?id=pTCPDwAAQBAJ&printsec=frontcover&source=gbs_atb#v=onepage&q&f=false
Scenario 3:
Defect Spotted: Severity 2 (example: Accessibility bug that user got indefinite loop of keyboard strokes when entering password and they have to kill browser instead navigating out)
Days before release: found 1 Day AFTER release in 50 days cycle (found by customer)
Protocol: I will reach to application owners, BA,Product Owners to prioritise fixing the bug as soon as they can
How to avoid it in future?
Image: User accessing the application part of beta testing and giving feedback.Source of Usability Testing (including accessibility/usability reporting): https://www.istqb.org/certification-path-root/usability-testing.html
https://engineers-hub.teachable.com/p/istqb-usability-testing-certification-training
https://uxplanet.org/why-is-it-important-to-do-usability-testing-5080a5640df3
How popular companies such as Google,Netflix dealing with Scenario 1,2,3:
DevSecOps model
Reference: It is a lengthy read of a novel type book. But it gives the scenarios I provided to you with clear industry examples from a story.
Sometimes condition arises where we discover a bug just before the release.The 1st job of a QA in this situation is to report that issue and update the dev team & the product owners.
The severity of the encountered issue plays a major role in adjusting the release timelines.
Further, software testing services usually perform a post mortem to find the root cause of the situation and develop/implement new processes to avoid these incidents again and execute efficient release cycles for future releases.
That would depend on if the bug is a showstopper. If so, stop the release. If not? Provide a workaround and create a new story to fix the bug during the next possible Sprint.
Discuss & let the business assess the priority & severity of it and go with the business decision at the release.
Post release, perform an RCA of that issue(s) to prevent it happening in future again.
Check the severity of the bug. If it is a very critical bug that will affect the reliability of the application, report the issue to the development team, call the attention of the business owner, and advise them against deploying the application.
You'll want to point out the implication and the severity of the bug on the application. You can perform an RCA (root cause analysis). You can ask for more time for the bug to be fixed and tested. Also, run a regression test until there are no regression errors in the application.
