Interview question on doing a production release with a bug

Question

There was an interview question as:

There is a production release planned tomorrow. QA found a critical bug. The dev team is working hard to fix it with extra effort but they aren't anywhere close to a solution. The marketing team is asking for the production release.

As the responsible QA person, should I reject the release or should I send the release to the marketing team with the known bug? If you reject the release it will affect your yearly evaluation.

What are the possible ways to handle this kind of situation?

Answer 1

as a QA responsible person should I reject the release or send the release to marketing team with known bug

Neither one. You should let all people involved know that there is such and such bug. You should communicate clearly what the bug means to customers and other people who use the product or are somehow involved (stakeholders). This might be a team effort as well.

Lastly, the person who asked this question told me that if I reject the release it will affect to my yearly evaluation too.

Hm, that's a strange system where "QA" people get punished for not releasing on time when the decision should not be with them in the first place, plus it's not only them who can do something about the product. I can see a potential for hiding important information from other people at such a workplace, because when people learn they can't single-handedly change something but they still get punished for it, it's just easier to hide information.

Answer 2

It depends, mainly on the risks. I might consider creating a taskforce to decide the go-no-go situation. Some example risks:

• Technical risks: Can we restore the issue later? Can the defect destroy data?
• Business risks: Can we lose clients/users due to this issue?
• Marketing risks: Will our growth halt if we do not release now?

Maybe you can communicate the known issue, maybe there is a workaround? Maybe not. If I would facilitate a good risk analysis and we make a collective decision based on data, I would expect it would have a positive effect on my yearly evaluation.

If my yearly eval would be based on the number of releases I would reject, I would try to change that metric. It seems gameable with risky consequences, mainly that I might let known defects go into production so I might get more money in the short term. This certainly might be an issue if I would have debt or other financial issues.

Answer 3

Neither. The correct answer here is to properly inform everyone who has a stake (so the marketing team and the developers who are working on the fix, and probably the QA members who discovered the bug and whoever you as QA lead for this project report to) and get a meeting scheduled ASAP to determine whether the bug is worth delaying the release for. The only delay you as the QA person should be introducing here is preventing the release until a formal go/no-go decision has been made.

In the event that the release is planned to go out with the bug, you should then coordinate with the marketing team to make sure users are properly informed that there is a known issue with the release, ideally including suggestions for workarounds.

In the event that the release is delayed due to the bug, you should then keep in close touch with the development team so that your QA team can start testing any potential fix as soon as it's available and minimize any further delays.

---

On a side note, the fact that QA rejecting releases will affect your yearly evaluation may or may not be a red flag. If it's a case of having to substantiate why you rejected the release as part of the evaluation, that's a good thing and shows that they hold their employees properly accountable for their responsibilities. If, however, it's an automatic negative on your evaluation, that's a huge red flag, because it means they either don't properly understand what QA is supposed to be about, or that they just don't care about QA, in which case I'd suggest steering clear of that company.

I would suggest if presented with such information that you ask how rejecting a release would impact your performance review, though I would strongly advise doing so after answering their question so that you avoid any impression that their response might impact your answer in some way.

Answer 4

QA found a critical bug.

I'm probably missing something here, but isn't the answer fully determined by that alone?  Doesn't the word ‘critical’ mean that it prevents release?  If it could be released, then it wouldn't be critical, merely serious.

If it weren't for that, then yes, you'd make sure all the relevant stakeholders were informed and consulted, and the company as a whole would make the decision whether to release, comparing the likely effects of releasing and not releasing.  But hasn't all that effectively been done already in the classification of the bug as critical?

Answer 5

There is a production release planned tomorrow. QA found a critical bug.

Been there, done that. This is how we handled a similar situation in past.

Go-live was scheduled around mid-night for a project where I was working as Lead QA. Around 2 AM, we found a critical bug. The best course of action - "communicate properly" the news to your Product Manager, Program Lead and other executive partners. A quick war-room was set-up and it was found that no matter how hard team try, fix was impossible. Next step- Discussion for another 1 hour to think-through the possibility of any work-around. Luckily, we found a work-around. Code changes were done along with thorough testing. This defect(with was now not a critical one because of the workaround) was published in release notes, as application went-live around 6 AM.

As the responsible QA person, should I reject the release or should I send the release to the marketing team with the known bug?

IMO, no matter how responsible QA you are, the actual decision-making lies with the 'client/customer' and key stack-holders. However, you can always assist and influence that 'decision'.

Answer 6

There is a production release planned tomorrow. QA found a critical bug. The dev team is working hard to fix it with extra effort but they aren't anywhere close to a solution. The marketing team is asking for the production release.

Are we talking about an Internet facing application like a web site ? Then knowingly pushing a seriously flawed application is not something you should doing.

If you reject the release it will affect your yearly evaluation.

If the release is flawed like you say, there is stronger likelihood that the company will get hacked as a result. That means lots time and effort to clean up, not to mention the reputation damage. Plus, the legal liability if your customer data is breached and exfiltrated by hackers.

To me this is a no-brainer. Ethically, my inclination would be to protect the company against a breach. Perhaps you have to take a stand against management to protect them against themselves.

If the worst scenario occurs what do you think your yearly evaluation will be ? You will be blamed for approving the damned thing and not evaluating the risks enough.

There are enough examples of breached companies to make them think twice. Just think how much it cost Equifax for not patching their systems in time against a known flaw. It's not something that only happens to others. A lot if not most companies have been hacked at some point and may still harboring backdoors in their systems, that they don't know about.

Seriously, don't compromise on security. I think the question is more about personal ethics or risk management than Q&A.

Answer 7

My approach would be:

1. Assess the importance of the change;

2. Give the developers a timeframe to attempt resolving the bug (e.g. 2 business days);

3. If the bug is still present, have the developers not include the change in the release branch (there is a pre-release branch right?), else prioritise testing/QA of the change;

4. Inform the marketing team what changes/features will be in the product.

As a multi-role developer when working in a small company, this is the approach I took. It used to annoy the marketing and sales person who wanted to say "We have feature x" while hiding that feature x didn't actually work.