17

There was an interview question as:

There is a production release planned tomorrow. QA found a critical bug. The dev team is working hard to fix it with extra effort but they aren't anywhere close to a solution. The marketing team is asking for the production release.

As the responsible QA person, should I reject the release or should I send the release to the marketing team with the known bug? If you reject the release it will affect your yearly evaluation.

What are the possible ways to handle this kind of situation?

  • 1
    Are you told whether you answer to the marketing team? – Corey Sep 11 at 18:28
  • 1
    @corey yes i said i will send an email to marketing team with a release rejected release note. – ChathuD Sep 12 at 3:25
  • 15
    "If you reject the release it will affect your yearly evaluation." -- presumably, this means that it will enhance your yearly evaluation. QA's job is to say no. – Pete Becker Sep 12 at 11:56
  • 5
    Every production release of every piece of software has bugs. This just happens to be one that you found just before you were going to release it. Really, the answer depends on what the organization means by "critical." Either it means "don't ship till it is fixed" or it is just emotive language that doesn't really mean anything. – alephzero Sep 13 at 2:00
  • 1
    I would answer with this story: I was a happy user of an online invoice software. Then it was "modernised". The result was so bad that I no longer use that software. When I asked for alternatives from my accountant, she was also familiar with the change "yeah, we get a lot of questions like that". I bet the vendor lost, not few, but majority of their customers. The difference in usability etc. is do big that the company must have realised they are launching an inferior product. Yet they did. That was a very bad idea. – Tero Lahtinen Sep 15 at 5:47
39

as a QA responsible person should I reject the release or send the release to marketing team with known bug

Neither one. You should let all people involved know that there is such and such bug. You should communicate clearly what the bug means to customers and other people who use the product or are somehow involved (stakeholders). This might be a team effort as well.

Lastly, the person who asked this question told me that if I reject the release it will affect to my yearly evaluation too.

Hm, that's a strange system where "QA" people get punished for not releasing on time when the decision should not be with them in the first place, plus it's not only them who can do something about the product. I can see a potential for hiding important information from other people at such a workplace, because when people learn they can't single-handedly change something but they still get punished for it, it's just easier to hide information.

| improve this answer | |
  • 24
    Strange system. One wonders if they want the answer: 'you incentivize the QA team not to find bugs. Seriously?" – richardb Sep 11 at 19:04
  • 4
    Tagging on to emphasize: members of the dev team are stakeholders too. They have the technical expertise to identify risks to the long-term health of the software systems and development teams, and have an obligation to bring that information to the table. Devs aren't hired to be yes-men who ship no matter what. The business/product people rely on their expertise to highlight technical risks like this. – Alexander - Reinstate Monica Sep 12 at 17:23
  • @richardb Next level QA is when they help prevent bugs from being made int the first olace, (which would mean they don't find them). But that is not what is discribed here. – DarcyThomas Sep 14 at 7:24
  • @richardb They incentivize the QA team to not reject releases. Big difference. This leads to "yes it has this bug which formats the user's hard drive and sets their computer on fire, but you incentivized me to never reject a release, so I didn't." – user253751 Sep 14 at 10:10
13

It depends, mainly on the risks. I might consider creating a taskforce to decide the go-no-go situation. Some example risks:

  • Technical risks: Can we restore the issue later? Can the defect destroy data?
  • Business risks: Can we lose clients/users due to this issue?
  • Marketing risks: Will our growth halt if we do not release now?

Maybe you can communicate the known issue, maybe there is a workaround? Maybe not. If I would facilitate a good risk analysis and we make a collective decision based on data, I would expect it would have a positive effect on my yearly evaluation.

If my yearly eval would be based on the number of releases I would reject, I would try to change that metric. It seems gameable with risky consequences, mainly that I might let known defects go into production so I might get more money in the short term. This certainly might be an issue if I would have debt or other financial issues.

| improve this answer | |
11

Neither. The correct answer here is to properly inform everyone who has a stake (so the marketing team and the developers who are working on the fix, and probably the QA members who discovered the bug and whoever you as QA lead for this project report to) and get a meeting scheduled ASAP to determine whether the bug is worth delaying the release for. The only delay you as the QA person should be introducing here is preventing the release until a formal go/no-go decision has been made.

In the event that the release is planned to go out with the bug, you should then coordinate with the marketing team to make sure users are properly informed that there is a known issue with the release, ideally including suggestions for workarounds.

In the event that the release is delayed due to the bug, you should then keep in close touch with the development team so that your QA team can start testing any potential fix as soon as it's available and minimize any further delays.


On a side note, the fact that QA rejecting releases will affect your yearly evaluation may or may not be a red flag. If it's a case of having to substantiate why you rejected the release as part of the evaluation, that's a good thing and shows that they hold their employees properly accountable for their responsibilities. If, however, it's an automatic negative on your evaluation, that's a huge red flag, because it means they either don't properly understand what QA is supposed to be about, or that they just don't care about QA, in which case I'd suggest steering clear of that company.

I would suggest if presented with such information that you ask how rejecting a release would impact your performance review, though I would strongly advise doing so after answering their question so that you avoid any impression that their response might impact your answer in some way.

| improve this answer | |
7

QA found a critical bug.

I'm probably missing something here, but isn't the answer fully determined by that alone?  Doesn't the word ‘critical’ mean that it prevents release?  If it could be released, then it wouldn't be critical, merely serious.

If it weren't for that, then yes, you'd make sure all the relevant stakeholders were informed and consulted, and the company as a whole would make the decision whether to release, comparing the likely effects of releasing and not releasing.  But hasn't all that effectively been done already in the classification of the bug as critical?

| improve this answer | |
4

There is a production release planned tomorrow. QA found a critical bug.

Been there, done that. This is how we handled a similar situation in past.

Go-live was scheduled around mid-night for a project where I was working as Lead QA. Around 2 AM, we found a critical bug. The best course of action - "communicate properly" the news to your Product Manager, Program Lead and other executive partners. A quick war-room was set-up and it was found that no matter how hard team try, fix was impossible. Next step- Discussion for another 1 hour to think-through the possibility of any work-around. Luckily, we found a work-around. Code changes were done along with thorough testing. This defect(with was now not a critical one because of the workaround) was published in release notes, as application went-live around 6 AM.

As the responsible QA person, should I reject the release or should I send the release to the marketing team with the known bug?

IMO, no matter how responsible QA you are, the actual decision-making lies with the 'client/customer' and key stack-holders. However, you can always assist and influence that 'decision'.

| improve this answer | |
2

There is a production release planned tomorrow. QA found a critical bug. The dev team is working hard to fix it with extra effort but they aren't anywhere close to a solution. The marketing team is asking for the production release.

Are we talking about an Internet facing application like a web site ? Then knowingly pushing a seriously flawed application is not something you should doing.

If you reject the release it will affect your yearly evaluation.

If the release is flawed like you say, there is stronger likelihood that the company will get hacked as a result. That means lots time and effort to clean up, not to mention the reputation damage. Plus, the legal liability if your customer data is breached and exfiltrated by hackers.

To me this is a no-brainer. Ethically, my inclination would be to protect the company against a breach. Perhaps you have to take a stand against management to protect them against themselves.

If the worst scenario occurs what do you think your yearly evaluation will be ? You will be blamed for approving the damned thing and not evaluating the risks enough.

There are enough examples of breached companies to make them think twice. Just think how much it cost Equifax for not patching their systems in time against a known flaw. It's not something that only happens to others. A lot if not most companies have been hacked at some point and may still harboring backdoors in their systems, that they don't know about.

Seriously, don't compromise on security. I think the question is more about personal ethics or risk management than Q&A.

| improve this answer | |
1

My approach would be:

  1. Assess the importance of the change;

  2. Give the developers a timeframe to attempt resolving the bug (e.g. 2 business days);

  3. If the bug is still present, have the developers not include the change in the release branch (there is a pre-release branch right?), else prioritise testing/QA of the change;

  4. Inform the marketing team what changes/features will be in the product.

As a multi-role developer when working in a small company, this is the approach I took. It used to annoy the marketing and sales person who wanted to say "We have feature x" while hiding that feature x didn't actually work.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.