3

This question has been asked to me in an interview with a really good company.Below i will provide the question in the form of our interaction(M:me & I:interviewer).Though there is no definitive ans but I need to know what idea/answer the interviewer really wanted:

I: The scenario is you and 2 others consist of the testing team. You,the lead, are the only one who can do automation, others can do manual testing only. You have nearly 10,000 bugs that have been raised and you have 4-5 weeks or less before this product is delivered. What will you do to ensure that the product is delivered in time?

M: Filter the bugs priority wise and retest them. In the meantime keep a log about what functionalities are facing more regression and thus start automating them. Similar or related bugs will be given to others for further testing.

I: Say suppose none of the bugs have been marked with any priority. What will you do?

M:I will filter with dates.In any kind of SDLC, even the agile one, the core components are developed first, if there are core bugs, they need to be fixed first.

I:(Disapprovingly)What if a very important functionality is added in a later sprint? Also how will you make use of your team-mates and your ability to automate.

M:Along with the date, as a tester I will have to know the core & important functionalities of the product till date.So keeping that in mind will find the core areas of each sprint to work on(about team mated answered the same thing as before).

I: Say suppose, the bugs haven't been marked with the timeline of each sprint. What will you do?

M:I will search the bugs list with keywords which represent the important functionalities without which the product cannot be released. I will pick the bugs from there.

I:(Again disapprovingly) With a keyword, you will get so many results, will you go through them one by one?

M:(slowly losing hope)I will just go through the title and decide.

I:Generally titles are not so explanatory, how will you handle?

M:I will start testing the product myself and search with similar bugs that I face, rather than trying to go through bugs because I need to take decision for the product delivery.

I: So you will ignore those many bugs? Stakeholders may not agree. (After this I totally lost it and just kept on blabbering and I don't remember what else was asked.Also everywhere the management/work of the 2 other manual testers was asked)

This was an interview for Sr SDET.

  • 3
    "What would you do?" for what? The question doesn't seem to have a goal. I would suggest clarifying what was the goal of the testers in this situation. – João Farias Aug 16 at 9:02
  • @ João Farias I edited the question. The "what would you do" part is really confusing. What she asked was, what will you do so that the product can be delivered in time. – Reema Aug 17 at 2:50
  • 2
    I feel like more clarification was needed from her side. Or perhaps that was what she expected. You knew nothing about the bugs, just the number of them. 10,000 is quite a number, so you're unlikely go go over them all in this time frame anyway. "What will you do so that the product can be delivered in time" That feels more like a question for a project manager. If the development process generates this number of bugs, the odd are something has been wrong for some time, and the project management has been blind to it or perhaps didn't have the information. – pavelsaman Aug 17 at 5:29
  • 2
    @pavelsaman is right, this is mainly a project management issue. As an interviewer I think the goal of the question was not to get a real and full answer but the see how do you cope with tough situations where there is no complete solutions. – Rsf Aug 17 at 7:39
4

In addition to what the other answers have said, I would say the interviewer is looking for how you, as a new addition to a team, would deal with a no-win situation. Frankly, I'd suspect that- at minimum - the company has found itself in this kind of situation in the past. At worst (I freely admit that I'm cynical) something similar is going to be facing whoever gets the position.

As an interviewer, I'd want something like this from the person I was interviewing:

First, I'd be wanting to know how these bugs are organized, particularly priority, severity, and risk. I'm assuming that I'm coming into this situation and not that I've been involved from the beginning, because this kind of situation suggests that things have gone badly wrong somewhere.

If the bugs are not organized in a way that involves priority, severity, and risk, I'd want to talk to the other testers, project management, and development to determine what problems they know of that pose the biggest risk to the projected deployment date.

If there is such an organization, I'd be looking to talk to testers, project management, and development to confirm the highest risk bugs. Ideally, I'd be looking for a way to build a list of bugs that must be fixed before the product can be released. With 10,000 bugs, that list is going to take some time to create, and that's assuming there are no bugs that testers haven't been able to find because the reported bugs are hiding or blocking them.

Once I have an idea of how bad the situation is, I can decide if - in my opinion - it's possible to release the application as planned. If most of the bugs are relatively low risk, and the high risk bugs seem to be reasonably easily fixed, I'd focus my team on the high risk bugs, and work with the project manager and any other team leads to get the highest risk (high severity, most likely to occur in the field, and/or block areas of the application) bugs fixed and tested.

If I can't see a way to release the product in time, I'd start talking to the project manager and my boss to see if there's a way to do a limited beta release of solid functionality or to delay the release. Since I'm new to the position, I don't know if there are any contract requirements or other factors outside my control that could force the release date to be immovable.

I'd also make sure that after the release I got with the leaders of all the teams involved to work out how such a situation came about and what actions we could take to prevent it happening again, as well as how we can work together to bring the bug backlog down.

Note that none of this has anything to do with the SDET role. It's clear from the question that the interviewer expects an SDET to also act as a test lead - I don't think this is a particularly good thing, and frankly, I'd be wanting to know if this is something the company expects from its SDETs.

It's worth remembering that even though interviews are high-stress situations, trying to think sideways and look at the implications of the questions you're asked rather than diving in. It's hard to do because you're stressed and trying to give your best, but if you can take a little time to mentally ask yourself what the interviewer is looking for with the question, you can usually give a better answer.

| improve this answer | |
  • 1
    I agree to what you said. I knew it was a no win situation from the question itself, but I just dived into it without thinking much (keep in mind this was the last interview of a 6 hr long onsite interview). But I do get the idea and i think I have to get into the practice of thinking outside for a bigger picture of the project, not just my assigned work or responsibility. – Reema Aug 18 at 6:13
  • @Reema - a 6 hour onsite interview is horribly draining. I'm not surprised you were having trouble thinking strategically at that point. I'm not sure anyone could do their best in that situation. – Kate Paulk Aug 18 at 12:00
1

The first thing that comes to mind is -- were these tests ever working before? If so then, don't panic. Something has changed either in the codebase or the testing framework that is probably causing groups of them to fail. Track that down and see if you can eliminate several thousand fails at a go. You're still going to need to go read over the ones passing again manually and double check them but maybe that will only take a few days.

If they never were checked before I'd still do something similar -- look for any commonality that might let you fix large groups at once.

Otherwise there is so much noise there it might cause you to miss something critical that is failing.

After that accept that you might not be able to get to everything and focus on the money maker code path. The things that have to work or the business folds. Then after you've cleared a few more of those, every other day or three look and see if there are anymore grouped failures like mentioned before and give a go at clearing a few more groups.

Note: Answering this from the point of view of an SDET -- someone that can fix the offending code base itself.

| improve this answer | |
1

If interviewer was mentioning bugs and not test failure ( if its test failure refer answer by @Lewis

First of all having 10000 active bugs in a product is really big red flag .

And you should never release such a product. But if the management decision is still to release then,

The answer the interviewer was expecting would be "severity"

The team should focus on fixing high severity bugs first if there are no priorities and keep low once on hold if it's not a urgent requirement and, is not affecting the actual business logic.

And concentrate on automating smoke test initially , then start automating all regression suites

Group the bugs and see where the bug clustering happens and rigorously test that module once the fix is made.

Before release manually test all the smoke test scenarios ( Critical businesses logic)

Also , having 10000 bugs can result in defect masking where these bugs are masking some critical bugs within the product.

So once the fix is made, more rigorous testing should be made around the modules to dig for more critical bugs

so if i was in the interview , i would answer like:

  1. 10000 bugs in any project would be a huge red flag , it shows there was no proper bug fix process and estimation in place. I would surely worry about defect clustering and defect masking , meaning there are possibility that most of the bugs are concentrated on a single module , and this much bugs may be masking any other critical bugs which will only be identified after fixing and retesting the module rigorously. And will recommend to push the release date further due to this reason.

Mean while as development team is busy with fixing the bugs , we will start automating the smoke test use cases and bug use cases . Once the fix arrives we would assign the retesting tasks to the manual testers and we our self do rigorous adhoc testing on the module to find any masked critical bugs.

  1. If there is no priority then it would be idle to revisit critical or high severity bugs first , and also to investigate bug life time and understand why bugs where not fixed for so long to help in improving the overall process in future.

About the low severity bugs we need to make a team decision on the timeline and on the release decision of whether to release the first version with these bugs but still documenting the same and workarounds where required. Also provide the next release date for the possible fix if possible.

So being a senior QA you should put forward your strong opinion to stay "NO" when you see red flags . Do not be too flexible

| improve this answer | |
  • "having 10000 active bugs in a product is really big red flag" Why? What's the direct relation between quality and bug documentation? "you should never release such a product" The question doesn't say anything about context. How can one say we shouldn't do X if we don't know anything about the product and its context? – João Farias Aug 17 at 8:13
  • @JoãoFarias Having 10000 bugs in open state is surely a red flag in the process and product irrespective of the context , it shows that the observation from testing is ignored completely and there is no proper bug fix effort estimation in the project. – PDHide Aug 17 at 8:29
  • @JoãoFarias Either choose to mark the bugs as no Fix needed , differed , or closed . Having 10000 bug to be revisited is surely indicates a really bad development strategy . Even a product being build from scratch wouldn't have so much open bugs. If its a product being build from scratch there will be milestone to be achieved and after which all bugs will be revisited and fixed . But surely product won't be released unitl bugs are revisited – PDHide Aug 17 at 8:32
  • 1
    Usually, there's no relationship between quality and bug count. On the other hand, everything has its limits, the number 10,000 seems to be so big that it suggests something is not quite right. However, it could be not only the product, but also the people who have generated this number of bugs - there could be hundreds of bugs for a simple typo in some marketing texts, there could be thousands of bug reports created automatically by some automation tool that failed because of reasons not related to the product. I'd say further investigation is needed before saying "don't release the product". – pavelsaman Aug 17 at 10:08
  • 2
    10000 bugs in backlog shows that no one knows what the contents of those bugs, it can be duplicates or critical bugs . You need to be triaged and closed or fixed properly – PDHide Aug 17 at 10:18
0

The other answers here are good if the point of the question is to give a concrete answer.

However, a lot of interviewers ask vague questions without a specific answer because they want to know how you think or to understand if you are making assumptions about the question. They want you to ask clarifying questions to them to get specifics. This helps guide your answer.

The scenario is you and 2 others consist of the testing team. You,the lead, are the only one who can do automation, others can do manual testing only. You have nearly 10,000 bugs that have been raised and you have 4-5 weeks or less before this product is delivered. What will you do to ensure that the product is delivered in time?

Some questions to ask:

  • How experienced are the manual qa testers?
  • Are the manual testers experienced in this project? Or are they new to the project as well?
  • Do all 10,000 need to be fixed before the delivery date?
  • Is there a bug tracker software the teams uses? If so, what?
  • How are the known bugs tracked? Do they have a priority, severity listed? Are they grouped/tagged by feature?
  • Are there any automated tests currently in use for software? If so, how many unit tests, integration tests, UI tests? Or, do I need to create all automated tests/framework from scratch in the 4-5 week timeframe?
  • How much testing are the developers responsible for? Are they creating unit/integration tests?
  • Are the 10,000 bugs UI bugs? Or a mix of bugs that can be tested using unit tests, integration tests, UI tests?
  • What devices need to be used for testing?
  • What is the level of quality we need to reach to satisfy users and stakeholders? How do stakeholders perceive quality?
  • How do stakeholders determine a successful project launch?
  • What is the teams definition of done?
  • Will the team have time after project release to fix bugs? Or are we moving on to the next project? How much time will we have if we do get time?
  • Is the team using Agile SDLC or Waterfall SDLC?

There's an endless number of questions you can ask to get clarification you need to provide a well thought out answer.

And, from the detailed conversation above, the interviewer kept prompting for specifics on how to include the manual testers in your plan. This gives you a big hint of what the interviewer is looking for: they don't want you to take on the full burden of testing this project yourself; they want to know as a Senior level SDET/QA Engineer how you mentor/lead a team of junior level testers.

Keep in mind, interviews shouldn't be an interrogation where you just answer their questions. Interviews should be a conversation where you can ask anything that helps to clarify their questions.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.