24

I know the best bet in this situation is for the testers to go through the process of reporting blocking-type problems initially show up, with the focus being on critical bugs. Since, this type of problem can high severely affect on schedules and indicates deeper problems in the software development process.

This situation will occur due to insufficient unit testing or insufficient integration testing, poor design, improper build or release procedures, etc. These are the major reasons.

What are the other steps which we have to follow as a QA if the software is so buggy?

  • 7
    Good question after a long time, Just reject the Build & report the critical bug list to the development team. Make sure to keep in loop with your manager & higher technical team; so that next time this situation will not be in. – Sophia Jun 29 '17 at 18:31
35

Continue testing in this situation would be counter-productive and may lead to a "deadlock" - when the reported issues would depend on each other recursively.

  • make sure the management is aware of the situation
  • sit down with the development team - determine the most critical and problematic areas and the step-by-step process to get out of the testing "deadlock"
  • follow the improved/fixed areas of the software in the "breadth-first" manner - from most critical to less critical
  • do the root-cause analysis - what led to this kind of situation, what can be improved to not let this happen again - was it a design/architecture issue, poor-documented software requirements, lack of different types of testing, communication issues etc.
  • 8
    +1 for root cause analysis and preventing it in the future. With decent unit test, such situation should have been detected – Peter M. - stands for Monica Jun 29 '17 at 21:44
10

Ideally, suspend the testing. And it's time to refer the Test Plan document.

This is the time when one should look at the Testing suspension and resumption criteria section in the Test Plan document.

I can refer to the following three points that are generally mentioned under testing suspension criteria and this scenario looks like falling under point 1 and/or 2:

  1. A bug is found which requires fixing before further testing is possible, or which renders further test cases obsolete.

  2. The developed software is not functioning well enough for the tests to be meaningful.

  3. Problems arise with test environment or access and/or setup is incomplete and not available.

As mentioned, the software can't really be tested because of critical bugs. So, the testing team should raise a red flag along with data (a list of well organized bugs found so far) in order to present their case.

And testing should be resumed once the conditions for the resumption criteria have met:

Resumption Criteria - Testing can be resumed only when

  1. Situation due to which testing was suspended has improved and it is feasible to go ahead with the testing activities.
8

What are the other steps which we have to follow as a QA if the software is so buggy?

This question reminded me of an interview question:

  • During one software security interview, one candidate was asked: "How would you fix a zero-day security bug?" This candidate replied, "I would start from fixing 100-day bugs."

100-day bugs is a made up term for older security bug comparing to zero-day bugs.

When a piece of software has accumulated too many bugs, it is almost certain that some bugs are more fundamental than the rest, e.g. some bugs have results of other bugs. In other words, mistakes made earlier lead to more mistakes later.

Apart from Aalok and alecxe have stated, is it possible for you to categorize the bugs so that you can start from the ones that most likely to have compounding effect?

Fixing them first and you may notice other bugs will be fixed in the process as well.

7

If you are a developer...

  1. Refactor code every time you touch it.

  2. Know the application's behavior before refactoring.

  3. Refactor as if you have to write unit tests (if you know how to write testable code).

  4. Include estimates, do change impact analysis, and let your manager know about it.

If you are a tester...

  1. Talk to the developer if they can batch bug fixes.

  2. Know the application behavior and prepare test cases.

  3. Don't blame developers for new bugs appearing because of new fixes. Be patient.

If you are a/the technical lead...

  1. This could be a nightmare if not taken care of soon.

  2. Talk to the developer(s) about estimates.

  3. Motivate the developer when code must change for the defect to be fixed. Change the code to be testable with automated tests.

  4. Appreciate automated tests, even if it increases development time.

Eventually, all of you are responsible for working or non-working software. So be patient and keep making it better every time you change it to fix something.

Anyways, I'm in the middle of doing something similar so sharing experience.

  • 1
    Apologies for formatting wrote it from phone app. – vendettamit Jun 30 '17 at 1:47
5

Often this situation happens to combine with a high pressure from management to get the product out, and they may not be receptive to QA telling them that it is unreleasable.

To make the best out of a bad situation, it is useful to find and follow through the fix for the bugs that most annoy the users. This is easier if the software is used internally in the company or in some close partner companies, but one can try to seek out a few example users anywhere.

It is surprising how well people can adapt to very buggy software when they have no choice. Just a few bug fixes can make a marked improvement even in a lousy piece of software.

5

The answer is Acceptance Tests. Your software should meet a baseline set of functionality (usually determined by the PO, but could be dev/QA combo) to be considered "done", and thus ready for QA.

4

In this situation the first thing I do is report the observed bugs, and label them in some way as blocking further testing.

Exactly how the labeling is done depends on the tool and the circumstances, but I make sure each bug that blocks further testing is clearly flagged as blocking further testing. For instance, a bug that makes it impossible to save data clearly prevents any further actions with that data.

Once I've flagged my testing as blocked, I notify my lead and project manager. They need to know that I can't progress further because of a situation not in my control. At that point it's their responsibility to make sure the blocking bugs are fixed so I can get back to work on the software.

Once my lead and PM have been notified, I work on something else until the blocking issues are fixed. There's never a shortage of things to do, so having something to work on isn't a problem.

In my opinion the most important step is making sure your lead and project manager know you are unable to continue testing and why.

3

An automated test is also an application that has a set of requirements, a design and a delivery date. Use the time wisely to continue developing the test application, because if the SW team do fix their bugs just one day before delivery, it will still be expected that you can run your tests and give the ok. Keep everyone regularly appraised of the test result status - not just x% of tests passed but also a breakdown of the application under test into sub-components.
In that way, everyone knows where the hotspots in the project are. Whilst there is little value for the SW team in running tests that rely on other code working, it may help you to improve your test application.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.