3

What's the best status of test case execution when minor error occurs, but the main goal of the test case is fulfilled. Let's consider this test scenario:

Test case title: Addition of object

  1. Press Add under list of objects
  2. Fill in form
  3. Press Save -> Confirmation should appear
  4. Press Yes in confirmation windows -> object should appear in object list

Now, let's say that step 4 is completed successfully, but step 3 is not because of lack of confirmation window. Object is created and that was the goal of this test case, but part (not crucial, but still) has failed. What should be the overall status of test case execution - Passed or Failed? AFAIK TFS doesn't provide any way to set status between failed and passed, something like "warning" status meaning "we had some minor problem with this test case, but it passes after all". What is the best way to tackle this kind of situation?

Improve this question

4 Answers 4

Reset to default
5

If you have specifications that say a confirmation must be displayed, the test fails.

If you have user requirements or user expectations that there will be a confirmation dialog, the test fails.

In the situation you describe, I would create a bug from the test, but it would have a lower severity rating because it doesn't interfere with functionality. The project leader would decide the priority of the bug fix.

That said, there is no "best way" to handle this or any other situation. There is only the way that works for your workplace and your team.

For instance, if you expected a confirmation, but there's no requirement that one display, you might need to revise your test steps.

Improve this answer
2

It may depend on the strategy of your project, but i'd say this test is failed. If the confirmation is part of your specifications/functional need, you can't make this test pass.

Imagine this is the only issue in your whole test plan, if you pass the test, then your indicators will be all green for this functionality. Thus, there is no reason to do rework, and this "minor error" could stay here forever.

Moreover, it's not a "minor error" to me. It could have major impact on the user experience.

Improve this answer
1

I would separate the test case into 2: The first one ends with the confirmation message and the second one continues from there on. I would create a bug, link it to the first test case and keep the test case in 'ready', while I would close the second test case. If this is not a major requirement for you, then the test cases can be prioritized, so that ones affecting the requirements would be run first.

Improve this answer
0

Best way it to talk to your internal customer and/or user interface designer find out what customers want, and what your team wants to deliver.

If they want confirmation dialog (consistently across your app), you fail the test.

If UI designer wants to avoid extra click to dismiss dialog (and add i.e. some message in the header or other less annoying place), you need to change requirements.

Any differences between requirements and implementation should start dialog (ideally, before implementation happened but you know that developers are often not the best in communication). Maybe implemented way is better, and requirements needs to change (because customer did not envision better way, and asked for inferior way). Maybe it is worse, and code needs to change.

But you will never know until you ask customers.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.