3

I am a bit confused as in a few books, negative testing is described as trying to break the system with providing inputs outside the defined group..which I understand and agree with ...and that a negative test is when product delivers an error when it should not or not deliver an error when it should.

But that does not make much sense, that deals with the actual result of the test while the positive-negative test is a matter of design. I cannot imagine how a negative test can result in a situation when it throws an error when it should not, because that is in my opinion a situation that can never happen if you design test with intention to break the system. I believe these authors confuse test result and purpose of the test.

If a fields accepts only letters and should show a message when putting numbers, I believe putting numbers is actually a positive test case as it verifies specified behaviour. But maybe I am missing something here..?

2

This question is the reason I prefer not to use "positive" and "negative" test case terminology.

There are some sources who maintain that testing expected error conditions is "negative" testing. Others claim that testing unexpected error conditions is "negative" testing.

a negative test is when product delivers an error when it should not or not deliver an error when it should

I agree with you that this is an incorrect statement: when the software generates an error in a situation that should not produce errors, or fails to generate an error in a situation where an error should occur, you have a failed test.

My preferred terminology covers things like:

  • Happy path/steel thread testing - tests that cover the essential functionality of a feature. In the case of an e-commerce app, this would be tests to add items to a cart and check them out.
  • Other requirement testing - tests that cover the stated and implied requirements other than the essential functionality. Using the e-commerce app example, this might include tests like ensuring that all pages in the system share a consistent look and feel, or that new features don't interact poorly with existing features.
  • Defined error testing - tests that cover any defined failure situations. Using the example above, a test that covers using an invalid credit card for a purchase would be an example of a defined error test. So would a test that attempts to purchase a quantity of -1 items.
  • Failure testing - tests that explore the system's ability to recover from unexpected error conditions. In the e-commerce app example, this might include ensuring the app recovers from a network failure partway through a transaction, or that it doesn't crash when given textual input in fields expecting numeric input.

My opinion is that only failure testing should be considered negative testing, but there isn't a consensus, so if you discuss negative testing, you should be prepared to define which of the accepted meanings you are using.

  • Thanks, I think the same. Actually if the e-commerce app has a defined requirement to show a message "Faield" when network disconnection occurs, then I believe this is not a negative test as the input/action is expected and described, which makes it not a unknown input. – John V Sep 8 '17 at 10:03
1

I do not see contradictions here. You have some definition of a valid input. Testing a valid input is defined as a positive case. (Those are basically the cases the system is designed for).

There is also nearly infinite number of cases how one can use the system in the way different from what the system's been originally designed for. The systems also might not be implemented for handling some of such teh cases but implemented for handling some other ones.

The fact the system has some predefined instructions for how to handle some of the "unexpected" cases does not make those tests positive. I would say the goal of the negative testing is to:

  1. Make sure the requirements for handling unexpected cases are met
  2. Minimize the number of unhandled cases (and their potential impact to the system)

As it seems to me, in other words, knowing how the system should react on an input does not define whether the test is positive or negative. It is rather defined by the "in scope/out of scope" section of requirements document or general knowledge of what the feature is intended to be applied to.

  • Well if the system has an instruction how to handle an invalid input, then it is just another class of valid (expected) inputs and therefore testing that is simply testing the expected behaviour, i.e. a positive test. – John V Sep 4 '17 at 17:17
  • @Pietross consider the following example: there is a separated collection of different types of garbage near your house. Assume you're dropping a glass bottle into a tank designed to collect organic debris. This is a sort of a negative test, despite that your action would not stop garbage recycling factory since they know how to handle such cases (apply various filters and chemical reagents). This seems to be unexpected behavior from your side. – Alexey R. Sep 4 '17 at 18:01
  • @Pietross After all the computers are strictly determenistic machines which means there is unlikely that you or your app will generate a really unexpected situation. Even if an exception is not handled (and not expected) by your app it throws it out to OS as an error code (or using any other mechanism that OS supports) and handle that case on OS level and then (which is less likely) on a bios level. Hence the error will somehow be handled eventually.Due to this logic negative tests should not exist as a class since we always know how even the errors unsupported by your app would be handled. – Alexey R. Sep 4 '17 at 18:05
  • What I mean is - if the specification says that such an input is handled, then it is not a negative test by definition. – John V Sep 4 '17 at 19:35
  • In my view negative testing is a testing in which our purpose is to break the system by inputting the INVALID values, irrespective of the fact that system has provisions to handle that invalid output. The main purpose of negative testing is to check whether application is handling invalid inputs gracefully or not. And to verify that it is handling invalid input gracefully it will obviously be mentioned in SRS that what application will output in such condition. – a Learner Sep 19 '17 at 18:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.