4

Let's say that I want to create an ad on some website. I am using Selenium IDE to test.

I understand how creating positive tests work, but I am confused about how to create good negative tests.

What makes a good negative test? Do I need to assert/verify that the data is correct or not? Is the test only considered negative testing if it shows that it failed?

3
  • 1
    Maybe you can give a clear example of a negative test-case you have issues with automating with Selenium? Commented Mar 5, 2017 at 16:48
  • Edited to clarify intent
    – Kate Paulk
    Commented Mar 6, 2017 at 12:57
  • Possible duplicate of Negative testing automation
    – ECiurleo
    Commented Mar 7, 2017 at 16:45

3 Answers 3

2

Yes. I treat positive and negative tests as basic tests. I treat them quite similarly and use them both in my tests. I assert that the things I should see are there and that things I shouldn't see are not there.
In the SeleniumIDE I use frequently use assertElementNotPresent for negative assertions.

Positive assertion examples:

  • the button is present
  • the text is present
  • the form is present
  • the input field exists
  • the submit button exists

Negative assertion examples:

  • the button is not present
  • the text is not present
  • the input field does not exist
  • the submit button does not exist

They often both have roles in a given test, for example:

I click on a button that opens up a form for 'add room'

  • the form is displayed
  • the 'add room' button does not exist

I enter correct data and submit the form Expectations:

  • I see a message that the room was added
  • I do not see the form anymore

I enter invalid data and submit the form

  • I see a message about invalid data
  • I see the form redisplay with an error message
  • I do not see the 'add room' button

Word of caution - always use a positive expectation when you have a negative one, otherwise ANY result including 404 and 500 http status code error pages may satisfy the negative expectation, but without a positive assertion, give you a false positive result.

I prefer to treat tests as 'happy path' and 'sad path' (happy - everything goes right, sad - user makes mistakes) rather than positive and negative tests as those words can get confusing (e.g. the element should NOT be there but is so is that a positive or negative test?). Happy tests can include both positive and negative assertions and so can Sad tests.

2
  • Something that is not present is not by default negative testing. Most of the time hiding elements is part of the "Happy" positive path and thus expected. Only when using invalid data the path is negative. Commented Mar 5, 2017 at 16:47
  • Sure. Although I have several tests where the element not being present is one of the correct assertions, e.g. a form is there instead of the add button. Commented Mar 5, 2017 at 23:30
1

First we need to define positive vs negative testing:

Positive testing is the type of testing that can be performed on the system by providing the valid data as input.

Negative Testing is a variant of testing that can be performed on the system by providing invalid data as input.

For example look at: http://www.guru99.com/positive-vs-negative-testing.html

So for your Advertisement example:

  • The positive test should check the advertisement is display when using valid data as it settings
  • The negative test would be to check that an error message is shown when using invalid data as it settings, the advertisement should not be displayed.

How you create those test with Selenium depends. First try todo it manually and then see if you can automate that same path. E.g. first write positive and negative test-cases.

Wonder if you really need to write tests for the negative paths. What is the chance a real user would execute it that way. If the chance is high make sure you have good test coverage.

1

When I write wrong data in some of the fields do I need to assert/verify right data about that field or I don't need to use assert/verify at all?

  • Yes, you will definitely to verify wrong data you have written in some of the fields. Imagine yourself as an end user, you would expect a software application to warn you that you have done something wrong AND even better to guide you out of this.
  • In addition to your question: the interesting difference between valid inputs and invalid inputs is: the collection of valid inputs is almost always smaller than the collection invalid inputs. E.g. the number of invalid inputs there are for a field is almost always infinite. You need to be very strategic when you select your invalid inputs for your negative tests.

Is it only considered negative testing if it shows that it failed?

  • It depends. What if the invalid input you have chosen is not recoganized as invalid? It is not uncommon for a software to accept invalid inputs, that is why we have negative tests: to make sure a software will fail when it is supposed to fail.

how to create negative test in selenium ide

  • It depends on how your software handles an invalid input. In general, provide a known invalid input then look for its associated failure message is the way to go.
  • Please bare in mind that Selenium IDE does not support error-handling well.

Not the answer you're looking for? Browse other questions tagged or ask your own question.