Negative testing automation

Question

Should we do negative test cases testing by automation. Generally I heard that we can do functional and regression automation. I am doing same. I am using selenium web driver for automation and doing all functional and regression testing using it. But thing is all tests are positive here.

So I would like to know that should we do automation for negative testing or it is better to do it manually? Because I think if we go for automation for negative testing then it will be lot of code to be done..

For Ex: I have register form and field is there called "Password" , Now validations for fields are "Password must be of minimum 6 characters and must contains letter,digit and special character"

So in above case positive automation will be easy as I will test direct by insert valid password , But for negative testing of that field will have lot of cases so should we do automation for that?

Please let me know best way for negative testing. Right now I am preferring manually.

Answer 1

Your password example is a classic example in favor of automation, once you have the infrastructure for entering a password and detecting rejection adding more cases should be simple and cost effective.

You use the term Negative Testing to describe a type of scenario, but keep in mind that it is functional the same way as any other sunny day scenario, entering a wrong password is something users are allowed to do and not something evil.

Real Negative tests are somewhat more meaningful when used in the code level, for example checking for NULL pointers- this is not something a user of a function should do (unless it is part of normal usage of course)

So, to summarize- you should do negative testing and you should build your automation in such a way that it is not requiring "lot of code to be done"

Answer 2

It doesn't matter so much what you'll be automating, rather why.

• Are these negative tests included in your regression sets?
• Are these tests executed manually every time you run your automated scenarios?
• Will it eventually save the test team time and resources should it be automated?

Seeing that testing frameworks can usually deal with expected errors, it's perfectly feasible to automate negative tests.

Answer 3

Yes!We should make negative test case be automation testcase.The keypoint is what we can do to make test case be automation.

1. create many negative case
2. using less time

The best solution is we can create a framework to automation creating negative test for our application.

Answer 4

One the main reasons of doing UI automation (more applicable in your case since you said you are using Selenium WebDriver) is to test your application with different types of datasets. This include those for negative testing as well.

In your example, I would use automation to parameterize my password with different datasets that include negative ones like-

1. Blank passwords

2. Password less than/equal to/greater than 6 characters

3. With only letters and digits (no special characters)

4. With repeating letters, digits and special characters

5. Starting your password with special characters

And so on....

Just because you are doing negative testing does not mean it needs additional code. You can use one function to input text in the password field and then parameterize the data with different positive and negative datasets. This means that there is 0 extra code and you are passing different datasets via a table, excel sheet or however you have setup your framework.

Always remember to do negative testing when doing automation just like how you will do so when you are doing manual testing. The only difference here is you can test a lot of input combinations in lesser amount of time using automation.

Also if you use tools like PICT, All Pairs, you can reduce the amount of input combinations used for test automation as well. These tools are used for combinatorial testing.

-Raj