I have a simple question, but it is still unanswered.

For example, I want to test a register feature, where the user must enter their name, email, and password.

The possible inputs are like this:

  • input name: valid (more than 2 characters), invalid, empty
  • input email: valid (contains "@"), invalid, empty
  • input password: valid (more than 8 characters), invalid, empty

How do I make a testcase so that the number of testcases can be as low as possible with the maximum quality of testing possible?

If I make all the testcases, the result will be like this:

  • valid name, valid email, valid password
  • valid name, valid email, invalid password
  • valid name, valid email, password empty
  • valid name, invalid email, valid password
  • valid name, invalid email, invalid password
  • name valid, email invalid, password empty
  • ...
  • name empty, email empty, password empty

If I make a testcase of all the possibilities, the total testcase is 3 * 3 * 3 = 27

Imagine if I tested a form with 10 types of input. Of course, the number of testcases produced can be tens of thousands.

What exactly is the correct way to make a testcase?

I saw on the internet, there is a technique called pairwise testing. But when I try to generate a testcase with pairwase testing using the following site:

The results are not the same (the number of testcases varies) between sites. So I am quite confused, what is the correct pairwise testing formula?

Then is there another technique that is better?

  • Testing two invalid fields at the same time is propably what you can leave out. I'd test valid name, valid email, and all password options, then valid name, password, and all email options, and finally valid email, valid password, and all name options. Than would be 9 cases according to your question. I'd, however, add one when all fields are empty and likely create more than just one negative input per field, so the total number would grow. – pavelsaman Nov 8 '20 at 7:52
  • Not to hawk a product, but there is a tool that tries to help solve this problem, and their intro deck does a fairly decent job of explaining how the problem manifests in the real world. Of course, they also present it in a way that their product solves it, so ya know, keep that in mind. hexawise.com/Hexawise_Introduction.pdf – corsiKa Nov 8 '20 at 22:00

"The value of any practice depends on its context"

N-wise testing is a technique, not a silver bullet to solve any problem. It's your responsibility as a professional to analyze how to best approach a specific problem.

Take the example of a pairwise analysis:

valid name, valid password
valid name, invalid password
invalid name, valid password
invalid name, invalid password

However, if the name and password validations are independent, it doesn't make sense to analyze these two variables together.

On the same token, if the password validation is performed by a third-party, you probably wouldn't need to explore it as you would explore name.

In summary, analyze your specific problem in order to decide the best approach. Perform risk-based analysis to understand your exploration space.

  • thank you for your answer. I think your answer is make sense – Samuel Ricky Saputro Nov 8 '20 at 11:39

"The essence of strategy is choosing what not to do."- Michael Porter

Everything in testing eventually comes down to one question- how it may affect business, as testing is fundamentally risk driven activity.

As with any effective strategy, an test strategy should not only cover what to test but more importantly what not to test.

As illustrated with simple login scenario, in real life complex applications there could be thousands of scenarios that can be created based on all possible combinations mathematically however that does not mean we should try to cover all of them in our testing unless and until it may affect business.

Example: For the same login functionality the level of testing will be done for an online banking application and pizza ordering application would be quite different.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.