Is equivalence partitioning test technique is really worth anything?

Question

I am teaching SW testing in a Computer Science department. As expected, I am conducting equivalence partitioning testing. For an exercise, I gave the class a set of 5 rules for code that grades password strength and asked them to list the equivalence classes for it. It's a simple set of rules; they are not that critical to my question - I am listing them here just so you see it's a rather simple set of rules:

• The password score (grade) can be between 0 and 10 (10 is a strong password)
• A password should be longer than 12 characters. The score is reduced by the distance from the actual length and 12.
• A password should include at least one upper case letter, one lower case letter, and one number. The score is reduced by 2 for every infraction.
• A password shouldn't be the words “password”, "admin" or "root". Violating this rule results in a score of 0.
• A password shouldn't be the same letter or number repeated for its entire length. This deducts 7 points.

The resulting number of equivalence classes is... 20. This is after I added some restrictions to avoid the number being even higher.

So my question: If for such simple code, I need 20 ECs, for an extensive system, the ECs will be an enormous number - rendering the EP test technique not very helpful.

Am I, and all the SW test textbooks, overplaying the usefulness of this technique? Is it useful except for trivial examples?

Answer 1

Ah the joys of combinatorial math.

As the other answers have implied, it's very easy for pretty much any technique to generate a massive number of test cases for any real-world application.

Equivalence partitioning is a way to group those test cases to help reduce the burden of testing. That doesn't change the fact that a real-world application is likely to have a large number of equivalence partitions - it's still easier to test (e.g.) that a password consisting of 12x the same character gets a low score than it is to test that a password of 12x0 gets a low score and a password of 12x1 gets a low score through every character allowed by the password input.

Regardless of the software, you're dealing with the fact that there are effectively an infinite number of pathways through it. In your example of a password strength module, the number of possible passwords may be finite, but with a minimum length of 12 and an unknown maximum length, it will be an unmanageably large number, particularly when you include all the potential invalid values. Compared to the many millions of possible password values, is 20 equivalence partitions such a burden?

Answer 2

for an extensive system

That's why we have the principles of modularization and composition as code quality criteria.

Conducting checking in a big module is not sound engineering.

Taking your own example, you probably would not create checks for the String library or the assert library - you are assuming they are bug-free (not really, but sound from an engineering point of view and your goal of checking your password evaluation code).

Going one step further, when you want to check your frontend displays an error when the password is weak, your check will just have an example - with a weak password - your password evaluation module quality is not the point of the frontend checks.

Dijkstra tried to promote the possibility of "proving" software, using structural analysis but admitted that for practical purposes is not useful.

Engineering != Math. Do what is necessary to build what you want to build. Use equivalence partitioning to fulfill your mission, not as a wall you have to overcome.

Answer 3

If for such simple code, I need 20 ECs, for an extensive system, the ECs will be an enormous number - rendering the EP test technique not very helpful.

Am I, and all the SW test textbooks, overplaying the usefulness of this technique? Is it useful except for trivial examples?

First, great question! And I wish more people would teach these topics in Computer Science courses. My CS courses never did!

No, you are not overplaying the usefulness of ECs. Even when you use it in combination with other techniques, like Boundary Value Analysis, these techniques can be very powerful. I've often taught these techniques to the dev teams I've worked on.

To me, what perspective are you coming from? A lot of devs dislike anything related to testing, so they throw out phrases like:

• "Testing is easy"
• "If it works, it works"
• "Why do we need tests? Users can find the bugs."
• "It works on my machine"
• "It works according to spec"

And a lot more phrases that escape me but are just as diminutive and minimizing in nature. I think all testers have heard phrases like these.

Are you trying to achieve "perfection?" Some level of exhaustive testing? Do you want to test via brute force? Trying to find a "magic number" that proves the software is completely tested (like O notation)?

Are you thinking testing is a bottleneck and takes too long? If we can't find all the tests or all the bugs, then we shouldn't do any testing?

Keep in mind, you'll never reach perfection and it's impossible to do exhaustive testing as it takes too much time!

The ultimate purpose of these techniques is to help reduce the amount of testing done while still proclaiming the software is tested at a reasonable coverage level. They are used to make the complex more simplified. These techniques speed up test cycles.

These testing techniques are engineering techniques. Are they simple to understand? Yep! So is a for loop and those get used all the time! I had CS professors that professed that "simple code is better, more secure, more performant, etc." Why make a technique more complicated when it works?

I'll leave you with this: Think of Testers like they are Editors (book editors, newspaper editors, film editors, etc)!

It's the job of a tester to ensure the final product matches the objective and goal; ensure consistency; features are functional by themselves and in combination with other features; to find mistakes; to ask questions, etc

Would you want to read a book or watch a movie without editing?