How does one unit test something that excludes everything except a finite set?

Question

For argument's sake, pretend a URL can only contain letters, numbers, and underscores.

How does one unit test a method that would check that a URL only contained those characters?

Surely people aren't going over entire character sets and testing every single possible character.

Example function under test:

function isValid(url) {
    var validUrlChars = new RegExp(/^[\w]+$/);

    return validUrlChars.test(url);
}

Answer 1

Do you really want to test all cases? I think not. The main reason to write unit-tests is being able to refactor the code in a later stage. Second it is to safeguard the public behavior of a class.

In this case I would start writing three unit-tests:

• A valid url
• An invalid url
• Empty or null input parameter (function should probably not return null or throw an error)

Code coverage

After this I would check code and branch coverage to be (near) 100%.

Mutation testing

If you really want to be sure your unit-test suite will catch coding errors due to changes in the future you could now perform an automated mutation testing run. Now try to write tests for any surviving mutations.

Adding more cases when needed

If some url is not catched the code is tested and it is easy to add another test-case in a later stage. This will be good enough in most situations. Keep in mind that test-code is also code and needs to be maintained, try to keep it as simple as possible. Also apply YAGNI.

Formal verification

If you need more thorough testing have a look at formal verification as a method to be 100% sure, but this is very effort expensive. I would only do this for extremely high risk situations.

Maybe also read this article:

Computer scientists can prove certain programs to be error-free with the same certainty that mathematicians prove theorems. The advances are being used to secure everything from unmanned drones to the internet.

Answer 2

white list, write a regular expression that verifies if a URL contains only alphanumerical characters or underscore.

Answer 3

If you can describe certain properties of your function, there are tools that can generate test cases automatically. These are smart enough to generate common edge cases (empty input, very long input, etc). See https://en.wikipedia.org/wiki/QuickCheck -- it has links to implementations for many languages.

If your function takes several parameters and it would be too expensive to test all combinations of them, there's a field of study called Combinatorial Test Case Generation (it even has its own conference). The idea is to generate the combinations that are most likely to find a bug. See http://www.pairwise.org/