How to execute test cases designed for white-box testing?

Question

I have read about white-box testing, and that it involves things like statement coverage, branch coverage, path coverage done at unit, integration levels etc. As I understand it, we derive test cases to obtain maximum coverage for statements, branches and paths.

But, I want to ask, what do we do with test cases built during white box testing? I mean, in black box testing we execute test cases on application under test in order to check whether they pass or fail. But, what do we do with the test cases of white box testing? How, where and when do we execute such test cases?

My second question is, what type of defects are found in white box testing?

Answer 1

To answer your second question first, defects found in white box testing are the same defects that could be found in black box testing. The main difference is that some defects can be more easily found by white box testing than black box.

White box testing can be more than just statement/branch/path coverage. It can also include evaluating application logic, running code complexity metrics, and pretty much anything else that involves evaluating the quality of the code itself. Precisely what you would be doing when performing white box testing depends on the application, organization, and situation.

That said, tests identified during white box testing can be run as black box tests where you perform the actions that would follow targeted paths through the application and observe the results. They can also be used as baseline information (when dealing with code complexity metrics, for instance), re-run to ensure that known paths through the software remain good... There are very few limits: like anything else, how you use white box test depends on the situation.

Answer 2

Why you run white box tests depends on your goal. If your goal is to maximize code coverage then you probably want to measure code coverage, which means you need to run your SUT (system under test) in an environment where you can measure code coverage, e.g. in an IDE or attached to special libraries that collect coverage metrics.

If your goal is more generally to do a good job of testing then you might run your white box tests at the same time that you run your black box tests. You won’t know how much code you covered, but you will know that your white box tests were designed with the aim to cover lots of code, and that may be good enough, at least in the short term.

Your SUT will likely change after you write your white box tests. In the long run, if you don’t measure your code coverage and scrutinize the results, you won’t know whether your white box tests are still effective. You may be trying to test code paths that no longer exist.

Answer 3

In addition to Kate and user246's answers, I would like to add that white box testing is normally coded and executed by the developers. Yes, white box testing doesn't have to be automated but it is more common for it to be automated.

Developers commonly use a unit-testing framework to support their white box testing.

Regarding the question: But what do we do with the test cases of white box testing?

White box testing in the form of automated unit testing can be executed over and over again and its scope can be modified based on demand.

Answer 4

Unit tests are more closer to the application code where defects hide so easier to design and catch issues on that level compared to black box tests which are many layered away from the issues.

On top of other existing good answers, I would like to add a simple unit test code example for reference:

testAdd()
{
    int x = 5;
    int y = -2;
    int expectedResult = 3;

    Calculator calculator = new Calculator();
    int actualResult = calculator.Add(x, y);
    Assert.AreEqual(expectedResult, actualResult);
}

one thing to keep in mind, the expected result should be hard-coded into your test case so that it does not change when the implementation changes.

Answer 5

Adding some more to the excellent answers here- unit tests test the system at the unit level, e.g. a class, function or small module. As such, they can exercise the internals of the unit in a way that a system test usually cannot (or should not). Examples would be using illegal values, non realistic inputs etc.

This is also their weakness- some bugs might be real but will never manifest in the real system such as measurements from a physical system. On the other hand, some things are difficult to simulate at the unit level such as the timing of messages0

Practically, you need both test levels: Unit tests are fast, cheap and cover a lot while system tests compliment them.