Where to start with white box testing of a massive project with multiple components

Question

I am very new to SQA and I am pretty much flooded with one of the biggest projects of my company. I am in a very problematic position. I simply just don't know where to start. That's why I am lagging far behind. Can you please help me out. My project is a server based project. And do I have to know the full code base to design my test cases?

It is a spring framework based project. And it is a huge project which also consists of five individual components. These projects communicate with each other via messaging. Now where to start with in this huge project?

Answer 1

There's no single answer for this - a lot depends on how your organization prefers to manage testing.

Here's how I'd approach a situation like this (using a manual version of the answer I gave here):

1. Who - who is the application intended for? Knowing who the users are tells you a lot about how they're likely to want to use the software and what parts of it they're going to use most.
2. What - what does the software do? What's it's main function?
3. When - and how often does it do what it should do? The tests you'll want for software that's needs to be available all the time is different than what you'd do for something likely to get occasional use.
4. Why - why is the software needed? This does overlap somewhat with what it does, but the difference here is that there's a reason the application exists, and that reason is going to guide your strategy.
5. Where - where is the software used? This doesn't just cover whether it's business or consumer software, it also covers things like is it web-based or desktop-based, will it be running over a LAN or WAN or will it be self-contained, and so forth. The environment - software, hardware, and human - tells you a lot about the conditions the application will need to handle, which in turn tells you where to focus your testing.

Once I have the overview, the next thing I'd do is start to explore the application, referring to any documentation that already exists. My goal here is to get a feel for the application and where manual tests will need to focus. Depending on how the application works, this can include inspecting the database and checking logs as well as interacting with the application itself.

At this point, if there's no guiding strategy for testing the application (which may be the case), I'll start to look for core functionality, with particular emphasis on the functions that must work and the areas of the application that will get most use. If I don't have that information, I ask for it. If I can't get answers, I guess based on the Who/What/When/Why/Where information I've already gathered.

If formal test plans and test cases are required, this is where I start writing them. My approach tends to be somewhat iterative with these: a test case will be relatively high level, and if I run into something anomalous (other than an obvious failure), I'll ask if that's what's supposed to happen. Sometimes it is - my understanding of the software won't be perfect. I update my test cases as I'm testing until they give an accurate description of the software behavior.

Answer 2

There is no possible single and perfect answer to your question. No matter how big or small the project you will have to understand its requirements and architecture.

Now you say you have to do white box testing so I believe you will have to be involved in code review and optimization. For this you will require to go through the code, the whole of it.

As far as where to start is concerned, well start with discussion with the other stakeholders of the project. Get to know your team and your project better. Then decide the priority and risks depending on time availability and budget. With this you will know where to begin and where to concentrate.

Once you have your strategy put together ask other stakeholders' feedback. Get another set of eyes on it to see if you missed put something.

Answer 3

The answer to this is simple and complex. You start by reviewing the requirement and writing what is called a Test Strategy and Impact Analysis. This is a general overview of the software, how it operates, the different functionality and, typically, I will assign a value to the 'importance' of each functionality.

By doing this you will naturally start to see pieces that should catch your eye as either critical pieces or overly complicated.

After this, you take your test strategy and compile a higher detailed version of the Test Strategy labeling what you want tested and how. Again, I will typically assign a value to each of these by order of importance. For example, a log in function will score much higher than an individual component since it impacts the most users.

Next, write out an extremely high detailed Tase Case Document that will contain your test cases. These will be what will be executed and what the result should be. Again, assigning a level of importance to each one.

Execute the test cases that you have written, filing defects for anything that you have found. Monitor which test cases found defects and keep track of them for later. After you have completed 100% run (all test cases ran, not all passed), re-run test cases that discovered a defect in the past until you have achieved 100% passed. If you run out of them, do exploratory and ad-hoc testing.

Once you achieve 100% passed, again, rerun all test cases that discovered a defect in the past to ensure that there was no issues with source control and re-run test cases based on their level of importance as marked above. During this run, stray from the written tests (while following their general guide) and think outside of the box some.

Answer 4

Since I can't add comments yet, I'll start with an answer:

Begin with the simplest case scenario. You said you have 5 individual modules. What's the minimum you can do in module 1 to provide the output to module 2, etc. Map that out on a piece of paper with each module being a square and an arrow going from one module to the next. That way if they can skip each other, you can draw routes, say from module 1 to 3, or from 4 back to 2.

Once you have that done you'll have your basic test bed. Then you can start adding "options" to each box. What are the basic categories of data you can change in each module, that will cause it to feed different data to different modules. This will give you a really nice picture of what the tests need to look like.

At my company this usually results in about a dozen different modules or "stops", each with 3 to a dozen different configuration options, which results in millions of paths. We then use a script that randomizes the paths and generates the tests. While we can't cover the millions of iterations, we can do a representative sample and then look for any glaring holes.