Test Data Bank creation: approach, important points, useful links, data generators, file storages?

Question

Being Senior QA, I often face one and the same problem: for every project I'm involved in I need to prepare / collect / generate various sort of testing data. It might be VERY different - below are examples I had to deal with during last year only:

1. XML files (application integration layer transport testing)
2. Set of as-much-as-possible video files with different format encoding (online converter testing)
3. ISBN number generation (publishing store testing)
4. Typical office-like set of docs (MS / Open Office, PDF, PS, etc.) of various versions / formats (testing proprietary online doc viewer similar to Google Docs)
5. ...And so on.

So... basically I often spend too much time for preparing / collecting things folks in the next room already have "on the shelf". That's why I'd like to organize a centralized solution, like network drive or share, with the set of test data files and / or generators available for the whole QA department.

Sure thing I tried to accomplish that many times, but I faced the following (the list is actually far from complete):

• Lack of data: new projects come, we need smth new OR current set of data is not enough.
• People still prepare data on their own and do not share it (for numerous reasons) - they're lazy or not motivated.
• It's quite difficult to maintain & keep the set up-to date: each team / QA tries to do the best - as a result, we have similar sets of data organized in different manner.
• Sometimes it's hard to decide which files should be kept as prepared set, and which ones better generate as required using appropriate software / converter / tool etc. At least we should keep in mind cost of storage, since it's not effective to store 100 x 250Mb files filled with dummy data for bandwidth load test.
• If we use generators - they should be flexible, customizable and produce desired output quickly and "as expected".
• When the storage becomes huge, the mean of inventory becomes important (for quite obvious reasons).

All in all, I'm asking for valuable advice from your experience on handling the above points, and in general - how achieve success in test data repository preparation and its further "keep going".

Any best practices, links, concepts, approaches, success stories - from handy fast format converter to examples of test bank maintenance - are highly appreciated. Thanks in advance!

Answer 1

Test data generators are especially useful when the data format is likely to change.

Here is a common problem I've encountered with test data: it isn't always clear which parts of the data are intentional and which are optional. For example, suppose you want to test a CSV file of names and addresses, where each record contains a first name, last name, street address, city, country, and postal code. When you boundary-check the first name and last name fields, you may not care what the other fields contain, as long as they're valid; in other words, the first name and last name values are intentional while the others are arbitrary.

You should care about this because someday someone will need to update your test data: they will need to adjust to a new format, or add some new test cases, or delete some obsolete test cases. If the intent of each test isn't clear, they may accidentally corrupt your tests.

There are different ways to address this. If your file format allows for comments, you can use them. You can also address it by generating your test data. Your generator can contain comments or (even better) be coded so that the intentional and arbitrary values values are explicit.

Another thing I like to maintain with the test data: expected values. Surprisingly, I have known testers who keep the expected results in their heads or expect the result to be obvious. That can be the right thing to do if the test group is very small, but it creates a steeper learning curve for the person who replaces them.

Answer 2

Some tools for test data generation - online and standalone:

1. Datagenerator - free tool, DB data / tables generation.
2. GenerateData.com - free online script-based data gen, different output formats, including CSV, Excel and SQL.
3. Spawner Data Generator - sample/test data for databases.

More links may be found here, but the above 3 I use more often and they're currently supported and being updated.

Answer 3

One more online Data Generator: TeDaGen. It supports on-the-fly generated output conversion to many different formats (XML, CSV, SQL, etc.), as well as many fields as options to generate.

Answer 4

At my current company, we are a small team.

We have designated a shared folder on the file system where we place these types of test assets. And we each maintain them on an as-needed basis. This approach works well enough for us.

At other companies where I have worked, I had a larger team.

In some of those cases, one of the roles was that of a "Test Lab Manager". I usually assigned the task of collection, documenting, and publishing these sorts of test assets to this person. In those cases, it worked well for us.

Over the years, we have used the following for storage/retrieval of these test assets: - shared folders - Microsoft SharePoint - a wiki - my blog

And this might help: http://www.allthingsquality.com/search/label/Test%20Data