Testing data updates to data stores

Question

I'm currently writing tests for a Data Registration system. In summary, a data provider will supply a daily update file that will be processed by the registration system and if they pass the validation checks, the registration system will update the data it stores for those records contained in the update file.

I've been given a list of 85 different update scenarios - that is 85 different ways the data can be updated in the database, which consists of a series of related tables. Having had a look at these scenarios, I don't think that it would be necessary to test all 85 scenarios. The system is however implemented on large scale and is a nationwide system. For confidentiality, I cannot reveal details about the system, so lets take an example of a Police motoring system. So lets say, for example, the system accepts a daily update file from Vehicle Insurance companies containing changes to their records for details of the cars and owners they cover. Lets say that their are 85 different forms of update they can send - So 85 different update scenarios. An example scenario may be a change of Address of owner. Another scenario may be Change of Address and car of owner. A third scenario may be a change of name of owner and license type of owner, etc... The daily update file may contain an update for a single record i.e a driver or thousands. From what I have seen, these update scenarios seem to be categorised - 20 scenarios related to updating owner details, 25 scenarios relating to adding new vehicle owner, etc....

So my question is how would I write tests for these update scenarios without having to write tests for all 85? Is it a simple case of choosing a subset of scenarios from each category and write test for those? So for example, In the 'update owner details' category, I could have one test that updates all the owners details using the assumption that if that works, then a test for updating the owners name only, will work.

The testing environment is manual.

Your input is appreciated.

Answer 1

This sounds like a reasonably standard data input problem.

Here's how I'd look at it (this method is pretty heavy on up-front work, but has the advantage of making life easier in the long run):

First, I'd gather as many input files I can get hold of. These will have the input types you need in the correct format.

Next, I'd go through them and split out classic examples for each of your 85 scenarios. I'm going to assume that some of these are actually combinations of others (e.g. Scenario 1: update name. Scenario 2: update address. Scenario 3: update name and address). The goal here is to build a library of file imports that cover all your scenarios - in my example above, you'd flag one file as covering Scenarios 1, 2, and 3.

Once I had my set of valid scenarios, I'd make copies of some to build my invalid import test files - files that I expect to be rejected because required fields are missing or there is something else invalid in what I'm sending. If these aren't part of your 85 scenarios, oops. Someone missed something when they put that list together.

Eventually, you'll have a large set of files that you can test without much difficulty.

How you choose your tests will depend on - among other things - whether you can access the database directly or not. If you can, try to beg a developer to write you a utility to export the data tables that should be updated. Then all you need to do is make sure that everything works correctly once (making sure you save your output files as baselines) then run a file compare between your baselines and the output files from each set of tests.

If you don't have that access, then I'd suggest importing one file at a time, choosing files that hit multiple scenarios at once, and checking after each import that all the data you expected to be entered is in the system. If you hit a problem, you drop to the files covering fewer of the scenarios until you locate the exact scenario that fails.

Depending on the exact nature of your data, you can probably smoke test in a very small number of file imports or data entry runs (I'd go for file imports if possible because it reduces the chances of errors) - say for every class of record covered (such as "customer", "user", "insurer") one valid new record with all data, one valid update with changes to every field (except the key, obviously), one invalid new record where every field that gets validated has some kind of error, one invalid update where every field that gets updated has some kind of error. The goal is to make your first pass as broad as possible so that you don't need any further passes if everything is good.

I'd add that if you can automate, and you have direct read access to the database, this is the kind of tests that begs for some automation support: a simple utility to send/set files for import, retrieve responses (or parse logs), extract the records you just pumped in and compare them to a baseline you've previously validated would give you a more useful regression test set and allow you to focus more on the edge cases (which is likely to be where most of your problems happen anyway).