We are migrating our production data (DB and filesystem) from a fairly complex data model to another fairly complex data model. The migration process is given by the specification as a set of mapping rules and transformation functions.
I'm planning a testing strategy for this migration process. To identify the riskiest areas here, I've read some lessons learned from migration testing and ETL tools testing in general and compared them with our migration process specification.
Where the things may go wrong?
- For backward compatibility reasons, the new data will be provided by the new application to the old modules through adapters. Old modules will do their job using migrated data as they used.
- The new data model will support some new application features.
- The new data model will no longer support some old (usually used infrequently) application features.
- Nullable fields in the old model are no longer nullable.
- Files in the filesystem are defined in a different format.
- Data that used to be decoupled in the old system will now be joined (e.g. customers with their projects) based on:
- automatic rules
- administrator rules
- New data formats will be used
What may go wrong?
- Values for data used by new features may have incorrectly defaulted.
- Adapter may translate new data to legacy format incorrectly.
- Not all entities may be migrated, e.g., some old client configurations cannot be found in the new system.
- Not all data (properties) may be migrated.
- New data model may use data formats may not be able to store some old data structure (e.g., VARCHAR vs. BLOB)
- Migrated files may not be readable for old modules
- etc.
My very first idea is to combine two testing strategies here:
Low-level testing strategy that will be relatively thorough but will have little coverage. The idea is to isolate a most representative sample of original data (that makes sense for the business, e.g., single customer and its documents). Then, for each mapping rule define real expected output for the corresponding part of the sample data. Automate checks.
High-level testing strategy that will be less thorough but will cover the majority of application functionalities and try to identify integration problems with old modules. The idea is to migrate all data first. Then perform user test scenarios that involve all functionalities of the new application and of the old modules.
What other strategies would you suggest to discover bugs for the identified risks?