The flat file has millions of records and the source query is joining at least 10 tables.

The Testing I have done is:

1. Structure validation of the file. Like name & number of fields, delimiter, naming convention, Header or Trailer records.
2. Check for duplicate records.
3. Select one row from the target file and for that record alone, run the source query and then compare the Source output and the target record in the flat file manually comparing each field one at a time.
4. Check for data truncation.

The concerns I have is:

1. I cant employ minus approach between source and target because target is in flat file.
2. I tried loading data into excel and compare but excel has record limitation and hence not all data can be loaded.
3. In the where clause of the source query outer join is used. How to ensure that data is loaded accordingly.
4. Is there a way to automate this testing process.
5. Though I have written test cases dedicating one test case for fields sourced from one table. Is there any better approach. Pls provide some samples.

Thanks a bunch for reading this through and waiting for the suggestions.

The flat file has millions of records and the source query is joining at least 10 tables.

The Testing I have done is:

1. Structure validation of the file. Like name & number of fields, delimiter, naming convention, Header or Trailer records.
2. Check for duplicate records.
3. Select one row from the target file and for that record alone, run the source query and then compare the Source output and the target record in the flat file manually comparing each field one at a time.
4. Check for data truncation.

The concerns I have is:

1. I can't employ minus approach between source and target because target is in flat file.
2. I tried loading data into excel and compare but excel has record limitation and hence not all data can be loaded.
3. In the where clause of the source query outer join is used. How to ensure that data is loaded accordingly.
4. Is there a way to automate this testing process.
5. Though I have written test cases dedicating one test case for fields sourced from one table. Is there any better approach. Pls provide some samples.

Thanks a bunch for reading this through and waiting for the suggestions.